Thinking about assessment

The education literature likes to make a distinction between assessment for learning and assessment of learning. The distinction is, in my view, a necessary insight, but the way that it is conceived is both too limiting and prone to confusion. In this rant I am going present a somewhat richer framework for discussing different types of assessment for different purposes.

Where I'm coming from

As I've mentioned before, I am training to be a high school math teacher, and I am enrolled in what I consider to be an outstanding program through Collin College. I must confess that when I signed up for the program, I, in my arrogance, did not think that I would learn much. I am pleased to report that I was dead wrong. I won't go into why I was wrong, but I will say that I go to bed thinking about the ideas that come up from class discussion and readings and I wake up thinking about them. I remain (very) critical of some of the argumentation and scholarship in the readings, but it is extremely helpful for me to read them. I'm gobbling them up and loving it.

I have been, and remain, highly critical of the kinds of testing and incentive systems that have been set up by NCLB even though I fully support the goal of keeping schools and districts accountable for how well they serve all students, particularly the ones who are at risk of being left behind. Please see my previous posts on the matter (and more to come). NCLB does appear to be reaching that stated goal but it distorts the educational system as a whole and hinders progress in other important areas. But this essay is about assessment (testing and similar things). Whether you are a critic or supporter of NCLB you will agree that it is has greatly intensified the amount and importance of (standardized) testing in schools.

The Educators' Complaint

The education literature makes a distinction between assessment of learning and assessment for learning. A similar distinction is also called summative assessment and formative assessment. I will not attempt to give a full definition of these here. I don't think that the definitions in the literature bear up under close inspection, and the fuller the definition the less enlightening it is. Instead here is the rough idea through examples. Assessment of includes things like the TAKS, end of term exams, and major examinations that determine a student's grade. Assessment for learning is the on-going assessment that teachers engage while teaching. These include asking questions of the class, seeing what sorts of questions students ask. These are considered for learning because they help the teacher adapt teaching to the particular student.

The problem with our increased emphasis on assessment of learning is that most of that assessment isn't pedagogically useful. Some even argue that it is harmful in and of itself beyond the misdirection of resources (although I have my doubts about that claim). NCLB is a reality (which really does appear to be meeting its narrow, but important, goals), but the concern among educators is that it leads to too much pedagogically useless assessment. I agree, but I think that we are talking about assessment in a far too limiting framework.

Distinguishing distinctions

When we look at assessment, and try to categorize it, I think that we need to be looking at two dimensions, instead of the one-dimensional approach in the of-for distinction. We need to ask

1. What is the form of the assessment?
2. What is the purpose of the assessment?

The current discussion seems to think that all standardized tests (form) serve only to assess what a student has learned and not to adjust teaching (purpose), while all of the less formal (form) assessments are only used to adjust teaching (purpose). Certainly there is a strong connection between form and function, but when looking at assessment it will be useful to look at these along these two not-quite-independent dimensions.

Three purposes

When it comes to considering the various purposes of assessment I think that it is helpful to consider three separate purposes, not just the two in the existing conceptualization.

1. Adjusting: to help adjust teaching to the needs of the particular student
2. Grading: to provide feedback to student and family, to assign grades and work as an incentive
3. Accounting: to evaluate the teaching of the teacher, school, district.

Accounting is what we see in the testing that follows from NCLB. It is about rating and evaluating schools and districts (and within districts it will be used to evaluate teachers). It is the school administrators who have the most to gain or lose by these test results. And they are typically done at the end of the school year. Although students who fail the test will be intensively tutored so that they will pass a retake, these tests are not used to help students directly.

Grading is typically the assessments that a course grade is based upon. These are presented to parents and students. These become part of a student's record and are intended to indicate how much the student learned. Of course these will also feed back on how a particular student is taught. A teacher can learn from these that a student is not meeting expectations and so can look for ways to help the student. One characteristic of grading assessment is that it (almost) never goes beyond what has been taught in class.

Adjusting is used primarily to help determine how to teach a particular student. These can range from everyday queries while teaching to see if students are getting it or not. But at the other extreme these can be the kinds of evaluations that are used to determine whether a student should be in a gifted and talented program or in special education. Those typically involve highly formalized exams, but are used exclusively for determining how best to teach an individual student. Homework may be part of a student's grade (usually to get them to do it), but is used primarily as a frequent check of whether something needs to be retaught.

Any particular assessment can (and often) will serve multiple purposes. But when looking at any particular assessment it is useful to keep those three purposes in mind.

Form follows function except for when it doesn't

If you've been talking about the differences between similes and metaphors in class you may ask for examples to help with the learning that day (adjusting). But you may also ask for examples of each on an end of term examination (grading). So the same form can be used for different purposes in different contexts. I've praised the MAP testing that PISD does. But I honestly don't know what they use it for. I would hope that they use it to help differentiate teaching (adjusting), but it may be used primarily to track teacher performance (accounting). So here is a particular standardized test administered exactly the same way could be used for entirely different purposes.

Some forms of assessment really are single purpose. Some like the Texas TAKS tests can't be used for much other than accounting, and then only a limited type. The test is designed to distinguish between students who have acquired the basic knowledge expected for the grade level from those who have not. It doesn't do a very good job of discriminating between students at the high end or very low end. It is hard for me to imagine a set of exams that is more narrowly focused on one purpose.

With understanding come solutions

This understanding of purposes can bring real, practical, recommendations. The TAKS serves little direct pedagogical purpose other than accounting, we could save a great deal of time and money (that could then go to actually improving education) by sampling. Not every student needs to take the TAKS in every subject. Consider fifth grade TAKS requirements. Students take Reading, Math and Science. Not counting make-ups and such, that takes three full days for the students' to complete. But if the goal is to measure a schools' performance, then have one third of the students take Reading, one third Math, and one third Science. Students would be randomly assigned with neither student nor school staff knowing which student gets which test until test day. All of the tests can then be given on the same day.

I believe that the framework I've introduced above, first separating form from purpose and then distinguishing three separate purposes for assessment, allows for a more useful discussion of assessment than is common. At least it helps me think about these things more carefully, and I hope it does the same for any readers I might have.

The Murder of James Poullion

The New York Times reports on the murder of an abortion opponent. Yes, that's right an abortion opponent was murdered, apparently because of his protests. This is news in the man bites dog sense. We are not surprised when an abortion supporter is murdered for position, but I was gobsmacked to read of this case.

Although this goes without saying, I will say it anyway. I absolutely condemn this murder and anything like it. The man charged with the murder should be prosecuted to the full extent of the law for his terrible crime.

With that said, it is unclear the extent to which James Poullion was murdered for his views on abortion or because he made a persistent nuisance of himself. Of course the latter doesn't justify murder, but it might help us understand the motives of the killer.

Like many protesters for any cause, Poullion sought attention and controversy. He would occasionally stake out a position at a Farmer's Market and cuss customers out. Poullion's protests were loud, gory, and generally obnoxious. He appeared to be getting more provocative as time went on, but always staying (just) within the law. The assistant prosecutor is reported by the Times to have said that the suspect was annoyed by Mr. Pouillion's protests, especially when they were near schools.

So while I unequivocally condemn the murder, I don't think that he was so much murdered because of his views, but because of his protesting style. Neither is any justification for murder, but we should be cautious about concluding that abortion supporters are just as like to murder their opposition as abortion opponents are.

Asymmetry of passion

We need to recognize that there is a fundamental asymmetry between supporters of legalized abortion and opponents. Abortion opponents (at least in their rhetoric) treat abortion as murder. For them the legal system and the courts have enabled mass systematic murder with no recourse within the legal system to stop or bring those perceived murderers to justice. While I'm offering no justification, it is not too difficult to imagine how a few people with those beliefs could turn to violence.

For supporters of legalized abortion (like me), there is no heinous crime that our opponents are involved in. We think that our opponents are wrong, and that much harm would be done if they got their way. But only though the most elastic stretches of hyperbole could we consider them as supporters of systematic murder. To me, opponents of legal abortion are not evil or criminal. They are not sinful or inhuman monsters. They are merely mistaken. It is this asymmetry that makes it so hard for me to belive that Pouillion was murdered because of his views on abortion.

Congratulations: I may be wrong

I have been ranting (particularly here and here) that Texas' implementation of NCLB is doing a disservice to above average students. I am perplexed, but delighted, to report evidence that I have been wrong. Apparently, Texas students have been making remarkable gains in passing Advanced Placement exams.

The TEA has reported strong gains in AP pass rates, and the gains among some minority groups are truly spectacular. Being the cynic that I am, I had first assumed that the results were a consequence of fewer students taking the exams. But, according to the report, these gains while the number of students taking the exams has increased. So this positive result does not (immediately) look like the result of statistical manipulation.

My skepticism remains, and there are a few things to check out. But at the moment we have some good news, and I will take it as such.

It will be interesting to learn how these gains were distributed throughout the State. Do they come from a few school districts, and are those districts doing something unusual? If anyone knows where I can get this data, please let me know.

Schools are not a battleground

I have expressed in my posts here numerous criticisms of parts of our public education system. I have also expressed views about religion (I'm an atheist) that many people in my community would find anathema. At the same time, I am training to be a high school math teacher here in north Texas. Naturally, this can raise some legitimate concerns among potential colleagues, employers and student families when I start working. The goal of this post is the persuade even those who oppose everything I've stood for here that they have nothing to worry about with me in the classroom.

Why I want to teach

First of all my reasons for going into teaching are that I love explaining things and transferring knowledge. I believe that education is important in our society to help deliver on the promise of equal opportunity. Slightly more controversially I believe that for democracy to work the public needs to be educated to at least the point where they can meaningfully participate in the decisions we face. And even more controversially, I believe that public education plays an assimilatory role, preventing extreme fragmentation of society along ethnic, religious or linguistic lines. But if one sentence could sum up the feelings that drive me to be a teacher it would be a line from an old song:

I hear babies crying, I watch them grow, they will learn more than I'll ever know. And I think to myself, what a wonderful world. —Louis Armstrong

But stepping down from those lofty reasons, my job as a math teacher will be to teach math. I am one of those weirdos who really loves math. But I also love history, science and language and literature. I love thinking and ideas. But given my skills and what needs there are to fill, I can make the biggest contribution by teaching math. Teaching math will be my job and my mission.

Make things easier, not harder

Anyone who cares deeply about education, and I hope that that is true of every teacher, will have strong opinions about how we do things. Some of those opinions will include disagreements. But it helps no one to stir up trouble just to make a point. It only makes it harder for everyone to do their job.

A few individuals with their own political or religious agendas like to fight their battles in the schools. Whether it is people who object to the addition of under God to the Pledge of Allegiance or people who want school led prayer, they oughtn't make things harder for schools to do their jobs. I have little sympathy for those people, even where I might agree with their cause in principle. Ideally, I would like to see the Pledge restored to its original form, but I'm not going to make a fuss about it. Forcing school districts to spend real time and real money fighting a largely symbolic battle is destructive to our educational goals, angers people, and needlessly divides our community.

My concerns about the impacts of accountability requirements will not stop me from doing my job and focusing my efforts where state law and policy tell me to focus my efforts. Forgive me for waxing philosophical again, but there is much to learn from understanding where various policies come from. The policies come from people who were elected to make those policies or from people who were appointed by those elected. I may, on occasion, think that I know better then those people, but my conceit does not entitle me to defy those policies with which I disagree. Just as nobody should pick and choose which laws they obey based on whether they like those laws (otherwise why have laws?), I won't cherrie pick those policies which I like.

If I sign a contract with a school or a school district, I do so willingly and see it as a commitment to upholding my end of the contract. That means following their rules. It would be dishonest to sign a contract with the intention of doing anything other than accepting their policies. This point is point is very important. It is actually something that I thought about a great deal before enrolling in a teacher training program, but by seeking to be a teacher I have firmly decided that the value of doing so greatly outweighs the distaste for implementing a few policies I may not like.

In and out of the classroom

I can see no reason to ever mention my religious or political views in the classroom or to students. There are some things about myself that if expressed in the capacity as a teacher would be inappropriate. My religious and political views certainly fall into that category. However, anyone reading these blog postings (if anyone actually is) will realize that my political, social, and religious views are emphatically not secret. I am happy to share them with anyone (outside of my role as a teacher) who asks.

By seeking out my postings or searching for my musings you are asking me what I think. I am not pushing or advertising my views. I am not addressing a captive audience, and I am absolutely not using a school, classroom, or any authority I might have as a teacher to express my views. By coming here, you have asked me what I think.

Student discovery

I cannot prevent my students from doing the same sort of search or link following that brought you to my postings. I will do nothing whatsoever to encourage them to find this, but that won't stop the student who decides to google all their teachers. So some students will learn my views on politics, social matters and religion. This isn't the problem it might seem.

First of all, I like math, and that will be made known to all of my students. This already makes me a kook in the eyes of many of them. So if I have other kooky ideas, that should come as no surprise. And having one kooky teacher who is an atheist will be balanced out by having a dozen others who have more conventional views. I would be over-estimating my status and influence if I thought that students discovering what my views are would change their views.

I also don't think that I should be singled out for making my opinions available on line. If you are active in your church or a civic organization, if you have ever contributed to a political campaign or are registered to vote for a particular party, then your religious and political views are available to anyone who knows how to search the Internet. You may not have gone into detail about your views, but much will be guessed from your affiliations. Should everyone who works for the public schools try to eradicate all public traces of their views on religion or politics? That would be both unwise and impossible.

Deletion and pseudonyms

Should I remove my postings on controversial matters from the sites I control (this blog and goldmark.org? Well, I could, but I have a posting history that goes back to 1986, and I do not have the ability to have everything I've said over the decades removed from where they can be searched. And given how my views, approaches, and spelling have changed over the past 20 years, I would much rather have someone read my more recent comments than my older ones. So if I attempt to remove what I can, then what is left paints a very distorted picture of my views. In truth there is no delete button on the Internet.

I should also make it clear that when I first became active on the Internet, we didn't use pseudonyms. The usernames under which we posted were the usernames that we had on some mainframe computer and were not under our control. Pseudonyms were possible, with effort and resources, but rare. So from the beginning I've posted under my own name. The other problem with pseudonyms is that they are very rarely anonymous. It is possible to operate truly anonymously on the net, but it is difficult. (Most people who think that they are operating anonymously are not.)

It is also not my style to hide from what I say. It would feel dishonest writing under a pseudonym. I should not feel embarrassed by what I say in a public arena. Of course over the decades I've said plenty of things that I'm embarrassed by, and maybe years from now I'll be embarrassed by what I write now. Nonetheless, I hope that my openness about my views and the explanation I've given above should make it clear that my various musings on the Internet will not interfere with my teaching.