No Child Gets Ahead - The evidence

In April I wrote a piece No Child Gets Ahead in which I argued that current implementations (and particularly in Texas) of the No Child Left Behind program is detrimental to the interests of the above average student. Let me also remind everyone that I consider the goals of NCLB laudable and important. Again, see that earlier rant for a defense of those goals.

Now there is increasing evidence that I am correct. Brighter students are not advancing at the rate one might normally expect of them. This was discussed in a New York Times opinion piece titled Smart Child Left Behind on August 28, 2009. The authors, Tom Loveless and Micheal Petrilli, refer at first to report by the Center for Educational Policy published in June 2009.

The Rosy CEP report

The CEP report asks the question in its title, Is the emphasis on proficiency shortchanging higher- and lower-achieving students? Their answer is no. But Loveless and Petrilli argue that the CEP's report is deeply flawed. After reading the report, I entirely agree that it is broken beyond repair. The most egregious error in that study is the exclusive use of state proficiency test scores. State proficiency tests are designed to measure skill at the grade proficiency level. They never test anything above grade level (which is where advanced students are). My anecdotal experience is that pretty much everyone in my daughter's gifted and talented program hit the ceiling (score 100%) of the state TAKS tests. State proficiency tests are not designed to measure learning beyond the grade level proficiency levels, and simply don't work to measure learning for the high level students. The CEP report pretty much spells out the flaw without realizing it

The main measure of student achievement for this study consists of data from the state tests in reading (or English language arts) and mathematics used for NCLB accountability. Although no large-scale test provides a complete picture of student achievement, we have analyzed state test results because these tests are given to nearly all students in a state, are intended to reflect each state’s academic content standards, and are designed to assess whether students have met their states’ expectations for performance at a particular grade level. [Emphasis mine.]

It appears that the CEP report measures success in a state by looking at state results in terms of the percentage of students (with each state) scoring proficient or above. This notion of counting the number of students who exceed a certain (minimal) standard as a way of seeing whether you are serving the higher performing students is entirely missing the point of the exercise. The question we are asking is Does the way we measure school success shortchange the top students?. The CEP's answer appears to be, Well if we count success according to the way NCLB measure it, then we have success..

Another astounding flaw in the CEP analysis is their use of states as their level of analysis. For them, a gain in a small state completely off sets a lose in a large state, even if it means a decline for millions of more students than there is a gain for. This is truly blushworthy error. even though they fully acknowledge it (on page 18). I could go on. These are not minor technical quibbles. These problems completely and utterly undermine the CEP conclusions.

Why worry

Before I go on to cite the evidence for my assertion that NCLB does shortchange the better students, let me spell out why I and so many others worry that it would do exactly that. I've outlined these reasons in my earlier rant and when combined with the actual level of these proficiency standards (see my rants, Race to the Bottom and No Comparison) there really is a concern. As I said before, people and systems do respond to incentive systems, so we should look very clearly at what we incentivize.

Two Scenarios

The NCLB incentive system rewards schools and districts for the number of students who pass (minimal) proficiency tests. The margin of passing or failing (how high above or below the passing cut-off) counts for nothing. Imagine a class with three students: Alice, Bob, and Charlie. And suppose that the proficiency level is considered met if a student scores a 70 on the crucial test. (Obviously I'm grossly simplifying the examples for the purposes of illustration.)

Now consider scenario 1: Alice scores a 78 and passes. Bob scores and 70 and passes, Charlie scores a 50 and fails. In this scenario, the class has two passes and one failure. That is what will be counted in determining the school's, district's, and (probably) teacher's rating.

Now consider scenario 2: Alice scores a 95 and passes, Bob scores a 69 and fails, Charlie scores a 62 and fails. This class has one pass and two failures. The school, district, and teacher will be marked down severely for this.

All of the incentives (and they are powerful incentives) of NCLB push for scenario 2 above scenario 1. But let's look which class is serving the students better. Both Alice and Charlie do much better in scenario 1 than they do in scenario 2. While Bob does slightly worse in scenario 1 then in 2.

Of course you may object that I could have set up an example where the class that did better on NCLB criteria would also be the one that we would all agree better served the students. But my example illustrates real choices that schools and teachers make every day.

Suppose that you are a teacher and you are confident that Alice will pass the exam with little extra effort from you. With more effort from you, she might learn a great deal, but she is already on a clear target to pass the exam. And suppose that Bob is a student who looks like he will pass the exam, but only if you put extra effort into preparing him. Finally, as a teacher, with all of your pre-tests and such, you determine that even with an extraordinary effort on your part, Charlie is unlikely to pass the exam. If you want to keep your job, and the school wants to keep property values high in its district, then you will focus your effort on Bob.

Response to Response to Intervention

In the excellent teacher training program that I am currently enrolled in, we have been studying the mechanisms by which we identify and help the struggling student, Bob, before he falls too far behind. It is a program (or framework) called Response to Intervention (RtI). It really looks like it should be effective at identifying students like Bob earlier and getting the teacher to devote more time to Bob's needs. But of course any additional time spent on Bob will be time taken away from Alice and Charlie (unless additional staff are provided or the school day is lengthened).

This is one thing that people always seem to forget. Anyone who says that we need to spend more time doing X (where X is "with struggling students", "in the library", "teaching math", "practicing bus evacuations", "taking tests", etc) needs to remember that that means spending less time doing something else. This applies to money as well as time. An additional dollar spent on X is a dollar taken away from something else. It's easy to say what we should spend more time or money on, but it's very hard to answer the question of where that time or money comes from.

How to find out

I've already explained that if we want to test whether the incentives set up by NCLB create a disservice to above average students, we can't measure that by counting how many states have an increase in the number of students reaching the proficient level in that state. So how do we check? First of all we will need to use measures that are (a) comparable across states, and (b) which accurately measure the skills of the above average student. Ideally, we would like to have (c) where the progress of individual students from year to year is measured.

Getting data that is comparable across states is difficult. NCLB allows each state to set its own minimum and proficient standards. Because both politicians and educators like to be able to boast about how well their students are doing, there is pressure to set these standards low. (There are also some good reasons to set them low.) As a consequence of this, there is an incentive to shy away from mechanisms that allow state standards to be compared with one another or have students from one state compared with those of another. (See my earlier rant, No Comparison.)

We also need achievement results that don't suffer from a ceiling effect. That is, it should assess the full range of student achievement including those students near the top. This can be difficult for a number of reasons. First of all, the state assessments for NCLB are completely unsuited for this; so any tests would need to be in addition to those required for NCLB. Secondly, most testing to see whether students have learned the material presented in class; thus they rarely can test students who are above grade level.

Fortunately, there have been an number of attempts to collect such data. In an earlier post, I discussed the Measure of Academic Progress produced by the Northwest Evaluation Association. This time, I will be looking at The Nation's Report Card: Writing 2007 produced by the National Center for Education Statistics (part of the US Department of Education). They developed a scale which should include most advanced students, and sampled school children from across the country. Details of their method can be found in the report. For our purposes, merely showing a chart on page 9 of their report should make the point.

Before NCLB went into effect nation wide (2002) there was no growth in 8th grade writing skills at the lowest levels, while there were gains at the highest level. After NCLB went into effect, there were gains significant at the lowest levels and stagnation at the upper levels. Now I admit that I did troll through reports to find the most dramatic example. But for all grades studied and in all areas we find that NCLB has led wonderful gains at the lower levels. These are important and valued achievements. At the same time, it has lead to a flattening of growth at the higher levels.

In all fairness?

As I've said elsewhere, gains in one place often have costs elsewhere. If we have to have a trade off of improvements for the top students or improvements for the bottom students, maybe we are redressing a prior imbalance by focussing on the struggling student. I will address this issue in a later post. Here I will say that the situation before NCLB was destructive and unjust, with the below average abandoned. NCLB needs to be credited with fixing that. But the current situation, in which the above average student is ignored by the educational system, is little better. But whether you think that the current situation is right or wrong, I hope that everyone realizes that it does shortchange the above average students. In future posts, I will try to elaborate on how I think we can develop an accountability system that establishes incentives which serve all students.

Call Me Uppity

Once again there is a complaint about "militant atheists". The latest to reach my door step was an opinion piece in today's (August 16, 2009) Dallas Morning News by Rod Dreher. (Note that the on-line version doesn't have the subheading that exists in the print edition Militant atheists think replacing religion with science will solve everything. But the title does talk about atheist fundamentalists)

The article itself is mostly arguing that science and religion don't need to conflict, just as long as each stays within its proper domain. I do not intend to deal with that here. It's something that I've been wanting to write about for a while, but here I want to write about the phrases militant atheism and atheist fundamentalist.

Let's look at what it takes to be a militant for any other cause or belief system. Environmentalists usually have to engage in or actively endorse violence or, at the very least, vandalism to earn the title. Racial supremacists often need to actually try to kill someone before they are called militants. Merely rehearsing race war in the woods with your friends on the weekends doesn't quality. Anti-capitalists have to riot to get called militants. As for religionists, in this country they can withdraw from the public school system, go door to door telling you you are damned to Hell, demand special holidays, prevent the sale of alcohol on their sabbath, disregard parts of the Constitution; and we call this business as usual. To be a militant, they actually need to kill someone. But when they do they are still not called militants religionists unless they are Muslim. They are just called crazy individuals acting alone. When a women kills her child because she believes it is possessed by demons we call her crazy (as she is), even if the belief in demon possession is reinforced in her church. When preachers and politicians talk about placing Jesus and the Bible at the heart of government or suggest that they are on a mission from God to transform the nation, we don't even bat an eye.

But what does it take for an atheist to be called militant? Well, openly declaring that you are an atheist (say wearing a t-shirt) might do the trick. If that doesn't do it then maybe taking the extreme step of encouraging other atheists to come out of the closet must be militant activism.

But the thing that will really get you labelled as a militant is stating that you find many religious beliefs silly. Let's be clear, we are not talking about go door to door ridiculing people's beliefs. We are not talking about putting up billboards making fun of their beliefs. We are talking about writing books and posting blogs. Maybe there is the occasional letter to the editor. The mere fact that we are now feeling free to openly challenge religious beliefs gets us called militant.

I try to respect people. And if their beliefs were private matters, I would have no reason to fuss about them. But we all know that religious beliefs have played a major role in history and are likely to continue to do so. They are influential ideas in the public sphere. Like any beliefs that matter, they must be open to criticism and even ridicule.

This is why we get called militant. People like me no longer bow to the taboo of criticizing religious beliefs. But please face up to what bothers you about that, and use the right word. Call me uppity.

Kool Aid is the answer

For some time now I'd been considering a question — not exactly an earth shattering one — which I've only just realized has been answered since long before I was born. The answer is Kool-Aid. You may think that any question whose answer is "Kool-Aid" would be a silly question, but it's merely a geeky question.

I am a fan of flavored water. In our house we drink filtered tap water, bottled flavored water (usually store brand), coffee, and a few juices. After going through a bout of intensive cola drinking after moving from Hungary to Britain in 1994, I simply stopped drinking the stuff in 1998 when my daughter was born. As a household we simply shifted to plain old ordinary water.

A few years ago I discovered unsweetened flavored bottled water. The flavoring (particularly any citrus-like flavoring) made the water seem more refreshing. So each time we go shopping, we pick up a dozen or so plastic bottles of flavored water. It is a regular staple of our household, and the only reason that I don't go through dozens of bottles of it a day is that I don't want to buy that many.

Now let me make it clear that I am not a particular fan of bottled water. I like water in a bottle, but the idea of buying water that gets hauled around in trucks for 1000 times the price of the built-in water delivery system that is in my house just offends my sense of efficiency. (I have neither training as an engineer nor as an economist, but I share their distaste for waste. So I like to think that I think like an engineer or economist.) You can think of this inefficiency in terms of money it costs you or in terms of environmental damage. In this, as in other cases, it works out to be the same.

So, I thought, wouldn't it be nice to just be able to purchase the distilled flavoring that is used for flavored water. It would be much cheaper/efficient to transport and store, and the water I would mix it with would be much fresher than that in a purchased bottle of water. So why doesn't someone go into the business of selling just unsweetened flavoring that I could add to the water at home? Well, Kool-Aid has been around since 1927 doing exactly that. Of course they add food coloring, which I could do with out. But I can live with a little food coloring to save on consuming stall transported water.

So on my next shopping trip, I'll be picking up a few ounces of Kool-Aid instead of a few pounds of water. It will take some experimentation to get the mixture right, and there is always the possibility that no Kool-Aid flavor does the trick, but I suspect that the flavoring that is used in Kool-Aid is pretty much the same flavoring that is used for flavored water.

Voters do need an explanation

One of the many astounding things that Governor Palin said when she announced that she would not be completing her term as Governor of Alaska was,

I think of the saying on my parents' refrigerator that says Don't explain: your friends don't need it and your enemies won't believe you anyway.

What Palin seems to have failed to grasp is that the world is not divided up simply into enemies and friends of Sarah Palin. There are people who voted for her as governor who may not fall into either category. They, along with every other citizen of Alaska, need an explanation. And more importantly to her future, everyone who will face a ballot with her name on it deserves an explanation.

This failure to try to address people who don't initially agree with her, but aren't (yet) dead set against her will, in my view, be her political undoing. She has a substantial and extremely committed group of supporters. But they are in a minority. She needs to stop insulting east coast elites or everything that isn't part of her vision of the real America if she hopes to win any votes there. Todd Purdum's profile of Palin in Vanity Fair includes many suggestions that she takes an if you're not with me, you are against me approach to people.

More than once in my travels in Alaska, people brought up, without prompting, the question of Palin’s extravagant self-regard. Several told me, independently of one another, that they had consulted the definition of narcissistic personality disorder in the Diagnostic and Statistical Manual of Mental Disorders — a pervasive pattern of grandiosity (in fantasy or behavior), need for admiration, and lack of empathy — and thought it fit her perfectly.

I am not for a moment saying that I agree with these anonymous interviewees. People have a strong inclination to attribute kookiness to those we end up in conflict with, particularly if they have strong personalities. But I do quote that text to suggest that Palin has tendencies in those directions. She tends to divide people up into enemies and friends more than most of us do; and she is lacking in a kind of intellectual empathy, the ability to understand how a reasonable person could hold a different view.

We must remember that she is the person who took an innocent question of where do you get your news from? to be an attack on the remoteness of Alaskans. Righteous indignation is not the appropriate response to questions like that, yet that appears to be how she responds to anyone how offers any kind of challenge to her. Yet that is exactly what we would expect to someone who sees enemies everywhere and can't understand that some challenges to her are in good faith.

I believe that unless she learns to overcome these habits of thinking, her political career is doomed. She may count on a strong base of support from people who don't have to work with her, but she lacks what it takes to build a broader coalition beyond that base of support. And she will find fewer and fewer people willing to work closely with her. Please keep in mind that I am not talking about her successfully reaching out to people like me. We are too far apart in too many ways for anything like that to happen, but she needs to reach out within the GOP. Her no explanations stand will not help her with that.