Standardized Testing and the Myth of the Meritocracy

Curmie is a little behind on his reading, or at least at writing about his reading, so it’s only now we discuss an article that appeared some three weeks ago in the Washington Post (or at least on their website). The original post, by Marion Berry (no, not the crooked DC mayor, that was Marion Barry), provides a single anecdote which, he claims, provides “A concise summary of what’s wrong with present corporately driven education change: Decisions are being made by individuals who lack perspective and aren’t really accountable.”

Well, as I became notorious for saying to my Asian Theatre class a few years ago, yes and no.

Let’s start with Berry’s credentials. According to Valerie Strauss at WashPo, he’s a “veteran teacher, administrator, curriculum designer and author.” According to his own website, he began his teaching career in 1952. That would make him roughly 80 years old. The last time he was actually in a classroom? Can’t say, but I’m willing to bet it wasn’t in this millennium. So while Strauss may see Berry as an authority, I see—at first glance, at least—an old guy who wants to sell more books and collect more speaking fees.

I really don’t want to denigrate Mr. Berry’s credentials, especially since I think he has a lot that’s good to say. But, much as I agree with his general assessment of standardized testing, it’s important to realize that a lot of his argument here is sheer crap.

Berry’s friend, Rick Roach, oh-so-courageously (Gasp!!!) took the Florida Comprehensive Assessment Test. The version Roach took was administered to sophomores. (Roach’s identity and the test in question are revealed here.) Miraculously, Roach lived to tell the tale. Of course, Berry reveals a lot about himself in his description of what makes Roach “successful”: “His now-grown kids are well-educated. He has a big house in a good part of town. Paid-for condo in the Caribbean. Influential friends. Lots of frequent flyer miles.” Ooooohh… I’m impressed.

Anyway, here’s Roach’s commentary:

I won’t beat around the bush. The math section had 60 questions. I knew the answers to none of them, but managed to guess ten out of the 60 correctly. On the reading test, I got 62%. In our system, that’s a ‘D,’ and would get me a mandatory assignment to a double block of reading instruction.

It seems to me something is seriously wrong. I have a bachelor of science degree, two masters degrees, and 15 credit hours toward a doctorate. I help oversee an organization with 22,000 employees and a $3 billion operations and capital budget, and am able to make sense of complex data related to those responsibilities....

It might be argued that I’ve been out of school too long, that if I’d actually been in the 10th grade prior to taking the test, the material would have been fresh. But doesn’t that miss the point? A test that can determine a student’s future life chances should surely relate in some practical way to the requirements of life. I can’t see how that could possibly be true of the test I took.

At first blush, this would seem a pretty damning indictment of the test.

Trouble is, some of those questions were revealed in a subsequent post: the reading test here and the math test here. OK, we can, perhaps, criticize these questions for relevance, but not for difficulty. We’re encouraged to see how we’d do. So I checked them out. Needless to say, I got them all right. That means I’m sufficiently well-educated to survive as an above-average high school sophomore. Somehow I don’t feel the urge to add that line to my CV. I do confess that I actually had to think about one of the math questions a little, but I ask you, Gentle Reader, to remember that I haven’t been in a math class at any level in almost 37 years.

Here’s the point: if those reading questions are in fact indicative of the difficulty of the test, then Mr. Roach’s score is indeed appalling. His claim not to have known any of the math questions reveals one of two things: he is—multiple Masters degrees, well-educated children, and Caribbean condo notwithstanding—a blithering idiot, or he’s a mendacious dirtbag, willing to say anything to advance an agenda. Either way, the fact that the likes of Mr. Roach occupy any sort of leadership role in the education establishment certainly dispels any myth of a meritocracy in these matters.

Roach’s comments also include this:

If I’d been required to take those two tests when I was a 10th grader, my life would almost certainly have been very different. I’d have been told I wasn’t “college material,” would probably have believed it, and looked for work appropriate for the level of ability that the test said I had.

From what I’ve seen of Mr. Roach, I kinda wish that had happened.

I’m also impressed (not!) by this quotation from a piece by Michael Winerip in the New York Times, cited with approbation by Mr. Berry: “As of last night, 658 principals around the state [New York] had signed a letter—488 of them from Long Island, where the insurrection began—protesting the use of students’ test scores to evaluate teachers’ and principals’ performance.” I am, in fact, disturbed by using standardized testing as the sole criterion to measure, well, anything or anyone: teachers, principals, schools, and students alike. But I’m more distressed by the fact that someone writing in the NYT would employ such an egregiously misplaced modifier, and that such a preposterous grammatical error would be quoted without comment by someone who purports to have solutions to all that ails American education. Irony abounds, to say the least.

Berry and Roach are absolutely correct in two areas. First, the tests are indeed designed by educationists (often Education majors who couldn't get an actual teaching job) who are granted virtual free reign, accountable to no one. Secondly, performance on a standardized test should never completely override a student’s achievements (or lack of them) for the academic year as a whole. I’ve been thinking about both these issues for a while—here’s a blog post I wrote six and a half years ago on the subject; my views haven’t changed in the interim.

Moreover, to say that cheating on these exams is endemic is rather like saying Tim Tebow is annoying. A couple of reports on recent cases are here and here.

But this doesn’t mean that standardized tests ought to disappear. At their best, they do distinguish between students from different schools, different backgrounds, different priorities. As I said recently, “there is [a] wide disparity between schools and their populations—being at the top of a weak class might be better or worse than being in the middle of a strong one. It’s useful to have some means of comparing such students.” Yes, it would be nice if the questions were devised by actual educators, if the scorers for these exams had real credentials, if there were legitimate oversight. But it’s not the test’s problem if Mr. Roach can’t handle algebra problems I was doing in 7th grade.

For all this, thanks are due to Mssrs. Roach and Berry for calling attention to some of the inadequacies in the system. Even though I strongly suspect that both of these guys are charlatans.