Four events from Curmie’s (much) younger days.
1. I was maybe ten. Someone at my school had bought a contraption (an Ur-version of a modern computer) that measured not only how accurately but how quickly students responded to a series of math and vocabulary questions. I was accused of cheating (how and why would I do that on a test that didn’t count for anything?) because I answered all the questions correctly in a time that would have been considered excellent for a college student.
2. I was in college. A friend was doing a psychology experiment on the effects of caffeine in various quantities. She’d read off a number and my job was to add 17 to it. She charted response time and accuracy. Then she bought me a coke, I drank it, and we repeated the process. I don’t remember how many times we did this. I do remember that every time, I answered correctly to every question and did so in a time a quantum step or two faster than any of the math majors she tested.
3. This was also in college, but in a different venue. Another friend would read off a three digit number, then start carrying on a conversation. After an appointed length of time—30 seconds, a minute, 3 minutes—she’d ask me to repeat the numbers. I did, accurately, every time. Then we repeated the test with letters: same result.
4. Because I did my MA in England, I was actually already teaching college when I took the GRE. My preparation for the math section was minimal: I did a little brushing up on algebra, but I didn’t take any test prep courses, and I hadn’t been in a math classroom in about five years. The night before the exam, I was at a party until about 4:00 before the 8:00 a.m. test, and yes, I’d had a couple of beers. With this unimpeachable regimen, I proceeded to get a perfect score on the math section of the GRE.
I don’t know why these related memories clicked into my mind yesterday morning, but I suspect it might have something to do with the ongoing debate about standardized testing: in particular, the comments of one John Barker. I’ve made it pretty clear over the years what I think of the increasing emphasis on standardized testing and the accompanying teach-to-the-test mentality that has infested public education in recent years. That is, whereas I grant the “objectivity” of these exams, I am skeptical of their accuracy, their relevance, their potential either to measure outcomes or to predict future performance, and even—in light of the cheating scandals we know about (and the certainty that there are those we don’t)—the integrity of the process. I’ve made these points repeatedly on this blog and its predecessor over a period of nearly eight years: here, here, here, here, here, and here, for example.
Mr. Barker, the chief accountability officer for the Chicago Public Schools, thinks otherwise. Well, duh. His job is to legitimize his own salary—well over twice mine, by the way—and to pretend that his ultimate boss, the despicable Rahm Emanuel, is something other than the venal corporate meat puppet he truly is.
The Barker quotation that’s attracting all the attention from the teaching profession is this: “My philosophy has always been that if it's a good test, teach to it.” This inanity encapsulates precisely the sort of folksy pseudo-sensibility that characterizes the accountability crowd. The implicit underpinnings of this argumentation are two-fold: that every student, everywhere, ought to be learning not merely the same basic concepts, but precisely the same thing, and a smug, unspoken assertion that the “good test” in question not only exists, but is employed universally. Needless to say, these self-serving rationales have something of the aroma of merde de taureau about them.
But I want to concentrate attention on another of Barker’s pronouncements: “I was watching the NFL combine last night, and these guys are running 40-yard dashes. If they haven't trained for that particular test, they're going to have a problem. But as you run the race in that particular test, you get better and you know more about yourself.” Seriously, he said that. Look, Gentle Reader, you and I both know that the comparison is inane. I’m pretty sure we can take as given that the sprints in question aren’t designed to increase participants’ self-knowledge.
And here’s where we return to the stories of Curmie’s youth. What, after all, was determined by the fact that I had scores that were (literally) off the charts on a couple of tests? That I was some sort of genius? Hardly. I was a bright enough guy, but all that was really determined was that I can do easy math really quickly and that I employed some basic mnemonic devices in memorizing number or letter sequences, even when I wasn’t intending to do so. I recall, for example, that one of the letter sequences happened to be a friend’s initials, a fact I noticed immediately and couldn’t “unremember” even though I’d been instructed not to try too hard to get the answers correct. And, of course, as an actor, I was used to finding (or creating) connections between seemingly unique data to aid in the process of (wait for it) memorization.
How’d I do so well on the GRE? Well, for one thing, they didn’t ask anything hard: no analytical geometry or calculus or probability, much less stuff I’d never seen before. Nobody does easy math better than I do. Lots of people do hard math better than I do. If you want somebody to add 17 to a two-digit number in a hurry, I’m your man. But when you start talking about natural logarithms and second derivatives, I’d really suggest that you look elsewhere. What these tests, any or all of them, didn’t show was whether I could memorize long strings of letters and numbers, or to do higher order math: addition (or even basic algebra) is a long way from calculus or set theory.
Thus, I’d like to look at the parallelism between that scouting combine and a standardized test in a slightly different light: namely, what is done with the data collected. Those NFL scouts at the combine understand that a time in the 40 provides a single piece of objective but only marginally relevant data. No offensive tackle is going to be asked to run 40 yards on a single play, and certainly not in a straight line. Even receivers are going to be sent in motion or not, to be positioned as a wide-out or in the slot, etc. Quickness means more than speed, and pure speed without strength (also objectively measurable) and savvy (not measurable) doesn’t amount to much.
That is, whereas a scout or a general manager might be interested that this player is two-tenths of a second faster than that one for forty yards, those times will be factored in with dozens of other bits of data—some objective, some subjective—in determining whom the team should draft. Crucially, the scouts analyze everything they can about a prospect: his strength and speed, sure, but also his work ethic, sense of teamwork, flexibility (in both the physical and attitudinal senses of the term), knowledge of the techniques of the game, etc. No player will be drafted or not based solely on his speed in the 40, and no one is going to be stupid enough to judge his college coach on the basis of that number.
Moreover, whereas it may be true that a particular prospect does a little better on the test because he’s “trained for it,” and someone else is “going to have a problem,” that very fact diminishes if not completely undermines the legitimacy of the result: since no one is going to be asked to run, unobstructed, for 40 yards (or to bench press stationary weights, or whatever) in a football game, the purpose of the test is to approximate the skill set actually required by a top-notch player in a manner that is objective and at least reasonably accurate. That a player “trains” for the test so that, hypothetically, he gets a faster start from a body position he’ll never use in a game situation only distorts the test results, rendering them even less valuable than they’d already been. The analogy to test preparation services—which, assuming they work at all, of course, benefit those who can afford them at the relative expense of those who can’t—seems obvious.
Yes, it tells you something about a student that s/he does really well on some exam, the same way it tells you something about a football player if he gets a great time in the 40. The difference is that NFL teams are smart enough to know that they’re seeing only a snapshot, whereas the educationists—especially those, like the good Mr. Barker, who have apparently never spent a day as an actual educator—place increasingly higher emphasis on those isolated moments in time. Would I rather have a football player who runs the 40 in 4.5 seconds than one who runs a 4.7? All other things being equal, sure. But all other things are never equal. Never. Allow me to repeat: never.
Not only are strength, explosiveness, balance, and a host of other variables just as important as speed, but there’s one more factor that in fact occupies the very center of the discussion, although Barker may be too dim-witted to understand. A 4.7 is a great time for a lineman; a 4.5 is average (by NFL standards) for a wide receiver. That doesn’t mean the wide receiver is “better,” only different. Don’t make a quarterback try to decide between the guy who’ll make great catches and the guy who’ll keep him vertical to throw the pass at all.
Similarly, whereas standardized testing can provide useful insight into a student’s preparation, such exams not only don’t measure everything (try creating an objective test for poetry, or intellectual curiosity, or kindness), they don’t really even measure what they measure. Looked at intelligently, which is to say skeptically, they provide some useful information. But that requires both work ethic and wisdom, two attributes conspicuously absent in most educationists. Student populations differ—class to class, neighborhood to neighborhood, year to year. The data I, as a university professor, compile for our bullshit assessment tells us far more about the caliber of students we’re attracting than about either my skills or the course structure.
A grain of salt would come in handy, in other words. Otherwise, we end up with wicked fast players who can’t… you know… catch or block or tackle. And that’s no way to win.
