Friday, October 28, 2011

Lies, Damned Lies, and Statistics




Somewhere in your school district is a group of people who care a lot more about standardized testing than you do. Since you're reading this, you may be a Slacker, which means it's possible that someone in your school district cares about lots of things more than you do. Still, I'm guessing it's true for everyone else as well, assuming, of course, that your name doesn't start with "Dr. Betsy," or "Dr. Dennis." Oh, dear, I hope your name doesn't start this way...

Because there are very smart people who care a great deal about standardized testing, there are lessons that can be gleaned from what your school is currently doing to game the system. I don't mean cheating, although people have tried that too (see here and here). I mean perfectly legitimate ways that schools use to improve test scores, many of which require little or no additional effort. Here are a few tricks, and ways to adapt them your own Slacker ways.

What they know:
Statistics matter - Schools must make (or "meet," as we all seem to say) Adequate Yearly Progress. If they do, they get to keep their funding, get to stay open, and get a little plaque celebrating their status as "Adequate." This is determined, however, through a more complicated formula than you might expect. Rather than a simple matter of reaching a passing score, it is tied to all sorts of statistical trickery. Like the near cousins lies and damned lies, statistics can be used to tell a story--and best of all, you choose the story. My school met AYP not by having our scores reach a certain threshold (they didn't), or by making significant improvements in scores (they may even have gone down). We met it through a morass of "safe harbor" and "confidence intervals" and other mathematical voodoo that defies simple explanation. If you'd like a complicated explanation, see the 54 page document called, and this is funny, "Understanding Your Adequate Yearly Progress (AYP)."
How you can use this:
Spin and statistics are closely linked to one another. You're probably already well-versed in some forms of comparative analysis--but everyone failed that quiz, or no one turned that in on time. However, you should experiment with some others as well--yes, I failed that, but right now I'm failing many fewer things than last (month, year, time I was in 9th grade...). Also, keep in mind that 87% of statistics are made up on the spot, so don't spend a lot of effort on the research.

What they know:
The report itself matters - You feel like you spend the entire month of April taking high-stakes tests. In truth, you spend three days in April taking high-stakes tests. You spend the rest of the month taking relatively low-stakes tests. Your school knows that only the Math and Reading PSSAs count toward AYP, and consequently get reported in the newspaper. We pretend to take the other tests seriously, but we've already shot our collective wad on the Math and Reading and basically just coast through the others.
How you can use this:
Kids with good grades do this sort of thing all the time. In your syllabus (if you've lost it, you can probably find the same stuff in the online grade program) is some kind of formula that tells you how much everything is worth. In there you will find the high-value stuff, and some general padding to make things look more legit. Many teachers communicate this by dong a "total points" calculation--10 point quizzes are worth exactly 10% as much as 100 point tests. Others use the "weighting" option to tweak the points so that quizzes have a certain value regardless of how many questions are on them. Either way, figure out what something's worth before you put effort into it, or not.

What they know:
Take care of yourself - Schools have studied data that says students do better on testing if they've had breakfast. So, we feed you breakfast on days of testing that we care about. We can't control how well-rested you are, but we'd take control over that too if we could.
How you can use this:
Take care of yourself. There are limits to what this can do for you, and I've probably tested those limits as much as you have. Skipping the studying entirely in favor of a good night's sleep means that you will ready for peak performance, but will have no information to write down on the paper. Still, it's a handy excuse for giving up and going to play video games.

What they know:
Demographics matter - However, not in the way you may think. They matter in a sort of Bizzaro, bass-akwards way. It probably started with the fact that standardized testing has a long history of problems with racial bias, which you can start checking out here. Rather than deal with the underlying problem, NLCB (we don't say "No Child Left Behind," or even "so-called No Child Left Behind" anymore because everyone knows the name doesn't match what the legislation has actually done to education) pretended to care through an unfunded mandate. For AYP calculation, if your school has a high enough number of minorities, or economically disadvantaged, or even special education (see below) students, these subgroups must meet the higher standard on their own. In other words, if your demographic is well-represented in a school, you no longer only belong to the student body as a whole, but to your a sub-group as well.
How you can use this:
You can't really. If you're a minority, you're entitled to any advantages you can get. If you're not, it's too late to start.

What they know:
IEPs matter - IEPs matter in standardized testing in much the same way demographics matter, that is: kind of backward. There are some adaptations made for the testing experience itself, but Pennsylvania has exempted itself from adapting the content of the PSSA. This means that you may be in 11th grade, taking the equivalent of 8th grade math, but you will still take the 11th grade math PSSA.
How you can use this:
You can't. Not here anyway. We'll do a whole thing on what IEPs can do for you, and how to get yourself one, but some other time. For this, it doesn't really matter.

What they know:
Slow and steady - The best way to prepare for any large and important task is to prepare continually and non-specifically. If your school has a "daily vocabulary word" or a "math problem of the day" or a "Drop Everything And Read" program, it may be something based on this theory. The College Board even publishes a Question of the Day for the SAT, probably to prove that they are our benevolent overlords.
How you can use this:
My father has a theory that if he would lift a small child every week at church that his strength would increase in line with the child's growth, so that eventually he'd be able to lift a full-grown person. He's never had the attention span to find out if this works, so he can just assume that it would. The merit of this theory is that a drop of water can level a mountain, given enough time and enough of its little friends. We have a similar axiom in music: ten 6-minute work sessions are much more likely to stick with you than an hour straight. It's a tough one for Slackers to make use of, especially since our efforts tend to come in the form of furious sprints from behind, rather than slow and steady progress. It is worth noticing, however, that if more benefit can be gained from the same amount of work, this is probably something we'd be interested in.

What they know:
Environment matters - If you take the SAT in a room with a marching band practicing right outside your window, there's going to be an incident report filed. For the PSSAs (the ones that that matter, anyway), our school converts the cafeteria to testing central. We adjust the bells so that there are no distractions during the tests. We provides nice long breaks, complete with thumping pop music to get the blood going. Even the cafeteria workers prepare lunch on those days in total silence so that nothing interferes with the testes' (testers'? testingers's??) efforts.
How you can use this:
Again, Slackers are rather sunk on this one. If we do our homework, it's going to be on the bus, or in front of the TV, or during chorus. These, as you know, are terrible places to try to concentrate, but they are better than skipping the work entirely (except for chorus. That should just never happen.) Still, if something is of high point value (see above), requires concentration (even from you), and your girlfriend isn't taking the same course that semester, you really should try to find a work-space that gives you a fighting chance.

What they know:
Cut everything else out - This sounds ridiculous, but it's more common than you think. A friend of mine was interviewing for an administrative position in a school district and was marveling to the interviewer about his school's test scores. He said, "Well, our kids don't play the flute." The truth is, we can make the test scores whatever you want, its just a matter of how much of the rest of it you're willing to cut away.
How you can use this:
Don't use this. Okay, maybe it's good to have a thought about how many activities you can add to your schedule and still contribute well to all of them, plus function in school. On the whole, however, don't do this. School is important, but the other stuff you do may prepare you for life just as well.

It's tough for Slackers to care about grades. In fact, if you want an excellent rant on getting rid of grades entirely, check out this exceprt from Robert Pirsig's book Zen and the Art of Motorcycle Maintenance: An Inquiry into Values (more on this another time as well). It's tough for real educators to care about standardized tests. The problem with grades and standardized tests is that our well-developed rants about them fail so badly at making them go away. For now, it's probably worth it to game the numbers so that they come out more in our favor. In fact, proving that we can do this so easily may be more effective than the rants ever could be.





No comments:

Post a Comment