A test result is just a test result

This past weekend was Catskillsconf - my favorite event of the year. I spent the weekend up in Ulster County with a bunch of my students. some great friends old and new, and Devorah. It was a great weekend but I was mostly offline.

As a result I missed a rather heated discussion in the CS Ed Facebook groups. The debate was over whether or not Strong AP CSP exam results are indicative of a good curriculum or good professional development (PD).

Further, anyone who thinks that a set of exam results can tell you that a particular PD sequence or curriculum is good shouldn't be allowed to call themselves a teacher and I don't want them anywhere near my kids.

I guess that language is strong, even for me but it's true.

Tests are designed to assess students and many tests don't even do that well. They shouldn't be used to measure something further removed.

The last time this idea made its rounds was using student test results for teacher evaluation. They do that in NY. A judge called the practice "capricious" and "arbitrary." My friend and former colleague Gary Rubinstein showed a year or so ago that standardized test scores varied enough from year to year so as to make the ratings useless ¹

Using test scores to say a PD sequence is good? What if that PD focused on test prep? What if it did nothing in particular? When I was at Stuy and when I was at Seward Park the Calc teachers' students had great AP results. I can tell you that none of those teachers had any quality PD specific to AP Calc.

Curriculum? My mentor and friend Danny Jaye told me many times - "A great teacher can save a class from a horrible curriculum but a great curriculum will do nothing for a class with a horrible teacher." He was right. Again, what if the curriculum encourages test prep. What if pressure on the teacher encourages them to focus on test prep regardless of curriculum or PD (see my recent post).

An AP test measures one thing - how well the student did on the AP test. There are so many variables that go into a class:

• Is it first period at 8:00am?
• Is it the last class of the day?
• Is it before lunch or right afterwards?
• What about right after Gym.
• What's the mix of students - every class is different
• Is the teacher teaching the subject once a day? Two times? Five times?
• How large is the class
• How many other classes are the kids taking?

The list goes on and on. There are so many contributing factors that you just can't say "Good test results = good curriculum" or "good test results = good PD."

Want to know if a curriculum is good - have experienced teachers who know their subject run it a few times through and ask them. Same for PD.

Teachers know education a lot better than test makers, curriculum developers and PD providers. How about listening to them for a change?