From MUSE: The Problem with Test Scores for Teacher Evaluation

New York announced plans last week to begin using standardized test scores in their teacher evaluation scheme.  Although officially the scores count for only 40% (insisted upon by Gov. Cuomo despite an agreement between schools and unions that they would count for 20%) of the basis for evaluation, it will in fact be the determining factor, according to a report on the language of the law by Diane Ravitch:

The remaining 60 percent is supposed to consist of classroom observations and other measures. All teachers must be rated on a scale from 0 to 100, using these multiple measures. This draconian point system will guarantee that a teacher with a perfect 60 out of 60 on teaching skill will nonetheless be judged "ineffective" if he or she is in the ineffective range on scores. As it now stands, the rating system is so bizarre that a teacher could be rated effective in all three categories and still be rated "ineffective" overall.

The agreement contains this strange sentence: "Teachers rated ineffective on student performance based on objective assessments must be rated ineffective overall." Unless I can't read plain English, this says that the 40 percent devoted to test scores overrides the other 60 percent. In other words, 40 percent is equal to 100 percent. The teacher who doesn't raise test scores is ineffective. Oh, and state education Commissioner John King—who taught for three years and founded charter schools—will have the ultimate authority to review every district's additional measures for rigor and quality.

Not only is the system put in place bizarre and misleading, but the use of standardized testing for teacher evaluation seems to be unsupported by research.  The Economic Policy Institute's April 29, 2010 Briefing #278 details a host of studies showing the unreliability of, the invalidity of, and the perverse incentives created by the use of the most widely used type of data analysis ("value-added modeling" or VAM) in teacher evaluation.  And VAM is used most widely because it is considered the most accurate form of analysis and the one that most effectively takes into account factors other than teacher performance.

Instead of reliance on solely "objective" numbers, which, despite some claims to the contrary, is not the way most professionals are evaluated, the Briefing calls for "[d]istricts seeking to remove ineffective teachers [to] invest the time and resources in a comprehensive approach to evaluation that incorporates concrete steps for the improvement of teacher performance based
on professional standards of instructional practice, and unambiguous evidence for dismissal, if improvements do not occur (p. 20)."

The Briefing is attached here (with my highlighting) and is worth a read..  It contains an excellent and understandable analysis of VAM and a shorter description of the other methods of data analysis.


This post was also published on MUSE '13