Advertisement

Opinion: Times reporters respond to Blowback critical of ‘Grading the Teachers’ series

Share

This article was originally on a blog post platform and may be missing photos, graphics or links. See About archive blog posts.

Below is a response by Times reporters Jason Felch, Jason Song and Doug Smith to the Aug. 25 Blowback article by UC Berkeley education professors Bruce Fuller and Xiaoxia Newton. In their piece, Fuller and Newton say The Times’ ‘value added’ method of evaluating teacher effectiveness in the Los Angeles Unified School District ignores classroom factors beyond an instructor’s control.

We’re happy to have credible experts debate the reliability of our statistical approach, but we would hope in doing so they would exercise the same care in their critiques as we have in our publications. Bruce Fuller and Xiaoxia Newton make several points that we’d like to respond to.

Advertisement

The authors say Richard Buddin’s approach “has yet to be openly reviewed by scholarly peers.” In fact, the reason we chose Buddin is because he has published several peer-reviewed studies using data from the Los Angeles Unified School District in major academic journals using the same ‘value added’ approach he employed for us. Buddin’s methods paper is not a formal academic publication. Nevertheless, we asked leading experts to review that work before choosing Buddin, and asked several other experts, including skeptics, to review his methods paper.

The authors say our approach and graphic fail “to take into account differences in student background, including English proficiency, parents’ education levels or home practices that affect children’s learning.” L.A. Unified did not provide student demographic information for this study, citing federal privacy laws. Demographic factors do have a large effect on student achievement, but these influences are largely included in the students’ prior-year test scores. Prior research (including Buddin’s own using L.A. Unified data) has shown that demographic factors are much less important after controlling for a student’s previous test scores. The technical report used results from Buddin’s previous Rand Corp. research to show that student demographics had small effects on teacher value added as calculated in this study. This earlier study (Buddin and Zamarro, 2009) ran through 2007 instead of 2009, but this pattern is likely to persist. The approach and results are discussed in the subsection of the technical report ‘How does classroom composition affect teacher value added” that begins on page 13. The key empirical results are in Table 7.

The authors cite the National Academy of Sciences report urging caution. We cited the same report in our story. There is a variety of other research available that suggests these estimates are reliable. See Kane and Staiger’s 2008 random assignment validation of value-added approaches, which found value added were “significant predictors of student achievement” and controlling for students test prior scores yielded “unbiased results.”

The authors claim our analysis “fails to recognize that test score across grade levels cannot be compared, given the limitations of California’s standardized tests.” California Standards Test scores have been used by many researchers in peer-reviewed value-added analysis for years. The district’s own researchers concluded the test was appropriate for such estimates, as the story mentioned. See this L.A. Unified report.

The authors claim, incorrectly, that The Times plans to “publish a single number, purporting to gauge the sum total of a teacher’s effect on children.” In our stories and Q&As, we have repeatedly made clear that value added should not be the sole measure of a teacher’s overall performance. And our database does not present a single number for anyone.

Finally, the authors point to various limitations of the value-added approach. These are well known and have repeatedly been disclosed by The Times. What the authors fail to note is that leading experts say value added is a far more thoroughly vetted, peer-reviewed and provably reliable tool than any other teacher evaluation tool currently available. Rather than comparing value added to a Platonic ideal of a perfect teacher evaluation, it should be weighed against classroom observations by principals, student surveys and the other “multiple measures” that are being considered. Under this analysis, most scholars agree that, warts and all, value added shines.

Advertisement