New York City teachers who received the highest ratings on their performance evaluations are unevenly distributed within the city school system and less likely to serve black and Hispanic students, state education officials said this week.
The trends were among several findings from an analysis of teacher evaluation data released this week by the State Education Department. Officials said that they hoped the data would nudge districts like New York City to revamp their teacher placement policies with a greater focus on recruiting and retaining top teachers in the most challenging schools.
“Effectiveness matters,” Senior Deputy Commissioner Ken Wagner said at the state Board of Regents meeting on Tuesday. “It has real impact on real kids in real classrooms.”
Chalkbeat reported its own analysis of teacher evaluation ratings from the 2013-14 school year last week, looking specifically at teachers working in struggling schools in the city’s Renewal Schools program, where high-needs students are overrepresented. The schools had twice as many teachers rated “ineffective” and “developing” as the city average, Chalkbeat’s report found.
The state’s analysis focused on a narrower measure of teacher performance, but found that similar disparities exist when sorted by student demographics. The city’s black students, for instance, were more than two times as likely to have a teacher rated “ineffective” or “developing” as white students.
“From an equitable access standpoint, that gap in access to effective teachers will catch some attention,” said Tim Daly, president of TNTP, an education nonprofit that works with districts on teacher recruitment and retention policies.
The state’s data is based on the 2012-13 ratings of about 40,000 math and English teachers in fourth through eighth grade throughout the state. The ratings were calculated using an “enhanced growth model,” which is designed to measure a teacher’s impact on student learning, controlling for outside variables such as a student’s socioeconomic status, special learning needs, previous years test scores, and attendance. Ratings were calculated for teachers in both district and charter schools.
The analysis found that city teachers, though unevenly distributed within the district, were three times more likely to earn “highly effective” ratings than other poor districts in the state, and twice as likely compared to affluent districts. In charter schools, Hispanic and black students were more likely have a higher-rated teacher, especially in math.
The measure on which these growth scores depend, however — standardized state tests — are under intense scrutiny.
An increasing number of parents are keeping their children from taking the tests this week, in part because they view them as unreliable indicators of student performance, while the state teachers union has encouraged test refusals as a way to undermine the evaluation system. This year the scores count for 20 percent of a district teacher’s evaluation, although changes to the evaluation system mean they could count for up to half next year.
The opt-out movement has startled state officials, who have warned that too many test refusals could damage their data-collecting abilities.
“A lack of full participation in New York State assessments hinders the State’s ability to collect complete data about students and educators, and subsequently could impact the analysis of equity issues,” officials wrote in a memo.
In city district schools, 17 percent of math teachers with lower growth ratings taught black students, 13 percent taught Hispanic students, 7 percent taught white students and 3 percent taught Asian students, according to state data. For English, 12 percent of lower-rated teachers taught black students, 9 percent taught Hispanic students, 5 percent taught white students and 4 percent taught Asian students.
One-third of black students and 39 percent of Hispanic students in charters were taught by a “highly effective” math teacher, compared to 10 percent and 14 percent of those student groups in New York City schools, according to the analysis. Hispanic charter school students had roughly the same share of top-rated English teachers.
Sean Corcoran, an economist at New York University, said that was consistent with other test-based charter school research, but student gains could be due to longer school days, stricter discipline policies, or any number of factors — not necessarily superior teachers.
“That’s one of the fundamental problems with test score based measures of teacher effectiveness,” Corcoran said. “It’s difficult to separate the teacher’s effectiveness from other factors, including those at the school and classroom level.”
Leo Casey, executive director of the Albert Shanker Institute and a former vice president for the United Federation of Teachers, said the department’s report left out important caveats. Growth models, for instance, are prone to volatility and bias toward teachers in high-poverty schools, limitations he said should have been noted more prominently.
But additional data provided by the state shows that racial and socioeconomic gaps in teacher equity were actually exacerbated when classroom observations and locally-selected assessments were factored into evaluation ratings. This could in part be explained by bias toward teachers in high-poverty in observations, which has been detected by researchers.
Daly said the findings should not be so quickly dismissed, particularly because it involves a large sample size.
“I’d say [the] best conclusion is that while there can be noise in this stuff at the micro level, on a statewide level it’s probably a good signal,” Daly said.
Daly’s organization, TNTP, advocates for higher pay and merit bonuses for recruiting and retaining top teachers and a quick firing trigger for weeding out the subpar ones. But Casey said that compensation and staff replacement aren’t enough to stem the churn of teachers who cycle through high-poverty schools.
“We are already replacing teachers as they leave of their own volition, and it makes no difference, because we are not addressing the reasons why they are not succeeding as teachers,” Casey said in an email.
On this, Daly and Casey found common ground. It was TNTP that concluded, in an influential 2012 research paper, that better working conditions at high-poverty schools, strong leaders who provide constructive feedback, and a few words of encouragement here and there significantly increased the likelihood of effective teachers staying.
The city already has some policies in place designed to improve teacher quality in struggling schools.
The new teachers contract allows for the promotion of effective teachers to higher-paid leadership positions. Chancellor Carmen Fariña also has the ability to give $5,000 bonuses to teachers working in “hard-to-staff” schools who have been rated developing or better, although a spokeswoman would not say if any payments have been authorized yet.
The spokeswoman, Devora Kaye, said the caliber of applicants seeking teaching positions in the city has gone up significantly in the last decade. In 2014, she said the city received 16,000 applications for certified teachers.
“In contrast to a decade ago, now every DOE hire must get state teacher certification, which requires completing rigorous coursework and testing,” Kaye said.