For two years, state leaders have trumpeted the claim that Tennessee is the fastest improving state in the nation in K-12 education, based on the state’s 2013 performance on the National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card.
It’s practically become a state slogan.
But while Tennessee has made significant progress compared with many other states, it might not be the education climber that its leaders declare.
The NAEP — administered every other year by the National Center for Education Statistics — is the only common yardstick that states have to see how they stack up against their neighbors. It reveals a lot about state and national trends. But the results also can be easy to misunderstand.
Here’s some points to keep in mind when examining the 2015 scores released on Wednesday:
It’s difficult to compare different grade levels and subjects. In 2013, Tennessee’s combined gains on all four subjects added up to 21.80 points, slightly less than the District of Columbia, but more than any other state. However, researchers say that you can’t see which state is “most improved” merely by adding, because the scales for the tests are different. A one-point gain on the fourth-grade math test might not be the same as a one-point gain on the eighth-grade reading test. In fact, the NAEP website warns users against comparing scale scores across subjects and grades. Combining the gains on all four tests would be much more complicated than the simple addition used by Tennessee officials.
It’s too soon to know what policies are to blame or credit for the 2015 results. In 2013, when Tennessee was in the early years of a massive K-12 overhaul spurred by the federal Race to the Top competition, arguments ensued among leaders, officials and politicians over who — or what policies — to thank for Tennessee’s gains. Those gains, though not the largest, were significant compared with other states.
However, researchers say day-of analysis is just speculation.
“It’s not that NAEP results can’t ever be used to identify the effects of policies. But in order to do that, you need to a really sophisticated analysis,” said Morgan Polikoff, a researcher from the University of Southern California.
It’s too soon to say if Tennessee’s 2013 results were due to the state’s switch to the Common Core State Standards, test-based teacher evaluations or numerous other policy changes over the last five years.
Tennessee isn’t the only state where leaders are quick to offer off-the-cuff conclusions tying results with particular policies. “This comes up every time NAEP results are released,” said Mike Hansen, deputy director of Brookings’ Brown Center of Education Policy. “Politicians or advocates use the data to further their agendas, even though the way they’re doing it is not good science.”
It’s helpful to zoom out. Tennessee’s gains in 2013 were an anomaly, when the nation as a whole was nearly stagnant. But both Tennessee and the nation have made large strides since students began taking the NAEP in the early 1990s. In particular, the report shows trends in achievement gaps among races, and how Tennessee stacks up historically against other states.
Polikoff advises using a big-picture approach. “It’s interesting and useful,” he said, “to look at overall trends.”