Are Children Learning

Here's why Indianapolis isn't one of the U.S. cities that will get comparison national test data

PHOTO: Scott Elliott

Nearly every big-city school district in the country is measured by an important national schools ranking — but Indianapolis is left out because of the odd way its schools are divided.

Six more cities were added to to the ranking this week, giving parents, teachers and school leaders in Denver, Milwaukee, Memphis and the other cities a chance to gain crucial information about how their students compare with their peers around the nation.

But the Indianapolis students are spread out over 11 districts, meaning the city is not being considered for the ranking.

There are now  27 U.S. cities participating for a special project of the National Assessment of Educational Progress, also known as NAEP or “The Nation’s Report Card.”

NAEP is an exam used to gauge national academic progress. It is given every other year to a sample of fourth- and eighth-grade students across the country.

Since 2002, NAEP has also been given to a larger sample of students in a group of cities as part of what it calls the Trial Urban District Assessment. That allows for comparison data, which shows how the cities rank against each other.

NAEP is a rigorous exam that has been given for decades. Its passing rate is usually lower than most state exams like Indiana’s ISTEP. But its wide scope and long track record makes it ideal for comparing the test performance of American kids across the country and over time.

“Cities wanted to be able to compare themselves across state lines with other big-city school districts that shared many of the same issues and challenges,” said Michael Casserly, the executive director of the Council of Great City Schools, which first suggested the urban ranking.

“We wanted to be able to tell whether or not the reforms we were pursuing were producing results,” Casserly added. With different districts trying different reform strategies, comparing results was one way to tell which were working and which weren’t, he said.

“NAEP gave us data at a level of detail across state lines that we couldn’t get any place else.”

The new group of cities added Tuesday included several that could be seen as potential peers for Indianapolis based on size and demographics, such as Denver, Milwaukee, Memphis and Fort Worth, Texas. Also added to the study were Las Vegas and Greensboro, the school district where Indianapolis Public Schools Superintendent Lewis Ferebee spent most of his career.

The city of Indianapolis, 13th largest in the U.S. with a population of more than 850,000, is actually bigger than all the cities that were added to the study and Indianapolis otherwise would qualify for the NAEP study based on other demographic factors it requires. But a NAEP spokesman said the study simply isn’t set up to do the study in a city that doesn’t have a single large school district.

The city’s unwieldy system spreads more than 140,000 public school students across 11 school districts, meaning no single Indianapolis district is large enough to qualify for the ranking.

The state’s largest school district, Indianapolis Public Schools, has an enrollment of 29,581 — much smaller than the main school districts of most cities the size of Indianapolis.

Indianapolis was one of the first cities in the country to create a unified city-county government. Small cities and townships in Marion County were merged into the city and share most services. The big exception was schools, which remained divided.

That means Indianapolis has no good way to know how its students are measuring up.

Most standardized tests are state-created exams that are not comparable beyond state borders. And national exams like the SAT and ACT are not taken by all students, which make them difficult to use as the basis of comparing scores. Even other academic comparison factors, like graduation rate, are calculated differently in different places.

Those sorts of comparisons were a primary goal of the shared Common Core standards and linked tests like PARCC as both were being developed. Indiana was an early adopter of Common Core and a leader in the group that developed PARCC. But Gov. Mike Pence withdrew the state from PARCC during his first year in office and the Indiana legislature ordered the state to withdraw from Common Core and write its own standards in 2014.

Pence and Republican leaders argued that following Common Core would ultimately lead to a loss of state control over standards. The U.S. Department of Education and President Obama supported Common Core and asked states to adopt the standards in return for release from some of the consequences of the federal No Child Left Behind law, which was evidence enough to some of them that Common Core was too connected to federal priorities.

Indiana’s new academic standards are considered strongly aligned to Common Core. But the state’s ISTEP test is not comparable to PARCC or other exams.

More states have followed Indiana’s lead by pulling out of Common Core and PARCC. In fact, that was one factor in Denver’s decision to volunteer for the NAEP study.

“Given the decrease in the number of PARCC states, and the lack of comparable districts in Colorado, DPS was interested in seeing how we compare to other large urban districts throughout the country,” Denver officials said in a statement.

Students in urban school districts tend to score lower than the national average on NAEP reading and math tests. But data from the past 12 years shows that big-city school districts are narrowing that gap. In addition, results show that the reading scores of some black and Latino boys in urban districts have improved more than the scores of black and Latino boys nationwide.

The urban district comparisons have revealed two ingredients necessary to improve student achievement, Casserly said: Programs that improve the quality and rigor of classroom instruction, and a district leadership team united around a common agenda.

The Indiana legislature is debating whether to replace ISTEP after 2017. One option is to use some sort of shared exam that other states use but given the politics surrounding the Common Core and PARCC, lawmakers don’t appear poised to return PARCC.

Here’s the list of urban school districts (including new additions marked *) participating in NAEP’s Trial Urban District Assessment:
*Shelby County Schools (Memphis)
*Clark County School District (including Las Vegas)
*Denver Public Schools
*Fort Worth Independent School District (Texas)
*Guilford County Schools (including Greensboro, N.C.)
*Milwaukee Public Schools
Albuquerque Public Schools (New Mexico)
Atlanta Public Schools
Austin Independent School District (Texas)
Baltimore City Public Schools
Boston Public Schools
Charlotte-Mecklenburg Schools
Chicago Public Schools
Cleveland Metropolitan School District
Dallas Independent School District
Detroit Public Schools
District of Columbia Public Schools
Duval County Public Schools (Jacksonville, Fla.)
Fresno Unified School District (California)
Hillsborough County Public Schools (Tampa, Fla.)
Houston Independent School District
Jefferson County Public Schools (Louisville, Ky.)
Los Angeles Unified School District
Miami-Dade County Public Schools
New York City Public Schools
School District of Philadelphia
San Diego Unified School District

New research

From an ‘F’ to an ‘A’, Tennessee now sets high expectations for students, says Harvard study

PHOTO: Lisegagne/Getty Images

Criticized for setting low expectations for students just a decade ago, Tennessee has dramatically raised the bar for standards that now rank among the top in the nation, according to a new analysis from Harvard University.

The state earned an “A” for its 2017 proficiency standards in a study released Tuesday by the same researchers who gave Tennessee an “F” in that category in 2009.

The researchers have been tracking state proficiency standards since 2006. Their latest analysis focused on changes since 2009 when, like Tennessee, most states began adopting Common Core academic standards, then began retreating one by one from the nationally endorsed benchmarks.

Did the exodus from a consistent set of standards cause states to lower expectations for students? The researchers say no.

“Our research shows that most all the states have actually improved their standards, and Tennessee has probably improved the most because its standards were so low in the past,” said Paul Peterson, who co-authored the analysis with Daniel Hamlin.

The grades are based on the difference between the percentages of students deemed proficient on state tests and the National Assessment of Educational Progress, or NAEP, the exam administered by the U.S. Department of Education to measure what students know in math and English language arts. The narrower the proficiency gap between those tests, the higher the grade a state received.

Tennessee’s 2009 proficiency gap was 63 percent, an amount that Peterson called “ridiculous” and “the worst in the country” compared to 37 percent nationally.

In 2017, Tennessee’s gap narrowed to less than 3 percent, compared to 9 percent nationally, under revised standards that reached classrooms last fall after the state exited the Common Core brand.

“It’s a dramatic improvement,” Peterson said of Tennessee’s work to align its standards with national expectations.

Interestingly, in other states, the study found virtually no relationship between rising proficiency standards and test score growth — a finding that the researchers called “disheartening.”

“The one exception was Tennessee,” Peterson said of the state’s academic gains on NAEP since 2011. “It has not only raised its standards dramatically, it saw some student gains over the same period.”

Since 2010, higher academic standards has been an integral part of Tennessee’s long-term plan for improving public education. The other two components are an aligned state assessment and across-the-board accountability systems for students, teachers and schools, including a controversial policy to include student growth from standardized test scores in teacher evaluations.

Tennessee poured millions of federal dollars from its 2010 Race to the Top award into training teachers on its new standards. The process began in 2012 with large-scale Common Core trainings and shifted last year to regional trainings aimed at equipping local educators to prepare their peers back home for Tennessee’s revised standards.

PHOTO: Marta W. Aldrich
Karyn Bailey (left), a facilitator from Williamson County Schools, coaches elementary school teachers during a 2017 exercise on Tennessee’s revised standards for English language arts as part of a two-day training at La Vergne High School, one of 11 training sites across the state.

Implementation really matters. You can’t just make the shift on paper,” said Education Commissioner Candice McQueen, who will take part in a panel discussion on the study’s findings Tuesday in Washington, D.C. “You have to do the hard work to implement it on the ground. And that is a long game.”

The Harvard study comes on the heels of a separate but related report by pro-Common Core group Achieve that says Tennessee is essentially being more honest in how its students are doing academically. The state was called out in 2007 by the U.S. Chamber of Commerce because Tennessee tests showed students doing well, while national tests reported otherwise.

Both analyses come as Tennessee tries to regroup after a problem-plagued return to statewide online testing this spring.

While supporters of Tennessee’s current policy agenda fear that headaches with the state’s standardized test could undo the policies it may be getting right, Peterson said a study like Harvard’s can provide a birds-eye view.

“What happens over a period of years is a better way to look at how a state is doing,” he said, “because things can fluctuate from one year to the next.”

The Harvard research is funded by the Bill & Melinda Gates Foundation and the Walton Family Foundation. (Chalkbeat is a nonprofit news organization and also receives funding from both foundations. You can find the list of our supporters here and learn more about Chalkbeat here.)

Regrouping

After another bumpy testing year, Tennessee likely will slow its switch to online exams

PHOTO: Marta W. Aldrich
Members of Tennessee's testing task force listen to a presentation by Mary Batilwalla, deputy commissioner over assessment for Tennessee's Department of Education. The group offered feedback on options for transitioning to online testing after more problems occurred this year.

Tennessee education leaders are rethinking their timeline for adopting computerized testing after a parade of technical problems bedeviled students taking the state’s TNReady exam for a third straight year.

Most students are scheduled to test online next school year under a three-year transition plan. But since keyboard testing had significant challenges this year with half that number of students, Education Commissioner Candice McQueen is backing off from that timetable.

And while there’s disagreement over exactly how to move ahead, there’s consensus about one thing.

“We have a credibility issue,” said state Rep. John Forgety, “and we need to get it right one time.”

McQueen floated three options for the 2018-19 school year to members of her testing task force during its Wednesday meeting in Nashville:

  •     Returning to paper testing across all grades for one year;
  •     Computer testing for high school students; paper testing for grades 3-8;
  •     Computer testing for grade 6 through high school; paper testing for grades 3-5

Off the table, however, is the option that districts had this year to give computer tests to more grades than required by the state.

The state ordered that all high school students take the test by computer this year, but about 40 percent of districts also chose to go digital for at least some of their students in grades 5-8.

The early thinking had been that letting districts test more students than required would expedite Tennessee’s online switch if local leaders felt ready. But state officials now believe the piecemeal approach only complicated the process.

“We feel very strongly” about this decision, Deputy Education Commissioner Mary Batilwalla told the task force. “The complexity is really too great for us to overcome in ensuring that we have a seamless delivery.”

The 30-member task force of educators and advocates has been McQueen’s sounding board on TNReady and other testing issues, and she sought the group’s feedback one week after the state’s messy testing season ended.

“We don’t want to introduce any additional complexity. We want to eliminate complexity, eliminate risk,” said McQueen, who also is turning to superintendents and upcoming focus groups for advice about how to improve their TNReady experience.

PHOTO: Marta W. Aldrich
Education Commissioner Candice McQueen speaks at a 2017 event as Gov. Bill Haslam looks on.

McQueen will decide about digital vs. paper — and for which grades — by late June. She is leaning toward keeping high schools online and putting all lower grades on paper tests, but it’s not a done deal, she told Chalkbeat.

“The feedback we’re getting is for more to go online than not, and that’s very meaningful to hear,” she said.

Her boss, Gov. Bill Haslam, has made it clear that Tennessee is committed to eventually adopting computerized testing.

“It’s not just that’s where the world is going; that’s where the world is,” Haslam said earlier in the week.

About 300,000 students took TNReady online this year — the most ever since a wholesale switch to computers failed in 2016 under Measurement Inc. McQueen fired that testing company, hired Questar as its successor, and unveiled a new game plan to gradually wade back in. That approach worked well last year for the 24 districts that did a trial run for high schools, although later scoring errors detracted from Questar’s debut.

This year marked the return to statewide online assessments, beginning with Tennessee’s oldest students. But challenges included a cyber attack and lousy internet service when a dump truck cut a main fiber optic cable — examples that demonstrate the risks of computerized testing.

There are benefits, too, however. Digital exams are quicker to score, offer more flexibility in the types of questions asked, and ultimately cost less. Returning to all paper testing would cost an extra $11 million in printing and shipping costs.

One big advantage of paper-and-pencil testing is a shorter testing period. Three weeks were allotted to TNReady this spring because schools had to rotate their students in and out of testing labs to use a limited number of computers. That requires a lot of coordination and time.

Task force members agreed that reverting to paper would be a step backward, especially with the state’s focus on the technical skills needed for college and careers and the significant investments made by school districts to prepare for online testing.

But they were adamant that Tennessee needs a win next time around to rebuild trust in a test that many consider broken.

“There has been a serious erosion in confidence in state testing, whether it’s online or on paper,” said Shawn Kimble, director of schools in Lauderdale County. “If we fail again, where does that leave us as a state?”