State rankings critiqued, more on the Southern surge, and education research for dummies

Welcome to Chalkbeat Ideas. We’re trying something a little different today: A roundup of interesting tidbits, data points, and links rather than a single piece. My plan is to do this once a month. Feel free to let me know what you think by replying to this email.

How should we judge states’ educational performance?

Mark Weber, a researcher and teacher, has a differing take from many on the Southern surge, which I also wrote about last month. Recall that Louisiana and Mississippi’s test scores look particularly strong when controlling for their high poverty rates, according to a widely cited Urban Institute analysis. 

Weber argues that this is the wrong way to evaluate states because lower income levels may be a result of state policies, particularly the historic quality of their schools. He points to his home state of New Jersey, which does well in raw scores but is less impressive in Urban’s adjusted comparisons. Such statistical adjustments are “wiping away one of the primary outcomes of good schooling that public education advocates want: higher incomes,” Weber writes.

I don’t quite agree with his analysis. Controlling for out-of-school factors, like poverty and income, is necessary to begin to evaluate the current quality of schools in a given community. 

But I think Weber is on to something. Numerous state policies can affect income levels, so statistically controlling for this fails to credit places that are doing a better or worse job improving families’ economic prospects. These demographic adjustments, in other words, are preferable for assessing the performance of a state’s school system, but not necessarily for judging the overall policy environment that supports student learning.

A separate issue, which Weber also notes and the Urban Institute acknowledges, is whether we can easily make these demographic adjustments between states. This is much more challenging than it might seem at first glance.

Speaking of the Southern surge, Todd Collins, a reading advocate in California, sent me a note suggesting that my piece failed to give Mississippi enough credit for its performance in eighth grade, noting that it looks better than most other states that have seen scores fall. Let me try to clarify.

Mississippi fourth graders reached the national average on NAEP reading in 2019. However, eighth graders in the state were still a bit below average in 2022 and 2024. That suggests that those fourth graders in 2019 failed to progress at the same speed as students elsewhere. We should try to understand why that is. (The pandemic doesn’t explain this because the comparison is to other states who were also affected.)

This is not to discount Mississippi’s success, however. While it’s true that fourth grade gains have not fully translated into eighth grade, even there the state has closed some of the gap with the nation and ranks well on the Urban Institute adjusted scores.

Have thoughts on a recent story or on what I should write about next? Feel free to respond to this email or leave a comment.

I recently explored the disconnect between the rhetoric and the data on higher education, noting that enrollment in four-year colleges has been relatively stable, despite frequent claims to the contrary. Preliminary data from this fall shows another uptick in college attendance. This would mark the third straight year of increases.

Another recent myth buster focused on the notion that voters no longer trust Democrats on education. That just isn’t true, according to most current polls. One potential reason why: A new survey from The Argument shows that a majority of Americans say that pandemic-era school closures were “largely necessary.” This challenges the common view that voters turned against Democrats over this.

Education abounds with confident, sometimes conflicting, claims about what “research shows.” I consider part of my job trying to sort through these claims and unearth interesting studies. Doing so is challenging, though. There is so much research and not all of it credible.

Tulane professor and education economist Doug Harris decided to try to address this vexing challenge. He helped organize a group of researchers to bring academic expertise to a wider audience. The goal is to summarize research on a variety of topics. Dozens of highly credentialed experts wrote and edited summaries on issues including teacher performance, charter schools, education funding, and student loans.

The project is called the Live Handbook. It launched online earlier this year, hosted by the Association for Education Finance and Policy, a professional group for researchers. In my view, the handbook is an incredibly valuable resource.

Still, synthesizing research remains a fraught, subjective endeavor, even when experts are involved. For instance, one section says that teachers’ observable traits matter “very little” for their performance. In fact, some studies find these traits matter quite a bit. "I would feel a little less confident [about the section] and I thank you for raising this,” says author Ben Backes, in response to my questions. “It makes me want to go back and play with the data myself." 

Harris says the goal is to continue to keep the Live Handbook up to date with the latest research findings, new topic sections, and responses to feedback.

Cellphone bans improve test scores modestly by their second year, according to a new study out of Florida. There were some growing pains, though: The first year saw an increase in suspensions. 

Immigration enforcement depresses test scores in a Florida district according to a different study by the same authors.

Who uses Arizona’s universal voucher-like education savings account? So far, it’s mostly students who had already been in private school, according to a new RAND paper. Less than 1 in 5 students switched from public schools to private education with an ESA.

What’s been billed as the “first public Christian school” has launched in Colorado. This is part of an effort by a conservative law firm to test the “legalities around the issue of whether a public school may provide religious education,” according to an email obtained by Chalkbeat’s Ann Schimke. 

Is the special education system broken? That’s the argument of a new think tank analysis, which Chalkbeat’s Erica Meltzer examined in a Q&A with the report’s author.

Reach me at [email protected].