How strong is the case against ed-tech?

Welcome to Chalkbeat Ideas, a new section featuring reported columns on the big ideas and debates shaping American schools. Forwarded this newsletter? Sign up here. Later this week, I’ll be at the Association for Education Finance and Policy conference in Chicago. If you’ll be there too, let me know and make sure to say hi!

Schools have been struggling for nearly a decade with stagnant or declining test scores. Some have blamed external factors like the pandemic or children’s screen use outside of school. But what if, in a sort of educational horror movie, the call is coming from inside the house?

Such is the provocative theory advanced by Jared Cooney Horvath, a Ph.D-holding neuroscientist who runs an education consulting company. Students’ learning and attention have been derailed by the proliferation of screens in schools. “When tech enters education, learning goes down,” he told a U.S. Senate committee earlier this year.

Horvath has offered a synthesis of two anxieties of the moment: poor test scores and rising screen time among children, including in school. It’s a message that is resonating with some parents, educators, and lawmakers. 

His Senate testimony has been viewed well over 2 million times on C-Span’s YouTube page. Clipped versions have ricocheted across Instagram. Prominent psychologist and tech critic Jonathan Haidt has promoted Horvath’s work. At least one tech-skeptical parent is attending school board meetings armed with copies of Horvath’s self-published book, “The Digital Delusion,” which is set to be re-released this summer by a major publisher. Meanwhile, some states are considering new restrictions on school-issued devices.

I wanted to figure out how strong the case against ed-tech really is, so I took a careful look at Horvath’s evidence. My takeaway: There’s no smoking-gun data showing that ed-tech is at the root of, or even contributing to, recent learning declines. But Horvath’s case should still give schools and educators some pause. Could the tech tools they’ve adopted be doing more harm than good?

In his book and Senate testimony, Horvath offers a three-pronged case against tech in schools. 

First, he notes a basic correlation. On a number of international tests, students who spend more time on school computers tend to do worse, he finds. Both U.S. and international scores also started to drop once schools began giving students laptops, Horvath says. 

Second, Horvath summarizes academic research — over 20,000 studies — on ed-tech tools, which he defines broadly. Startlingly, in his telling, they appear to harm learning. “Most general-use educational technologies perform below the effectiveness of ordinary classroom instruction,” Horvath said in his written Senate testimony.

Third, screens are not consistent with learning best practices, says Horvath, drawing from his neuroscience background. Human relationships are an important part of learning, which digital tools can’t match. Screens are also just distracting for students. 

Plus, tech can make learning frictionless, which seems like a good thing but isn’t, Horvath contends. For instance, it’s easy to take notes on a computer. Yet the process of hand writing notes requires more consideration of what is being written, and that can help commit it to memory.

Across different strands of evidence, "the same exact pattern pops up again and again,” Horvath tells me — that ed-tech is actively harmful. 

Yet this case has a number of significant holes.

His first point about correlation is just that: correlational. It does not show tech has caused lower achievement. Such declines in the U.S. and around the world have coincided with a number of other factors, including the pandemic and the growth of screens outside of school. Researchers are still struggling to pin down the key causal factors here. Horvath readily acknowledges this limitation.

His summary of research is also questionable. The overall impact of ed-tech in the studies he cites is actually positive. Horvath says the effect is too small to justify the use of tech because it’s smaller than the effect of “typical classroom instruction.” Taking that into account turns the ed-tech results negative, he says. This is a key methodological decision on which Horvath’s conclusion rests.

But it’s simply not the right approach, says Elizabeth Tipton, a Northwestern University statistician, who counts herself as an ed-tech skeptic. Horvath’s benchmark is wildly unrealistic, she says. In carefully designed studies, in which two groups are both receiving some sort of instruction, most effects are quite small. In that context, an effect above zero would be encouraging. 

Perhaps more importantly, says Tipton, it’s not clear the vast array of research that Horvath is citing should be relied on at all. There are many poorly done studies in education. To do a good summary, low-quality papers need to be ruthlessly weeded out. Another challenge is that tech is constantly changing, making old studies quickly obsolete. She’s worried that there’s a “garbage in, garbage out” problem in Horvath’s summary, which is really an overview of multiple overview papers.

"We are so far from the original data, we have no idea what we're studying at this point," says Tipton. She’s not aware of a high-quality summary of research on ed-tech broadly. This would be hard to do because the uses of tech are so diverse.

Horvath acknowledges limits in his cause and effect evidence, but he says that’s inevitable in social science. We should look at the weight of the data and the overall coherence of the case, he argues. 

Horvath is perhaps on the strongest ground in his summary of cognitive science research. Although it’s debated among researchers, there is some evidence that students learn less from reading and from taking notes on screens. Daniel Willingham, a cognitive psychologist at the University of Virginia, says he’s particularly concerned that tech opens up new avenues for distraction. “If kids have a Chromebook and they can find a way out of whatever they’re doing so they can do something more fun with the Chromebook, they will,” he says.

For his part, Horvath argues the burden of proof should not be on him, but on the purveyors of ed-tech. “We are much closer to proving that it's harmful than we are to proving it’s helping,” he says.

Reach me at [email protected].