A Parable:

At the turn of the century, sprinting became an increasingly popular sport.

Concerned that his athletes might be unprepared for competition, one progressive coach made an unusual decision: instead of focusing solely on running, he devoted a significant portion of training to teaching his runners how to use a stopwatch. They practiced starting it cleanly, stopping it precisely, and reading split times with confidence.

His reasoning: races are timed, so athletes should be fluent with the timer.

By the end of the season, his runners were excellent timekeepers – but none qualified for competition.

The Point:

Some tools exist solely to measure performance. They do not create it.

To be sure, athletes must be familiar enough with a stopwatch that it doesn’t confuse or distract them. But beyond that basic threshold, time spent training the tool is time not spent training the skill.

State academic tests – whether digital or paper-based – are no different. These are instruments of measurement: they capture performance, but they do not create it.

Of course, there is a difference between tools that are part of a skill and tools that merely display it. A pianist must train on a piano, because the instrument is the skill. But a stopwatch is not running – it merely records it.

Digital tests fall into the second category. When assessing reading comprehension, writing, or scientific reasoning, the screen is not the skill – it is simply the interface through which the skill is expressed.

THE MODE EFFECT

In a recent study, two equivalent groups of fifth-grade students were taught the same material over the same period. The only difference was how their learning was assessed: one group took paper-based tests, the other digital exams.

By the end of the semester, students tested on paper improved by roughly 14%, while students tested on computers declined approximately 5%.

This is a textbook example of the Mode Effect: the well-documented tendency for students to perform worse on digital assessments than on equivalent paper-based tests. As the authors noted:

• “…some students who demonstrated strong understanding during lessons and performed well on the pre-test experienced sharp score declines in subsequent tablet-based assessments.”

The students didn’t fail to learn - the testing medium interfered with how they could demonstrate their learning.

Faced with this problem, testing systems have tried to adapt. Some invest heavily in refining digital questions to eliminate formatting effects. Others change the construct itself: reducing long-form reading to skimming, or replacing writing with editing tasks.

The irony here is difficult to ignore: enormous time, effort, and expense are being devoted to mitigating a problem introduced by the testing medium itself – a problem that would be immediately resolved by returning to paper.

WHAT SHOULD WE DO?

Some tests are not designed to measure individual learning. The NAEP, for example, is used to track broad trends across schools and states. In these cases, as long as the mode effect is consistent, the testing medium is less important.

But if the goal is to understand what students actually know and can do, then introducing a medium that systematically distorts performance is, at best, counterproductive.

In my opinion, any test that directly impacts a child’s future options (e.g. the SATs, MAP, ACTs) should return to analog formatting - this ensures the most accurate measurement of a child’s development. However, because digital testing offers administrative efficiency, scalability, and rapid analysis, the return to analog formats may not happen immediately.

So the question becomes: if digital testing is required (at least in the short term), how much digital training do students need?

MORE TECH ≠ BETTER DIGITAL TESTING

There is no question that students need basic familiarity with the testing environment. A student who cannot type, navigate a screen, or select responses will be at a disadvantage.

But beyond basic familiarity, the evidence is remarkably consistent:

• A 2018 review found that student access to and familiarity with digital tools did not improve digital test performance. In some cases, the relationship was actually negative.
• A large-scale study (>25,000 participants) found that taking one digital practice test improved performance – likely by reducing initial friction. But additional practice tests provided no further benefits, and excessive exposure was associated with declining scores.
• A 2016 study found that computer-based training improved performance on digital writing tasks (where typing is directly relevant), but harmed performance in math (where typing is largely irrelevant).
• Studies of digital language learning consistently find that computer familiarity explains little-to-none of the variance in performance.
• A 2025 study showed that increased digital familiarity improves student comfort and preference for online testing. However, in controlled comparisons, students do not perform better in their preferred testing medium. In other words, preference is uncorrelated with performance, and mode effects persist.

Taken together, these findings suggest digital testing follows a threshold model.

Students need enough familiarity with the tool to avoid interference. But beyond that point, additional exposure provides no benefit – and may even be harmful.

SO NOW THEN…

Just because digital testing is the new norm, it does not follow that schooling needs to be redesigned around this format.

We should teach each subject and skill in whatever way best supports learning – whether that’s discussions, worksheets, problem-solving, or direct instruction. Then, shortly before the digital exam, we provide brief, targeted exposure to the testing platform.

That’s it.

If I knew my students had a digital science exam on December 30th, I wouldn’t put them on screen all year. I would teach the material using the most effective and proven pedagogies. Then, a few days before the test, I would run short (10-20min) sessions to familiarize them with the digital format. One to three exposures is enough.

When a measurement tool changes, it’s tempting to believe everything else must change with it. It doesn’t. We should continue to teach in the ways that best support learning – and adapt only briefly to the final tool.

The screen is not the skill.