Skip to main content

Answer Sheet: Why One-Size-Fits-All Metrics for Evaluating Schools Must Go

For several decades now, education policymakers have been obsessed with data-driven accountability — usually with standardized test scores as the key metric. The approach has failed to achieve any of the goals supporters have championed, such as closing the achievement gap, and has instead brought us things like pep rallies to get students excited to take standardized tests and methods to evaluate teachers based on the scores.

Such accountability tactics need to be reevaluated now as a new school year begins and millions of students and teachers are still trying to work through the academic and mental health setbacks caused by the coronavirus pandemic. This piece looks at the one-size-fits-all metrics and what could be used instead.

This was written by Simone Ispa-Landa and Wendy Espeland, both of Northwestern University. Ispa-Landa is a Northwestern sociologist and a former Public Voices fellow through the OpEd Project, an effort to widen the range of voices and ideas heard in the public square. Espeland is also a Northwestern sociologist and the author (with Michael Sauder) of the award-winning book “Engines of Anxiety: Academic Rankings, Reputation, and Accountability.”

 

By Simone Ispa-Landa and Wendy Espeland

These days, accountability means metrics. We are a nation obsessed with lists and rankings, and not just for dishwashers and other consumer products. We track our steps, rate our sleep, and go to hospitals with the “best ratings.”

In the world of education, school leaders in a few lucky towns can proudly announce that their school has been “ranked as the best in the state” by U.S. News & World Report. But for most of the other 91,328 public school communities in the United States, falling lower in the rankings can be a source of shame and chagrin — even if educators are providing students with the most excellent education possible.

In education, it’s not just committed, creative public school teachers who complain they are being punished by quantitative accountability methods. Law schools live and die by rankings based on metrics that may not match their mission or capture how well they serve their students and the public good. When they cater to underrepresented students or focus on public interest law, they may fall in the rankings despite accomplishing their goals. Professors may try to write in ways that will boost the number of “likes” on Twitter, even if no one actually reads their work.

In public schools, poor test scores can result in school closure, lower home values, and teacher layoffs. Trying to evaluate complex and varied institutions according to a universal metric that seems neutral and objective is a trend that has long alarmed experts, who warn that accountability measures can create perverse incentives.

In our research, we find that, across institutions, school leaders are pressured to devote enormous time and energy to “improving the numbers,” even when this comes at the expense of making changes that, in private, they acknowledge would be far more impactful for students. Because rankings and other measures change how school leaders do their work and make decisions, current accountability policies have far-reaching implications for school discipline and student mental health at a moment of intense national crisis in child and youth well-being.

Despite the problems of No Child Left Behind, the 2002 K-12 education law which punished schools with poor metrics, the use of performance metrics has spread. Schools are now evaluated by a series of quantifiable standards of success that now incorporate everything from test scores to school discipline. The state of Illinois illustrates the dilemma that many schools face. In Illinois, the State Board of Education created a single metric to capture a school’s use of problematic and racially biased discipline.

This tactic may sound like an excellent solution to the pernicious problem of school pushout for students from racially minoritized backgrounds, and activists whom we admire fought hard for such measures.

Yet, the metric is based on just three factors: the rate of out-of-school suspensions, out-of-school expulsions, and the differences in these rates for White students and students of color. Since its adoption in 2015, Illinois has ranked all public schools according to how they do on this one metric. Schools that fall within the top 20 percent are put on a highly publicized “bad” list and must submit plans to remediate the problem. Local news outlets are eager to report the results, often to the embarrassment and dismay of hard-working administrators.

Since the rankings are based on these measures only, school leaders are incentivized to engage in tactics to “bring the numbers down.” Transferring students to alternative schools that provide an inferior education can become an attractive option, as can switching to in-school suspension, which is not counted in the metrics, but is still likened by many teachers to a “jail” or “prison” within the school. Teachers may also be incentivized to “look the other way” when students act out since this will inevitably bring down the numbers of students who are referred to administrators and thereby become eligible for suspension. Yet, ignoring distracting behavior further weakens learning and the bonds of trust that can occur when a teacher shows concern for an adolescent student’s missteps.

In addition, the metric does not capture all the times when a sensitive teacher is able to help a student process the trauma of being racially profiled, navigate a potentially explosive conflict with a peer, or handle the disappointment of romantic rejection. Yet, all these can powerfully affect students’ mental health and the likelihood that they will react to stressors in ways that put them afoul of school rules.

What are savvy administrators to do? In the schools we have studied, smart, caring administrators engage in intense and time-consuming efforts to create data systems that will force their fellow educators to mark, measure, and quantify the good work they are doing — work that is not included in the metric, but can be used in school board meetings and media bulletins. Inevitably, the data systems they create will confine and quantify qualitatively different kinds of interactions with students into narrow, prearranged categories with positive labels that are sure to impress outsider evaluators — things like “preventive intervention” or “social-emotional learning.” These data systems will also add to the workload of already overburdened educators, who will have to learn yet another system for data entry.

To be sure, such efforts can look disingenuous; it can seem like administrators are cynically and singularly focused on their image. But, it’s understandable that administrators do these things; in many cases, they are acting to preserve their institutions.

Experts at the American Academy of Pediatrics have been warning that children’s mental health has been worsening for years. Recently, they declared child and adolescent mental health a national emergency. Knowing the toll that untreated mental health issues can take on students’ learning and behavior, school administrators are desperately looking for ways to help. And they are right to do so: youths who lack supportive mental health services are more likely to fall victim to harsh and exclusionary school discipline and school pushout.

Some administrators are coming up with promising and creative ways to improve student mental health — but these don’t factor into the rankings, leading them further into the spiral of time-consuming efforts to mark, measure, and quantify.

At one school, administrators are offering students a way to clear detentions by going to art therapy, changing the schedule so students are less rushed and stressed during passing periods, and urging deans of discipline and social workers not to punish students for passionately “venting” in colorful language in private office hours. Such efforts may ultimately help “reduce the numbers” in school discipline, but even if they do not, it would be a pity if they had to be scrapped, all because they do not manage to make their way into a single metric to which everyone is now beholden.

We should acknowledge that one-size-fits-all metrics do not fairly measure what matters most in many schools. Right now, what matters most is finding ways to address and improve students’ mental health so they can get back on track with learning. We should reward schools for innovation, for creating programs that will take time to evaluate.

Simple numbers promote simple solutions and can prevent promising programs with long-term positive implications from taking root. Before we head into another school year, let’s look at dismantling the ranking systems that are burdening our administrators with busywork and preventing authentic improvement.

Let’s have the new school year be one in which educators can focus on cutting-edge, evidence-based improvements to meet students’ needs instead of catering to the rankings.

This blog post has been shared by permission from the author.
Readers wishing to comment on the content are encouraged to do so via the link to the original post.
Find the original post here:

The views expressed by the blogger are not necessarily those of NEPC.

Valerie Strauss

Valerie Strauss is the Washington Post education writer.

Simone Ispa-Landa

Simone Ispa-Landa is an Assistant Professor at the School of Education and Social Policy and (by courtesy) Sociology at Northwestern University. Ispa-Landa’...

Wendy Espeland

Wendy Espeland is a Professor of Sociology at Northwestern University. She works in the areas of organizations, culture, and law. Her book, The Struggle for...