Last week this blog highlighted Advocates for Children of New York’s new report documenting that more than 10 percent of the over one million students in the New York City Public Schools—101,000 students—are homeless. These students are living in shelters, doubled up with friends or relatives, or living in cars and parks. What are the academic challenges for these homeless children and other children living in families with minimum wage employment, unemployment, unstable housing, food insecurity and inadequate medical care?

Although federal law continues to require that states measure the quality of schools and school districts with standardized tests, all sorts of research documents that students’ standardized test scores are indicators of their life circumstances and not a good measure of the quality of their public schools. Students concentrated in poor cities or scattered in impoverished and remote rural areas are more likely to struggle academically no matter the quality of their public school.

Here are just two examples of this research.

In 2017, Katherine Michelmore of Syracuse University and Susan Dynarski of the University of Michigan studied data from Michigan to identify the role of economic disadvantage in achievement gaps as measured by test scores: “We use administrative data from Michigan to develop a… detailed measure of economic disadvantage… Children who spend all of their school years eligible for subsidized meals have the lowest scores, whereas those who are never eligible have the highest. In eighth grade, the score gap between these two groups is nearly a standard deviation.” “Sixty percent of Michigan’s eighth graders were eligible for subsidized lunch at least once during their time in public schools. But just a quarter of these children (14% of all eighth graders) were economically disadvantaged in every year between kindergarten and eighth grade… Ninety percent of the test score gap we observe in eighth grade between the persistently disadvantaged and the never disadvantaged is present by third grade.”

In How Schools Really Matter: Why Our Assumption about Schools and Inequality Is Mostly Wrong, Douglas Downey, a professor of sociology at The Ohio State University describes academic research showing that evaluating public schools based on standardized test scores is unfair to educators and misleading to the public: “It turns out that gaps in skills between advantaged and disadvantaged children are largely formed prior to kindergarten entry and then do not grow appreciably when children are in school.” (How Schools Really Matter, p. 9) “Much of the ‘action’ of inequality therefore occurs very early in life… In addition to the fact that achievement gaps are primarily formed in early childhood, there is another reason to believe that schools are not as responsible for inequality as many think. It turns out that when children are in school during the nine-month academic year, achievement gaps are rather stable. Indeed, sometimes we even observe that socioeconomic gaps grow more slowly during school periods than during summers.” (How Schools Really Matter, p. 28)

In the context of this research, Downey examines the six indicators the Ohio Department of Education uses to evaluate public schools when it releases annual report cards on school performance. Although the state has ceased branding public schools with “A-F” letter grades, Downey explains that the state of Ohio continues to ignore the role outside-of-school variables in students’ lives when it blames educators and schools for low aggregate test scores:

“The report card for schools is constructed from six indicators and not a single one of them gauges performance independent of the children’s nonschool environments. First is achievement, which is based on the percentage of students  who pass state tests… By far, the biggest determinant of whether a school produces high or low test scores is the income level of the students’ families it serves… Second is the extent to which a district closes achievement gaps among subgroups. But performance on this indicator can also be influenced by factors out of the school’s control… Third, schools are gauged by the degree to which the school improved at-risk K-3 readers… Of course, it is much easier to make progress on this indicator if serving children who go home each evening to reinforce the school goals. Fourth, schools are evaluated on their progress, an indicator based on how much growth students exhibit on math and reading tests. This kind of indicator is better than most at isolating how schools matter, but again, growth is easier in schools where students enjoy home environments that also promote learning… Fifth, the graduation rate constitutes a component of the district’s (rating)… but this is only a measure of school quality if the likelihood of a child’s on-time graduation has nothing to do with the stress they experience at home, the access they have to health care, or the quality of their neighborhood.  Finally districts are evaluated on whether their students are prepared for success.  This indicator gauges the percentage of students at a school viewed as ready to succeed after high school… and is determined by how well the students performed on the ACT or SAT and whether they earned a 3 or higher on at least one AP exam… These report cards ‘are designed to give parents, communities, educators, and policymakers information about the performance of districts and schools,’ but what they really do is mix important factors outside of school with what is going on inside the schools in unknown ways.” (How Schools Really Matter, pp. 115-116)

What these reports and many others demonstrate is that we cannot expect that no child will be left behind merely because Congress passes a law declaring that schools can make every American child post proficient test scores by 2014. No Child Left Behind’s (and now the Every Student Succeeds Act’s)  policies—which have branded schools unable quickly to raise aggregate test scores as “failing schools”— have unfairly targeted school districts located in poor communities. In 2017, the Harvard University testing expert, Daniel Koretz published The Testing Charade: Pretending to Make Schools Better in which he shows that ameliorating opportunity gaps in the lives of children is not something schools can accomplish by themselves.

Koretz explains: “One aspect of the great inequity of the American educational system is that disadvantaged kids tend to be clustered in the same schools. The causes are complex, but the result is simple: some schools have far lower average scores—and, particularly important in this system, more kids who aren’t ‘proficient’—than others. Therefore, if one requires that all students must hit the proficient target by a certain date, these low-scoring schools will face far more demanding targets for gains than other schools do. This was not an accidental byproduct of the notion that ‘all children can learn to a high level.’ It was a deliberate and prominent part of many of the test-based accountability reforms… Unfortunately… it seems that no one asked for evidence that these ambitious targets for gains were realistic. The specific targets were often an automatic consequence of where the Proficient standard was placed and the length of time schools were given to bring all students to that standard, which are both arbitrary.” (pp. 129-130) Koretz continues: “(T)his decision backfired. The result was, in many cases, unrealistic expectations that teachers simply couldn’t meet by any legitimate means.” (p. 134)