NEPC Resources on Assessment
NEPC Review: Education System Alignment for 21st Century Skills: Focus on Assessment (The Brookings Institution, November 2018)
A Consumer’s Guide to Testing under the Every Student Succeeds Act (ESSA): What Can the Common Core and Other ESSA Assessments Tell Us?
State-Level Assessments and Teacher Evaluation Systems after the Passage of the Every Student Succeeds Act: Some Steps in the Right Direction
NEPC Review: High Stakes for High Achievers: State Accountability in the Age of ESSA and High Stakes for High Schoolers: State Accountability in the Age of ESSA
A new report that scores and ranks national education systems based on their efficiency has been receiving considerable media attention on both sides of the Atlantic. Efficiency is measured based on test scores, and resource use is analyzed in terms of teacher wages and pupil-teacher ratios. Looking across the 30 countries, the model predicts that, in order to get a 5% increase in PISA scores, teacher wages would have to go up by 14% or class sizes would have to go down by 13 students per class.
This policy brief summarizes the academic literature on the impact of class size and finds that class size is an important determinant of a variety of student outcomes, ranging from test scores to broader life outcomes. Smaller classes are particularly effective at raising achievement levels of low-income and minority children.
Policymakers should carefully weigh the efficacy of class-size policy against other potential uses of funds. While lower class size has a demonstrable cost, it may prove the more cost-effective policy overall.
NEPC Review: School Choice and School Performance in the New York City Public Schools: Will the Past be Prologue?
Over the past decade, big city school systems have scaled-up choice initiatives with remarkable speed. In School Choice and School Performance in the New York City Public Schools, Brookings contends that school choice and competition contributed to improved test scores and graduation rates in New York City since the universal high school choice reform began in 2004. However, after the report’s lengthy introduction describing and extolling choice and competition, only four pages are dedicated to discussing results, and many of those contentions are problematic.
Using two different approaches, researchers from Mathematica Policy Research conclude that Knowledge Is Power Program (KIPP) students scored higher than comparison students not attending KIPP schools by an amount equivalent to 11 months of additional learning in math and about eight months in reading. The impact was unevenly distributed across KIPP schools, and a number of factors were identified that were weakly related to this variation in effectiveness.
This National Bureau of Economic Research working paper purports to examine the extent and effects of sorting students into classrooms by test scores. It then claims to explore the effect of sorting on overall student achievement as well as on low achievers, high achievers, gifted, special education and Limited English Proficient students. The paper uses standardized Texas state test scores as the measure of learning growth.
This descriptive work urges U.S. policymakers to consider the English system of school inspections as a way of expanding our understanding of student achievement. Such an innovation is timely, according to the report, because the No Child Left Behind legislation is coming up for reauthorization. Using data largely from the English Office for Standards in Education (Ofsted), the report outlines the English system, including the rubrics it uses in judging teaching, some characteristics of its inspectors, and how much such a system might cost states in the United States.
In this review of The New Teacher Project’s (TNTP) “fact sheet” on value-added (VA) analysis, Di Carlo argues that while VA models are sophisticated and have a lot of potential, we have no idea how they are best used or whether they will work. It is likely, he says, that poor models implemented in the wrong way would “penalize” critically large numbers of teachers for reasons beyond their control, as well as generate estimates that are too unstable to be useful for any purpose, even low-stakes decisions.
This NEPC Policy Memo presents the text of a letter from Drs. Burris and Welner to Secretary of Education Arne Duncan. The letter was invited by Secretary Duncan during a phone conversation with Dr. Burris. It offers concrete guiding principles for evaluation of educators and suggestions for a way forward.
The research on which the Los Angeles Times relied for its August 2010 teacher effectiveness reporting was demonstrably inadequate to support the published rankings. Using the same L.A. Unified School District data and the same methods as the Times, this study probes deeper and finds the earlier research to have serious weaknesses.
The Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project seeks to validate the use of a teacher’s estimated “value-added”—computed from the year-on-year test score gains of her students—as a measure of teaching effectiveness. Using data from six school districts, the initial report examines correlations between student survey responses and value-added scores computed both from state tests and from higher-order tests of conceptual understanding. The study finds that the measures are related, but only modestly.
NEPC Review: U.S. Math Performance in Global Perspective: How Well Does Each State Do at Producing High-Achieving Students?
A report from Harvard’s Program on Education Policy and Governance and the journal Education Next finds that only 6% of U.S. students in the high school graduating class of 2009 achieved at an advanced level in mathematics compared with 28% of Taiwanese students and more than 20% of students in Hong Kong, Korea, and Finland. Overall, the United States ranked behind most of its industrialized competitors. The report compares the mathematics performance of high achievers not only across countries but also across the 50 U.S. states and 10 urban districts.
High Stakes, but Low Validity? A Case Study of Standardized Tests and Admissions into New York City Specialized High Schools
This is a study of the admissions process at a select group of New York City public high schools. It offers the first detailed look at the admissions practices of this highly regarded and competitive group of schools, and also provides a window into the broader national debate about the use of standardized tests in school admissions. According to New York State law, admission to these schools must be based solely on an exam. The exam used is called the Specialized High Schools Admissions Test (SHSAT). This study makes use of the individual test results from 2005 and 2006.
The successor to No Child Left Behind remains to be shaped, but one change seems certain: School success will depend on whether students’ test scores increase, as opposed to just requiring scores above an adequate yearly progress threshold. Growth modeling approaches appear to allow for this policy shift. And this would likely be an improvement over the AYP approach in the current NCLB. Yet like many new technologies, it’s being oversold.
NEPC Review: Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure
This study examines the relationship between high-stakes school accountability and its effects upon student test scores and school policies. The authors seek to understand the extent to which accountability sanctions and incentives for the poorest-performing schools in Florida explain subsequent changes in school practices and policies as well as achievement — measured by state assessment data, Stanford-10 assessment data and surveys of public school principals.
Publisher New York: Greenwood Publishing
Page Numbers 36-41
Summary American high schools have long separated students, often using tests, and then provided them with different educational experiences and opportunities. This entry examines these interrelated practices of assessment and curricular differentiation.
Assessment, High Stakes, and Alternative Visions: Appropriate Use of the Right Tools to Leverage Improvement
Institution: Simon Fraser University
This brief examines the theoretical basis behind high-stakes accountability, the intended and unintended consequences of such systems, and proposed alternative reform models. It also reviews existing research on all models, although the research is scant for some alternatives. As a caution to research consumers, the brief also details the highly political nature of much related contemporary research.
This series of policy briefs examining education reform in Florida finds that the results of the state's aggressive school-reform program have been mixed, and that the state's actions often do not match its rhetoric when it comes to implementing reforms.
Each of the following sections can be found in downloadable format below. The separate Executive Summaries can be found here.
Author: Craig Bolon Source: Education Policy Analysis Archives In 1998 Massachusetts began state-sponsored, annual achievement testing of all students in three public school grades. The state is treating scores and ratings as though they were precise educational measures of high significance. A review of tenth-grade mathematics test scores from academic high schools in metropolitan Boston showed that statistically they are not.
Publisher Washington, DC: American Educational Research Association
Publisher Educational Researcher, 29(7)
Page Numbers 4-14
Publisher Educational Testing Service
Summary What does it mean to test for what students really know? This presentation discusses the teaching-the-test literature, providing examples of not only what students cannot do when they have been prepared for a specific format but also some examples from performance assessments illustrating how teaching-the-test problems have been addressed. On the other side, examples of ETS research on test bias are also provided, and these contradictions are considered.
Publisher Educational Researcher, 24
Page Numbers 25-32
Publisher Educational Researcher, 22
Page Numbers 10-13
Summary This is a response to Cizek's criticisms to Shepard's article, "Psychometricians' Beliefs About Learning" (1991). This response asserts that each of Cizek's complaints reflects a serious misreading of the original article.
Commentary: What Policymakers Who Mandate Tests Should Know About the New Psychology of Intellectual Ability and Learning
Publisher Boston: Kluwer Academic Publishers
Publisher Educational Researcher, 20
Page Numbers 2-16
Summary This article examines beliefs that psychometricians hold about learning, borrowing both methodological approach and perspective from recent research on teaching thinking, which suggests that teachers' classroom practices can be understood in terms of their beliefs or implicit theories about instruction and learning.