NYU Wagner School: Urban Education Research Project

Introduction

Wagner faculty Amy Ellen Schwartz and Leanna Stiefel lead several research projects studying vital policy issues in urban elementary and secondary education. Current projects include a study of segregation and achievement among immigrant school children in New York City; an analysis of the test score gap between white, black, Hispanic and Asian children; and an investigation of useful and reliable measures of school performance. Recently completed studies include evaluations of the achievement effects of popular school reforms and the equity of school finance distributions. Much of this research focuses on the problems of urban schools and policies and reforms aimed at addressing those problems. Recent support for Professors Schwartz and Stiefels' research has come from the US Department of Education, New York State Education Finance Research Consortium, the Russell Sage Foundation, the Rockefeller Foundation and the Spencer Foundation. The following pages provide more details on each of these projects.

IMMIGRANTS AND EDUCATION

Summary:

While an estimated 10.4 percent of the U.S. population is now foreign-born, the highest percentage since 1930, we know surprisingly little about the experience of foreign-born students. We know little about the characteristics of their peers, the resources in their schools, their impact on their peers and schools, and the impact that their schools and peers have on their achievement. Thus, there is scant evidence to guide policymakers and educators in their efforts to provide a sound education for immigrant students. Our research is aimed at filling that gap. In particular, our research addresses three central questions: (1) how does the movement of students between schools affect the segregation, isolation and exposure of immigrant students? (2) how does that mobility change school resources? And (3) how are student experiences and academic achievement, including, but not limited to performance on tests, affected by the concentration of immigrants in schools and by school resources?

Publications:

• "Immigrant Children and Urban Schools: Evidence from New York City on Segregation and its Consequences for Schooling," Brookings-Wharton Papers on Urban Affairs, 2002 (Ingrid Gould Ellen, Katherine O'Regan, Amy Ellen Schwartz and Leanna Stiefel).

TEST SCORE GAPS: CAUSES AND CURES

Summary:

Black and Hispanic school children score lower, on average, on achievement tests than do white and Asian children. Immigrant students often score higher than native-born students and girls outperform boys in early grades. Why do we see these test score gaps and, more importantly, what can be done to close them? In conjunction with the Education Finance Research Consortium (EFRC), a joint effort between university researchers and the New York State Education Department, we are studying the test score gaps in all New York State elementary and middle schools, describing patterns and identifying characteristics associated with schools in which test score gaps are small. In future work, we hope to develop models of school success and we will work with the EFRC to implement findings in state schools.

SCHOOL PERFORMANCE: USEFUL AND RELIABLE MEASURES

Summary:

The recently passed federal No Child Left Behind Act (NCLB) requires states to test school children annually and use these test results to evaluate the performance of schools. Where performance is lacking school districts are required to provide alternative schooling options. Although it may seem that test results, in and of themselves, are adequate measures of performance, test results alone are unlikely to provide useful policy guidance. School-level average test scores are too aggregated, focus attention on achievement levels, rather than achievement gains (or losses) and, further, do not account for differences in the students attending those schools. What, then, is the alternative? How can useful, reliable, and understandable performance measures be derived from test results? Without such measures, public support for NCLB may well deteriorate. Because this is the first time the federal government has tied funding to school outcomes in a large way, the stakes are high to get the measures right. We are leading a three year project to evaluate alternative performance measures using unique data from two states, over five years. The results of this study will likely contribute major new insight the national debate on school performance and school accountability laws. In addition, because NCLB requires that test results be separated by subgroups of students (racial, income, language and disabled subgroups), the work of this project will benefit from the complementary work of the team's research on test score gaps.

Publications:

• “Better than Raw: A Guide to Measuring Organizational Performance with Adjusted Performance Measures,” Public Administration Review, forthcoming (Ross Rubenstein, Amy Ellen Schwartz, and Leanna Stiefel).

• “Using Adjusted Performance Measures for Evaluating Resource Use,” Public Budgeting and Finance, Volume 19, No. 3, Fall 1999 (Leanna Stiefel, Amy Ellen Schwartz and Ross Rubenstein). Recipient of Joseph S. Wholey Distinguished Scholarship Award.

• “Conceptual and Empirical Issues in the Measurement of School Efficiency,” National Tax Journal, Proceedings from 91st Annual Conference, 1998, pages 267-274. (Amy Ellen Schwartz, Ross Rubenstein, and Leanna Stiefel).

• “Measuring School Efficiency: Lessons from Economics, Implications for Practice,” in David H. Monk, Herbert J. Walberg, and Margaret C. Wang, eds., Improving Educational Productivity, Information Age Publishing Inc., 2001 (Amy Ellen Schwartz and Leanna Stiefel).

• “Measuring School Efficiency Using School-Level Data: Theory and Practice,” in Margaret Goertz and Allan Odden, eds., School-Based Financing, Corwin Press, 1999 (Leanna Stiefel, Amy Ellen Schwartz, and Ross Rubenstein).

• “Academic Performance, Characteristics and Expenditures in New York City Elementary and Middle Schools,” (Leanna Stiefel, Amy Ellen Schwartz, Patrice Iatarola, Norm Fruchter), Condition Report, Education Finance Research Consortium, New York State Education Department, April 2000.

• “Determinants of School Performance in New York City Elementary Schools: Results and Implications for Resource Use,” (Leanna Stiefel, Amy Ellen Schwartz and Patrice Iatarola), submitted as Condition Report, Education Finance Research Consortium, New York State Education Department, May 2001.

EVALUATIONS OF SCHOOL REFORMS

Summary:

Large scale school reforms are notorious for their lack of systematic, rigorous evaluations, and thus the progress that American schools can make in educating all children, and especially urban children, is hindered. We have developed methodologically rigorous methods that use administrative data to evaluate the effect of important urban school reforms on schooling outcomes. These evaluations are important not only for their specific findings, but also because they show that relatively inexpensive designs (compared to large controlled experimental designs) are highly feasible and reliable. Some recent evaluations are indicated below.

Publications:

Evaluation of the effect of New York City's school-based budgeting reform on school performance:

• "School Budgeting and School Performance: The Impact of New York City's Performance Driven Budgeting Initiative," Journal of Education Finance, Fall 2003 (Leanna Stiefel, Amy Ellen Schwartz, Dae Yeop Kim, Carole Portas).

Evaluation of the effect of New York City's Annenberg Challenge Grant on School Performance:

• "The Impact of School Reform on Student Performance: Evidence from the New York Networks for School Renewal Project," Journal of Human Resources, forthcoming (Amy Ellen Schwartz, Leanna Stiefel, Dae Yeop Kim)

Cost-Effectiveness Study of New York City Small High Schools:

• "High School Size: The Effects on Budgets and Performance in New York City," Educational Evaluation and Policy Analysis, Spring 2000 (Leanna Stiefel, Robert Berne, Patrice Iatarola and Norm Fruchter).

DISTRIBUTION AND EQUITY IN SCHOOL FINANCE

Summary:

Over the past 30 years, almost every state in the nation has been involved in a school finance law suit challenging its funding mechanisms. These law suits have been brought on the basis of the US and state constitutions and they have been successful in about half the states. Professors Stiefel and Schwartz have been active and central participants in the academic as well court research that has influenced these cases. Most recently, in conjunction of Vice President Bob Berne, Professors Stiefel and Schwartz undertook extensive analysis of the New York State education finance system in support of the Campaign for Fiscal Equity's court case against the state. Much of their work was instrumental in the lower court ruling, by Justice Leland DeGrasse, in favor of the plaintiffs. After an intermediate court ruling for the state, the case is now before New York's highest court, the Court of Appeals. Selected writing on the issue of distribution is highlighted below.

Publications:

• "Intradistrict Equity of Public Education Resources and Performance," Economics of Education Review, February 2003 (Leanna Stiefel and Patrice Iatarola).

• The Measurement of Equity in School Finance: Conceptual, Methodological, and Empirical Dimensions, Johns Hopkins University Press, 1984 (Robert Berne and Leanna Stiefel).

• "Concepts of Equity: 1970 to Present," in Helen F. Ladd, Rosemary Chalk, and Janet S. Hansen, eds., Equity and Adequacy in School Finance: Background Papers, National Academy of Science Press, 1999 (Robert Berne and Leanna Stiefel).

• "Does the New York State School Finance System Have a Disparate Racial Impact on New York City School Children? The Contribution of Statistical Analysis," working paper, Wagner School, 2001 (Leanna Stiefel, Amy Ellen Schwartz, and Robert Berne).

• "Education Finance," in Fred Thompson and Mark Green, eds., The Handbook of Public Finance, Marcel Dekker, Inc., 1998, pages 447-482 (Amy Ellen Schwartz, Leanna Stiefel, and Ross Rubenstein).

• "School Districts and Spending in the Schools," Selected Papers in School Finance, 1997-99, National Center for Education Statistics, US Department of Education, 1999, pp. 55-83 (Amy Ellen Schwartz).

IMPORTANT FUTURE RESEARCH AREAS

We are interested in research that will help urban schools provide a first rate education to all of their students. Important questions we are poised to investigate, pending needed support include:

Closing the Achievement Gap: What explains the disparity in academic achievement across socioeconomic and demographic groups? Which school policies or resources might be used to reduce or eliminate these disparities? To be specific, why are there such significant differences in the test performance of white, black, Asian and Hispanic students in New York City and State? Why is there a gap in performance between girls and boys; native and foreign born students; or between poor and non-poor students? What can city and state school policymakers and administrators do to address these test score gaps? We hope to investigate both what can be done and how much it will cost to make these changes.

Accountability and Performance: As federal, state and district policymakers increasingly implement performance-based accountability reforms aimed at school improvement we are interested in exploring the impact of these reforms on urban students and schools. What will be the effect of the new accountability systems required by No Child Left Behind legislation on the achievement of urban students? What impact does the introduction of high-stakes testing have on student promotion or retention; on referrals to special education; on the duration of time students spend in programs aimed at limited English proficiency?

Evaluations of School Reforms: Each year, New York City implements a wide range of reforms in school policies, curricula, and organization, many of which are aimed at improving student performance and/or reducing school costs. How effective are these reforms at achieving their objectives? We hope to use the data and expertise we have developed to evaluate such reforms in New York City as the creation of small schools or schools, after school programs, or lengthening of the school day