Empirical Roadblocks Conference:
Measuring non-cognitive predictors of school success from early childhood to adolescence: Empirical roadblocks
and their solutions

Funded by the Spencer Foundation

On March 24, 2008, a group of 19 nationally-recognized experts and influential researchers gathered for a working conference at New York University and hosted by the Institute of Human Development and Social Change and Dr. C. Cybele Raver and moderated by Dr. Stephanie Jones. The goal of this specially-selected group, each with strong track records of productivity and collaboration, was to agree upon and to draft "best practice" solutions to the empirical challenge of measuring and analyzing children's "non-cognitive predictors" of their educational success, including over time and across developmental stages.

Our field's next empirical challenge

Early investments in children's opportunities to learn have been claimed to lead to long-term benefits in both social and cognitive domains.  A number of researchers have placed particular emphasis on measurement of children's skills in "non-cognitive" domains, focusing, for example, on children's self-regulation, their relationships with teachers, and their motivation as important predictors or later school outcomes (Raver, 2002; Raver, Garner, & Smith-Donald, 2007). In recent informal discussions, this group of researchers found that their work had hit upon a major empirical "roadblock." Driven by the disciplinary tradition of specialization within specific developmental periods, most have expertise in modeling individual differences between children at a given time point, on a given set of measures.  In short, measurement development led them to be able to clearly distinguish a child with high levels of impulse control, for example, from other children with lower levels of impulse control, at a given point in time.  However, they found that they had few metrics that allowed them to compare that child's performance later in his or her development with his or her own performance at an earlier time.

A small group of investigators focusing on poverty-related risks and children's emotional and behavioral development in the contexts of policy and prevention science was convened at NYU to addresss this challenge. Together, the community has engaged in empirical research cutting across psychological, sociological and econometric traditions, focusing on the questions outlined above within different developmental periods spanning from early childhood through adolescence. This group includes senior leaders in the field such as J. Lawrence Aber, Greg Duncan, Sean Reardon, Todd Little, Hiro Yoshikawa, Monica Rodriguez, Pamela Morris, as well as several "rising stars" in the field, including Stephanie Jones, Christine Li-Grining, and Rodolfo Mendoza-Denton.  

These measurement issues have particular salience as prevention science, education research, and policy research move forward in testing the impact of educational and clinical interventions using randomized, or experimental design. Specifically, lessons learned from randomized experiments testing the impact of early investments suggest that children's gains are often not detected until much later in children's development.  Yet it is generally difficult to capture the magnitude of those gains when the metric used to assess children's status at different time points can't be counted upon to appropriately model children's growth, over time.

Future work

Since this working conference, several members of this group have since been writing a paper summarizing their discussion and recommendations for best practices.  Information about their work will eventually be made available here and on the Spencer Foundation site.

Thank you to our funder!

This meeting was made possible by the generous financial and intellectual support of the Spencer Foundation.  The Foundation is committed to supporting high-quality investigation of education through its research programs and to strengthening and renewing the educational research community through its fellowship and training programs and related activities.

Participants and Related Websites                   Proceedings