Arum Research Calls Out “Limited Learning” on College Campuses

Over the past decade, policy makers have focused intently on assessing what elementary and secondary school students are learning through regular standardized testing. Higher education has, for the most part, escaped this testing culture, as parents, students, and employers take for granted the notion that a college education will prepare students for the twenty-first century labor market.

A forthcoming book on what students actually learn in college, however, questions whether students at the nation’s colleges and universities are in fact acquiring the skills necessary to compete in the global marketplace. Economists, sociologists, and educators agree that future labor markets will demand sophisticated critical thinking, problem solving, and writing skills from workers, the very skills that a college education is presumed to develop in students.

Richard Arum, professor of sociology and education at New York University with joint appointments at FAS and NYU Steinhardt, and Josipa Roska, assistant professor of sociology at the University of Virginia, embarked on a multi-year, longitudinal study of more than 2300 undergraduate students at 24 universities across the country. Using a newly developed, state-of-the-art measurement tool, the College Learning Assessment (CLA), the research pair measured the extent to which students improved on these higher order skills. The CLA is a tool developed by the Council for Aid to Education, a national nonprofit based in New York.

Unlike standardized tests such as the GRE or GMAT, the CLA is not a multiple choice test. Instead, the multi-part test is a holistic assessment based on open-ended prompts. “Performance Task” section prompts students with an imagined “real world” scenario, and provides contextual documents that provide evidence and data. The students are asked to assess and synthesize the data and to construct an argument based on their analysis.

The results of Arum and Roska’s research are troubling: 45 percent of the students in their sample demonstrated no significant gains on the CLA between their freshman and sophomore years of college. The average student’s performance on the CLA rose by only 7 percentile points, suggesting that college curricula and instructional strategies are not developing students’ higher order thinking skills.

The researchers found significant differences across subgroups of students, with a big gap between the performance of black and white students on the CLA. They also found that students at more selective universities (those with traditional liberal arts curricula) tend to score better on the CLA.

In some ways, Arum says, the results of the study are not surprising. Previous research has shown that the average student spends only 13 hours per week studying, far less time than is spent on social pursuits. Likewise, faculty incentives within higher education (promotion, tenure) are aligned with research pursuits, rather than the quality of undergraduate instruction. According to Arum, this misalignment of goals results in far less attention to teaching and learning than is necessary to cultivate higher order skills in students.

These findings should concern policymakers, Arum argues, since the CLA has been adopted by the Organization for Economic Cooperation and Development (OECD). In 2016, the OECD will use the CLA for a cross-national study of systems of higher education. Arum surmises that, compared to that of other industrialized countries, higher education in the U.S. will appear to be less effective than many have assumed at producing the kinds of workers needed to compete in a global marketplace.

“The U.S. higher education system has been living off its international reputation gained from offering outstanding doctoral programs,” Arum asserted. “In the future, U.S. higher education will increasingly be held accountable for demonstrating measurable improvement in undergraduate learning.”

Their findings will be published this fall in a new book, Academically Adrift: Limited Learning on College Campuses (University of Chicago Press). Funding for their research was provided by the Ford, Lumina, Carnegie, and Teagle Foundations.

This post appears in the following categories: Faculty