Bill & Melinda Gates Foundation Postsecondary Success

By Jeffrey J. Selingo

Higher education policymaking has long been based on assumptions by experienced officials who often had inadequate information. But over the last decade-plus, as the data collected on students and the ability to analyze it has improved, so too has its use in shaping policymaking. Better education data has had another byproduct as well: attracting a new generation of higher-education researchers, particularly economists, to the discipline.

Among this new group of researchers is Judith Scott-Clayton, an associate professor of economics and education at Teachers College of Columbia University. “I’ve always liked the interaction between quantitative analysis and policy questions,” Scott-Clayton said. “There are questions that need to be answered, there’s data available to answer those questions, so let’s go.”

Answering those questions in higher education wasn’t always her goal. “Higher education research kind of found me, rather than the other way around,” said Scott-Clayton, who is also a senior research associate at the Community College Research Center and a research associate of the National Bureau of Economic Research.

As a Ph.D. student in public policy at Harvard University’s Kennedy School of Government, she was pulled into a project by a faculty member, Susan Dynarski, to study the complexity of the Free Application for Federal Student Aid (FAFSA). The form had grown over the years to more than 100 questions and four pages of instructions. There was rising concern among higher-education officials that the FAFSA was increasingly a barrier for students who were the first in their family to go to college. By one estimate, almost one-third of all undergraduates don’t file the form, and two million of those would have qualified for a Pell Grant if they had.

But policymakers believed simplifying the form might end up disqualifying some low-income students from receiving a Pell Grant while allowing less needy students to get aid. “Those were real concerns,” Scott-Clayton said, “but what we didn’t know was how big are these concerns.” Working with Dynarski, Scott-Clayton coded every question on the FAFSA to test how aid would change if questions were eliminated. What she found was startling: that with nearly 90 percent accuracy, eligibility for the Pell Grant could be determined with just a handful of questions.

The resulting study and publicity surrounding it added to a chorus of calls to simplify or ditch the form altogether—most prominent among them a June 2014 New York Times op-ed by Senators Lamar Alexander and Michael Bennet that advocated for scrapping all the questions on the form except for two (and the pair introduced legislation to do just that soon afterward). And while the form hasn’t been simplified to the point that Scott-Clayton and her colleagues had hoped, several key changes have been made to federal aid policies in recent years. In 2016-17, overall FAFSA filings increased for the first time in four years.

“Even when you have the data on your side, you need to have patience to wait for policy changes,” Scott-Clayton said. “Sometimes things sit around and then the stars align, so it’s good to have ideas packaged and ready for when that moment comes.”

***

Such patience is critical in a divided political era where policy changes may take years to come to pass. The current political environment also requires widening the reach of higher-education research to new audiences because states or institutions often pick up policy ideas before the federal government does. Beyond applying Big Data, another key difference between today’s higher-education researchers like Scott-Clayton and previous generations is how they share their research. It’s not unusual to see higher-education researchers publishing in the mainstream press or appearing on national radio and television programs. And in a social media age, many of them also have thousands of followers on Twitter.

For her part, Scott-Clayton is active on Twitter and a regular on the conference circuit. She is also frequently quoted in the press, and for a year, wrote a monthly post for the now-defunct New York Times Economix blog. She credits Dynarski for instilling in her an appetite to talk often and publicly about her research. “Sue was always passing off opportunities to me,” Scott-Clayton said. “What she’d tell me is that ‘you might not always feel like it, but you are the expert on this. If they are not talking to you, they’re going to talk to someone else.’”

Writing for the New York Times, in particular, brought Scott-Clayton a new level of exposure, but also an appreciation for how to explain sometimes complicated research in new ways. “I found it to be so challenging to write something that would be interesting to readers but that I can also imagine my senior colleagues appreciating,” she said. As a result, she came to understand how an exhaustive academic paper written for a journal could be sliced for easy public consumption into a plainspoken blog post, a simple graphic, or a short tweet.

Dynarski is now a professor of education, public policy, and economics at the University of Michigan at Ann Arbor and a regular contributor to the New York Times. She said that Scott-Clayton, who was her first doctoral student, asks questions that those in the field for a long time who get “stuck in lines of research” don’t often think to ask.

“While there are some folks wowed by the data in education but not particularly interested in education policy, Judith is in this because she wants to improve education,” Dynarski said. “She is asking questions, using whatever methods to answer those questions, and writes really clearly to explain her results to a broad audience.”

***

Beyond her work on the FAFSA, Scott-Clayton’s research has drawn attention to how students are placed into college remediation, what the impact of federal work study on job outcomes after graduation is, and whether the requirements of a state’s merit-based scholarship encourage degree completion (her research focused on West Virginia). While on the surface Scott-Clayton’s research portfolio might seem wide-ranging and disparate, she said her areas of study are all connected in the ways they use large data sets to quantify how certain policies affect student outcomes.

“What inspires me when I’m sitting at my desk, crunching numbers, is the feeling that I have access to some information that very few other people, or maybe nobody, has access to,” Scott-Clayton said, “and that I’m looking at a question that very few other people, or maybe nobody else, is looking at.”

Take, for example, her research on how students are placed into remedial education. While there is a bevy of research on college-entrance exams, such as the SAT and ACT, and an entire test-prep industry that helps students prepare for them, there was relatively little research on placement tests given to incoming students that determined whether they required remedial classes. Most of the media coverage of remedial education focused on the large percentage of college students in remedial education, but few college officials or policymakers ever asked if the students actually belonged in the classes.

“This was a problem—that there are actually a lot of students in remedial courses that don’t need to be there—that was much less visible,” Scott-Clayton said, “And it was a case where quantifying the problem and being able to put numbers on it, I think really did change the conversation.”

Scott-Clayton’s studies have found that community colleges have funneled tens of thousands of entering students unnecessarily into remedial classes through placement tests. Up to a third of students who tested into remedial courses because of college placement tests could have passed college-level classes with a grade of B or better. What’s more, students are rarely told about the high stakes of these tests—that a remedial course can often delay or derail their plans for a college degree. Scott-Clayton’s research has shown that placement decisions would be just as sound if colleges relied on high school GPAs instead.

Influenced by the research by Scott-Clayton and others, dozens of institutions and state systems have eliminated placement tests for remedial courses or use them as one of many factors. Last year, one of the largest university systems in the country, the 23-campus California State University system, dropped placement exams in math and English as well as the noncredit remedial courses that more than 25,000 freshmen had been required to take each fall.

“It does seem like the research is holding up and that students’ lives are being changed for the better,” Scott-Clayton said. “That’s very rewarding.”

In the nearly 10 years since she earned her Ph.D., Scott-Clayton has discovered how much higher-education policy viewed as arcane intersects with almost every other aspect of the college experience. But for higher-education researchers today to have impact on policy they need to move out of silos that might focus them in one particular area.

“What a lot of people are starting to realize is that these problems can’t be solved in isolation,” Scott-Clayton said. “Problems with financial aid or remediation or course registration, all these problems interact with each other, and if you can solve just one of them, that might push the needle a little bit, but if you really want to have transformative change, you have to deal with multiple challenges at the same time.”

Read about other higher education leaders who are making a difference.