Huge Study Confirms Science Ending! (Sort Of)

Uh oh, this doesn’t look good. Study in Nature shows massive decline in “disruptive” science.

January 19, 2023. I've always been obsessed with the limits of science. How far can science go? Can it keep giving us deep insights into nature forever, or will it eventually run into a wall? In my 1996 book The End of Science, I argued that we're already hitting a wall. The era of profound discoveries is over.

My claim that science is ending seems nutty if you look at the sheer quantity of science. More scientists are churning out more research papers than ever. But what about the quality of these papers? How many reveal anything truly consequential? 

A big new study in Nature addresses this question by examining 45 million papers published over the past six decades. Three scholars, led by economist Russell Funk of the University of Minnesota, find sharp declines in so-called "disruptive" research in biology, physics, social science and technology. The authors distinguish "disruptive" papers from those that merely build upon or "consolidate" previous knowledge. Citations of disruptive papers are less likely to cite earlier research, presumably because they render it obsolete. 

An analysis of 3.9 million patents produced similar results. Papers and patents "are increasingly less likely to break with the past in ways that push science and technology in new directions," the authors conclude. This study corroborates others showing declines in science's productivity (see graph below).

The End of Science focuses on "pure" science, which yields insights into nature, rather than applied science, which gives us new technologies and medicine. The Nature authors do not distinguish pure from applied science. But their distinction between "disruptive" and consolidating science is useful. It reminds me of Thomas Kuhn's distinction between revolutionary and normal science. Revolutionary science challenges the prevailing paradigm, or status quo, whereas normal merely extends it.

In The End of Science, I argue that science has entered a period of permanent normality; there will be no more insights into nature as revolutionary as the theory of evolution, the double helix, quantum mechanics, relativity and the big bang. Why not? Because these profound discoveries are true. Put them together, and they form a map of reality that, like our maps of the Earth, is unlikely to undergo significant changes. Science, in other words, is a victim of its own success.

The Nature authors cite The End of Science but reject its thesis. They note that "science and technology do not appear to have reached the end of the 'endless frontier.'" Science still produces disruptive advances, they argue, albeit at a diminished rate compared to total research. As examples of "highly disruptive work," they cite the detection of gravitational waves seven years ago and the recent development of covid vaccines.

But these examples support my argument that science is no longer revolutionary. Einstein predicted gravitational waves a century ago. And while I'm grateful for covid vaccines, they are an application of work in molecular biology dating back to the discovery of the double helix in 1953. In other words, neither example is remotely revolutionary.

A news story on the new Nature study says "no one knows" why disruptive science has "plummeted over the last half century." Actually, the Nature scholars cautiously blame the decline of disruptive science on the increasing competitiveness of science, which is leading to an emphasis on quantity over quality. To counter this trend, universities and funding agencies should give researchers more time to "step outside the fray, inoculate themselves from the publish or perish culture, and produce more truly consequential work." Admirable goals, but do they have any chance of being implemented? 

Ironically, I'm less pessimistic than I used to be about science's future, mainly because I've spent the last two-plus years studying quantum mechanics. The more I learn about the theory, the less sense it makes. The vast edifice of modern physics, which rests on quantum physics, seems wobbly, unstable, ripe for a revolution. Perhaps quantum computing will catalyze this revolution.  

Or not. The bursting of the tech bubble might take quantum computing down with it. I guess I'm still a pessimist at heart.

Updates: Commentary on the "disruptive science" study keeps rolling in. See "What Happened to All of Science's Big Breakthroughs?" by William Broad of The New York Times; "Are we living in the last days of the scientific age?" by Mark Sumner of Daily Kos; and "Science Is Losing its Ability to Disrupt" by Anjana Ahuja of Financial Times. Ahuja writes:

The US science writer John Horgan, who penned The End of Science in 1996, believes the era of blockbuster discoveries is over. "[The Nature study] dovetails with my analysis that science is hitting a wall," he tells me, adding that he finds no pleasure in vindication. "I want to see revelation after revelation — forever! But sadly, that's not the way scientific discovery works." Horgan argues there are only so many truths to uncover, only so many paradigms to shift — and thus a finite pot of disruption to exhaust.

Graph from 2020 paper “Are Ideas Getting Harder to Find?” by economists Nicholas Bloom et al.