Smaller teams produce more innovative research, study finds

A new analysis of more than 65 million papers, patents and software projects found that smaller teams produce much more disruptive and innovative research than larger teams, which more often develop and consolidate existing knowledge.

University of Chicago researchers examined 60 years of publications and found that smaller teams were far more likely to introduce new ideas to science and technology, while larger teams more often developed and consolidated existing knowledge. Their findings have been published in Nature.

While both large and small teams are essential for scientific progress, the findings suggest that recent trends in research policy and funding towards big teams should be reassessed.

“Big teams are almost always more conservative. The work they produce is like blockbuster sequels; very reactive and low-risk,” said study co-author James Evans, a professor of sociology and director of the Knowledge Lab at the University of Chicago. 

“Bigger teams are always searching the immediate past, always building on yesterday’s hits. Whereas the small teams, they do weird stuff — they’re reaching further into the past, and it takes longer for others to understand and appreciate the potential of what they are doing.”

Knowledge Lab is a research centre that combines ‘science of science’ approaches from sociology with the explosion of digital information now available on the history of research and discovery. By using advanced computational techniques and developing new tools, Knowledge Lab researchers reconstruct and examine how knowledge over time grows and influences our world, generating insights that can fuel future innovation.

The Nature study collected 44 million articles and more than 600 million citations from the Web of Science database, 5 million patents from the US Patent and Trademark Office, and 16 million software projects from the Github platform. Each individual work in this massive dataset was then computationally assessed for how much it disrupted versus developed its field of science or technology.

“Intuitively, a disruptive paper is like the moon during the lunar eclipse; it overshadows the sun — the idea it builds upon — and redirects all future attention to itself,” said study co-author Lingfei Wu, a postdoctoral researcher with the University of Chicago and Knowledge Lab. “The fact that most of the future works only cite the focal paper and not its references is evidence for the ‘novelty’ of the focal paper. Therefore, we can use this measure, originally proposed by Funk and Owen-Smith, as a proxy for the creation of new directions in the history of science and technology.”

The analysis shows that both small and large teams play important roles in the research ecosystem, with the former generating new, promising insights that are rapidly developed and refined by larger teams. Some experiments are so expensive, like the Large Hadron Collider or the search for dark energy, that they can only be answered by a single, massive collaboration. But other complex scientific questions may be more effectively pursued by an ensemble of independent, risk-taking small teams rather than a large consortium, the authors argue.

“In the context of science, funders around the world are funding bigger and bigger teams,” said Evans. “What our research proposes is that you really want to fund a greater diversity of approaches. It suggests that if you really want to build science and technology, you need to act like a venture capitalist rather than a big bank — you want to fund a bunch of smaller and largely disconnected efforts to improve the likelihood of major, path-breaking success.

“Most things are going to fail, or are not going to push the needle within a field. As a result it’s really about optimising failure,” Evans added. “If you want to do discovery, you have to gamble.”

Across papers, patents and software products, disruption dramatically declined with the addition of each additional team member. The same relationship appeared when the authors controlled for publication year, topic or author, or tested subsets of data, such as Nobel Prize-winning articles. Even review articles, which simply aggregate the findings of previous publications, are more disruptive when authored by fewer individuals, the study found.

The main driver of the difference in disruption between large and small teams appeared to be how each treat the history of their field. Larger teams were more likely to cite more recent, highly cited research in their work, building upon past successes and acknowledging problems already in their field’s zeitgeist. By contrast, smaller teams more often cited older, less popular ideas, a deeper and wider information search that creates new directions in science and technology.

“Small teams and large teams are different in nature,” Wu said. “Small teams remember forgotten ideas, ask questions and create new directions, whereas large teams chase hotspots and forget less popular ideas, answer questions and stabilise established paradigms.”

Image credit: ©stock.adobe.com/au/Gajus