“If you want to have good ideas you must have many ideas,” said the chemist Linus Pauling, according to molecular biologist Francis Crick. “Most of them will be wrong, and what you have to learn is which ones to throw away.” Two researchers who in 2013 created a computational model exploring the relationship between good and bad ideas added a caveat to Pauling’s dictum: “And the best way to get lots of ideas is to first generate a few that are as different as possible and then strategically build on them.”
But what does that sorting process look like, in practice? What proportion of our ideas end up being good ones, and how does one analyze something as subjective as idea quality, anyway?
One interesting answer comes from a small experiment conducted by John Kirwan, a professor of rheumatology at the University of Bristol. After considering Pauling’s thesis, Kirwan decided to find out for himself what percentage of the ideas he’d generated in the last 23 years of his academic career fit his personal definition of a good idea: an important concept that resulted in at least one publication that contributed significantly to the field of scientific inquiry.
As he chronicled in the journal Nature, Kirwan went through his computer files and found folders for 185 distinct projects, each containing data, drafts, correspondence, and other materials. Using this information and the dates in the materials, he scored the time and effort he put into each project idea. Then he classified each on a spectrum of progress, from ideas that didn’t move beyond the concept stage to those that produced multiple publications. He also assigned each project a subjective “scientific importance” ranking, heavily influenced by whether or not a project led to quality research from other scientists.
Kirwan has had a successful academic career. In his time in academia, 55 of his published works have been cited at least 55 times each, with a cumulative 10,000 citations. Yet his analysis revealed that 75% of the 185 ideas he found in his archives did not lead to to any publications at all. By his own assessment, only 2.7% of his ideas met all his criteria to be considered “especially good.”
Does he consider the other 97.3% a waste? Not at all.
“The issue here is to recognize that in science (and indeed, perhaps in life as a whole) we have lots of ideas that don’t work out. You cannot tell initially if the idea will work or not—you need to explore it and do some work on it in order to find out,” Kirwan wrote to Quartz. “This is a necessary process…we are not wasting our time in exploring ideas that turn out not to work—we are helping to generate good ideas.”