"The Great Influenza" IV: Interpreting ExperimentsIn The Great Influenza: The Story of the Deadliest Pandemic in History, John M. Barry devotes considerable attention to the challenges faced by the scientists trying to identify the pathogen behind the "Spanish flu" of 1918.
At one point, Barry notes:
Not all scientific investigators can deal comfortably with uncertainty and those who can may not be creative enough to understand and design the experiments that will illuminate a subject to know both where and how to look. Others may lack the confidence to persist. Experiments do not simply work Regardless of design and preparation, experiments especially at the beginning, when one proceeds by intelligent guesswork rarely yield the results desired. An investigator must make them work. The less known, the more one has to manipulate and even force experiments to yield and answer.
Which raises another question: How does one know when one knows? In turn this leads to more practical questions: How does one know when to continue to push an experiment? And how does one know when to abandon a clue as a false trail?
No one interested in any truth will torture the data itself, ever. But a scientist can and should torture an experiment to get data, to get a result, especially when investigating a new area. A scientist can and should seek any way to answer a question: if using mice and guinea pigs and rabbits does not provide a satisfactory answer, then trying dogs, pigs, cats, monkeys. And if one experiment shows a hint of a result, the slightest bump on a flat line of information, then a scientist designs the next experiment to focus on that bump, to create conditions more likely to get more bumps until they become either consistent and meaningful or demonstrate that the initial bump was mere random variation without meaning. [pp. 263-264]