The Positive Impact of Negative Data
Posted on February 22, 2016
Irena Nesterenko
The heart of scientific research is hypothesis-driven experimentation. Although we phrase the hypothesis as a relationship between variables with a predicted outcome, the rigorously-controlled experiment that follows is actually designed to disprove something — to reject the null hypothesis, which states that there is no such relationship between the tested variables. Positive data supports our hypothesis and rejects the null hypothesis, and best of all, positive results afford us the graphs and pictures that we so desperately need in order to publish, to include as preliminary data in grant applications, and to fuel further investigation.
Negative or non-confirmatory results are scientifically sound outcomes that fail to reject the null hypothesis. Often they are not pursued further or hidden in supplemental figures. At worst, negative results are omitted without acknowledgement or discussion, or even “fudged” to appear as positive data. But I like to think on the bright side… At best, the results that fail to reject the null hypothesis can inspire new avenues of research as they force us to refine or restructure the initial hypothesis.
So, if positive and negative data are two sides of the same coin, both necessary to advance scientific inquiry, then why do we rarely, if ever, see negative results published? Reviewers and readers alike clamor for statistically significant, easily digestible data, causing a publication bias towards the “sexy” results that clearly move the field forward. But the reality is, a great deal of the results we obtain are non-confirmatory and often are relegated to a “file drawer” of unpublished and largely forgotten data.
Many argue that the time and effort invested in writing up negative data is a lost opportunity to write about positive data to publish in conventional journals. However I believe this is the beginning of the slippery slope to confirmation bias. Exploring why negative results are taking place can open new avenues and lead to breakthroughs in the field as we challenge accepted models. It can open new lines of communication between research groups experiencing the same recurring roadblocks. It’s the ethical way to handle the results that disappoint and confound us; transparency helps us all!
Indeed, the idea that transparency allows us to work better and smarter has been touted in science, politics, and journalism. What may seem like an idealist abstraction has very real consequences in medicine: peer-reviewed, published clinical trial data informs how doctors manage the course of their treatments. In cases of confirmation bias, mild side effects may be accidentally omitted in studies where the pressure to validate new therapies is high. To increase transparency in publishing, several journals now publish all documents and communications from the review process, and some journals even make the raw data available to allow for independent peer review outside of the publishing pipeline.
Stringent review and careful oversight in the reporting of negative results will silence the petty complaint that those who publish them are only doing so to pad their resumes with publications. Negative results are simply insufficient without rigorous documentation and reporting of experimental assumptions and parameters, and as such, should be reviewed with the same scrutiny we turn towards our positive results.
This idea gained momentum in the public eye over the last 15 years, bringing the advent of several journals committed to publishing rigorously-reviewed negative or contradictory results (a selection of actively-publishing journals listed below). Professional social media site ResearchGate is an excellent platform to ask questions, share negative data, and forge collaborations worldwide. Working together, using a transparent and rigorous approach, we can illuminate what seems inscrutable and conquer what seems insurmountable when we work alone.
All Results Journals, published by SACSIS*
F1000Research, published by same
Journal of Articles in Support of the Null Hypothesis, published by the Reysen Group
Journal of Negative Results in BioMedicine, published by BioMed Central
Journal of Pharmaceutical Negative Results, published by Wolters Kluwer
New Negatives in Plant Science, published by Elsevier
http://www.journals.elsevier.com/new-negatives-in-plant-science
PLOS Missing Pieces Collection, published by PLoS
http://collections.plos.org/missing-pieces
Irina Nesterenko is a graduate student researcher and a NINDS T32 fellow in the laboratory of Dr. Olga Razorenova in the Molecular Biology and Biochemistry Department at UCI. Her current research is focused on the role of low-oxygen conditions in the survival of therapeutic stem cells in stroke, towards the development of improved cell-based therapies.
* Author volunteers with the All Results Journals as an English copyeditor.