Navigation und Service

Science learns from its mistakes too

27/2018, 12.09.2018

A mathematical model shows that even seemingly inconclusive studies speed up the gain in knowledge

Scientific studies should always be published irrespective of their result. That is one of the conclusions of a research project conducted by the “German Centre for the Protection of Laboratory Animals” at the German Federal Institute for Risk Assessment (BfR), the results of which have now been published in the journal “PLOS ONE”. Using a mathematical model, the scientists examined the influence that individual benchmarks have on further research when preparing the studies. “The research community should do everything possible to maintain social trust in science,” says BfR President, Professor Dr. Dr. Andreas Hensel. “This also means that results have to be understandable and reproducible so that false conclusions can be refuted easily. Our study shows that we achieve better results when seemingly inconclusive studies are published”.

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0202762

Investigations show that scientific studies have a better chance of getting published if they have a desired “positive” result, such as measuring an expected effect or detecting a substance or validating a hypothesis.  “Negative” or “null” results, which do not have any of these effects, have a lesser chance of publication.

It goes without saying that scientists also have a great interest in achieving meaningful results that are worthy of publication, thus advancing research. The great significance the publishing of a study in journals has on reputation and future sponsorship further intensifies this interest. The result of this can be, however, that studies are published the results of which are not reproducible and which therefore only appear to be “positive”.

These seemingly positive results then lead to further studies which build on the supposedly proven effect. The well-established practice among publishers of mainly publishing studies with positive results thus favours studies which do not stand up to scrutiny and therefore entail further unnecessary studies.

The mathematical model presented in the publication shows how the mechanism of “false positive” results can be broken through. If all studies – irrespective of their results – were to be published after complying with good scientific practice, a false result could be disproven more quickly.

This means that a seemingly negative result is not a drawback but rather a gain in knowledge, too. An animal test, for example, which cannot prove the efficacy of a new drug, would then not be a failure in the eyes of science but rather a valuable result which prevents unnecessary follow-on studies (and further animal tests) and speeds up the development of new therapies.

As it turned out, an additional criterion helps to facilitate the knowledge gain when preparing studies: in biomedical studies, a scientifically argued, sufficiently high number of test animals for a single experiment increases the likelihood of achieving correct and reproducible results at the first attempt. In the long run, unnecessary follow on tests with animals based on false assumptions can be avoided in this way. Ultimately therefore, the use of more test animals in a single experiment can reduce the total number of animals used.

The calculations of the BfR research team are based on biomedical research with laboratory animals, but the results can in general be applied to the life sciences.

The background of the study is the reproducibility crisis bemoaned in the life sciences and psychological research. Depending on meta-research, between 51 and 89 percent of the results published in bioscientific studies cannot be reproduced by other researchers. Neuroscientific studies show that shortcomings in the statistical evaluation of experiments are often a reason why studies cannot be reproduced.

About the Bf3R

The German Centre for the Protection of Laboratory Animals (Bf3R) was set up in 2015 and is an integral part of the German Federal Institute for Risk Assessment (BfR). It coordinates all activities throughout Germany to reduce animal experiments to an absolute minimum and to guarantee the best possible protection for laboratory animals. Further aims are to stimulate research and promote scientific dialogue worldwide.

About the BfR

The German Federal Institute for Risk Assessment (BfR) is a scientifically independent institution within the portfolio of the Federal Ministry of Food and Agriculture (BMEL) in Germany. It advises the Federal Government and Federal Laender on questions of food, chemical and product safety. The BfR conducts its own research on topics that are closely linked to its assessment tasks.

This text version is a translation of the original German text which is the only legally binding version.

Up