When is a jellyfish plague not (necessarily) a jellyfish plague? When time-poor scientists selectively cite the literature to make it look like the oceans are flooded with jellies – even when it’s far from clear that they really are.

What does scientists being in a rush have to do with jellyfish populations? Let’s start from the beginning.

The identification of patterns and trends in nature happens through the accumulation of consistent observations, published in scientific reports. Once observed, the emerging patterns are usually reported in narrative reviews, which often stimulates a flurry of research activity in that field.

Eventually, the purported patterns are formally tested using “meta-analyses” of the published literature, to either confirm the pattern and establish it as theory, or refute it.

This path from the primary observations to theory can be traced through a network of citations.

Science, however, is done by humans and citation practices are subject to errors of bias and accuracy. Citation practices that are biased in a particular direction have the potential to lead to the identification of false patterns and flawed theory.

Enter the jellies

In the 1990s and 2000s, reports began to appear in the scientific literature of increased jellyfish populations in some parts of the world’s oceans. Various reviews reported the possibility that jellyfish blooms might be increasing globally. Over time, these became increasingly assertive about the existence and extent of the trend, until researchers were asking what to do about the increasingly “gelatinous state” of the oceans worldwide.

The question of whether the global jellyfish boom was real or not was tested by two meta-analyses – which came to opposite conclusions. A 2012 study concluded that populations were increasing globally because they found evidence for increasing populations in 62% of large marine ecosystems tested (although low certainty was assigned to two-thirds of these). The following year, another study found that only 30% of populations were increasing. It concluded that jellyfish populations wax and wane over several decades.

So, in reality, the scientific community is still divided over whether there really has been a sustained global increase in jellyfish numbers.

What about perception?

We wanted to know whether the perception of a global increase in jellyfish blooms was at least partly due to poor citation practices in the scientific literature. Our research, published in Global Ecology and Biogeography, suggests that it was.

Citation practices can be flawed in several ways:

  • Unsupported citations are when authors cite sources that contain no evidence that could possibly support the author’s claim.
  • Selective citations happen when a paper is cited to support a claim but contrasting evidence provided in the same paper is ignored, or when authors choose to cite earlier papers that have since been refuted.
  • Ambiguous citations happen when an author’s sentence contains multiple phrases, but the citations used to support each phrase are clustered at the end of the sentence, preventing readers from telling which is which.
  • Empty citations are when authors cite a paper that cites another paper as evidence for the claim, rather than the original source (also called “lazy author syndrome”).

We comprehensively searched the literature for papers, published before the two meta-analyses, that issued statements regarding trends in jellyfish populations. We classified each statement according to its affirmation and direction (that is, whether it said jellyfish are “increasing”, “may be increasing”, “decreasing”, or “not sure”), as well as their geographic extent (global, multiple regions, or one region).

We then assessed the papers cited as evidence of the statement, to see whether the citations were accurate or whether they fell into one of the categories of flawed citations outlined above.

A (jelly)fishy tale?

Of 159 papers that had issued statements about trends in jellyfish, 61% claimed that populations were increasing (27% at the global scale and 34% in multiple regions) and 25% asserted that populations may be increasing. Only 10% of papers said the data was equivocal. Just one reported that populations were decreasing (but at a local scale).

Most concerning was that only 51% of papers cited were considered to provide unambiguous support for the statements made by the authors. Almost all statements based on unsupportive citations were those claiming that jellyfish were increasing globally (despite the fact that it would have been impossible to make any claims about global trends before the first global meta-analysis was published in 2012). And in all cases, selective citations were biased towards claims that jellyfish populations were increasing.

Pressure to publish in prestigious journals and win research funds may lead some scientists to make claims that reach beyond the evidence available. In most cases, however, citation errors are not overt attempts to distort the evidence. Rather, they probably arise because increasing academic workloads reduce the time available to evaluate papers accurately and to keep abreast of the almost exponential increase in the volume of literature being published.

As scientists, we need to ensure that our claims are always supported by robust evidence because it is apparent that poor citation practices – and, in particular, selective citation of the literature – can distort perceptions within a research field.

Kylie Pitt, Associate Professor, Griffith University; Carlos Duarte, Adjunct professor, King Abdullah University of Science and Technology; Cathy Lucas, Associate Professor, Marine Biology & Ecology Research Group (MBERG), University of Southampton; Charles Novaes de Santana, Postdoctoral research associate, University of Zurich; Marina Sanz-Martín, Researcher, Department of Global Change Research, University of Barcelona, and Rob Condon, Assistant Professor in Biological Oceanography, University of North Carolina Wilmington.

This article first appeared on The Conversation.