Unreliable research

How scientists pressed for time inadvertently made it seem like there was a global jellyfish boom

Flawed citation practices can perpetuate scientific ideas even before they've been fully established as true.

When is a jellyfish plague not (necessarily) a jellyfish plague? When time-poor scientists selectively cite the literature to make it look like the oceans are flooded with jellies – even when it’s far from clear that they really are.

What does scientists being in a rush have to do with jellyfish populations? Let’s start from the beginning.

The identification of patterns and trends in nature happens through the accumulation of consistent observations, published in scientific reports. Once observed, the emerging patterns are usually reported in narrative reviews, which often stimulates a flurry of research activity in that field.

Eventually, the purported patterns are formally tested using “meta-analyses” of the published literature, to either confirm the pattern and establish it as theory, or refute it.

This path from the primary observations to theory can be traced through a network of citations.

Science, however, is done by humans and citation practices are subject to errors of bias and accuracy. Citation practices that are biased in a particular direction have the potential to lead to the identification of false patterns and flawed theory.

Enter the jellies

In the 1990s and 2000s, reports began to appear in the scientific literature of increased jellyfish populations in some parts of the world’s oceans. Various reviews reported the possibility that jellyfish blooms might be increasing globally. Over time, these became increasingly assertive about the existence and extent of the trend, until researchers were asking what to do about the increasingly “gelatinous state” of the oceans worldwide.

The question of whether the global jellyfish boom was real or not was tested by two meta-analyses – which came to opposite conclusions. A 2012 study concluded that populations were increasing globally because they found evidence for increasing populations in 62% of large marine ecosystems tested (although low certainty was assigned to two-thirds of these). The following year, another study found that only 30% of populations were increasing. It concluded that jellyfish populations wax and wane over several decades.

So, in reality, the scientific community is still divided over whether there really has been a sustained global increase in jellyfish numbers.

What about perception?

We wanted to know whether the perception of a global increase in jellyfish blooms was at least partly due to poor citation practices in the scientific literature. Our research, published in Global Ecology and Biogeography, suggests that it was.

Citation practices can be flawed in several ways:

  • Unsupported citations are when authors cite sources that contain no evidence that could possibly support the author’s claim.
  • Selective citations happen when a paper is cited to support a claim but contrasting evidence provided in the same paper is ignored, or when authors choose to cite earlier papers that have since been refuted.
  • Ambiguous citations happen when an author’s sentence contains multiple phrases, but the citations used to support each phrase are clustered at the end of the sentence, preventing readers from telling which is which.
  • Empty citations are when authors cite a paper that cites another paper as evidence for the claim, rather than the original source (also called “lazy author syndrome”).

We comprehensively searched the literature for papers, published before the two meta-analyses, that issued statements regarding trends in jellyfish populations. We classified each statement according to its affirmation and direction (that is, whether it said jellyfish are “increasing”, “may be increasing”, “decreasing”, or “not sure”), as well as their geographic extent (global, multiple regions, or one region).

We then assessed the papers cited as evidence of the statement, to see whether the citations were accurate or whether they fell into one of the categories of flawed citations outlined above.

A (jelly)fishy tale?

Of 159 papers that had issued statements about trends in jellyfish, 61% claimed that populations were increasing (27% at the global scale and 34% in multiple regions) and 25% asserted that populations may be increasing. Only 10% of papers said the data was equivocal. Just one reported that populations were decreasing (but at a local scale).

Most concerning was that only 51% of papers cited were considered to provide unambiguous support for the statements made by the authors. Almost all statements based on unsupportive citations were those claiming that jellyfish were increasing globally (despite the fact that it would have been impossible to make any claims about global trends before the first global meta-analysis was published in 2012). And in all cases, selective citations were biased towards claims that jellyfish populations were increasing.

Pressure to publish in prestigious journals and win research funds may lead some scientists to make claims that reach beyond the evidence available. In most cases, however, citation errors are not overt attempts to distort the evidence. Rather, they probably arise because increasing academic workloads reduce the time available to evaluate papers accurately and to keep abreast of the almost exponential increase in the volume of literature being published.

As scientists, we need to ensure that our claims are always supported by robust evidence because it is apparent that poor citation practices – and, in particular, selective citation of the literature – can distort perceptions within a research field.

Kylie Pitt, Associate Professor, Griffith University; Carlos Duarte, Adjunct professor, King Abdullah University of Science and Technology; Cathy Lucas, Associate Professor, Marine Biology & Ecology Research Group (MBERG), University of Southampton; Charles Novaes de Santana, Postdoctoral research associate, University of Zurich; Marina Sanz-Martín, Researcher, Department of Global Change Research, University of Barcelona, and Rob Condon, Assistant Professor in Biological Oceanography, University of North Carolina Wilmington.

This article first appeared on The Conversation.

We welcome your comments at letters@scroll.in.
Sponsored Content BY 

The ordeal of choosing the right data pack for your connectivity needs

"Your data has been activated." <10 seconds later> "You have crossed your data limit."

The internet is an amazing space where you can watch a donkey playing football while simultaneously looking up whether the mole on your elbow is a symptom of a terminal diseases. It’s as busy as it’s big with at least 2.96 billion pages in the indexed web and over 40,000 Google search queries processed every second. If you have access to this vast expanse of information through your mobile, then you’re probably on something known as a data plan.

However, data plans or data packs are a lot like prescription pills. You need to go through a barrage of perplexing words to understand what they really do. Not to mention the call from the telecom company rattling on at 400 words per minute about a life-changing data pack which is as undecipherable as reading a doctor’s handwriting on the prescription. On top of it all, most data packs expect you to solve complex algorithms on permutations to figure out which one is the right one.

Source: giphy.com
Source: giphy.com

Even the most sophisticated and evolved beings of the digital era would agree that choosing a data pack is a lot like getting stuck on a seesaw, struggling to find the right balance between getting the most out of your data and not paying for more than you need. Running out of data is frustrating, but losing the data that you paid for but couldn’t use during a busy month is outright infuriating. Shouldn’t your unused data be rolled over to the next month?

You peruse the advice available online on how to go about choosing the right data pack, most of which talks about understanding your own data usage. Armed with wisdom, you escape to your mind palace, Sherlock style, and review your access to Wifi zones, the size of the websites you regularly visit, the number of emails you send and receive, even the number of cat videos you watch. You somehow manage to figure out your daily usage which you multiply by 30 and there it is. All you need to do now is find the appropriate data pack.

Promptly ignoring the above calculations, you fall for unlimited data plans with an “all you can eat” buffet style data offering. You immediately text a code to the telecom company to activate this portal to unlimited video calls, selfies, instastories, snapchats – sky is the limit. You tell all your friends and colleagues about the genius new plan you have and how you’ve been watching funny sloth videos on YouTube all day, well, because you CAN!

Source: giphy.com
Source: giphy.com

Alas, after a day of reign, you realise that your phone has run out of data. Anyone who has suffered the terms and conditions of unlimited data packs knows the importance of reading the fine print before committing yourself to one. Some plans place limits on video quality to 480p on mobile phones, some limit the speed after reaching a mark mentioned in the fine print. Is it too much to ask for a plan that lets us binge on our favourite shows on Amazon Prime, unconditionally?

You find yourself stuck in an endless loop of estimating your data usage, figuring out how you crossed your data limit and arguing with customer care about your sky-high phone bill. Exasperated, you somehow muster up the strength to do it all over again and decide to browse for more data packs. Regrettably, the website wont load on your mobile because of expired data.

Source: giphy.com
Source: giphy.com

Getting the right data plan shouldn’t be this complicated a decision. Instead of getting confused by the numerous offers, focus on your usage and guide yourself out of the maze by having a clear idea of what you want. And if all you want is to enjoy unlimited calls with friends and uninterrupted Snapchat, then you know exactly what to look for in a plan.

Source: giphy.com
Source: giphy.com

The Airtel Postpaid at Rs. 499 comes closest to a plan that is up front with its offerings, making it easy to choose exactly what you need. One of the best-selling Airtel Postpaid plans, the Rs. 499 pack offers 40 GB 3G/4G data that you can carry forward to the next bill cycle if unused. The pack also offers a one year subscription to Amazon Prime on the Airtel TV app.

So, next time, don’t let your frustration get the better of you. Click here to find a plan that’s right for you.

Source: giphy.com
Source: giphy.com

This article was produced by the Scroll marketing team on behalf of Airtel and not by the Scroll editorial team.