Unreliable research

How scientists pressed for time inadvertently made it seem like there was a global jellyfish boom

Flawed citation practices can perpetuate scientific ideas even before they've been fully established as true.

When is a jellyfish plague not (necessarily) a jellyfish plague? When time-poor scientists selectively cite the literature to make it look like the oceans are flooded with jellies – even when it’s far from clear that they really are.

What does scientists being in a rush have to do with jellyfish populations? Let’s start from the beginning.

The identification of patterns and trends in nature happens through the accumulation of consistent observations, published in scientific reports. Once observed, the emerging patterns are usually reported in narrative reviews, which often stimulates a flurry of research activity in that field.

Eventually, the purported patterns are formally tested using “meta-analyses” of the published literature, to either confirm the pattern and establish it as theory, or refute it.

This path from the primary observations to theory can be traced through a network of citations.

Science, however, is done by humans and citation practices are subject to errors of bias and accuracy. Citation practices that are biased in a particular direction have the potential to lead to the identification of false patterns and flawed theory.

Enter the jellies

In the 1990s and 2000s, reports began to appear in the scientific literature of increased jellyfish populations in some parts of the world’s oceans. Various reviews reported the possibility that jellyfish blooms might be increasing globally. Over time, these became increasingly assertive about the existence and extent of the trend, until researchers were asking what to do about the increasingly “gelatinous state” of the oceans worldwide.

The question of whether the global jellyfish boom was real or not was tested by two meta-analyses – which came to opposite conclusions. A 2012 study concluded that populations were increasing globally because they found evidence for increasing populations in 62% of large marine ecosystems tested (although low certainty was assigned to two-thirds of these). The following year, another study found that only 30% of populations were increasing. It concluded that jellyfish populations wax and wane over several decades.

So, in reality, the scientific community is still divided over whether there really has been a sustained global increase in jellyfish numbers.

What about perception?

We wanted to know whether the perception of a global increase in jellyfish blooms was at least partly due to poor citation practices in the scientific literature. Our research, published in Global Ecology and Biogeography, suggests that it was.

Citation practices can be flawed in several ways:

  • Unsupported citations are when authors cite sources that contain no evidence that could possibly support the author’s claim.
  • Selective citations happen when a paper is cited to support a claim but contrasting evidence provided in the same paper is ignored, or when authors choose to cite earlier papers that have since been refuted.
  • Ambiguous citations happen when an author’s sentence contains multiple phrases, but the citations used to support each phrase are clustered at the end of the sentence, preventing readers from telling which is which.
  • Empty citations are when authors cite a paper that cites another paper as evidence for the claim, rather than the original source (also called “lazy author syndrome”).

We comprehensively searched the literature for papers, published before the two meta-analyses, that issued statements regarding trends in jellyfish populations. We classified each statement according to its affirmation and direction (that is, whether it said jellyfish are “increasing”, “may be increasing”, “decreasing”, or “not sure”), as well as their geographic extent (global, multiple regions, or one region).

We then assessed the papers cited as evidence of the statement, to see whether the citations were accurate or whether they fell into one of the categories of flawed citations outlined above.

A (jelly)fishy tale?

Of 159 papers that had issued statements about trends in jellyfish, 61% claimed that populations were increasing (27% at the global scale and 34% in multiple regions) and 25% asserted that populations may be increasing. Only 10% of papers said the data was equivocal. Just one reported that populations were decreasing (but at a local scale).

Most concerning was that only 51% of papers cited were considered to provide unambiguous support for the statements made by the authors. Almost all statements based on unsupportive citations were those claiming that jellyfish were increasing globally (despite the fact that it would have been impossible to make any claims about global trends before the first global meta-analysis was published in 2012). And in all cases, selective citations were biased towards claims that jellyfish populations were increasing.

Pressure to publish in prestigious journals and win research funds may lead some scientists to make claims that reach beyond the evidence available. In most cases, however, citation errors are not overt attempts to distort the evidence. Rather, they probably arise because increasing academic workloads reduce the time available to evaluate papers accurately and to keep abreast of the almost exponential increase in the volume of literature being published.

As scientists, we need to ensure that our claims are always supported by robust evidence because it is apparent that poor citation practices – and, in particular, selective citation of the literature – can distort perceptions within a research field.

Kylie Pitt, Associate Professor, Griffith University; Carlos Duarte, Adjunct professor, King Abdullah University of Science and Technology; Cathy Lucas, Associate Professor, Marine Biology & Ecology Research Group (MBERG), University of Southampton; Charles Novaes de Santana, Postdoctoral research associate, University of Zurich; Marina Sanz-Martín, Researcher, Department of Global Change Research, University of Barcelona, and Rob Condon, Assistant Professor in Biological Oceanography, University of North Carolina Wilmington.

This article first appeared on The Conversation.

We welcome your comments at letters@scroll.in.
Sponsored Content BY 

The perpetual millennial quest for self-expression just got another boost

Making adulting in the new millennium easier, one step at a time.

Having come of age in the Age of the Internet, millennials had a rocky start to self-expression. Indeed, the internet allowed us to personalise things in unprecedented fashion and we really rose to the occasion. The learning curve to a straightforward firstname.surname@___mail.com email address was a long one, routed through cringeworthy e-mail ids like coolgal1234@hotmail.com. You know you had one - making a personalised e-mail id was a rite of passage for millennials after all.

Declaring yourself to be cool, a star, a princess or a hunk boy was a given (for how else would the world know?!). Those with eclectic tastes (read: juvenile groupies) would flaunt their artistic preferences with an elitist flair. You could take for granted that bitbybeatlemania@hotmail.com and hpfan@yahoo.com would listen to Bollywood music or read Archie comics only in private. The emo kids, meanwhile, had to learn the hard way that employers probably don’t trust candidates with e-mail ids such as depressingdystopian@gmail.com.

Created using Imgflip
Created using Imgflip

And with chat rooms, early millennials had found a way to communicate, with...interesting results. The oldest crop of millennials (30+ year olds) learnt to deal with the realities of adolescent life hunched behind anonymous accounts, spewing their teenage hormone-laden angst, passion and idealism to other anonymous accounts. Skater_chick could hide her ineptitude for skating behind a convincing username and a skateboard-peddling red-haired avatar, and you could declare your fantasies of world domination, armed with the assurance that no one would take you seriously.

With the rise of blogging, millennial individualism found a way to express itself to millions of people across the world. The verbosity of ‘intellectual’ millennials even shone through in their blog URLs and names. GirlWhoTravels could now opine on her adventures on the road to those who actually cared about such things. The blogger behind scentofpetunia.blogspot.com could choose to totally ignore petunias and no one would question why. It’s a tradition still being staunchly upheld on Tumblr. You’re not really a Tumblr(er?) if you haven’t been inspired to test your creative limits while crafting your blog URL. Fantasy literature and anime fandoms to pop-culture fanatics and pizza lovers- it’s where people of all leanings go to let their alter ego thrive.

Created using Imgflip
Created using Imgflip

Then of course social media became the new front of self-expression on the Internet. Back when social media was too much of a millennial thing for anyone to meddle with, avatars and usernames were a window into your personality and fantasies. Suddenly, it was cool to post emo quotes of Meredith Grey on Facebook and update the world on the picturesque breakfast you had (or not). Twitter upped the pressure by limiting expression to 140 characters (now 280-have you heard?) and the brevity translated to the Twitter handles as well. The trend of sarcasm-and-wit-laden handles is still alive well and has only gotten more sophisticated with time. The blogging platform Medium makes the best of Twitter intellect in longform. It’s here that even businesses have cool account names!

Self-expression on the Internet and the millennials’ love for the personalised and customised has indeed seen an interesting trajectory. Most millennial adolescents of yore though are now grownups, navigating an adulting crisis of mammoth proportions. How to wake up in time for classes, how to keep the boss happy, how to keep from going broke every month, how to deal with the new F-word – Finances! Don’t judge, finances can be stressful at the beginning of a career. Forget investments, loans and debts, even matters of simple money transactions are riddled with scary terms like beneficiaries, NEFT, IMPS, RTGS and more. Then there’s the quadruple checking to make sure you input the correct card, IFSC or account number. If this wasn’t stressful enough, there’s the long wait while the cheque is cleared or the fund transfer is credited. Doesn’t it make you wish there was a simpler way to deal with it all? If life could just be like…

Created using Imgflip
Created using Imgflip

Lo and behold, millennial prayers have been heard! Airtel Payments Bank, India’s first, has now integrated UPI on its digital platform, making banking over the phone easier than ever. Airtel Payments Bank UPI, or Unified Payment Interface, allows you to transfer funds and shop and pay bills instantly to anyone any time without the hassles of inputting any bank details – all through a unique Virtual Payment Address. In true millennial fashion, you can even create your own personalised UPI ID or Virtual Payment Address (VPA) with your name or number- like rhea@airtel or 9990011122@airtel. It’s the smartest, easiest and coolest way to pay, frankly, because you’re going to be the first person to actually make instant, costless payments, rather than claiming to do that and making people wait for hours.

To make life even simpler, with the My Airtel app, you can make digital payments both online and offline (using the Scan and Pay feature that uses a UPI QR code). Imagine, no more running to the ATM at the last minute when you accidentally opt for COD or don’t have exact change to pay for a cab or coffee! Opening an account takes less than three minutes and remembering your VPA requires you to literally remember your own name. Get started with a more customised banking experience here.

This article was produced by the Scroll marketing team on behalf of Airtel Payments Bank and not by the Scroll editorial team.