Unreliable research

How scientists pressed for time inadvertently made it seem like there was a global jellyfish boom

Flawed citation practices can perpetuate scientific ideas even before they've been fully established as true.

When is a jellyfish plague not (necessarily) a jellyfish plague? When time-poor scientists selectively cite the literature to make it look like the oceans are flooded with jellies – even when it’s far from clear that they really are.

What does scientists being in a rush have to do with jellyfish populations? Let’s start from the beginning.

The identification of patterns and trends in nature happens through the accumulation of consistent observations, published in scientific reports. Once observed, the emerging patterns are usually reported in narrative reviews, which often stimulates a flurry of research activity in that field.

Eventually, the purported patterns are formally tested using “meta-analyses” of the published literature, to either confirm the pattern and establish it as theory, or refute it.

This path from the primary observations to theory can be traced through a network of citations.

Science, however, is done by humans and citation practices are subject to errors of bias and accuracy. Citation practices that are biased in a particular direction have the potential to lead to the identification of false patterns and flawed theory.

Enter the jellies

In the 1990s and 2000s, reports began to appear in the scientific literature of increased jellyfish populations in some parts of the world’s oceans. Various reviews reported the possibility that jellyfish blooms might be increasing globally. Over time, these became increasingly assertive about the existence and extent of the trend, until researchers were asking what to do about the increasingly “gelatinous state” of the oceans worldwide.

The question of whether the global jellyfish boom was real or not was tested by two meta-analyses – which came to opposite conclusions. A 2012 study concluded that populations were increasing globally because they found evidence for increasing populations in 62% of large marine ecosystems tested (although low certainty was assigned to two-thirds of these). The following year, another study found that only 30% of populations were increasing. It concluded that jellyfish populations wax and wane over several decades.

So, in reality, the scientific community is still divided over whether there really has been a sustained global increase in jellyfish numbers.

What about perception?

We wanted to know whether the perception of a global increase in jellyfish blooms was at least partly due to poor citation practices in the scientific literature. Our research, published in Global Ecology and Biogeography, suggests that it was.

Citation practices can be flawed in several ways:

  • Unsupported citations are when authors cite sources that contain no evidence that could possibly support the author’s claim.
  • Selective citations happen when a paper is cited to support a claim but contrasting evidence provided in the same paper is ignored, or when authors choose to cite earlier papers that have since been refuted.
  • Ambiguous citations happen when an author’s sentence contains multiple phrases, but the citations used to support each phrase are clustered at the end of the sentence, preventing readers from telling which is which.
  • Empty citations are when authors cite a paper that cites another paper as evidence for the claim, rather than the original source (also called “lazy author syndrome”).

We comprehensively searched the literature for papers, published before the two meta-analyses, that issued statements regarding trends in jellyfish populations. We classified each statement according to its affirmation and direction (that is, whether it said jellyfish are “increasing”, “may be increasing”, “decreasing”, or “not sure”), as well as their geographic extent (global, multiple regions, or one region).

We then assessed the papers cited as evidence of the statement, to see whether the citations were accurate or whether they fell into one of the categories of flawed citations outlined above.

A (jelly)fishy tale?

Of 159 papers that had issued statements about trends in jellyfish, 61% claimed that populations were increasing (27% at the global scale and 34% in multiple regions) and 25% asserted that populations may be increasing. Only 10% of papers said the data was equivocal. Just one reported that populations were decreasing (but at a local scale).

Most concerning was that only 51% of papers cited were considered to provide unambiguous support for the statements made by the authors. Almost all statements based on unsupportive citations were those claiming that jellyfish were increasing globally (despite the fact that it would have been impossible to make any claims about global trends before the first global meta-analysis was published in 2012). And in all cases, selective citations were biased towards claims that jellyfish populations were increasing.

Pressure to publish in prestigious journals and win research funds may lead some scientists to make claims that reach beyond the evidence available. In most cases, however, citation errors are not overt attempts to distort the evidence. Rather, they probably arise because increasing academic workloads reduce the time available to evaluate papers accurately and to keep abreast of the almost exponential increase in the volume of literature being published.

As scientists, we need to ensure that our claims are always supported by robust evidence because it is apparent that poor citation practices – and, in particular, selective citation of the literature – can distort perceptions within a research field.

Kylie Pitt, Associate Professor, Griffith University; Carlos Duarte, Adjunct professor, King Abdullah University of Science and Technology; Cathy Lucas, Associate Professor, Marine Biology & Ecology Research Group (MBERG), University of Southampton; Charles Novaes de Santana, Postdoctoral research associate, University of Zurich; Marina Sanz-Martín, Researcher, Department of Global Change Research, University of Barcelona, and Rob Condon, Assistant Professor in Biological Oceanography, University of North Carolina Wilmington.

This article first appeared on The Conversation.

We welcome your comments at letters@scroll.in.
Sponsored Content BY 

How sustainable farming practices can secure India's food for the future

India is home to 15% of the world’s undernourished population.

Food security is a pressing problem in India and in the world. According to the Food and Agriculture Organization of the UN (FAO), it is estimated that over 190 million people go hungry every day in the country.

Evidence for India’s food challenge can be found in the fact that the yield per hectare of rice, one of India’s principal crops, is 2177 kgs per hectare, lagging behind countries such as China and Brazil that have yield rates of 4263 kgs/hectare and 3265 kgs/hectare respectively. The cereal yield per hectare in the country is also 2,981 kgs per hectare, lagging far behind countries such as China, Japan and the US.

The slow growth of agricultural production in India can be attributed to an inefficient rural transport system, lack of awareness about the treatment of crops, limited access to modern farming technology and the shrinking agricultural land due to urbanization. Add to that, an irregular monsoon and the fact that 63% of agricultural land is dependent on rainfall further increase the difficulties we face.

Despite these odds, there is huge potential for India to increase its agricultural productivity to meet the food requirements of its growing population.

The good news is that experience in India and other countries shows that the adoption of sustainable farming practices can increase both productivity and reduce ecological harm.

Sustainable agriculture techniques enable higher resource efficiency – they help produce greater agricultural output while using lesser land, water and energy, ensuring profitability for the farmer. These essentially include methods that, among other things, protect and enhance the crops and the soil, improve water absorption and use efficient seed treatments. While Indian farmers have traditionally followed these principles, new technology now makes them more effective.

For example, for soil enhancement, certified biodegradable mulch films are now available. A mulch film is a layer of protective material applied to soil to conserve moisture and fertility. Most mulch films used in agriculture today are made of polyethylene (PE), which has the unwanted overhead of disposal. It is a labour intensive and time-consuming process to remove the PE mulch film after usage. If not done, it affects soil quality and hence, crop yield. An independently certified biodegradable mulch film, on the other hand, is directly absorbed by the microorganisms in the soil. It conserves the soil properties, eliminates soil contamination, and saves the labor cost that comes with PE mulch films.

The other perpetual challenge for India’s farms is the availability of water. Many food crops like rice and sugarcane have a high-water requirement. In a country like India, where majority of the agricultural land is rain-fed, low rainfall years can wreak havoc for crops and cause a slew of other problems - a surge in crop prices and a reduction in access to essential food items. Again, Indian farmers have long experience in water conservation that can now be enhanced through technology.

Seeds can now be treated with enhancements that help them improve their root systems. This leads to more efficient water absorption.

In addition to soil and water management, the third big factor, better seed treatment, can also significantly improve crop health and boost productivity. These solutions include application of fungicides and insecticides that protect the seed from unwanted fungi and parasites that can damage crops or hinder growth, and increase productivity.

While sustainable agriculture through soil, water and seed management can increase crop yields, an efficient warehousing and distribution system is also necessary to ensure that the output reaches the consumers. According to a study by CIPHET, Indian government’s harvest-research body, up to 67 million tons of food get wasted every year — a quantity equivalent to that consumed by the entire state of Bihar in a year. Perishables, such as fruits and vegetables, end up rotting in store houses or during transportation due to pests, erratic weather and the lack of modern storage facilities. In fact, simply bringing down food wastage and increasing the efficiency in distribution alone can significantly help improve food security. Innovations such as special tarpaulins, that keep perishables cool during transit, and more efficient insulation solutions can reduce rotting and reduce energy usage in cold storage.

Thus, all three aspects — production, storage, and distribution — need to be optimized if India is to feed its ever-growing population.

One company working to drive increased sustainability down the entire agriculture value chain is BASF. For example, the company offers cutting edge seed treatments that protect crops from disease and provide plant health benefits such as enhanced vitality and better tolerance for stress and cold. In addition, BASF has developed a biodegradable mulch film from its ecovio® bioplastic that is certified compostable – meaning farmers can reap the benefits of better soil without risk of contamination or increased labor costs. These and more of the company’s innovations are helping farmers in India achieve higher and more sustainable yields.

Of course, products are only one part of the solution. The company also recognizes the importance of training farmers in sustainable farming practices and in the safe use of its products. To this end, BASF engaged in a widespread farmer outreach program called Samruddhi from 2007 to 2014. Their ‘Suraksha Hamesha’ (safety always) program reached over 23,000 farmers and 4,000 spray men across India in 2016 alone. In addition to training, the company also offers a ‘Sanrakshan® Kit’ to farmers that includes personal protection tools and equipment. All these efforts serve to spread awareness about the sustainable and responsible use of crop protection products – ensuring that farmers stay safe while producing good quality food.

Interested in learning more about BASF’s work in sustainable agriculture? See here.

This article was produced by the Scroll marketing team on behalf of BASF and not by the Scroll editorial team.