A superficial reading of the history of vaccination might lead you to believe that it is simple. Dried smallpox pustules had been used for 1,000 years to inoculate people against smallpox before the first successful vaccine trial, conducted by Edward Jenner in 1796 on a single eight-year-old boy.
A more detailed reading, however, reveals two significant risks both extremely relevant in the current pandemic.
The first is that bad vaccines do not just fail to protect, they can cause direct harm to patients. Some make subsequent infection with the disease they are intended to protect against more serious.
The second risk is that trust in vaccines is easily damaged and slow to recover. People feel anxious about interventions given to the well – the first anti-vaccination movement appeared just a few years after Jenner’s successful trial.
The Russian government appeared to be taking both these risks when in August 2020, President Vladimir Putin, announced the registration of a new Covid-19 vaccine: Sputnik V.
Sputnik V
While Putin said that it had gone through “all the necessary trials”, the registration certificate said that it had been trialled on just 38 participants. The international responses ranged from concern to outrage and, since that announcement, everything about Sputnik has seemed worthy of detailed scrutiny, as I argued in a recent essay in The BMJ.
In September 2020, the first peer-reviewed Sputnik V data was published in the prestigious medical journal, The Lancet: two studies each of 38 people who all seemed to develop a robust immune response with no serious problems.
Very quickly inconsistencies were found in the paper by an Italian scientist, Enrico Bucci. He runs a research integrity company and posted an open letter noting that the results on several graphs seemed identical between participants – more than might be expected by chance. Bucci and several others (including myself) wrote to The Lancet requesting access to the data from which the figures were generated to resolve the issue.
We expected to have the backing of The Lancet with this request since they are enthusiastic about data transparency. Their website declares: “The Lancet journals will continue to hold authors and editors accountable for the data published in our pages, and we encourage our readers to do the same.” The Sputnik team responded that the patterns in the data were “coincidences” but confirmed that they would make individual participant data available on request.
Despite these assurances and several requests, neither The Lancet nor the Sputnik team have provided any further data.
We might expect The Lancet to be cautious when it comes to papers on Covid-19 or vaccines. In the summer of 2020, they published – and then retracted – a major Covid-19 study based on a flawed dataset. They made a similar mistake on a 1998 paper fraudulently linking the MMR vaccine to autism, which contributed to huge increases in measles rates around the world.
Despite this history and the errors in the previous paper, in February The Lancet published an interim report on a much larger study of thousands of people. This was accompanied by favourable editorials. One announced that: “Sputnik V Covid-19 vaccine candidate appears safe and effective” and added that “another vaccine can now join the fight”.
Once again, Bucci and other internationally regarded scientists identified many minor errors – surprising in a study of this importance. Data tables that did not add up and more anomalous graphs. Not evidence of fraud but a concerning lack of rigour on the part of both the Sputnik team and The Lancet.
These errors and the uncritically glowing editorials are particularly concerning given that Sputnik was developed at an institution in a country with no significant track record of vaccine development and at the time of publication of the phase 3 trial, Sputnik had not been submitted to a major regulator. The European Medicines Agency did not begin reviewing the Sputnik team’s application until a month after The Lancet publication.
Meanwhile, the publications in The Lancet have been used very effectively by the Sputnik V marketing team online, in every press release, and in several interviews. While 16 countries had authorised Sputnik V before the phase 3 trial publication, more than 40 have authorised it since, mainly low and middle-income countries without effective regulators. Understandably, they may have had to rely on The Lancet’s vetting of the science. But the peer-review process is not adequate to evaluate a new vaccine in the way that a regulator can.
Regulator’s scrutiny
At most journals, following inspection by the editorial team and a statistician, peer review is undertaken over a few hours, by a few anonymous, unpaid experts, without publicly declared interests and without access to underlying data.
By contrast, the major regulators (such as the European Union’s European Medicines Agency, the Food and Drug Administration in the United States and the United Kingdom’s Medicines and Healthcare products Regulatory Agency) typically use named teams of in-house and external experts, all with declared interests.
They work full time for many months with unlimited access to all the non-clinical, clinical and manufacturing data. They frequently inspect research and manufacturing sites. If they choose to exercise it, they have the power to look at individual patient notes to confirm data. A journal has no such power, which is why using the words “safe and effective” in the title of an editorial about an unauthorised drug is so unusual.
Finally, the regulatory output is far more transparent than peer review. The European Medicines Agency has published thousands of pages of data and analysis from the vaccines submitted to them.
The European Medicines Agency may declare Sputnik to have a favourable risk-benefit balance despite the errors in the published papers. If so, this will be a boost to global health. But this episode will still raise questions about The Lancet’s commitment to open data and to the wider claims they make about “applying scientific knowledge to improve health and advance human progress”.
If it is not authorised, then we will need to ask more serious questions about how many people have been harmed by a misguided faith in peer review, and how much damage has been caused to public confidence in the vaccines that are actually safe and effective.
Prior to publication, The BMJ provided The Lancet with a list of allegations contained in Dr Chris van Tulleken’s essay – which are the same allegations contained in this article. The BMJ received the following response from Emily Head, media relations manager:
This research was independently peer reviewed by international experts on Covid-19 and vaccines, including a statistical reviewer. At the Lancet journals, our editors treat communication with authors as confidential, and details of peer review including dates and peer review comments are not shared publicly.
All publicly available information for Lancet articles is published with the article, in the Supplementary Materials or Linked Articles sections on the article webpage. In addition, explanations of any errors that have been corrected within an article are provided in the Department of Error notice.
Our policies on peer review, data access, and corrections are available here: https://www.thelancet.com/publishing-excellence
Christoffer van Tulleken is an Honorary Associate Professor at the University College London.
This article first appeared on The Conversation.
