The go-to software that scientists have been using to study brain activity is flawed and could call into question at least 3,500 of 40,000 studies conducted in the last decade. According to a paper published in mid-June in the Proceedings of the National Academy of Sciences that made headlines around the world, Swedish neuroscientists found that the software used in functional Magnetic Resonance Imaging, or fMRI, threw up false positive rates of up to 70%. This means, that there is a 70% chance that a positive result from an fMRI study was wrong.
What is fMRI and why is this important?
While a regular MRI shows us the structure of a brain and whether there are abnormal growths or bleeds in it, an fMRI measures brain activity. Like in a regular MRI, the subject of an fMRI study is put in a machine through which a strong magnetic field pulsates. This then measures the tiny changes in blood flow through the brain.
Neuroscientists measure brain activity on the basis that when a certain part of the brain is activated it receives more blood flow, which influences the spins of molecules in the brain, which in turn is captured as an electromagnetic signal in an fMRI.
Now, this technique has been used on the last 25 years to measure brain activity for everything from regular movement to the experience of pain, from how much marijuana dulls neural activity to how dogs react when they recognise human faces. Here’s a look at actor and science communicator Alan Alda take an fMRI-based lie detector test to show how it works.
Here’s the problem. During an analysis of an fMRI the images of the brain are divided into box-like regions called voxels and each voxel is scanned for activity by software. So, the software is interpreting what it sees in the images.
The authors of the new paper looked at resting-state fMRI data of 499 healthy people, gathered from data databases from around the world and found that the software used in fMRI analysis are flawed and generate false positives. They might even indicate that there is brain activity where there is none. The average false positive rate from the software was 5% but could be as much as 70% in some cases, the Swedish scientists found.
At least 40,000 research papers have been published in the last 25 years of fMRI research and the new development could call the validity of many of these studies into question. But as Discover Magazine blogger Neuroskeptic points out a 70% chance of finding at least one false positive does not mean that 70% of positive results are false. It is also possible that much of the research holds good and only a minority of positives will be false.
The stormy reaction to the Proceedings of the National Academy of Sciences paper, with news reports talking about flaws and glitches while neuroscientists defended their work, forced one of the authors Thomas Nichols to clarify that the deceptive statistics generating false positive results were likely to be found in only 3,500 studies and didn't call into question all 40,000 fMRI studies.
While it is difficult to figure out which of the studies threw up false positive results and even tougher to redo the ones that might be wrong, the authors of the paper have asked that scientists using fMRI techniques validate their findings and make sure that they are able to reproduce them.