Neuroscientists often make groundless boasts of having discovered things in brains that are not actually there. What is going on is pareidolia, people reporting seeing something that is not there, after wishfully analyzing large amounts of ambiguous and hazy data. It's like someone eagerly analyzing his toast every day for years, looking for something that looks like the face of Jesus, and eventually reporting he saw something that looked to him like the face of Jesus. It's also like someone eager scanning the clouds, looking for animal shapes, while trying to confirm his belief that after an animal dies, his ghost goes up and lives in the clouds.
On this blog I have provided very many examples of neuroscientist pareidolia. Some examples can be found in my posts here:
Neuroscientists Claim "Drifting Representations," But It's Mainly Just Their Pareidolia
Pareidolia Progress: Neuroscientists Get Better and Better at Seeing Things in the Brain That Are Not There
Pareidolia Helps Neuroscientists Getting Nowhere Trying to Show a Brain Basis for Memory
Some Brain Wave Analysts Are Like "Face of Jesus in My Toast" Claimants
Normally when neuroscientist pareidolia occurs it is relatively harmless. There is the epistemic harm and intellectual harm caused when bad evidence is provided of things that are not true, but little other harm. However, one type of neuroscientist pareidolia can cause bad psychological harm in the relatives of those who are in a coma. I speak of the case of neuroscientists making shoddy claims of having discovered consciousness in comatose patients.
We read some examples in a Harvard Medicine article entitled "The Covert Consciousness Dilemma." The article is written by someone falling "hook, line and sinker" for some very dubious claims that were never well-supported. Early in the article the writer states this:
"Around the same time, Edlow heard about a paper published in Science that stunned him. The 2006 study focused on a patient who sustained extensive brain injuries following a traffic accident. Six months later she was still in a vegetative state. She showed no evidence of purposeful behaviors, such as following commands or making intentional movements. But then the researchers put her in an fMRI machine. While scanning her brain, they asked her to imagine walking through her house, moving slowly from room to room. And they told her to imagine playing a game of tennis, swinging for the ball with forehand and backhand strokes... Her brain responded to the commands with activity patterns that matched those of a healthy person, as if to say, I’m still here."
The 2006 study referred is the very low-quality paper "Detecting Awareness in the Vegetative State," which fails to provide any decent evidence justifying its title. The paper provides no robust evidence that any consciousness was detected in the single subject it studied. The paper claims that "Speech-specific activity was observed bilaterally in the middle and
superior temporal gyri,
equivalent to that observed in healthy volunteers listening to the
same stimuli (fig. S1)." The Figure S1 referred to is one of those extremely misleading visuals in which neuroscientists artificially color-highlight particular regions of the brain that showed only tiny differences such as 1 part in 200, to create the false impression that there was some big difference in brain activity in some region which merely had a difference such as 1 part in 200.
Here is the Figure1 of the paper
This visual is not a visual produced directly from any brain scan. The visual was produced through a process of artificial construction designed to make tiny differences such as 1 part in 200 look like big differences. Such methods are misleading. A correct visual showing the brain activity would show differences that cannot even be recognized by a human.
The authors of the paper fail to caption their Figure 1 correctly. They say that it shows what occurred during "tennis imagery in the patient," even though they have no evidence that any tennis imagery was occurring in the mind or brain of their comatose patient when this scan was taken. Looking closely at Figure 1 above, we fail to even see any close match. The highlighted regions in the first row look significantly different from the highlighted regions in the second row. No good evidence has been provided that any such thing as "tennis imagery" or consciousness was occurring in the patient.
There is actually no evidence in the literature of neuroscience that there exists any such thing as a neural correlate of imagination. If you scan someone's brain while asking him to keep a blank state of mind, and compare the scans to scans taken while the person was asked to imagine something, you will see no difference if the person kept perfectly still. There will be variations here and there such as 1 part in 200, but such variations can be explained as mere random variations, rather than something that resulted from an act of imagination. Muscle movements do produce blips in EEG readings and fMRI scans, but such blips are not evidence of thinking.
The paper "Neural substrates of envisioning the future" describes a study looking for neural correlates of imagination. 21 subjects were brain scanned while they did three tasks: remembering an event in their past (such as a birthday in the past), imagining an event in their future (such as a future birthday they would have), or imagining an event in someone else future. Figure 1 shows unimpressive results, with percent signal changes no greater than about 1 part in 200, which are the kind of differences one would expect to get from random fluctuations, even if people don't use their brains for imagining or remembering. We read of a "statistically indistinguishable pattern of activity across time while subjects envisioned their personal future (SF) and recollected the past (SR) in response to a series of event cues (e.g., Birthday)." The results when people were "imagining a future individual in similar scenarios" also fail to show any clear sign of neural involvement, with percent signal changes no greater than about 1 part in 200.
The low-quality paper "Detecting Awareness in the Vegetative State" fails to provide any decent evidence for consciousness in the comatose. To have decent evidence for such a thing, you would need a much larger sample size than the paper's way-too-small sample size of a single subject. The authors are guilty of runaway pareidolia when they claim this about their comatose patient "Moreover, her decision to
cooperate with the authors by imagining particular tasks when asked to do so represents a clear
act of intention, which confirmed beyond any
doubt that she was consciously aware of herself
and her surroundings." They have provided no decent evidence of either imagination, intention, cooperation or consciousness in the comatose patient they brain scanned. The authors engage in the old fallacy of claiming that "significant activity was observed" in particular brain regions when the subject was asked to imagine something. All brain regions are continually active in any living patient with a heartbeat, so showing that "significant activity was observed" in particular brain regions does nothing to show consciousness, intention or imagination.
We then have in the Harvard Medicine article an emotional quote from neurologist Brain Edlow saying that this low-quality paper "just blew my mind," followed by the groundless claim that this paper "completely reframed the way that we think about consciousness and opened up the possibility that a substantial number of patients may be conscious even if we cannot detect purposeful behavior at the bedside." Once again, we have a brain expert getting all super-excited over the flimsiest of evidence.
The Harvard Medical article then refers us to a 2024 paper with a larger size, the paper "Cognitive Motor Dissociation in Disorders of Consciousness." The paper is one of those deals in which you have a claim of a fairly large sample size, but the claim is made by adding up the study group sizes in a bunch of papers, which all used very small samples.
The two sentence abstract of the paper starts out by making unfounded claims, saying this:
"Patients with brain injury who are unresponsive to commands may perform cognitive tasks that are detected on functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). This phenomenon, known as cognitive motor dissociation, has not been systematically studied in a large cohort of persons with disorders of consciousness."
No such phenomenon has ever been established by high-quality studies, and this paper is not any such high-quality study. The "detection" it refers to are very likely example of pareidolia or errors-to-be-expected, in which over-eager researchers see things that are really there, because of a biased interpretation of borderline, varying data in which an over-eager analyst could see 101 things that are not really there.
The authors claim, "We detected cognitive motor dissociation in 60 of the 241 participants (25%) without an observable response to commands, of whom 11 had been assessed with the use of fMRI only, 13 with the use of EEG only, and 36 with the use of both techniques." The visual below shows the type of devices used.
The paper incorrectly states, "Cognitive motor dissociation is an established phenomenon in which persons with severe brain injury who are behaviorally unresponsive to commands show brain activation on functional magnetic resonance imaging (fMRI) or electroencephalography (EEG) when presented with cognitive tasks, such as motor imagery commands." No robust evidence has ever shown the existence of such a "cognitive motor dissociation" phenomenon.
Supplementary Figure S4 (in the Supplementary Information of the paper) gives us an indication of how weak the evidence of this paper is. The figure refers to a CRS-R scale that is a measure of how deeply someone is in a coma. According to an AI overview, "A total score of ≥ 8 generally indicates a transition from a vegetative state to a minimally conscious state, while a score of ≥ 10 helps identify patients who have achieved emergence from a minimally conscious state." The Figure S4 graph below shows no robust evidence of any responses from any patients with a score of less than 11 on this CRS-R score. Below is the graph:
The left half of the graph plots responses in those being in stronger and deeper comas. The right half of the graph shows responses in those who are not fully comatose. The claimed evidence for responses (as detected by fMRI or EEG activity) is found in the bar sections that are light blue or pink.
But as you can see by closely studying this graph, there is almost no pink or light blue color in any of the bars on the left half of the graph. Any of the very small pink areas we see in the left half of the graph above can easily be explained as being mere pareidolia or garden-variety analysis error from over-eager researchers eager to report "responses" after analyzing data that did not actually show any good evidence of a response.
So we see the "secret sauce" behind this paper "Cognitive Motor Dissociation in Disorders of Consciousness." It may have some got some evidence of responses from the subjects that underwent scanning by fMRI scanning or EEG scanning. But almost all of those claimed responses come from those not fully comatose (those scoring higher than 10 on the CRS-R scale). No strong evidence has been produced of responses or cognition or imagination or will in any of those who are fully comatose (the people graphed on the left half of the graph above).
Normally the misleading language and "see what you are hoping to see" pareidolia of brain experts is fairly harmless. But in this case such pareidolia is extremely harmful. The people doing low-quality studies like this are causing the relatives of those in a coma to needlessly fear that their relatives are enduring what many would regard as a fate worse than death: a fate of being conscious in a body when you cannot open your eyes and cannot move your muscles. There is no robust evidence that anyone in a coma is suffering such a fate. "Eager for another paper publication" brain experts are causing needless mental anguish in the relatives of those in comas, by groundlessly causing them to fear their relatives are undergoing some horrible fate, when there is no sound evidence basis for such fears. When studying such papers, we should always remember that brain experts studying the ever-varying data from fMRI scans and EEG readings have a long history of frequently claiming to see things that are not really there, and a long history of conjuring up phantasms that don't really exist.
You cannot actually tell whether someone is imagining something by looking at EEG readings taken from such a person's brain or by looking at fMRI scans. Studies that claim to provide evidence for such a thing are often studies that improperly leverage EEG traces of muscle movements. If you give someone an instruction such as "imagine yourself playing tennis," a person might well make subtle movements matching such thoughts, movements that show up in EEG readings. Evidence of muscle movements are not evidence of imagination. No one disputes that muscle movements produce EEG blips.
A look at one of the papers referenced by the "Cognitive Motor Dissociation in Disorders of Consciousness" paper shows the kind of arbitrary and convoluted statistical rigamarole that was going on to gin up some of these claims of "responses" in comatose patients being spoken to: "We used two complementary methods to determine the significance of differences in the frequency content of the EEG signal between the task and rest conditions: a univariate (frequency-by-frequency) approach and a multivariate approach. For each subject, both analyses were applied on a channel-by-channel basis to each run individually and to all runs combined.
For the univariate approach, we used a z-statistic, the Two Group Test (TGT) (Bokil et al., 2007), as implemented by the Chronux toolbox routine, two_group_test_spectrum (http://chronux.org), with a cutoff of p≤0.05 by jackknife method. Because spectral estimates within 2 Hz of each other are correlated by the taper functions, a difference identified by the TGT was only considered significant if it was present for all frequencies contiguously over a range greater than 2 Hz. This implies significance over at least two neighboring but non-overlapping windows of the multi-taper estimate and is indicated in figures with a rectangle drawn around the results (Figure 3B). Spectral differences over ranges narrower than 2 Hz represent only a trend to significance. To compensate for multiple comparisons (60 frequencies per channel in 29 or 37 channels), the False Discovery Rate (FDR) (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001) was applied to the TGT p-values determined from analyses of all runs combined.
To look for spectral differences that might only be apparent if combinations of frequencies are considered, we employed a multivariate approach, Fisher’s linear discriminant (FLD) (Fisher, 1936). This approach has been used successfully for classification of EEG responses to motor imagery (Hung et al., 2005; Bai et al., 2007). To limit dimensionality, we binned the log spectra from 4 to 24 Hz into 2 Hz windows, reducing the spectrum to 10 values. The FLD was then defined as the linear combination of these quantities that maximized the ratio of the power variance between the conditions to the power variance within the conditions. To determine the significance of the FLD, we used a shuffle method: we recomputed the FLD from 1000 shuffles of the two conditions, and determined the p-value as the fraction of shuffled datasets that yielded an equal or larger value to the actual FLD. To take into account the possibility that neighboring snippets had similar spectra because of a slowly changing underlying brain state (rather than the task) (Menzer et al., 2010), the shuffled datasets kept the snippets from the nine-second-response period after each command together during all shuffles. To control for multiple comparisons (since the FLD was applied separately to each channel), the FLD p-value was only considered significant for a channel if it was less than an FDR-corrected rate of 0.05. This is shown as an asterisk on the summary figures (e.g. channel Oz in Figure 3B). For each subject, this analysis was applied to each run individually, and to all runs combined."
Of course, when researchers are free to use "keep torturing the data until it confesses" methods like these (methods as arbitrary, convoluted and opaque as the researchers may wish), it is hardly a surprise that there might be conjured up here and there a little so-called "response evidence," even when data from the brains of utterly unconscious coma patients is being analyzed.
So many ways to conjure up phantasms that don't exist
No comments:
Post a Comment