In Neuroscience Papers Bluffing Is More Common Than Candor

The Cornell Physics Paper Server at arxiv.org is mainly useful for finding papers on physics, but it also includes many papers on quantitative biology and computer science. Below are some observations I made after searching for papers with "memory" or "thought" in the title.

Occasionally a neuroscience paper will have a little candor regarding the vast gulf between the claims neuroscientists make and the low-level data they observe. One paper gives us some indications that what neuroscientists observe on a low level is something totally different from the stability we see in long-term memories.  We read the following, in which "turn over" refers to demise and replacement:

"The building blocks of the brain are in constant flux at the subcellular, cellular and circuit level. Synaptic and non-synaptic proteins are mobile [1] and rapidly turn over on the scale of hours to days [2]. Individual synapses continuously change their size and strength both in vitro and in vivo [3–5]. Most notably, however, the mature brain appears to continuously rewire itself, even without experimental intervention [6,7]. This is evident from the perpetual turnover of dendritic spines, small protrusions from the parent dendrites of most cortical neurons that are commonly used as proxies for excitatory synapses. Depending on the cell types and brain regions investigated, dendritic spines are gained and lost at rates ranging from approximately 1% per day in primary visual cortex [8] over approximately 5% per day in the CA1 region of hippocampus [9] to up to approximately 15% per day in primary somatosensory cortex [10] (but see [6,11,12] for potential pitfalls of these quantifications)." 

Another paper refers to it as a "fundamental enigma" that memories can last for even weeks (which is not surprising, given the facts above). Using the acronym LTM for long-term memory, the paper says, "A fundamental enigma is how the physical substrate for storage of LTM can nonetheless be preserved for weeks, months, or a lifetime."

The paper here suggests that there is no understanding of how a brain could ever translate episodic experience or learned knowledge into neural states. The paper errs only in using the term "largely" when it should have used the word "totally." We read this:

"Codifying memories is one of the fundamental problems of modern Neuroscience. The functional mechanisms behind this phenomenon remain largely unknown."

The paper "Long Term Memory: Scaling of Information to Brain Size" by Donald R. Forsdyke is a paper of unusual candor.  We read the following about patients whose brain regions consisted almost entirely of watery fluid rather than neurons:

"The journal Science, under the title 'Is your brain really necessary?' (Lewin 1980), described a series of 600 cases with residual ventricular enlargement that had been studied in Britain by paediatrician John Lorber (1915-1996). Again, while long-term memories were not directly assessed, intelligence quotients (IQs) were. Amazingly, in 60 of Lorber’s cases, ventricular fluid still occupied 95% of cranial capacity. Yet half of this group had IQs above average. Among these was a student with an IQ of 126 who had a first class honours degree in mathematics and was socially normal....The drastic reduction in brain mass in certain, clinically-normal, hydrocephalic cases, seems to demand unimaginable levels of redundancy and/or plasticity – superplasticity. How much brain must be absent before we abandon these explanations and look elsewhere?...Regarding the human brain’s 'massive storage capacity' for object details, Brady et al. (2008) have also challenged 'neural models of memory storage and retrieval.' ...The unconventional alternatives are that the repository is external to the nervous system, either elsewhere within the body, or extra-corporeal. The former is unlikely since the functions of other body organs are well understood. Remarkably, the latter has been on the table since at least the time of Avicenna and hypothetical mechanisms have been advanced (Talbot 1991; Berkovich 1993; Forsdyke 2009; Doerfler 2010). Its modern metaphor is 'cloud computing.' ” 

But such candor and willingness to challenge fossilized dogmas is rare. What is more common is for neuroscience papers to give us bluffing, in which an author pretends to have something he doesn't have, like a poker player with a weak hand acting as if he has a royal flush.  An example is the paper "Neural origins of self-generated thought: Insights from intracranial electrical stimulation and recordings in humans."  The paper would have us believe that it is presenting some evidence that brains produce thinking.

But when we look at the visuals, we see no substantial evidence for such a thing.  Figure 1 and Figure 2 gives us the usual deal in which some tiny difference in signal strength is shown in a very bright color such as bright red or bright blue.  But in Figure 3 we get some hard numbers. We have some graphs showing brain signal differences during thinking, and we can see from the visuals that the percent signal change was never more than a tenth of one percent, never more than 1 part in 1000.   Of course, such a tiny difference in signal strength is no robust evidence at all that brains are producing thought, but is merely the kind of difference we would expect from chance variations.

A recent example of a bluffing neuroscience paper is the 21-page paper "Memory Systems of the Brain," which seems to be bluffing us in the sense that it provides no compelling evidence for such systems.  We have no discussion of how a brain could translate learned knowledge or experiences into neural states or synapse states. We have no discussion of how a brain could store a memory for decades, or even for a single year. We have no discussion of how a brain could retrieve a memory. 

How does the paper manage to fill up 21 pages without any such things? The paper follows various space-filling strategies used by similar papers:

(1) An historical approach is taken in which pages are filled up with a discussion of the history of human thinking about memory. 

(2) Lots of space is used up with a discussion of different types of memory. For example, there is a discussion about the difference between short-term memory, working memory and long-term memory. 

(3) There is a discussion of a handful of cherry-picked case histories, carefully chosen to make us believe in neuroscientist dogmas about a brain storage of memories. 

There are innumerable case histories that could be quoted, but neuroscientists tend to spend excessive time citing the cases of patient H.M and patient K. C.  The author of the "Memory Systems of the Brain" paper repeats the incorrect claim so often made about patient H.M, a claim that he was unable to form new memories. The paper states that patient H.M. "became unable to consciously recollect new events in his life or new facts about the world."  This is not entirely correct. A 14-year follow-up study of patient H.M. (whose memory problems started in 1953) actually tells us that H.M. was able to form some new memories. The study says this on page 217:

"In February 1968, when shown the head on a Kennedy half-dollar, he said, correctly, that the person portrayed on the coin was President Kennedy. When asked him whether President Kennedy was dead or alive, and he answered, without hesitation, that Kennedy had been assassinated...In a similar way, he recalled various other public events, such as the death of Pope John (soon after the event), and recognized the name of one of the astronauts, but his performance in these respects was quite variable."

Another paper (""Evidence for Semantic Learning in Profound Amnesia: An Investigation With Patient H.M."tells us this about patient H.M.") states this:

"We used cued recall and forced-choice recognition tasks to investigate whether the patient H.M. had acquired knowledge of people who became famous after the onset of his amnesia. Results revealed that, with first names provided as cues, he was able to recall the corresponding famous last name for 12 of 35 postoperatively famous personalities. This number nearly doubled when semantic cues were added, suggesting that his knowledge of the names was not limited to perceptual information, but was incorporated in a semantic network capable of supporting explicit recall. In forced-choice recognition, H.M. discriminated 87% of postmorbid famous names from foils. Critically, he was able to provide uniquely identifying semantic facts for one-third of these recognized names, describing John Glenn, for example, as 'the first rocketeer' and Lee Harvey Oswald as a man who 'assassinated the president.' Although H.M.’s semantic learning was clearly impaired, the results provide robust, unambiguous evidence that some new semantic learning can be supported by structures beyond the hippocampus proper."

Patient K.C. was a patient who had extensive brain damage in a motorcycle accident, but could still remember learned information well. However, he was unable to provide autobiographical recollections of events before his injury.  But a study of a patient with a similar problem  (patient Y.K.) suggests the possibility that memory of experiences was not lost, but merely the ability to recall such information in the form of a first-person narrative. In one source we read the following:

"For example, one patient (Y.K.) was reported to have some knowledge of remote incidents in his life but was unable to 'remember' them (

Hirano and Noguchi, 1998

, 

Hirano et al., 2002

). Using the Remember and Know procedure (

Tulving, 1985

), Y.K. assigned K responses to all of his remote recollections, indicating that he had knowledge of the events as facts but could not actually place himself mentally at the scenes where the events occurred."

The "Memory Systems of the Brain" paper seems to hint that there is no understanding of how a brain could store a memory, when it states this: "It remains unclear how neuronal cooperativity in intact networks relates to memories or how network activity in the behaving animal brings about synaptic modification "  Before stating that, the paper makes this claim: "Clinical evidence indicates that damage to the hippocampus produces anterograde amnesia."  But while there are a few famous cases of patients with impaired recall of past experiences after hippocampus damage, there are vastly more cases of people who could recall previous memories fairly well after the total removal of the hippocampus. 

The "Memory Systems of the Brain" paper conveniently fails to mention the main research paper on the hippocampus and memory: the paper "Memory Outcome after Selective Amygdalohippocampectomy: A Study in 140 Patients with Temporal Lobe Epilepsy." That paper gives memory scores for 140 patients who almost all had the hippocampus removed to stop seizures.  Using the term "en bloc" which means "in its entirety" and the term "resected" which means "cut out," the paper states, "The hippocampus and the parahippocampal gyrus were usually resected en bloc."  The paper refers us to another paper  describing the surgeries, and that paper tells us that hippocampectomy (surgical removal of the hippocampus) was performed in almost all of the patients. 

The "Memory Outcome after Selective Amygdalohippocampectomy" paper does not use the word "amnesia" to describe the results. That paper gives memory scores that merely show only a modest decline in memory performance.  The paper states, "Nonverbal memory performance is slightly impaired preoperatively in both groups, with no apparent worsening attributable to surgery."  In fact, Table 3 of the paper informs us that a lack of any significant change in memory performance after removal of the hippocampus was far more common than a decline in memory performance, and that a substantial number of the patients improved their memory performance after their hippocampus was removed. 

In light of these results, it is objectionable for the "Memory Systems of the Brain" paper to have made this claim:  "Clinical evidence indicates that damage to the hippocampus produces anterograde amnesia."  The paper should merely have stated that there are a small number of famous cases of patients who had both hippocampus damage and anterograde amnesia, but that removal of the hippocampus generally does not produce either anterograde amnesia or even a very severe decline in memory performance. 

We should remember that nothing is proven by a few cases in which a small number of people had some brain damage and also a memory problem. We do not know in such cases whether there is a causal relation between the brain damage and the memory problem. If I scanned enough data in hospital records, I could surely find cases in which someone had a toothache before dying suddenly. But that would not at all prove that toothaches can produce sudden death. 

The "Memory Systems of the Brain" paper presents no good evidence that memories are stored in particular parts of the brain. But it does make this claim that it completely fails to back up with any evidence: "Memories are stored in the brain in a distributed pattern in the outer layer of the cortex, related to the area of the brain that initially processed them."  At the end of this statement, the paper makes a reference to another neuroscience paper, as if such a thing had been established by that paper.  The paper is the paper "Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory" by Larry R. Squire.  That paper fails to state any such claim that memories are stored in the outer layer of the cortex, and does not at all provide any substantial evidence to back up such a claim. 

The "Memory Systems of the Brain" paper does cite another paper co-authored by Squire, the paper "Structure and function of declarative and nondeclarative memory system." When I examine the paper in question, I find it does not actually make the claim that memories are stored in the outer layer of the cortex, and merely weakly says that the neocortex "is believed to be the permanent repository of memory."  The paper in question  does not establish any claim about a storage place of memory, and  merely mentions some small-effect experiments with monkeys that had damage to various regions of their cortex. None of the monkeys had any more than a small deficit in their memory performance after such damage.  For example, we are told in one case of cortical damage, performance declined from 79% correct to 67% correct, and in another such case  performance declined from 79% correct to 77% correct; and it is noted that cortex-damaged monkeys  "were unimpaired at learning and retaining single-object discriminations."  It is not very unlikely that you might get such results purely because of chance variations, particularly if you were using a small sample size less than 15. 

In this paper "Structure and function of declarative and nondeclarative memory systems" we learn that the authors are relying on absurdly underpowered cortex memory studies. For example, Figure 8 refers us to an experiment using only 5 monkeys with cortex lesions, which is way too few for a reliable experimental result. The minimum for a moderately reliable result is 15 subjects per study group.  

Below are some other examples of weak elements in the "Memory Systems of the Brain" paper:

(1) We are told on page 15 that neuroimaging shows that certain regions of the brain show "common activity" when memories are formed. This is irrelevant, because all regions of the brain are active during normal consciousness.

(2) We are told on page 18 that a paper showed "increased activity in the amygdala" for those who learned better. The paper in question actually only showed that those with higher levels of stress hormones in the amygdala tended to remember more.  But that does nothing to show that the amygdala stores memories, but merely shows we remember better when emotionally aroused. You could do a similar test showing that people remember more what they experience when their heart rate is 130 beats per minute rather than a normal rate of only 65 beats per minute.  But that would do nothing to show that memories are stored in the heart. 

(3) On page 14 we are told, "Imaging studies have also illuminated the contributions of distinct prefrontal regions to encoding and retrieval."  At the end of this statement there is a reference to three papers. The first of these papers used only 6 subjects per experiment, way too small a sample size to be a reliable result (15 subjects per study group has been suggested as a minimum for reliable results, and Kelly Zalocusky PhD hints that 31 subjects per study group may be needed). The second of these papers suffered from the defect of judging strength of memory based on subjective "confidence levels" rather than objective accuracy, and also the very large defect of failing to specify how many subjects were used for the experiments (we are told 14 subjects gave their permission to be tested, but not told how many subjects actually participated; and the graphs suggest that maybe only half that many participated). The third of the papers presents no original research. 

Containing some very dubious assertions, some references to weak research and some troubling omissions (such as no mention of the supremely relevant research of John Lorber or the short lifetimes of synapse proteins), the paper "Memory Systems of the Brain" is an example of a bluffing neuroscience paper (in the sense that its title suggests something the paper does not deliver). The author does nothing to describe a system of the brain capable of encoding memories. He does nothing to describe a system in the brain capable of storing memory information. He does nothing to describe in the brain a system capable of preserving memory information for decades. He does nothing to describe a system in the brain capable of retrieving memories. So he does not describe any such thing as a memory system in the brain.  Nature never told us that brains store memories; it was merely neuroscientists who told us that, without any good evidence for such a claim.