Papers Claiming Brain Memory Storage Keep Citing Poor Science Papers

Let us look at all the flaws, internal contradictions and incorrect statements in a 2022 paper purporting to provide evidence of engrams (cells or matter in the brain which are claimed to store memories).  The paper is entitled "The essence of the engram: Cellular or synaptic?" The second sentence of the paper makes this very untrue claim: "In the last few years it has been shown that simple association memories can be encoded by a subset of the neuronal population called engram cells." This is an example of what I call an achievement legend, which is when scientists boast of having achieved something that was not actually achieved. A careful and sufficiently critical examination of all papers claiming to provide evidence for engrams will show that none of them was good experimental science. Research claiming to provide evidence for engrams is plagued by Questionable Research Practices, and fails to be robust scientific evidence.  

Early on the paper makes this unfounded claim: "Over the last few decades, numerous discoveries have been made regarding the properties of memory ranging from identification of the molecular mechanism(s) underlying memory formation to establishing the different temporal phases of memory." Scientists are actually completely lacking in any understanding of any molecular mechanisms underlying memory formation.  There is no scientist who can even give a credible explanation of how something as simple as the phrase "my dog has fleas" could be stored in a brain.  Contradicting the previously quoted statement, the paper says a bit later,   "the physical basis of memory is elusive."  

We then read this untrue statement: "There is now a substantial body of evidence based on recently developed techniques, including optogenetics, chemogenetics, electrophysiology, and multiphoton confocal imaging, to suggest that memory for basic types of behavioral learning such as contextual fear conditioning is maintained in a population of neurons referred to as engram cells [4], [5], [6], [7], [8]." Notice the hedging in the language. People often use the term "substantial" when they don't have much of anything. If your mother asks you, "Have you finished your term paper?" you might say, "I've done substantial work" when you haven't done much of anything.  The use of the word "suggest" indicates uncertainty. An examination of the studies referred to shows that none of them establish any robust evidence for any such things as engram cells. 

Let's look at exactly the studies referred to above. Before discussing this, let me list some of the types of study defects I previously listed in my post "The Seven Sins of 'Memory Engram'  Experiments."  The sins I mentioned were as follow:

• Sin #1: assuming or acting as if a memory is stored in some exact speck-sized spot of a brain without any adequate basis for such a “shot in the dark” assumption.
• Sin #2: Either a lack of a blinding protocol, or no detailed discussion of how an effective technique for blinding was achieved.
• Sin #3: inadequate sample sizes, and a failure to do a sample size calculation to determine how large a sample size to test with.
• Sin #4: a high occurrence of low statistical significance near the minimum of .05, along with a frequent hiding of such unimpressive results, burying them outside of the main text of a paper rather than placing them in the abstract of the paper.
• Sin #5: using presumptuous or loaded language in the paper, such as referring in the paper to the non-movement of an animal as “freezing” and referring to some supposedly "preferentially activated" cell as an "engram cell." 
• Sin #6: failing to mention or test alternate explanations for the non-movement of an animal (called “freezing”), explanations that have nothing to do with memory recall.
• Sin #7: a dependency on arbitrarily analyzed brain scans or an uncorroborated judgment of "freezing behavior" which is not a reliable way of measuring fear.

Here are the papers referenced:

• Reference 4 in the "Essence of the Engram" paper refers to the paper “Optogenetic stimulation of a hippocampal engram activates fear memory recall.” We see in Figure 3 of that paper that inadequate sample sizes were used. The number of animals listed in that figure (during different parts of the experiments) are 12, 12, 12, 5, and 6, for an average of 9.4. That is not anything like what would be needed for a moderately convincing result, which would be a minimum of 15 or 20 animals per study group. So the study is  guilty of Sin #3. The study is also guilty of Sin #7. The experiment relied crucially on judgments of fear produced by manual assessments of freezing behavior, which were not corroborated by any other technique such as heart-rate measurement. The study does not describe in detail any effective blinding protocol, so it is also guilty of Sin #2. The study is also guilty of Sin #6. The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part of the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs. 
• Reference 5 in the "Essence of the Engram" paper is a reference to the 2013 study "Creating a false memory in the hippocampus." When we look at Figure 2 and Figure 3 of that  paper, we see that the sample sizes used were paltry: the different groups of mice had only about 8 or 9 mice per group. Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. A sample size calculation would have revealed the defect, but the authors failed to do such a calculation.  No convincing evidence has been provided of creating a false memory. The paper also judged fear in rodents by subjective judgments of "freezing behavior," which is not a reliable way to measure fear in rodents. A reliable way to measure fear in rodent is to measure heart rate, which consistently spikes very sharply when rodents are afraid.  The study also failed to use any blinding protocol. 
• Reference 6 in the "Essence of the Engram" paper is a reference to the study "Bidirectional switch of the valence associated with a hippocampal contextual memory engram."  We see in that paper 5 or 6 results reported with a borderline statistical significance of only "< 0.05," so this paper is  guilty of Sin #4. No detailed description is given of how an effective blinding protocol was achieved, and only the skimpiest mention is made of blinding, so this paper is guilty of Sin #2.  The study used only "freezing behavior" to try to measure fear, without corroborating such a thing by measuring heart rates.  So the paper was guilty of Sin #7.  The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs.  So the study is also guilty of Sin #6. 
• Reference 7 in the "Essence of the Engram" paper is to a  paper by Ramirez and Liu  published in Nature, one entitled, “Activating positive memory engrams suppresses depression-like behaviour.” Figure 2 of the paper says that in one group there were only 6 mice used, and elsewhere the paper states that a control group had only 3 mice. These sizes are way below the 15 or 20 animals per study group (control and non-control) recommended as a minimum for a reliable experimental result. The authors claim to have counted differences in the degree to which mice “struggled” when presented with a maze – again something involving a subjective interpretation in which a researcher might tend to see whatever he wants to see. The authors' interpretation of what is going on is speculative. The authors do not present any solid evidence that they actually activated a memory by optogenetic stimulation.
• Reference 8 in the "Essence of the Engram" paper is to the paper "Memory engrams: Recalling the past and imagining the future." The paper is not a paper that presents original research, but one that simply cites previous research, such as the bad studies mentioned above.  The paper is co-authored by Susumu Tonegawa, who co-authored several of the poorly designed and unreliable studies mentioned above. 

So let's summarize what has gone on in this crucial part of the "Essence of the Engram" paper. The authors have made the claim that there is a "substantial body of evidence" for engrams, and have cited five different papers as references. Not one of those papers presents any robust original research supporting claims that engrams exist. The first four papers cited are low-quality science studies that failed in numerous ways to be good experimental science. The fifth paper is co-authored by the co-author of several of these low-quality science studies, and presents no new research.  This is what goes on all the time in literature referring to engrams.  You have either papers presenting low-quality science experiments, or you have papers that refer to low-quality science experiments.  Nowhere will you find any solid research presenting any robust evidence for engrams.  

 Later in the "Essence of the Engram" paper we read this untrue claim: "Han et al. [14] provided the first causal evidence that engram cells are qualitatively different from non-engram cells."  The reference is to the low-quality science paper "Selective Erasure of a Fear Memory." In Figure 1 of that paper we see a larger-than average sample size was used for two groups (17 and 24), but that a way-too-small sample size of only 4 was used for the corresponding control group. You need a sufficiently high number of animals in all study groups, including the control group, for a reliable result.  The same figure tells us that in another experiment the number of animals in the study group were only 5 or 6, which is way too small. Figure 3 tells us that in other experiments only 8 or 9 mice were used, and Figure 4 tells us that in other experiments only 5 or 6 mice were used. So this paper is guilty of Sin #3. No mention is made in the paper of any blinding protocol, so this paper is guilty of Sin #2. Figure 4 refers to two results with a borderline statistical significance of only "< 0.05," so this paper is also guilty of Sin #4.  The paper relies heavily on judgments of fear in rodents, but these were uncorroborated judgments based on "freezing behavior," without any measure of heart rate to corroborate such judgments. So the paper is also guilty of Sin #7. 

I could go on and on here, but you can get the idea. What goes on in these kind of review papers are endless references to defective, poorly designed research. 

Contrary to the previous claims I quoted from the paper "The essence of the engram: Cellular or synaptic?" is the paper's statement that "the physical form of memory is elusive," and also the paper's statement that "it remains unclear however whether the engram is essentially cellular in nature or whether it is best described in terms of the changes in synaptic strength of contacts made onto and by engram cell."  Such statements show that the authors are just engaging in hand waving and guesswork. It's kind of like someone saying, "I know there are extraterrestrial spaceship bases in the solar system, but I don't know whether they are on the moon or on Mars." 

We then have a repetition of the groundless legend that some MIT group headed by Tonegawa did something to help establish the reality of engrams in the brain.  See my post here for why such claims are groundless. 

The paper then has about 10 diagrams that look like the ones below, with different color variations:

We have the caption "Different phases of memory formation and retrieval." The circles represent neurons, and the lines represent synapses. There are two line thicknesses, and we are told one thickness represents "weak synaptic strength," and the other thickness represents "strong synaptic strength." As a diagram trying to depict a neural formation of memories, the diagrams are a joke.  A neuron has an average of about 7000 synaptic connections with other neurons. And synapses can have any of a thousand different strengths. There is no way even the simplest learned information could be stored in the brain through anything like the shown diagrams. 

It is, in general, wrong to try to explain information storage by appealing to a mere process of strengthening. Strengthening is not storage. We know of many ways in which information can be stored, and none of them are cases of strengthening.

Below are some examples:

1. People can store information by writing using a paper and pen. This does not involve strengthening.
2. People can store information by using a typewriter to type on paper. This does not involve strengthening.
3. People can store information by drawing pictures or making paintings. This does not involve strengthening.
4. People can store information by taking photographs, either by using digital cameras, or old-fashioned film cameras. In neither case is strengthening involved.
5. People can store information by using tape recorders. This does not involve strengthening.
6. People can store information by using computers. This does not involve strengthening.

So basically every case in which we are sure information is being stored does not involve strengthening. What sense, then, does it make to claim that memory could be stored in synapses through strengthening?

In all of the cases above, information is stored in a rather similar way. Some unit capable of making a particular type of impression or mark (physically visible or perhaps merely magnetic) moves over or strikes a surface, and a series of impressions or marks are made on the surface. Such a thing is not at all a process of strengthening.

Consider a simple example. You have a friend named Mary, and you one day learn that Mary has a black cat. Now let us try to imagine this knowledge being stored as a strengthening of synapses. There is no way we can imagine such knowledge being stored by a strengthening of synapses. If you happened to have stored in your brain the knowledge that Mary has a black cat, it could conceivably be that a strengthening of synapses might allow you to more quickly remember that Mary has a black cat. But there is no way that the fact of Mary having a black cat could be stored in your brain through a strengthening of synapses.

Similarly, a simple example of a new memory (often tested in neuroscience experiments) is when a mouse is trained to fear a shock plate. There is no way we can imagine such knowledge being stored by a mere strengthening of synapses.

On and on the paper "The essence of the engram: Cellular or synaptic?" goes, continuing again to make observational claims that are not well founded.  An example is its claim, "Studies of engram cells have greatly expanded our understanding of the mechanisms underlying memory, but several questions remain unanswered." No, there have been no studies showing that there are any such things as "engram cells," and neuroscientists have no understanding whatsoever of any physical mechanism underlying memory.  Our neuroscientists are simply guilty of pretending to understand things they don't have any understanding of.  

The lack of any scientific basis becomes apparent when we look at how such papers define an "engram cell."  We read of no special characteristic of such a cell, such as a different appearance. We read of no physical change going on to make a cell an "engram cell." So how is an engram cell defined by such papers? It is typically defined as a cell that is part of some group of neurons and synapses that undergo "increased activation" when a memory is retrieved.  Almost all neurons in the brain transmit nerve impulses continuously, largely in a random fashion. So anyone scanning activity levels in the brain will always be able to find various cells in various parts of the brain having "higher activation." And anyone scanning the strengths of synapses will be able to find that a certain percentage (say 5%) are stronger than other ones.   So limiting yourself to checking electrical activity levels and synapse strengths, what is the difference between the observational result expected under the nonexistence of engrams (no brain storage of memories) and the observational result expected under the existence of engrams (brain storage of memories)? There isn't one. 

Our "Essence of the engram" paper then tells us, "Recent studies suggest that reactivation of engram cells induces the retrieval of memory and vice versa [4], [34], [42], [97], [98]. " I already explained in the bullet list above why the Reference 4 is to a junk science study. Here are the other   papers referenced:

• Reference 34 is to the 2015 paper "Engram cells retain memory under retrograde amnesia." When we look at the end of the supplemental material, and look at figure s13, we find that the experimenters were using a number of mice that was equal to only 8 in one study group, and 7 in another study group.  Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. The paper failed to use a blinding protocol, an essential for a paper like this to be taken seriously.
• Reference 42 is the 2016 paper "Memory retrieval by activating engram cells in mouse models of early Alzheimer’s disease."  The paper states that “No statistical methods were used to predetermine sample size.” That means the authors did not do what they were supposed to have done to make sure their sample size was large enough. When we look at page 8 of the paper, we find that the sample sizes used were merely 8 mice in one group and 9 mice in another group. On page 2 we hear about a group with only 4 mice per group, and on page 4 we hear about a group with only 4 mice per group. Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. The study therefore provides no convincing evidence of engram cells.
• Reference 97 refers to the paper "Encoding of contextual fear memory in hippocampal–amygdala circuit." It's a junk science paper that used way-too-small study group sizes of only 6 mice and 11 mice.  The paper also used judgments of rodent freezing behavior to try to measure fear in animals, and all papers that use that faulty technique are junk science papers, for reasons I discuss at length here. The paper failed to use a blinding protocol, an essential for a paper like this to be taken seriously.
• Reference 98 refers to a paper that describes no original research, but merely references work by others. 

It is very clear what is occurring in papers such as this one.  Triumphal narratives are being given of scientists making progress in understanding a physical basis of memory. When we closely examine the papers that are cited to back up these boasts, we find that they are invariably weak shoddy studies guilty of Questionable Research Practices.

This is largely how false narratives are perpetuated in science: by people citing poorly designed scientific research, and claiming that such research showed something, when the research failed to show any such thing because the research was so poorly done. 

The idea that human memories (which can last for 60 years) are stored in synapses (as maintained by the "Essence of the engram" paper criticized above) is untenable because synapses are so unstable, and are built from protein molecules that only last an average of a few weeks or less. An individual synapse and a dendritic spine do not last for years, and consist of proteins that only last for two weeks or less.  A 2019 paper documents a 16-day examination of synapses, finding "the dataset contained n = 320 stable synapses, n = 163 eliminated synapses and n = 134 formed synapses."  That's about a 33% synapse disappearance rate over a course of 16 days. The same paper refers to another paper that "reported rates of [dendritic] spine eliminations in the order of 40% over an observation period of 4 days." 

An additional reason for rejecting the synaptic theory of memory storage is that according to such a theory a memory could only be formed after a synapse was strengthened by proteins (something requiring at least minutes for protein synthesis). But humans can form a new memory instantly. Imagine if someone walks into your workplace naked or firing a gun. It wouldn't take you minutes to form a permanent memory of that. The memory would form instantly. But new proteins (such as would be needed to strengthen a synapse) could never form instantly. We know that the synthesis of new proteins requires minutes.  If forming new memories required the synthesis of new proteins, the brain would never keep up with sensory experiences which keep coming at you continuously. I can watch a 30-minute television drama, and then tell you every major thing that happened in the show. I wouldn't be able to do that if each new thing I saw required the synthesis of a new protein which required several minutes. 

In his Nautilus post “Here's Why Most Neuroscientists Are Wrong About the Brain,” C. R. Gallistel (a professor of psychology and cognitive neuroscience) points out the absurdity of thinking that mere changes in synapse strengths could store the complex information humans remember. Gallistel writes the following:

"It does not make sense to say that something stores information but cannot store numbers. Neuroscientists have not come to terms with this truth. I have repeatedly asked roomfuls of my colleagues, first, whether they believe that the brain stores information by changing synaptic connections—they all say, yes—and then how the brain might store a number in an altered pattern of synaptic connections. They are stumped, or refuse to answer....When I asked how one could store numbers in synapses, several became angry or diverted the discussion with questions like, 'What’s a number? ' ”

What Gallistel describes sounds dysfunctional: a pretentious neuroscientist community that claims to understand how memory can be stored in a brain, but cannot give anything like a plausible answer to basic questions such as “How could a number be stored in a brain?” or “How could a series of words be stored in a brain?” or “How could a remembered image be stored in a brain?” Anyone who cannot suggest plausible detailed answers to such questions has no business claiming to understand how a brain could store a memory, and also has no business claiming that a brain does store episodic or conceptual memories.

                        Click to see left column more clearly

Postscript: The paper "Prevalence of Mixed-methods Sampling Designs in Social Science Research" has a Table 2 giving recommendations for minimum study group sizes for different types of research. The minimum subjects for an experimental study are 21 subjects per study group. Most of the studies mentioned above are experimental studies that used only about half of this minimum number.  The "case study" type mentioned below is a different type of study in which you merely document one or a few occurrences of some condition or situation, without trying to show a cause.