Showing posts with label citation of low-quality research. Show all posts
Showing posts with label citation of low-quality research. Show all posts

Saturday, August 23, 2025

A Neuroscientist Lacking Any Credible Explanation for Memory Resorts to "Contribution" Hand-Waving

Neuroscientists lack any credible explanation for human memory. Neuroscientists cannot credibly answer any of these questions:

  • How is a human able to ever instantly learn all of the many different things that humans can learn?
  • How is a human ever able to retain memories for decades, something that should be impossible from units such as synapses, which are built from proteins which have average lifetimes of only a few weeks or less?
  • How could a brain ever store a memory when nothing in a brain seems to bear any resemblance to some component capable of writing information?
  • How could a brain ever read a memory when nothing in a brain seems to bear any resemblance to some component capable of reading memory information?
  • How could there possibly be memories stored in brains, when there has been the most careful microscopic examination of so many thousands of brains of very recently deceased people, and the most careful microscopic examination of so many thousands of chunks of brain tissue from living people, without any trace of learned information ever being discovered from such examination? 
  • How could there possibly be memories stored in brains, when no one has ever credibly discussed any encoding system whereby episodic memories or learned knowledge could be converted to neuron states or synapse states? 
  • How could a human ever instantly remember lots of relevant facts about a person, place or event as soon as he hears the name of such a person, place or event, something seemingly impossible using a brain lacking any of the things that enable fast recall of stored information (addresses, indexes and sorting)?

When writing papers trying to persuade us that they have some understanding of such matters, neuroscientists engage in various types of bluffs, vacuous hand-waving and misleading tricks. They include these:

  • The most common trick is to abundantly cite junk science studies. So a neuroscientist might write some paper filled with mentions of experimental neuroscience papers. If you actually examine these papers, you will find they are almost all very low-quality papers guilty of very bad examples of Questionable Research Practices, such as the use of way-too-small study group sizes. 
  • Another trick used by empty-handed neuroscientists discussing memory is the trick I can call  concurrent process description. It works like this: using jargon-laden language, a neuroscientist will describe some type of biochemistry activity or electrical activity going on when a person learns or remember, typically some type of activity that is constantly occurring in the brain. An attempt will be made to insinuate that such activity explains something going on during learning or recall.  So we will read statements such as "While you are learning your school lessons, neurotransmitters are traveling across synapses, and new proteins are being synthesized that may strengthen synapses." The fallacy is that such things are occurring in the brain constantly, regardless of whether you are learning anything or forming any new memory or recalling anything. You do not explain things that occur only at some times by referring to types of events that occur constantly in the brain, even when learning and recall is not occurring. 
  • The writer may take a kind of approach we may call the "neural miscellany" approach. The approach consists of trying to mention a large variety of neuroscience terms referring to different types of structures or cells in brains or different anatomical parts in the brain or different chemical processes in brains, and so forth. These mentions never add up to a decent explanation for any part of memory, but by given these jargon-laden mentions, some impression of understanding may be created. 
  • Vague claims are made that memories are "processed" or "handled" by some particular part of the brain. Typically some very specific reference will be made to some particular fraction of the brain's anatomy, or some particular structures in the brain. No one should be impressed or persuaded by such claims, which typically are completely vague about what this alleged processing was. An example is this claim in Alberini's paper (discussed below): "These memories are processed by the medial temporal lobe-dependent memory system, which consists of the hippocampal region (CA fields, dentate gyrus, and subicular complex) and the adjacent perirhinal, entorhinal, and parahippocampal cortices." 
  • To try to create some greater impression of substance or knowledge, references to anatomy or biochemistry will be mixed up with psychology references to details of the phenomena of memory, such as the difference between explicit memory and implicit memory, the difference between episodic memory and muscle memory, the difference between learning and recall,  and the difference between short-term memory and long-term memory. Such psychology references do not involve any neural explanation of memory, but by using them a writer will help to create more of an impression of knowledge on the topic of memory. 
  • Another technique is to engage in what we can call "contribution" hand-waving. The technique involves mentioning various parts of the brain or chemicals in the brain, and vaguely claiming that such things "contribute" to memory. No actual explanation is going on when such hand-waving occurs. Claims about hundreds of possible "contributions" may be made without explaining how something occurs. Contribution is not causation. For example, my eyeglasses contribute to the overall process by which I see things and remember things I have seen. But my eyeglasses do not do anything to explain the mystery of memory creation or learning. 

We see an example of such "contribution" hand-waving in the recent paper "Not just neurons: The diverse cellular landscape of learning and memory" by neuroscientist Christina M. Alberini. Alberini has no credible tale to tell to explain how there could occur any of the phenomena of memory. What she mainly does is to make unwarranted or not-very-relevant claims about this or that thing "contributing" to memory or learning. 

Near the beginning of the paper, we have a concurrent process description by Alberini, who mentions "gene expression" and "chromatin regulation" while discussing learning.  Gene expression and chromatin regulation are constantly occurring events in the brain, and there is no evidence they occur more often or differently when a person learns or recalls. 

Alberini makes the statement below, which makes unwarranted boasts, and finally ends with a confession of ignorance:

" These memories are processed by the medial temporal lobe-dependent memory system, which consists of the hippocampal region (CA fields, dentate gyrus, and subicular complex) and the adjacent perirhinal, entorhinal, and parahippocampal cortices. This system can store memories of single episodic experiences for as long as we live—a process of very long-term storage that still lacks an understanding of its biological underpinnings. The implicit memory system, on the other hand, stores and recalls unconscious and automatic memories. These include habits, skills, priming, and simple forms of memories. One example of an implicit type of memory is procedural memory, which guides the execution of skills and tasks without conscious retrieval. These memories include tying shoes, riding a bike, driving a car, skiing, playing the piano, etc. Procedural memories are stored long term through a learning phase characterized by many repetitions while all the relevant neural systems work together to produce the action automatically. Implicit procedural learning is essential for developing any motor skill, and the brain regions involved in forming and storing these memories include the striatum, basal ganglia, cerebellum, and limbic system.10 Implicit memories, once established as long-term representations, can last a lifetime. As for the explicit types of memory, the biological underpinning of this very long-lasting memory storage is not yet understood." 

We have here some examples of the tricks discussed in my bullet list above. Psychology references having nothing to do with neural explanations for memory are mixed up with some specific references to brain anatomy, without any mention of how such brain anatomy can explain such phenomena. We have claims of brain memory storage, which have no specifics of how such a thing could occur: "Procedural memories are stored long term through a learning phase characterized by many repetitions while all the relevant neural systems work together to produce the action automatically." That is the vaguest hand-waving. In the underlined phrases, we have confessions that neuroscientists do not actually have any understanding of how long-lasting memory storage can occur. It makes no sense for someone to be confidently asserting (as Alberini does in the quote above) that brains store memories, and also to confess as she does that we do not understand how long-term memory storage occurs. If you do not understand how long-term memory storage occurs, you should have no confidence that brains store memories, particularly given that short-term memory storage is also not understood. 

Alberini then makes these faulty claims, which involve falsehood, bluffing and speculation:

"Long-term memories do not form instantly upon learning. They are initially fragile, and, in fact, in their early phase, they can be disrupted relatively easily by biological, pharmacological, or behavioral interference. However, over time, or via repeated learning, they build strength and become resistant to disruption through a process known as memory consolidation—a collection of biological changes taking place in several brain regions of the relevant memory system, which may include experience repetitions or reactivations, and eventually result in long-lasting, stable representations."

The first sentence is untrue. Many long-term memories do form instantly upon learning. If you are told of the death of your child or the death of your parent, you will instantly form a permanent new memory for the rest of your life. You do not need repeated notifications of such a death before the memory becomes permanent. The claim that you do not learn something until repeated exposures to it is untrue. Anyone can learn the plot of a movie by watching it a single time, and he may remember that plot for years, even though he has only seen the movie once. Learning something might require repeated exposures, but it very often does not. Scientists define a long-term memory as anything you remember for days or longer. Even many not-very-interesting things can be added to long-term memory after a single sensory experience. I often remember trivial little things I read about or experienced days, weeks, months or years ago, even if I never thought about such things between the time I read or experienced them and the time I remembered them. So neuroscientists deceive us when they make some generalization that learning requires multiple exposures. 

Why do neuroscientists make obviously false claims like those in the quote above? It is because the reality of instant learning is one that defies all claims that learning occurs through brain mechanisms. The brain has nothing like any component that could account for the instant learning that so commonly occurs in a person's life.  When neuroscientists speculate about how memory formation could occur, they speculate about sluggish, very slow processes such as protein synthesis, which would require many minutes. So for neuroscientists, the reality of instant learning is a scandal they must sweep under the rug, by telling us deceits such as the deceit that learning requires multiple sensory exposures. 

Alberini is bluffing when she makes the vague hand-waving statements below, speaking as if she knows things she does not really know: 

"Memory storage refers to the process of holding the learned information. When needed, memories are recalled or retrieved, and this process can temporarily return the memory to a labile state, during which the memory restabilizes—a process known as memory reconsolidation. Each of these memory processes requires the contributions of multiple brain regions, cell populations, and biological pathways that become activated and functionally engaged following learning and evolve over time." 

These "sound like I understand things" claims are contradicted by her previous confession: "As for the explicit types of memory, the biological underpinning of this very long-lasting memory storage is not yet understood." "Labile" means "easily altered." There is no evidence that recalling or retrieving a memory causes a memory to become "labile." To the contrary, verbally recalling a memory will make it less likely to be forgotten. 

In the quote below, Alberini asks some good questions, which she follows with a false boast that is the opposite of the truth:

"What mechanisms underlie the formation and storage of memory? Are the biological mechanisms recruited to form and store different types of memories similar or different? Where do they occur? How can they explain memory storage that lasts for a lifetime? Are they different at different ages? What mechanisms underlie memory recall? And so on. Over these 40 years, monumental progress has been made."

No, the truth is that no progress has been made in answering these questions. No claims of such progress will hold up to critical scrutiny. Typical claims of progress involve appeals to poorly-designed junk science studies such as rodent studies using way-too-small study group sizes and a bad, unreliable "freezing behavior" method for judging how well a rodent remembered. 

The Figure 1 of Alberini's paper is a laughable visual. She has taken a "double bullet list" approach. We have a picture of a brain, with the word "Memory" superimposed over it. On the left of this picture is a bullet list of neuroscience terms, referring to parts of the brain or chemical processes of the brain. On the right of this picture is another bullet list of neuroscience terms, referring to other parts of the brain or chemical processes of the brain. The figure has arrows pointing from these bullet lists to the picture of the brain.  Bullet lists are not explanations.  None of the items listed in the bullet list do anything substantive to explain how a human could form a memory, preserve a memory or instantly recall a memory. 

Alberini makes the untrue claim here: "Far from being just glue or filler cells, astrocytes have been shown by numerous studies to actively contribute to learning and memory through several mechanisms (Figure 2)." The Figure 2 claims that astrocytes do "computing, encoding and storing information." There is no good evidence that any part of the brain does any such thing as encoding memories or storing learned information. No neuroscientist has any credible tale to tell of how any brain component could do such things, and microscopic examination of brain tissue always fails to provide any evidence that such things occur in the brain, with not a single speck of learned information ever being found through microscopic examination of brain tissue. Alberini makes the untrue claim that "recent findings provided compelling evidence for the direct roles of astrocytes, similar to those played by neurons, in computing, encoding, and storing information." 

Here are some examples of the bad studies she cites to try and back up such boasts:

  • "Lactate produced by glycogenolysis in astrocytes regulates memory processing." This is a junk science rodent study using way-too-small study group sizes much smaller than 15, and usually much smaller than 10. 
  • "Lactate from astrocytes fuels learning-induced mRNA translation in excitatory and inhibitory neurons." This is another junk science rodent study using way-too-small study group sizes much smaller than 15, and usually much smaller than 10. 
  • "Astrocyte-Neuron Lactate Transport Is Required for Long-Term Memory Formation."  This is another junk science rodent study using a way-too-small study group size of only 7 mice. 
  • "Astrocytic β2 Adrenergic Receptor Gene Deletion Affects Memory in Aged Mice." This is another junk science rodent study using a way-too-small study group sizes such as only 3, 5 and 6. 

These are the results of my examination of a random selection of four papers that Alberini claims as evidence that astrocytes have something to do with memory. We may guess that the other studies she claims are equally bad examples of low-quality neuroscience. 

As a general rule, we should not trust the generalizations that neuroscientists make about human mental performance, because neuroscientists have a long history of mischaracterizing the mental abilities of humans, by making human minds sound much more weakly-performing than they are, so that neuroscientist explanations sound less far-fetched.  As an antidote to such mischaracterizations, remember the facts of your own mental abilities, and also study the very important topic of human best mental performances. A study of human best mental performances will typically blast into smithereens the type of generalizations that neuroscientists like to make about how human minds perform.  You can find many posts about human best mental performances by reading posts like the ones available at the links below:


Below are some results from the annals of the World Memory Championships that show how falsely neuroscientists speak when they depict memory formation as a slow process or claim human minds require multiple sensory exposures before something is memorized. 

Discipline 5, Speed Numbers: Wei Quinru was able to recall 642 digits memorized in a 5-minute period (Korea Open Memory Championship 2024). Four  people were able to each recall more than 800 digits memorized in a 5-minute period (2021 World Memory Championships).
Discipline 6, Dates and Fictional Events:  Prateek Yadav memorized in 5 minutes dates corresponding to 154 fictional events (2019).  Several other contestants memorized in 5 minutes dates corresponding to 700+ fictional events (2021 World Memory Championships).
Discipline 7, "Hour Cards" Card Memorization:  Kim Su Rim memorized 2530 cards in 60 minutes.  
Discipline 8, Random Words:  Prateek Yadev memorized 335 random words in 15 minutes. Several others in 2021 memorized more than 500 random words in 15 minutes. 
Discipline 9, "Spoken Numbers":  Ryu Song I was able to recall 547 decimal digits that had been read at a rate of one per second (WMSC World Championship 2019).  Tenuun Tamir and several other Mongolian or Chinese contestants were able to recall more than 600 decimal digits that had been read to him at a rate of one per second. 
Discipline 10, "Speed Cards":  Munkhshur NARMANDAK memorized 981 cards in five minutes, and several others memorized more than 600 cards in five minutes. 


Things Scientists Never Did

Postscript: The brain is an organ of constant molecular turnover, as the proteins that it is built from have average lifetimes no greater than a few weeks. Synapses (claimed to be a storage place of memory) are attached to short-lived dendritic spines known to have average lifetimes much shorter than years (and typically not six months).  So how could a human ever remember anything for decades, if memory storage occurred in the brain? Neuroscientists have no credible answer. When asked about such things they appeal to "consolidation," typically using the kind of vacuous, vague hand-waving we got in the quote above, where a scientist referred to "
memory consolidation—a collection of biological changes taking place in several brain regions of the relevant memory system, which may include experience repetitions or reactivations."

But it simply is not true that to remember something for decades, you need  periodic reactivations every few years that recharge or reactivate a memory that won't last years without being strengthened.  You can learn trivial little things and remember them decades later, even with no reactivations of the memory. 

An experience I had recently showed this. Someone in my family recently got a teapot, which caused me to recall the beginning of a children's song I had heard only a few times, in a house I have not lived in for about 25 years. I remembered the first line: "I'm a little teapot, short and stout." I tried to remember the second line, but at first I could not. Later I remembered both of the first two lines of the song I had not heard or sung or remembered in about 25 years:

I'm a little teapot, short and stout
Here is my handle, here is my spout

In 2025 (as I noted at the end of the post here) I had a recollection which proved the ability of the mind to recall very old memories that have not been recalled in 50 years. For some reason I recalled a book I had read about 50 years ago, and never since: the science fiction book "Galaxies Like Grains of Sand" by Brian Aldiss. I remembered some lines from the book. I wrote them down on paper like this:

"The mirror of the past lies shattered. The fragments you hold in your hand."

After I wrote this recollection of something I had not read, thought of or heard quoted in fifty years, I borrowed the book on www.archive.org.  I see that the lines were these (almost exactly as I remembered them)

"The long mirror of the past is shattered...Only a few fragments are left, and these you hold in your hand." 

Below is a quote on the same topic from an earlier post discussing why brains cannot be the storage place of very old memories:

"I know for a fact that memories can persist for 50 years, without rehearsal. Recently I was trying to recall all kinds of details from my childhood, and recalled the names of persons I hadn't thought about for decades, as well as a Christmas incident I hadn't thought of for 50 years (I confirmed my recollection by asking my older brother about it). ...Upon looking through a list of old children shows from the 1960's, I saw the title 'Lippy the Lion and Hardy Har Har,' which ran from 1962 to 1963 (and was not syndicated in repeats, to the best of my knowledge). I then immediately sung part of the melody of the very catchy theme song, which I hadn't heard in 53 years. I then looked up a clip on a youtube.com, and verified that my recall was exactly correct."

A scientific study by Harry Bahrick was entitled “Semantic memory content in permastore: Fifty years of memory for Spanish learned in school.” It showed that “large portions of the originally acquired information remain accessible for over 50 years in spite of the fact the information is not used or rehearsed.” The same researcher tested a large number of subjects to find out how well they could recall the faces of high school classmates, and found very substantial recall even with a group that had graduated 47 years ago. Bahrick reported the following:

"Subjects are able to identify about 90% of the names and faces of the names of their classes at graduation. The visual information is retained virtually unimpaired for at least 35 years...Free-recall probability does not diminish over 50 yr for names of classmates assigned to one or more of the Relationship Categories A through F."

While researching tests of very long-term memory, I found a 1989 scientific paper ("On the Course of Forgetting in Very Long-Term Memory") on an interesting experiment that tested people by asking which of four titles was an actual title of a TV show. All of the actual TV shows were ones that only ran for one year. This is a good technique for testing very old memories, because in the 1970's and 1980's in the US if a TV show ran for only one year it would almost never be shown in other years, and there would tend to be no references to it in popular culture. (This was before services such as Netflix, which might allow a program running only one year to be re-watched by many.)  Here is an example of some of the questions asked to the experiment's subjects:

1974
 Which of the following was a T.V. show? (a) Mandrake, (b) Shipmates (c) Private Nelson, (d) Lucas Tanner 
1978
 Which of the following was a T.V. show? (a) Gaslight Alley, (b) Cutting Corners (c) Black Knight, (d) Kaz 
1981
 Which of the following was a T.V. show? (a) Dateline Miami, (b) The Conductor (c) Discovery, (d) McClaine 's Law 

In these tests the tested subjects (231 in all) scored as shown below (25% is the result expected by chance, if no recall occurred).


We see here a good retention of trivial information that was learned between 7 and 15 years ago, with people scoring about 60% correctly, much higher than the 25% expected by chance. 

In my case in late August, 2025 not only did I recall that Lucas Tanner was an old TV show, but I also recalled the name of the show's star (David Hartman), and that the show was about a teacher. I have not watched this 1974-1975  show or read or heard any mention of this show in the past 50 years, nor have I ever thought about the show in the past 50 years. So it certainly is false that remembering something for decades requires periodic reactivations of the memory. Many people can remember meaningless and trivial things for 50 years, without any reactivations of the memory. Such abilities cannot be explained through any credible theory of brain activity, given the molecular and structural turnover in the brain. 

Tuesday, May 20, 2025

The Groundless Myth of Concept-Selective Regions in the Brain

 Here is one way that a type of scientific myth can arise:

 (1) Scientists very interested in providing evidence for some untrue claim (which they may believe to be true) may do poorly-designed research guilty of various types of Questionable Research practices such as way-too-small study sizes, a lack of pre-registration, a lack of a proper blinding protocol, and poor, unreliable measurement techniques. Such scientists then write up a paper incorrectly claiming that their research supports the  claim. 

(2) Papers such as these get endlessly cited in the popular press and also in other scientific papers. Typically when a scientific paper cites the poor-quality research, no mention will be made of any of the factors that disqualify the cited paper as being an example good robust evidence. Whatever the paper claimed to show will simply be cited by some other paper as a fact that was established by research. 

In my previous post I documented a case of this type of citation bungling going on. In my 2024 post "Papers Claiming Brain Memory Storage Keep Citing Poor Science Papers" I examined each of the references at the end of this sentence in a scientific paper:  "There is now a substantial body of evidence based on recently developed techniques, including optogeneticschemogeneticselectrophysiology, and multiphoton confocal imaging, to suggest that memory for basic types of behavioral learning such as contextual fear conditioning is maintained in a population of neurons referred to as engram cells [4][5][6][7][8]." For each one of the studies cited at the end of this sentence, I wrote a separate paragraph showing that the cited study was not an example of robust scientific research, but instead a low-quality study guilty of various types of Questionable Research Practices such as way-too-small study sizes, a lack of pre-registration, a lack of a proper blinding protocol, and poor, unreliable measurement techniques.

Let us look at another example of this type of misconduct.  In the recent preprint "MINDSIMULATOR: EXPLORING BRAIN CONCEPT LOCALIZATION VIA SYNTHETIC FMRI," which you can read here, we have the following spurious claim:

"Numerous neuroscience studies have illustrated that specific regions of the visual cortex exhibit concept selectivity. When individuals receive visual stimuli related to particular concepts (such as places, bodies, faces, words, colors, and foods), the respective cortical regions exhibit significant activation (Epstein & Kanwisher, 1998; Sergent et al., 1992; Jain et al., 2023; Pennock et al., 2023; Kanwisher et al., 1997; Allen et al., 2022). These regions are termed visual concept-selective regions and play a vital role in advancing the understanding of brain visual cognition."

This claim that there are "concept-selective" regions of the brain has no basis in fact. It is not true that particular regions of the brain become more active when someone is seeing some particular type of visual stimulus.  All of the papers cited above are examples of very low-quality research severely guilty of Questionable Research Practices. Let us look at each of them. 

Before discussing them,  I should explain why brain imaging studies using small study group sizes are worthless. An article on neursosciencenews.com states this: "A new analysis reveals that task-based fMRI experiments involving typical sample sizes of about 30 participants are only modestly replicable. This means that independent efforts to repeat the experiments are as likely to challenge as to confirm the original results."  The paper "Prevalence of Mixed-methods Sampling Designs in Social Science Research" has a Table 2 giving recommendations for minimum study group sizes for different types of research. According to the paper, the minimum number of subjects for an experimental study are 21 subjects per study group. The same table lists 61 subjects per study group as a minimum for a "correlational" study. 

In her post “Why Most Published Neuroscience Findings Are False,” Kelly Zalocusky PhD calculates that the median effect size of neuroscience studies is about .51. She then states the following, talking about statistical power (something that needs to be .5 or greater to be moderately convincing): 

"To get a power of 0.2, with an effect size of 0.51, the sample size needs to be 12 per group. This fits well with my intuition of sample sizes in (behavioral) neuroscience, and might actually be a little generous. To bump our power up to 0.5, we would need an n of 31 per group. A power of 0.8 would require 60 per group."

A study with a statistical power of .5 is considered only modestly replicable, something that will be replicated about half of the time if you try to replicate it.  A study with a statistical power of .8 is considered fairly good evidence.  If we describe a power of .5 as being modestly replicable, it therefore seems that about 31 subjects per study group is needed for an experimental neuroscience study to be worthy of consideration.  

Now let us look at list at the studies cited above as evidence for the claim that there are "concept-selective" regions in the brain. 
  • "Epstein & Kanwisher, 1998":  This is a reference to the paper "A cortical representation of the local visual environment,"  which you can read here. The study used a way-too-small study group size of only 9 subjects. So it provided no real evidence for  "concept-selective" regions in the brain. The reported "superior activations" were only about 1%, which can easily be explained as mere random variations. 
  • "Sergent et al., 1992": This is a reference to the paper "Functional neuroanatomy of face and object processing: a positron emission tomography study," and you can read the abstract here. The paper is behind a paywall, and the abstract makes no mention of the number of subjects used. Given that it is almost invariably true that the abstract of an experimental neuroscience paper will list the number of subjects used whenever it has a halfway-decent study group size, we may presume with high confidence that the study group size was too-small for anyone to be claiming the study as good evidence for concept-selective regions in the brain. 
  • " Jain et al., 2023" :   This is a reference to the paper " Selectivity for food in human ventral visual cortex," which you can read here. The paper claims to have found "two food-selective regions in the ventral visual cortex," but the claim is groundless, because it is based on brain-imaging experiments using a way-too-small study group size of only 8 subjects. 
  • "Pennock et al., 2023":  This is a reference to the study "Color-biased regions in the ventral visual pathway are food selective," which you can read here. It's another paper making claims similar to the Jain paper. But the claims are just as groundless, because they are also based on rain-imaging experiments using a way-too-small study group size of only 8 subjects. 
  • "Kanwisher et al., 1997":  This is a reference to the paper "The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception," which you can read here. The paper used a too-small study group size of only 15 subjects, a size too small for decent evidence to be claimed.  As discussed above, a minimum for a study like this to be taken seriously is about 30 subjects. For a long discussion of the weakness of Kanwisher's research on this topic, read my post here
  • "Allen et al., 2022":  This is a reference to the paper "A massive 7T fMRI dataset to bridge  cognitive neuroscience and artificial intelligence,"  which you can read here. The paper merely describes a dataset created by scanning 8 subjects, and makes no claims that any evidence was produced of concept-selective regions in the brain. 
So it is clear that the paper "MINDSIMULATOR: EXPLORING BRAIN CONCEPT LOCALIZATION VIA SYNTHETIC FMRI," which you can read here, was guilty of citation misconduct. The paper claimed that "numerous neuroscience studies have illustrated that specific regions of the visual cortex exhibit concept selectivity," and cited only the papers in the bullet list above to support this claim. But none of the papers cited provided any good evidence to back up such a claim. 

This type of thing is what constantly occurs in neuroscience literature. Again and again and again we have papers claiming that some grand result was established by neuroscience researchers. There follows a list citing a set of papers. But a careful examination of the papers cited will show that none of them provided any good evidence for the grand result claimed.  The citation of low-quality research is extremely abundant in neuroscience papers. When the citation of low-quality research becomes common, we have a situation in which the neuroscience literature serves to propel and propagate myths and legends, groundless boasts of achievements. 

The practice of citing poor-quality research occurs so abundantly in neuroscience papers that you should never assume the truth of any claim made in a neuroscience paper, merely because it is followed by some list of paper citations that do not discuss the details of the papers cited. When people have good evidence to back up a claim, they tell us about the details of such evidence. A sentence listing a bunch of neuroscience papers without giving us any details about such papers should in general be something regarded with high suspicion. 

If you are writing a neuroscience paper making Claim X, and you know of five well-designed high-quality studies providing strong evidence for Claim X, then you might do something like providing a bullet list in which each of those studies is described as one of the bullets in the bullet list.  You would provide details such as saying "Walker and Miller in 2017 did a well-designed pre-registered study in which a strong Claim X effect was reported in 50 subjects, who were compared to 50 control subjects using a stringent blinding protocol."  Not being worried about your readers reading the studies you mentioned, you would provide links allowing your reader to conveniently open up each of those studies, without copying their titles into the search page of Google Scholar.  

But if you knew of no such high-quality studies, but only low-quality studies, you might merely list those low-quality studies in a single sentence that provided no details about those studies, and had no links to the studies. That way only the most diligent readers of your papers would be able to find out that the studies you had cited were very low-quality studies. If you did that, you would be following the pattern we so often see in neuroscience papers citing poor-quality studies. 



The word "selection" refers to a choice made by a conscious agent. 
There is no robust evidence that any region of the brain is "concept-selective." There is no robust evidence that any region of the brain activates more strongly when certain types of things are shown to a person. People select things, but brain regions don't select things. Claims of "concept-selective" brain regions are another example of biologists making deceptive use of the words "selective" or "selection." Biologists have doing that for well over a century, by using the not-literally-true term "natural selection" to refer to some postulated "survival of the fittest" effect that is not actually selection, because it does not involve a choice by a conscious agent. 

Monday, August 5, 2024

Papers Claiming Brain Memory Storage Keep Citing Poor Science Papers

Let us look at all the flaws, internal contradictions and incorrect statements in a 2022 paper purporting to provide evidence of engrams (cells or matter in the brain which are claimed to store memories).  The paper is entitled "The essence of the engram: Cellular or synaptic?" The second sentence of the paper makes this very untrue claim: "In the last few years it has been shown that simple association memories can be encoded by a subset of the neuronal population called engram cells." This is an example of what I call an achievement legend, which is when scientists boast of having achieved something that was not actually achieved. A careful and sufficiently critical examination of all papers claiming to provide evidence for engrams will show that none of them was good experimental science. Research claiming to provide evidence for engrams is plagued by Questionable Research Practices, and fails to be robust scientific evidence.  

Early on the paper makes this unfounded claim: "Over the last few decades, numerous discoveries have been made regarding the properties of memory ranging from identification of the molecular mechanism(s) underlying memory formation to establishing the different temporal phases of memory." Scientists are actually completely lacking in any understanding of any molecular mechanisms underlying memory formation.  There is no scientist who can even give a credible explanation of how something as simple as the phrase "my dog has fleas" could be stored in a brain.  Contradicting the previously quoted statement, the paper says a bit later,   "the physical basis of memory is elusive."  

We then read this untrue statement: "There is now a substantial body of evidence based on recently developed techniques, including optogeneticschemogeneticselectrophysiology, and multiphoton confocal imaging, to suggest that memory for basic types of behavioral learning such as contextual fear conditioning is maintained in a population of neurons referred to as engram cells [4][5][6][7][8]." Notice the hedging in the language. People often use the term "substantial" when they don't have much of anything. If your mother asks you, "Have you finished your term paper?" you might say, "I've done substantial work" when you haven't done much of anything.  The use of the word "suggest" indicates uncertainty. An examination of the studies referred to shows that none of them establish any robust evidence for any such things as engram cells. 

Let's look at exactly the studies referred to above. Before discussing this, let me list some of the types of study defects I previously listed in my post "The Seven Sins of 'Memory Engram'  Experiments."  The sins I mentioned were as follow:

  • Sin #1: assuming or acting as if a memory is stored in some exact speck-sized spot of a brain without any adequate basis for such a “shot in the dark” assumption.
  • Sin #2: Either a lack of a blinding protocol, or no detailed discussion of how an effective technique for blinding was achieved.
  • Sin #3: inadequate sample sizes, and a failure to do a sample size calculation to determine how large a sample size to test with.
  • Sin #4: a high occurrence of low statistical significance near the minimum of .05, along with a frequent hiding of such unimpressive results, burying them outside of the main text of a paper rather than placing them in the abstract of the paper.
  • Sin #5: using presumptuous or loaded language in the paper, such as referring in the paper to the non-movement of an animal as “freezing” and referring to some supposedly "preferentially activated" cell as an "engram cell." 
  • Sin #6: failing to mention or test alternate explanations for the non-movement of an animal (called “freezing”), explanations that have nothing to do with memory recall.
  • Sin #7: a dependency on arbitrarily analyzed brain scans or an uncorroborated judgment of "freezing behavior" which is not a reliable way of measuring fear.

Here are the papers referenced:

  • Reference 4 in the "Essence of the Engram" paper refers to the paper “Optogenetic stimulation of a hippocampal engram activates fear memory recall.” We see in Figure 3 of that paper that inadequate sample sizes were used. The number of animals listed in that figure (during different parts of the experiments) are 12, 12, 12, 5, and 6, for an average of 9.4. That is not anything like what would be needed for a moderately convincing result, which would be a minimum of 15 or 20 animals per study group. So the study is  guilty of Sin #3. The study is also guilty of Sin #7. The experiment relied crucially on judgments of fear produced by manual assessments of freezing behavior, which were not corroborated by any other technique such as heart-rate measurement. The study does not describe in detail any effective blinding protocol, so it is also guilty of Sin #2. The study is also guilty of Sin #6. The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part of the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs. 
  • Reference 5 in the "Essence of the Engram" paper is a reference to the 2013 study "Creating a false memory in the hippocampus." When we look at Figure 2 and Figure 3 of that  paper, we see that the sample sizes used were paltry: the different groups of mice had only about 8 or 9 mice per group. Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. A sample size calculation would have revealed the defect, but the authors failed to do such a calculation.  No convincing evidence has been provided of creating a false memory. The paper also judged fear in rodents by subjective judgments of "freezing behavior," which is not a reliable way to measure fear in rodents. A reliable way to measure fear in rodent is to measure heart rate, which consistently spikes very sharply when rodents are afraid.  The study also failed to use any blinding protocol. 
  • Reference 6 in the "Essence of the Engram" paper is a reference to the study "Bidirectional switch of the valence associated with a hippocampal contextual memory engram."  We see in that paper 5 or 6 results reported with a borderline statistical significance of only "< 0.05," so this paper is  guilty of Sin #4. No detailed description is given of how an effective blinding protocol was achieved, and only the skimpiest mention is made of blinding, so this paper is guilty of Sin #2.  The study used only "freezing behavior" to try to measure fear, without corroborating such a thing by measuring heart rates.  So the paper was guilty of Sin #7.  The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs.  So the study is also guilty of Sin #6. 
  • Reference 7 in the "Essence of the Engram" paper is to a  paper by Ramirez and Liu  published in Nature, one entitled, “Activating positive memory engrams suppresses depression-like behaviour.” Figure 2 of the paper says that in one group there were only 6 mice used, and elsewhere the paper states that a control group had only 3 mice. These sizes are way below the 15 or 20 animals per study group (control and non-control) recommended as a minimum for a reliable experimental result. The authors claim to have counted differences in the degree to which mice “struggled” when presented with a maze – again something involving a subjective interpretation in which a researcher might tend to see whatever he wants to see. The authors' interpretation of what is going on is speculative. The authors do not present any solid evidence that they actually activated a memory by optogenetic stimulation.
  • Reference 8 in the "Essence of the Engram" paper is to the paper "Memory engrams: Recalling the past and imagining the future." The paper is not a paper that presents original research, but one that simply cites previous research, such as the bad studies mentioned above.  The paper is co-authored by Susumu Tonegawa, who co-authored several of the poorly designed and unreliable studies mentioned above. 

So let's summarize what has gone on in this crucial part of the "Essence of the Engram" paper. The authors have made the claim that there is a "substantial body of evidence" for engrams, and have cited five different papers as references. Not one of those papers presents any robust original research supporting claims that engrams exist. The first four papers cited are low-quality science studies that failed in numerous ways to be good experimental science. The fifth paper is co-authored by the co-author of several of these low-quality science studies, and presents no new research.  This is what goes on all the time in literature referring to engrams.  You have either papers presenting low-quality science experiments, or you have papers that refer to low-quality science experiments.  Nowhere will you find any solid research presenting any robust evidence for engrams.  

 Later in the "Essence of the Engram" paper we read this untrue claim: "Han et al. [14] provided the first causal evidence that engram cells are qualitatively different from non-engram cells."  The reference is to the low-quality science paper "Selective Erasure of a Fear Memory." In Figure 1 of that paper we see a larger-than average sample size was used for two groups (17 and 24), but that a way-too-small sample size of only 4 was used for the corresponding control group. You need a sufficiently high number of animals in all study groups, including the control group, for a reliable result.  The same figure tells us that in another experiment the number of animals in the study group were only 5 or 6, which is way too small. Figure 3 tells us that in other experiments only 8 or 9 mice were used, and Figure 4 tells us that in other experiments only 5 or 6 mice were used. So this paper is guilty of Sin #3. No mention is made in the paper of any blinding protocol, so this paper is guilty of Sin #2. Figure 4 refers to two results with a borderline statistical significance of only "< 0.05," so this paper is also guilty of Sin #4.  The paper relies heavily on judgments of fear in rodents, but these were uncorroborated judgments based on "freezing behavior," without any measure of heart rate to corroborate such judgments. So the paper is also guilty of Sin #7. 

I could go on and on here, but you can get the idea. What goes on in these kind of review papers are endless references to defective, poorly designed research. 

Contrary to the previous claims I quoted from the paper "The essence of the engram: Cellular or synaptic?" is the paper's statement that "the physical form of memory is elusive," and also the paper's statement that "it remains unclear however whether the engram is essentially cellular in nature or whether it is best described in terms of the changes in synaptic strength of contacts made onto and by engram cell."  Such statements show that the authors are just engaging in hand waving and guesswork. It's kind of like someone saying, "I know there are extraterrestrial spaceship bases in the solar system, but I don't know whether they are on the moon or on Mars." 

We then have a repetition of the groundless legend that some MIT group headed by Tonegawa did something to help establish the reality of engrams in the brain.  See my post here for why such claims are groundless. 

The paper then has about 10 diagrams that look like the ones below, with different color variations:



We have the caption "Different phases of memory formation and retrieval." The circles represent neurons, and the lines represent synapses. There are two line thicknesses, and we are told one thickness represents "weak synaptic strength," and the other thickness represents "strong synaptic strength." As a diagram trying to depict a neural formation of memories, the diagrams are a joke.  A neuron has an average of about 7000 synaptic connections with other neurons. And synapses can have any of a thousand different strengths. There is no way even the simplest learned information could be stored in the brain through anything like the shown diagrams. 

It is, in general, wrong to try to explain information storage by appealing to a mere process of strengthening. Strengthening is not storage. We know of many ways in which information can be stored, and none of them are cases of strengthening.

Below are some examples:
  1. People can store information by writing using a paper and pen. This does not involve strengthening.
  2. People can store information by using a typewriter to type on paper. This does not involve strengthening.
  3. People can store information by drawing pictures or making paintings. This does not involve strengthening.
  4. People can store information by taking photographs, either by using digital cameras, or old-fashioned film cameras. In neither case is strengthening involved.
  5. People can store information by using tape recorders. This does not involve strengthening.
  6. People can store information by using computers. This does not involve strengthening.
So basically every case in which we are sure information is being stored does not involve strengthening. What sense, then, does it make to claim that memory could be stored in synapses through strengthening?

In all of the cases above, information is stored in a rather similar way. Some unit capable of making a particular type of impression or mark (physically visible or perhaps merely magnetic) moves over or strikes a surface, and a series of impressions or marks are made on the surface. Such a thing is not at all a process of strengthening.

Consider a simple example. You have a friend named Mary, and you one day learn that Mary has a black cat. Now let us try to imagine this knowledge being stored as a strengthening of synapses. There is no way we can imagine such knowledge being stored by a strengthening of synapses. If you happened to have stored in your brain the knowledge that Mary has a black cat, it could conceivably be that a strengthening of synapses might allow you to more quickly remember that Mary has a black cat. But there is no way that the fact of Mary having a black cat could be stored in your brain through a strengthening of synapses.

Similarly, a simple example of a new memory (often tested in neuroscience experiments) is when a mouse is trained to fear a shock plate. There is no way we can imagine such knowledge being stored by a mere strengthening of synapses.

On and on the paper "The essence of the engram: Cellular or synaptic?" goes, continuing again to make observational claims that are not well founded.  An example is its claim, "Studies of engram cells have greatly expanded our understanding of the mechanisms underlying memory, but several questions remain unanswered." No, there have been no studies showing that there are any such things as "engram cells," and neuroscientists have no understanding whatsoever of any physical mechanism underlying memory.  Our neuroscientists are simply guilty of pretending to understand things they don't have any understanding of.  

The lack of any scientific basis becomes apparent when we look at how such papers define an "engram cell."  We read of no special characteristic of such a cell, such as a different appearance. We read of no physical change going on to make a cell an "engram cell." So how is an engram cell defined by such papers? It is typically defined as a cell that is part of some group of neurons and synapses that undergo "increased activation" when a memory is retrieved.  Almost all neurons in the brain transmit nerve impulses continuously, largely in a random fashion. So anyone scanning activity levels in the brain will always be able to find various cells in various parts of the brain having "higher activation." And anyone scanning the strengths of synapses will be able to find that a certain percentage (say 5%) are stronger than other ones.   So limiting yourself to checking electrical activity levels and synapse strengths, what is the difference between the observational result expected under the nonexistence of engrams (no brain storage of memories) and the observational result expected under the existence of engrams (brain storage of memories)? There isn't one. 

Our "Essence of the engram" paper then tells us, "Recent studies suggest that reactivation of engram cells induces the retrieval of memory and vice versa [4][34][42][97][98]. " I already explained in the bullet list above why the Reference 4 is to a junk science study. Here are the other   papers referenced:

  • Reference 34 is to the 2015 paper "Engram cells retain memory under retrograde amnesia." When we look at the end of the supplemental material, and look at figure s13, we find that the experimenters were using a number of mice that was equal to only 8 in one study group, and 7 in another study group.  Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. The paper failed to use a blinding protocol, an essential for a paper like this to be taken seriously.
  • Reference 42 is the 2016 paper "Memory retrieval by activating engram cells in mouse models of early Alzheimer’s disease."  The paper states that “No statistical methods were used to predetermine sample size.” That means the authors did not do what they were supposed to have done to make sure their sample size was large enough. When we look at page 8 of the paper, we find that the sample sizes used were merely 8 mice in one group and 9 mice in another group. On page 2 we hear about a group with only 4 mice per group, and on page 4 we hear about a group with only 4 mice per group. Such a paltry sample size does not result in any decent statistical power, and the results cannot be trusted, since they very easily could be false alarms. The study therefore provides no convincing evidence of engram cells.
  • Reference 97 refers to the paper "Encoding of contextual fear memory in hippocampal–amygdala circuit." It's a junk science paper that used way-too-small study group sizes of only 6 mice and 11 mice.  The paper also used judgments of rodent freezing behavior to try to measure fear in animals, and all papers that use that faulty technique are junk science papers, for reasons I discuss at length hereThe paper failed to use a blinding protocol, an essential for a paper like this to be taken seriously.
  • Reference 98 refers to a paper that describes no original research, but merely references work by others. 

It is very clear what is occurring in papers such as this one.  Triumphal narratives are being given of scientists making progress in understanding a physical basis of memory. When we closely examine the papers that are cited to back up these boasts, we find that they are invariably weak shoddy studies guilty of Questionable Research Practices.

Citation of Junk Science Studies

This is largely how false narratives are perpetuated in science: by people citing poorly designed scientific research, and claiming that such research showed something, when the research failed to show any such thing because the research was so poorly done. 

The idea that human memories (which can last for 60 years) are stored in synapses (as maintained by the "Essence of the engram" paper criticized above) is untenable because synapses are so unstable, and are built from protein molecules that only last an average of a few weeks or less. An individual synapse and a dendritic spine do not last for years, and consist of proteins that only last for two weeks or less.  A 2019 paper documents a 16-day examination of synapses, finding "the dataset contained n = 320 stable synapses, n = 163 eliminated synapses and n = 134 formed synapses."  That's about a 33% synapse disappearance rate over a course of 16 days. The same paper refers to another paper that "reported rates of [dendritic] spine eliminations in the order of 40% over an observation period of 4 days." 

An additional reason for rejecting the synaptic theory of memory storage is that according to such a theory a memory could only be formed after a synapse was strengthened by proteins (something requiring at least minutes for protein synthesis). But humans can form a new memory instantly. Imagine if someone walks into your workplace naked or firing a gun. It wouldn't take you minutes to form a permanent memory of that. The memory would form instantly. But new proteins (such as would be needed to strengthen a synapse) could never form instantly. We know that the synthesis of new proteins requires minutes.  If forming new memories required the synthesis of new proteins, the brain would never keep up with sensory experiences which keep coming at you continuously. I can watch a 30-minute television drama, and then tell you every major thing that happened in the show. I wouldn't be able to do that if each new thing I saw required the synthesis of a new protein which required several minutes. 

In his Nautilus post “Here's Why Most Neuroscientists Are Wrong About the Brain,” C. R. Gallistel (a professor of psychology and cognitive neuroscience) points out the absurdity of thinking that mere changes in synapse strengths could store the complex information humans remember. Gallistel writes the following:

"It does not make sense to say that something stores information but cannot store numbers. Neuroscientists have not come to terms with this truth. I have repeatedly asked roomfuls of my colleagues, first, whether they believe that the brain stores information by changing synaptic connections—they all say, yes—and then how the brain might store a number in an altered pattern of synaptic connections. They are stumped, or refuse to answer....When I asked how one could store numbers in synapses, several became angry or diverted the discussion with questions like, 'What’s a number? ' ”

What Gallistel describes sounds dysfunctional: a pretentious neuroscientist community that claims to understand how memory can be stored in a brain, but cannot give anything like a plausible answer to basic questions such as “How could a number be stored in a brain?” or “How could a series of words be stored in a brain?” or “How could a remembered image be stored in a brain?” Anyone who cannot suggest plausible detailed answers to such questions has no business claiming to understand how a brain could store a memory, and also has no business claiming that a brain does store episodic or conceptual memories.

structure of bad science paper
                        Click to see left column more clearly

Postscript: The paper "Prevalence of Mixed-methods Sampling Designs in Social Science Research" has a Table 2 giving recommendations for minimum study group sizes for different types of research. The minimum subjects for an experimental study are 21 subjects per study group. Most of the studies mentioned above are experimental studies that used only about half of this minimum number.  The "case study" type mentioned below is a different type of study in which you merely document one or a few occurrences of some condition or situation, without trying to show a cause. 

minimum sample sizes