Today's neuroscience is guilty of promoting many a groundless triumphal legend. One of those groundless socially-constructed triumphal legends is the clam that researcher Susumu Tonegawa did something to show a physical basis for memory. Tonegawa recently died, and the Transmitter magazine has a worshipful article repeating some of these groundless legends.
The article starts out by quoting false boasts about the very low-quality 2015 paper "Engram cells retain memory under retrograde amnesia" co-authored by Tonegawa. The boasts are made by a co-author of the paper. When we look at the end of the supplemental material, and look at figure s13, we find that the experimenters were using a number of mice that was equal to only 8 in one study group, and 7 in another study group. Such a paltry sample size does not result in any decent statistical power, and we should have no confidence in any paper using such way-too-small study groups sizes. The paper failed to use a blinding protocol, an essential for a paper like this to be taken seriously. The paper had a thorough reliance on an utterly reliable technique for trying to judge recall in rodents: the worthless method of trying to judge "freezing behavior." All research papers relying on that method are examples of junk science, for reasons I thoroughly explain in my post here.
Google Gemini infographic (pardon its spelling errors)
Next the Transmitter article makes this untrue claim: "In a series of papers in the 2010s, Tonegawa and his team showed that simply activating a subset of cells via optogenetics could reactivate a memory, change its valence and even create a false memory." The claim has no basis in fact.
Let's take a look at some of the schlock work that Tonegawa and his collaborators produced on this topic, all of which is very low-quality research work utterly unworthy of praise:
- The reference in the quote above to reactivating a memory is a reference to the very low-quality 2012 paper "Optogenetic stimulation of a hippocampal engram activates fear memory recall." Figure 2 tells us that in one of the groups of mice there were only 5 mice, and that in another group there were only 3 mice. Figure 3 tells us that in two other groups of mice there were only 12 mice. Figure 4 tells us that in some other group there was only 5 mice. Such paltry sample sizes does not result in any decent statistical power, and the results are no good evidence of anything, because the study group sizes are way-too-small for any reliable result to be claimed. The paper has a very big reliance on the use of "freezing behavior" judgments, not a reliable method for measuring fear or recall in rodents. No convincing evidence has been provided of artificially activating a fear memory by the use of optogenetics.
- The reference in the quote above to creating a false memory is a reference to the very low-quality science 2013 paper "Creating a False Memory in the Hippocampus" co-authored by Tonegawa, which you can read here. When we look at Figure 2 and Figure 3, we see that the sample sizes used were paltry: the different groups of mice had only about 8 or 9 mice per group. Such a way-too-small sample size does not result in any decent statistical power, so the results have no weight, because the study group size is way too small for any reliable result to be honestly claimed. The paper had a thorough reliance on an utterly reliable technique for trying to judge recall in rodents: the worthless method of trying to judge "freezing behavior." The paper makes no use of a blinding protocol, an essential for a paper like this to provide serious evidence of an effect. No convincing evidence has been provided of creating a false memory.
- The claim in the quote above about switching a valence is a reference to the 2015 low-quality science study "Bidirectional switch of the valence associated with a hippocampal contextual memory engram" co-authored by Tonegawa. We see in that paper 5 or 6 results reported with a borderline statistical significance of only "< 0.05," so this paper is guilty of p-hacking. No detailed description is given of how an effective blinding protocol was achieved, and only the skimpiest mention is made of blinding, so this paper fails to convince us effective blinding was achieve. The study used only "freezing behavior" to try to measure fear or recall, without corroborating such a thing by measuring heart rates. So the paper has done nothing to reliably measure fear or recall in the mice it studies. The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice "freeze" after stimulation, that this is a sign that they are recalling some fear memory stored in the part of the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs. Some of the study group sizes are good, but others are too-small. Because of all of these problems, no reliable evidence has been produced of a brain storage of memory in mice.
- Tonegawa co-authored the 2016 paper "Memory retrieval by activating engram cells in mouse models of early Alzheimer’s disease." This very low-quality paper states that “No statistical methods were used to predetermine sample size.” That means the authors did not do what they should have done to make sure their sample size was large enough. When we look at page 8 of the paper, we find that the sample sizes used were merely 8 mice in one group and 9 mice in another group. On page 2 we hear about a group with only 4 mice per group, and on page 4 we hear about a group with only 4 mice per group. Such a paltry sample size does not result in any decent statistical power, and we should have no confidence in any paper using such way-too-small study groups sizes. The paper had a thorough reliance on an utterly reliable technique for trying to judge recall in rodents: the worthless method of trying to judge "freezing behavior." The study therefore provides no convincing evidence for any of its main claims.
- A Tonegawa study claimed to have “identified engram cells” in the prefrontal cortex. It was a study entitled “Engrams and circuits crucial for systems consolidation of a memory.” In Figure 1 (containing multiple graphs), we learn that the number of animals used in different study groups or experimental activities were 10, 10, 8, 10, 10, 12, 8, and 8, for an average of 9.5. In Figure 3 (also containing multiple subgraphs), we have even smaller numbers. The numbers of animals mentioned in that figure are 4, 4, 5, 5, 5, 10, 8, 5, 6, 5 and 5. None of these numbers are anything like what would be needed for a moderately convincing result, which would be a minimum of 15 or 20 animals per study group. No detailed description is given of an effective blinding protocol. The study relies on judgments of freezing behavior of rodents, which is not a reliable way of measuring fear or recall in rodents.
- Tonegawa co-authored a study "Brain-wide mapping reveals that engrams for a single memory are distributed across multiple brain regions." It is a very low-quality piece of work using way-too-small sizes such as only 7 mice and only 9 mice. The study relies on judgments of freezing behavior of rodents, which is not a reliable way of measuring fear or recall in rodents.
- The church-like neuroscientist belief community (in which professors serve like priests) is a community that has long been addicted to very low quality methods of neuroscience research, largely so that it can maintain the illusion that its cherished but easily debunked belief dogmas are true.
- Junk cognitive neuroscience research is more the rule than the exception these days, even at laboratories of major universities such as Harvard and MIT.
- Leading journals such as Nature and Cell are routinely publishing very low-quality research in cognitive neuroscience. Such journals do not have published research standards guaranteeing high-quality research work in neuroscience.
- Peer-reviewers of submitted papers in cognitive neuroscience tend to be other researchers following research practices as bad as the methods of papers they are asked to review. Such peer-reviewers don't like to reject papers for being guilty of the same research methodology sins that the peer reviewers themselves are guilty of. So peer review does very little to prevent the publication of junk low-quality studies making false claims in their titles and abstracts.
When "freezing behavior" estimations go on, things typically work like this. A rodent will be trained to fear some thing such as a shock plate that gives the rodent a shock when the rodent steps on it. Then later the rodent will be placed in a cage that includes the fear stimulus such as the shock plate. The researchers will attempt to record what percentage of some time (say, a minute or 3 minutes) that the rodent was immobile when placed in such a case. This will be called a "freezing percentage," and will be claimed as a measure of how well the rodent remembered the fear stimulus.
The technique makes no sense. In the real world, rodents don't usually freeze and become immobile when they are afraid. They are much more likely to flee. I know that from years of observing how mice act in the presence of shrieking humans, in an apartment where mice would occasionally appear. So trying to judge recall of a fearful stimulus by judging how much time a rodent was immobile in a cage makes no sense as a way of measuring fear or recall. The thing that utterly destroys the credibility of all "freezing behavior" graphs is that they can be produced in any of more than a dozen ways. A researcher can put a rodent in the cage for three minutes and graph the whole three minutes. Or he can graph only the first 30 seconds, or only the first minute, or only the first two minutes. In each ten seconds of such a three minutes, the researcher can count it as "moving" if the rodent moves one second during that period; or the researcher can count two seconds of movement as being mobility; or the researcher can use three seconds, or four seconds, or five seconds.
There are no prevailing standards for how "freezing behavior" is judged. With there being a dozen different possibilities of how "freezing behavior" can be judged and graphed, with each having a possibility of success of about 50% (and with pre-registration -- a commitment to an exact methodology before gathering data -- being rare in neuroscience, as a neuroscientist recently confessed), it will be almost certain that the researcher will be able to choose some analysis method that will show the desired difference in "freezing behavior," even if the memory intervention being tested had no real effect. This is a large part of the reason why "freezing behavior" judgments are worthless as evidence for an increase or decrease in memory in rodents.













