The arXiv science paper server at https://arxiv.org/ is a widely used resource for finding and reading scientific papers. On its home page we read, "arXiv is a free distribution service and an open-access archive for 1,780,158 scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics." It has become something of a custom for physicists to upload "preprints" of physics papers to this server. Although mainly associated with physics papers, the server also has a huge number of quantitative biology papers.
An interesting way to use the arXiv server is simply to search for a topic, and see how high the paper count is (in other words, how many papers the server has on a particular topic). Such a method gives a rough idea of how much work has been done on a particular topic. It is not at all true that you can prove something is really science by doing a search for some topic and getting a high paper count. For example, when I search for papers with the word "string" in the title, on October 23, 2020, I get a count of 12,766 papers, a large fraction of which are papers expounding versions of string theory. But string theory is a speculative edifice that is not at all "science with a capital S," and has no observational basis.
While we can't tell that something is science just by searching for a topic and getting a high paper count, if we search for a topic and get a very low count, that is a reason for suspecting that the topic may not be any such thing as "science with a capital S." That's what happens when I search for the topic of "engram." An engram is an alleged brain location where a memory is stored, or some kind of "memory trace" in the brain. When I search for papers having "engram" or "engrams" in their title, using the arXiv science paper server, the server gives me a count of 0 such papers.
Could it be that the arXiv science paper server just doesn't have many papers on biology? No, it has tons of papers on quantitative biology. Below are a few examples of paper counts when I search for some biology topics:
|
Topic |
Number of papers on arXiv server having that topic in their title |
|
cancer | 1115 papers |
|
COVID-19 | 1738 papers |
|
brain | 2046 papers |
|
tissue |
708 papers |
|
engram |
0 papers |
|
engrams |
0 papers |
So how come the server gives us no papers when we search for "engram" as the topic? Maybe it's because engrams aren't really science with a capital S.
There's another way to do a search on the arXiv server. You can search for any use of the search topic in the abstract of the paper. When I do such a search, I get only 5 papers. Four of the five papers have no solid observational grounding, and are the kind of mathematical speculation papers that scientists write when they attempt to substantiate very doubtful speculations such as string theory or dark energy or primordial cosmic inflation. The only paper built upon observations is a paper entitled "Recording and Reproduction of Pattern Memory Trace in EEG by Direct Electrical Stimulation of Brain Cortex." The paper does not actually provide robust evidence that any such thing as a memory trace was detected. To do such thing, you would need to have a study group of at least 15 animals, but we read in the paper that "the experiments were performed on 5 outbred male rats." Using such a too-small study group, you have too high a chance of a false alarm.
There is another "preprint paper server," one more oriented toward biology papers. It is called bioXriv, and bills itself as "the preprint server for biology." When I use that server to look for papers that contain "engram" in the title, I get only 6 papers. Below is a comparision with other topics:
Topic | Number of papers on biorXiv server having that topic in their title |
cancer | 2777 papers |
COVID-19 | 376 papers |
brain | 2651 papers |
tissue | 1021 papers |
engram | 6 papers |
|
engrams |
8 papers |
The first of these six papers using "engram" in its title is a speculative paper with no observational grounding. The second of these six papers uses study group sizes of only 5, which are way too small to provide any robust result. The third paper has a similar problem, using study group sizes of only 8, way too small to provide any robust result. The fourth paper is a mouse study that fails to mention anywhere how many mice were used, which typically occurs only when some way-too-small study group size was used. The fifth paper suffers from the same problem, the only difference being that it vaguely suggests that way-too-small study group sizes of only 4 were used. The sixth paper uses way-too-small study group sizes of only about six.
Now let's look at the eight papers using "engrams" in their title. The first paper has "schematic" visuals based on imaginary hypotheticals. The second paper tries to use the word "engrams" as much as it can, but provides no physical evidence for such a thing. The third paper was a rodent study using study group sizes of only about 8, way too small for a robust result. The fourth paper was a rodent study using study group sizes of only about 5, way too small for a robust result. The paper confesses, "Data collection and analysis were not performed blind to the conditions of the experiments," a major procedural defect. The fifth paper is a theoretical paper not providing any observational results. The sixth paper and the seventh paper used way-too-small study group sizes of only 5. The eighth paper is merely a theoretical work based on mathematical simulations.
So the only six papers on the biorXiv server mentioning "engram" in their title fail to provide any robust evidence of engrams. Its the same thing for the 8 papers using "engrams" in their title. All in all, we have in these very low server counts (and the weaknesses of the papers coming up in the searches) a strong suggestion that engrams (supposed neural storage sites for memories) are not any such thing as well-established science, and that the evidence for engrams is merely very weak evidence rarely conjured up by scientists clumsily trying to provide some evidence for something they want to believe in. Engrams are not an example of science with a capital S.
My criticisms of such papers for using too small study group sizes is partially based on the guideline in the paper "Effect size and statistical power in the rodent fear conditioning literature – A systematic review," which mentions an "estimated sample size to achieve 80% power considering typical effect sizes and variances (15 animals per group)," and says that only 12% of neuroscience experiments involving rodents and fear met such a standard.
None of these papers I have referred to (on either preprint server) claims to have used a blinding protocol for both data gathering and data analysis. Most of them make no claims about blinding, which is usually a sure sign that no blinding protocol was followed. One paper makes a brief claim to have used a blinding protocol for experimentation, but makes no such claim for data analysis. Another paper claims briefly to have used a blinding protocol for statistical analysis, but makes no such claim in regard to experimentation and data gathering. None of these papers describes in detail a specific blinding protocol.
When blinding protocols are not thoroughly implemented, there is a large chance of bias and scientists reporting hoped-for effects that are not really there. Unless a paper describes in detail a blinding protocol, you should be rather skeptical that any halfway-decent blinding protocol was used. Similarly, if someone says, "I paid all my taxes," but doesn't release his tax forms, you should be rather skeptical that he did pay all his taxes.
The failure of experimental neuroscientists to adequately follow blinding protocols is a huge problem in contemporary neuroscience research, as big as the failure of most such neuroscientists to use adequate study group sizes. Be suspicious of junk science wherever you find experiments not using proper blinding protocols. A PLOS Biology article tells us, "Recent analyses have found, for example, that 86%–87% of papers reporting animal studies did not describe randomisation and blinding methods, and more than 95% of them did not report on the statistical power of the studies to detect a difference between experimental groups."
Ian Stevenson MD once made some candid comments relevant to the topic of engrams, stating this:
"Neuroscientists and psychologists cannot tell us either how we store memories or how we retrieve them. Suggestions that experiences leave 'traces' in the brain (whether in altered neural networks or otherwise) have not so far led to further understanding."
No comments:
Post a Comment