Neuroscientists Keep Writing Papers Bluffing Us About Memory Encoding

 " We do not yet have a systematic theory of how information is encoded in the chemical and electrical activity of neurons, how it is fused to determine behavior on short time scales, and how it is used to adapt, refine, and learn behaviors on longer time scales. " -- "BRAIN 2025: A Scientific Vision," document co-authored by many scientists (link). 

If we are to believe in the claim that brains store human memories, we must have a credible account of four things: encoding, neural storage of very old memories, the instantaneous formation of memories, and the instantaneous retrieval of memories. The theory that human memories are stored in the brain fails in regard to each of these things.

There exists no plausible theory as to how a brain could store memories lasting for 50 years, but we know humans can remember many things for that long. The most popular idea of brain memory storage claims that memories are stored in synapses, but the proteins in synapses have an average lifetime of less than two weeks, meaning such a theory falls short by a factor of 1000 when it comes to explaining memories that persist for 50 years. As for memory retrieval, there is no theory explaining how humans could possibly recall instantly things they learned many years ago, and haven't thought about in years. 

You may hear the name of some obscure historical or cultural figure you learned about decades ago, and haven't heard about or thought about since that time. You may then instantly recall something about that person. But if that memory was stored somewhere in your brain, how could you instantly find the exact little location where that memory was? Doing that (for example, instantly finding a memory in storage spot 834,220 out of 1,200,000) would be like instantly finding a needle in a mountain-sized haystack. If a brain had an indexing system, or a coordinate system, or a neuron numbering system, there might be a faint hope for explaining instantaneous memory retrieval; but the brain has no such things. As for the instantaneous formation of memories, there is no theory that can account for it in a brain. The prevailing theory that memories are stored by synapse strengthening (which would involve protein synthesis requiring minutes) fails to account for memories that humans can form instantly.

When we consider the issue of memory encoding, we find a difficulty as great as the difficulties just discussed. Encoding is supposedly some translation that occurs so that a memory can be physically stored in a brain, so that it might last for years. The problem is that human memories include incredibly diverse types of things, and we have no idea how most of these things could be stored as neural states. Consider only a few of the types of things that can be stored in a human memory:

• Memories of daily experiences, such as what you were doing on some day
• Facts you learned in school, such as the fact that Lincoln was shot at Ford's Theater
• Sequences of numbers such as your social security number
• Sequences of words, such as the dialog an actor has to recite in a play
• Sequences of musical notes, such as the notes an opera singer has to sing
• Abstract concepts that you have learned
• Memories of particular non-visual sensations such as sounds, food tastes, smells, pain, and physical pleasure
• Memories of how to do physical things, such as how to ride a bicycle
• Memories of how you felt at emotional moments of your life
• Rules and principles, such as “look both ways before crossing the street”
• Memories of visual information, such as what a particular person's face looks like

How could all of these very different types of information ever be translated into neural states so that a brain could store them?

Our neuroscientists have told us again and again that the brain does such an encoding, but there is no real evidence that any such thing takes place. What we have evidence for is merely evidence that humans remember things. If you are someone who believes that memories are physically stored in brains, then you may claim that memory encoding occurred at such and such a rate whenever you observe people learning something at such and such a rate. But merely observing evidence of learning or memory is not acquiring any actual evidence that encoding has occurred. There remains the possibility that our memories are not stored as neural states, the possibility that our repository of memory is some spiritual or psychic facility that is non-neural and non-biological.

Such a possibility should not seem remote when we consider that there is no workable theory as to how learned knowledge and experiences could be encoded so that they might be stored in a brain. No matter what theory we may create to account for the encoding of learned knowledge and episodic experience so that they can be stored in a brain, such a theory will always end up sounding ridiculous after we examine the theory in detail and consider its requirements and shortcomings. Let's look at some possibilities, and why they fail.

Theory #1: Direct writing of words and images

First, let's consider the simplest theory of encoding we can imagine – that a memory is stored in the brain so that it appears in a neural form pretty much as we see it in our minds. Under this theory, when you memorized some series of words, this would cause a sequence of microscopic little letters to become stored in your brain; and when you experienced some visual experience, this would get stored as some tiny little image in your brain. So, for example, under this theory, if someone memorized the sentence, “There may be green aliens in the center of the galaxy,” then after the person died, some scientist might examine that person's brain with an electron microscope, and actually find some tiny little words in some neurons, words that directly spelled out, “There may be green aliens in the center of the galaxy.” And under this theory, if someone was given a picture of a toy purple pony, and asked to memorize it, then after the person died, a scientist might be able to examine the person's brain under an electron microscope, and the scientist might say, “Aha, I see in his neurons a tiny little image of a toy purple pony.”

This theory may immediately provoke giggles, and it is rather easy to think of some reasons why it does not work. They are these:

1. If memory worked in such a way, we would surely have already discovered such easily-recognizable memory traces. But no such things have been seen, even though a great deal of human neural tissue has been examined at very high magnification. When we look at brain tissue at the highest magnification, we see no tiny little letters or tiny little images of animals, cars, and persons.
2. For a brain to be able to write words that we memorized in this type of direct manner, it would seem that the brain would need some very precise write mechanism, capable of forming the exact characters of the alphabet in brain tissue; but no such brain capability is known to exist.
3. It seems that if such a theory were true, recalling some words would be like reading. But recalling words is almost never like reading, and we don't see in our mind's eye some stream of letters as we recall some words we memorized.
4. For a brain to be able to read words that we memorized in this type of direct manner, it would seem that the brain would need some very precise reading mechanism, capable of reading the exact characters of the alphabet stored in very tiny letters written in brain tissue; but no such thing is known to exist. We don't have tiny little “micro-eyes” in our brains that might allow us to read tiny microscopic letters stored in our brains.
5. Alphabets of modern languages are relatively recent inventions. For example, the English alphabet is not many  thousands of years old. There has no been no change in brain structure for many thousands of years, and we can think of no  reason why brains would have some ability allowing it to write an alphabet that only came into existence centuries or a few thousand years ago, not many thousands of years ago. 

Theory #2: Brain storage of words and images using some unknown non-binary coding or translation protocol

Now, let's consider a different theory of memory encoding – the idea that instead of directly storing words and images (so that we could directly read the words and directly see the images), the brain uses some type of unknown coding or translation protocols. For example, it could conceivably be that words that we learn are somehow translated into proteins or chemicals or electrical states, using some as-of-yet undiscovered translation scheme.

For example, such a scheme might work a little like this:

Item	How the item might be represented
Letter “A”	Some particular neural arrangement of atoms, chemicals or electricity
Letter “B”	Some other neural arrangement of atoms, chemicals or electricity
Letter “C”	Some other neural  arrangement of atoms, chemicals or electricity

Such a scheme might work a little like the Morse code, in which particular letters are translated into some sequence of dots, dashes, or dots and dashes. Some particular arrangement of atoms, chemicals or electricity might work like a dot in the Morse code, and some other particular arrangement of atoms, chemicals or electricity might work like a dash in the Morse code.

Or there could be some higher-level translation system based on particular words rather than letters. For example, we can imagine something like this:

Item	How the item might be represented
Word “sun”	Some particular neural arrangement of atoms, chemicals, proteins or electricity
Word “man”	Some other neural arrangement of atoms, chemicals, proteins or electricity
Word “move”	Some other neural  arrangement of atoms, proteins chemicals or electricity

There is one giant problem with such a theory. All of the languages that we use are fairly recent innovations, having been created in only the last few percent of the time that humans have existed. For example, back in the Roman Empire people used Latin, but the English we use today has only been in use for less than 1200 years. The alphabet used for English is less than 1000 years old, and its alphabetic predecessor (the Latin alphabet) is only a few thousand years old. It is generally acknowledged even by Darwinism enthusiasts that very complex evolutionary innovations cannot arise in only a few centuries of time or a few thousand years. So we could never explain how the brain could naturally possess some elaborate translation system based on such a relatively recent innovation as the English language and the English alphabet.

Scientists strain our credulity whenever they talk about novel functional genes accidentally appearing even over the course of a million years. Think, then, on how much greater a problem there would be in explaining how hundreds of novel functional genes could have appeared in less than 3000 years, to perform some translation operation involving characters and words that have existed for less than 3000 years. To assume such a thing would be to assume evolution working thousands of times faster than the rate we would predict from known mutation rates.

There is also no evidence that any such great burst of genetic novelty has occurred. Although the half-life of DNA is 512 years, we have enough samples of human DNA from ancient Rome and ancient Egypt to know that there has been no big change in the DNA of humans during the past 3000 years. So it seems impossible that there could be any genetic capability (arising in the past few thousand years) that would allow humans to neurally store information using some encoding mechanism specifically tailored to the letters and words of the English language that have existed for less than 3000 years.

Another difficulty with the theory of encoding just mentioned is that if it existed, we would see big differences in the genes of people who spoke different languages. According to such a theory, we would expect that Chinese people would have one group of genes corresponding to proteins or RNA molecules needed to translate Chinese words into neural states, and that English speaking people would have some other quite different set of genes corresponding to proteins or RNA molecules needed to translate English words into neural states (particularly since the Chinese language and alphabet is so different from the English language and alphabet). But there exists no such difference in the genes of Chinese speaking people and English speaking people.

There is also the difficulty that there is no sign in the human genome that any such genes exist for performing such an elaborate operation of encoding human learned knowledge and episodic experience so that it can be stored in neurons or synapses (and there would need to be many hundreds or thousands of types of genes dedicated to performing such a task if it was done).

Theory #3: Binary writing of words and images

Now, let's consider a theory of memory encoding that perhaps the words we memorize and the images we remember are stored in binary format. We know that computers store information in binary format, so when it is suggested that the brain may use a similar format, this may sound reasonable to the average person (although it isn't, a brain being radically different from an electronic computer).

This possibility actually has all of the difficulties of the previous possibility. What goes on when your computer stores words in binary format is the following:

1. First individual letters in the words are converted into decimal numbers (such as 13, 19, and 23) using a particular translation table called the ASCII code.
2. Then, those numbers are converted from decimal to binary using a decimal-to-binary conversion routine.

So if we are to believe that the brain does binary encoding like a computer, we would need to believe that built into the brain on a low level is some type of translation scheme like the one below, a scheme in which letters are translated into decimal numbers.

ASCII table used by a computer to store encoded information

In addition, we would also have to believe that the brain has some kind of capability to translate the numbers in such a system into binary numbers. Alternately, we could believe that the brain has a scheme for directly translating characters into binary, but the overall complexity of such a translation mechanism would be every bit as great as a system in which characters are converted into decimal, and then into binary.

We have the following difficulties involved with such an idea:

1. If memory worked in such a way, we would surely have already discovered such easily-recognizable memory traces. We would have discovered tiny little traces in the brain that resemble binary coding. But no such things have been seen, even though a great deal of neural tissue has been examined at very high magnification.
2. For a brain to be able to write words that were memorized in this type of direct manner, it would seem that the brain would need some very precise write mechanism, capable of writing binary traces; but no such thing is known to exist.
3. For a brain to be able to read words that we memorized in this type of direct manner, it would seem that the brain would need some very precise reading mechanism, capable of reading in binary; but no such thing is known to exist.
4. Since the alphabets of human languages are only a few thousand years old, there would have been no time for the human body to have evolved some complex biological mechanism capable of converting specific alphabetic characters to binary.
5. We can imagine no way in which a brain could achieve the translation effect in which words are translated into binary. As far as we know, there is nothing anything like an ASCII table in your brain, nor is there anything like a facility for translating English letters directly into binary, nor is there anything like a facility for translating English letters into decimal, and then translating decimal numbers into binary. There are no genes in the genome that perform such tasks.

The Scientists Who Keep Falsely Talking About Memory Encoding and Memory Representations in Brains

The simple fact is that there is not the slightest evidence of memory encoding or memory representation in the human brain. So why it that we keep getting scientific papers claiming to have seen memory encoding or memory representation in the human brain? It's pretty much a "they're talking like they had what they need to have but don't have" kind of affair.  Something similar might be going on if some street beggar were to try to attract women by frequently using finance terms. So someone without $20 to his name might speak like this:  "I think that WEALTH MANAGEMENT is very important, and I am slowly BUILDING MY FORTUNE in a way that will give me a HUGE BONANZA OF WEALTH once my INVESTMENT PLANS are actualized."

But how can someone write a paper making frequent references to memory encoding and memory representation in the brain, even though there is no evidence that such things exist?  The visual below shows the general approach typically used. 

A key element is what can be called chance-mining. Chance-mining typically involves grandly trumpeting some little result that might easily be obtained by pure chance.  I can give an example of chance-mining. Suppose you are doing a study trying to prove what we may call "the Abracadabra Hypothesis," which you define as "the hypothesis that saying 'Abracadabra' before flipping a coin will increase the chance of a Heads result." Any test involving a sufficiently large sample size will fail to support this Abracadabra Hypothesis. That's because something called the Law of Large Numbers says that the more random trials you do, the more the result will tend to match the result expected by chance. But it is very easy to get a test result deviating from what is expected by chance, if you use a very small sample.  For example, it is easy to flip ten coins and get 6 or 7 Heads results, even though the expected chance result is 5.

So if you want to do a paper claiming to support the Abracadabra Hypothesis, you can try a test using a small number of coin flips such as only 10 or 20. If that fails, you can file the results in your file drawer, and start a new experiment, also using a small number of coin flips. Probably before long you will have some result you can claim as "statistically significant." Chance-mining like this misleads people, because it creates the impression that there is a causal relation or causal influence when there is none. Saying "Abracadabra" actually has no influence on the result of coin flips. 

Similar chance-mining occurs in quite a few neuroscience papers.  The papers may make quite a few references to neural encoding or neural representation, but they will not be backed up by references to well-designed studies using an adequate sample size.  Typically the paper will present some research that is utterly unreliable for multiple reasons, such as the use of way-too-small study group sizes or unreliable techniques for trying to measure recall in rodents. 

A example of such a paper is the paper here, entitled "Ventral hippocampus neurons encode meal-related memory." The title is unjustified. No robust evidence has been provided of any neural encoding of memory. There are two gigantic flaws in the study:

(1) The study group size is way too small, consisting of only 5 animals. 

(2) No reliable method was used to measure memory performance in rodents. We read that a "foraging-related spatial memory task" apparatus was constructed to test memory in mice. We see a picture of a weird device that the paper does not name. This was not any well-established protocol for testing memory in mice. We fail to have any diagram explaining why such a device would reliably test memory in mice. 

What usually occurs in such papers claiming evidence of "memory encoding" or "memory representation" is that there will be a combination of two main sins, either one of which is to disqualify a paper as good evidence:

(1) Almost always there will be some study group size that is way too-small for any reliable evidence to be honestly claimed 

(2) Usually there will be a reliance on a completely unreliable measurement for estimating fear or recall in rodents, the method of trying to judge "freezing behavior." All papers relying on this method are junk science, for reasons I discuss here. 

What largely goes on in such poorly written science papers is pareidolia.  Pareidolia is what occurs when someone keeps scanning some stream of random data, looking for something he can call a representation. An example is a person who eagerly scans his toast every day, looking for some sign of a representation, and who one day announces he has found the face of Jesus in his toast, or maybe the face of the Virgin Mary, or maybe the face of his dead wife. 

The neuroscientist eagerly scanning brain scan data or brain wave data looking for something he can call a little bit of "representation" or "encoding" is someone who we can no more trust than someone eagerly scanning the clouds every day, looking for something that looks like the shape of an animal. 

An example from 2025 of a pure pareidolia paper is the junk science paper "Movie-watching evokes ripple-like activity within events and at event boundaries." We have scientists looking at EEG readings obtained in ten epilepsy patients while the patients watched movies.  Looking at the ever-changing line squiggles that are EEG readings, the authors claim to have seen "ripples" that they claim occur more often during "event boundaries." The claim that there are such "ripples" in the data is as much see-what-you-want-to-see pareidolia as the claim that there are particular points in movies that are "event boundaries." The supposed identification of these "ripples" occurred not by humans staring at the EEG readings, but by some computer program which is a black box of convoluted spaghetti code. Computer programming is the best friend of the scientist trying to see things that are not really there. You can run some computer program that is supposedly seeking some obscure thing in noisy data; and if the first analysis run fails, you can keep tweaking the code until it reports finding what you were hoping to find. 

"Seeing What Was Not There" Pareidolia of Astrobiologists Resembles "Seeing What Was Not There" Pareidolia of Neuroscientists

 Phosphine is a gas mainly produced on Earth by living things. On September 14, 2020, a scientific paper entitled "Phosphine gas in the cloud decks of Venus" claimed to have detected the "apparent presence" of phosphine in the atmosphere of Venus. The lead author was Jane S. Greaves. The claimed abundance level was very tiny, only about 10 parts per billion. Following their long-standing tendency to hype like crazy any story that may serve as clickbait to produce more page views and advertising revenue, a host of web sites began proclaiming that life or a sign of life had been discovered at Venus.  

I may have been the first one in the blogosphere to make a substantive criticism of this claim, since I published on the next day a post entitled "No, They Haven't Detected Life at Venus," in which I cited several reasons for doubting the idea that the paper had provided any evidence of life at Venus.  I said on September 15, 2020 that an alternate explanation was that "an error in interpretation could have occurred in spectral data that is hard-to-interpret because of overlapping signals from a variety of different gases in the atmosphere of Venus," and that such a possibility was "not very unlikely." I also pointed out the substantial chance that merely geological processes could have produced phosphine, and pointed out the lack of any plausible scenario for life on Venus, given the incredibly hostile conditions both on its hot-enough-to-melt-lead surface and its clouds (having almost no water but lots of sulfuric acid vapor). 

Days after my post, we began seeing on some sites such as www.liveScience.com and the National Geographic web site some articles (such as this one) questioning whether any sign of life had really been discovered on Venus.  On October 19, 2020 there appeared a scientific paper with the title "Re-analysis of the 267-GHz ALMA observations of Venus: No statistically significant detection of phosphine."  The paper doesn't merely question whether evidence of life on Venus has been found. The paper tells us that no robust evidence of phosphine has been discovered in the atmosphere of Venus. 

The five authors of the scientific paper provide a critique of the  September 14, 2020 paper claiming evidence for phosphine in the atmosphere of Venus,  saying that the paper used a dubious statistical technique that "leads to spurious results." The five authors state that the data actually provides "no statistical evidence for phosphine in the atmosphere of Venus." 

Four additional recent papers (published after the widely discussed "phosphine at Venus" paper) said that there is no phosphine in the atmosphere of Venus. One paper by a single author states, "There is thus no significant evidence for phosphine absorption in the JCMT Venus spectra." Another paper with many co-authors is entitled, "No phosphine in the atmosphere of Venus." A third paper states there is "no statistical evidence for phosphine in the atmosphere of Venus."  Another paper says, "These findings, along with the recent papers by Encrenaz et al. (2020), Snellen et al. (2020), Lincowski et al. (2020), and Villanueva et al. (2020) undermine the reported detection of PH3 [phosphine] by Greaves et al. (2020a,b) and its possible biogenic origin."  A news account of this paper says, "The team concluded that what the scientists probably saw was just sulfur dioxide, which is a common gas around Venus and would not indicate the possible presence of life."

What was going on in the Greaves paper claiming phosphine on Venus seems to have been pareidolia fueled by wishful thinking.  Scientists looked at some noisy borderline data produced at the limits of observations, and interpreted the data in a way so that they could claim something they were eagerly hoping to find. Pareidolia typically occurs when someone looks at noisy, hard-to-interpret frequently-changing data, and then claims to see some important thing that is not really there.  The example of someone claiming to have found a Jesus image in his toast is an example of pareidolia. Another example of pareidolia is shown below:

A few years later we had another case resembling the subsequently-discredited report of detecting phosphine on Venus. It was another case of a scientist claiming to have found something of great significance in the field of astrobiology, the search for life in outer space.  In 2023 Nikku Madhusudhan and four other scientists created quite a stir. They authored a paper entitled "Carbon-bearing Molecules in a Possible Hycean Atmosphere." Researching a planet called  K2-18 b revolving around another star, the paper claimed to have found "potential signs of dimethyl sulfide (DMS), which has been predicted to be an observable biomarker in Hycean worlds." The term "Hycean worlds" refers to planets in other solar systems that may be entirely covered by an ocean. The term "biomarker" refers to something that may be a sign of life. A very simple compound, dimethyl sulfide is not any type of building block of life. But on Earth dimethyl sulfide is sometimes produced by life. 

But there were some reasons why the attempt to insinuate a biomarker was very dubious. One reason was that the claims about "potential signs of dimethyl sulfide" was a kind of "reading tea leaves" affair, in which scientists were analyzing the faintest of faint signals, rather like someone squinting at something on the horizon miles away. That type of observation offers plenty of opportunity to see what you want to see, by interpreting marginal hard-to-interpret just-barely-detectable data in some way that fits your cherished desires, rather than interpreting that data in a hundred other ways. 

Then there is the fact that when scientists do observations like this, they are picking up signals from many different chemical sources, with the signals being all mixed up. It's a recipe for false alarms, rather like someone in a very crowded high school cafeteria trying to listen to what someone at a different cafeteria table far away is saying. 

Despite the paper's failure to detect water, and its weak mention of a mere mention of "potential signs of dimethyl sulfide," the world's "give us an inch and we'll take a mile" science news press began publishing a flood of misleading stories falsely claiming that some promising sign of life had been found. After the "sugar rush" of this flood of misleading stories, other scientists got busy examining the data on the distant planet K2-18 b, to see whether there was any decent evidence for dimethyl sulfide. In 2024 scientists produced a paper arguing that K2-18 b was not a "Hycaean" planet covered by an ocean, but instead a gas planet like Neptune with no ocean. The paper was "JWST Observations of K2-18b Can Be Explained by a Gas-rich Mini-Neptune with No Habitable Surface" authored by Nicholas F. Wogan and others. 

Then in early 2025 there was published the paper "A Comprehensive Reanalysis of K2-18 b's JWST NIRISS+NIRSpec Transmission Spectrum." It reanalyzed the data on K2-18 b and says "we find no statistically significant or reliable evidence for CO2 or DMS [dimethyl sulfide]." The paper had 16 authors, as compared to only five authors of Madhusudhan's paper. The 16 authors had found that Madhusudhan's claims about dimethyl sulfide at K2-18 b were unfounded. 

In April 2025 Madhusudhan released a new paper, based on some new observations. He claimed to have found stronger evidence for dimethyl sulfide at the planet K2-18 b. For a few days the world's science news sites spread a Madhusudhan-crafted narrative that the "strongest evidence yet for extraterrestrial life" had been found. But Madhusudhan's claims were soon shot down by other scientists. An An Ars Technica article soon appeared entitled "Skepticism greets claims of a possible biosignature on a distant world." We read this:

"The last issue is whether, if dimethyl sulfide is really present on K2-18b, it was produced by life as it is here on Earth. The answer appears to be 'possibly not': A 2024 paper indicates it's possible to produce the chemical through light-activated reactions."

Referring to Madhusudhan's team, an Atlantic article soon stated this:

"The chemical [dimethyl sulfide] is one of several that could be responsible for the signal they found. And while it's the most likely one according to their models, others disagree." 

The article notes that dimethyl sulfide was found "in the dead, icy spray of a comet," meaning it isn't any reliable biomarker. "Abiotic" refers to something not involving life.  One paper is entitled "On the abiotic origin of dimethyl sulfide: discovery of DMS in the Interstellar Medium." Another paper is entitled "Evidence for Abiotic Dimethyl Sulfide in Cometary Matter." In the Atlantic article we read a quote by astronomer Ignas Snellen stating that Madhusudhan's framing of his research is "irresponsible nonsense."

A National Geographic page interviews some experts about Madhusudhan's recent claims. Some excerpts:

" 'I'm pretty skeptical of this claim, and I wish the press coverage better reflected the skepticism of the astronomical and astrobiological community,' wrote astrobiologist Joshua Krissansen-Totton of the University of Washington in an email....Another researcher, astronomer Ryan MacDonald at the University of Michigan went further, criticizing the three sigma claim as 'statistical hacking' on Bluesky....'The simplest explanation of this planet is a very thick gas-giant atmosphere with no habitable surface,' says exoplanet scientist Nick Wogan of NASA Ames. ...And we already know that nature can produce DMS [dimethyl sulfide] without life. Last year, chemist Nora Hänni at the University of Bern and her colleagues found DMS on comet 67P—not exactly a habitable world. Other researchers have found it in interstellar space. And last year, chemist Eleanor Browne of the University of Colorado, Boulder and her colleagues showed that DMS can be produced in light-fueled chemical reactions in lab experiments with synthetic atmospheres.

'There's no reason to understand [DMS] as a unique consequence of life,' says Mathis. 'I just, for the life of me, cannot figure out exactly what the argument is about: why they think this could even potentially be indicative of life, given that we've seen abiotic sources.' ”

An article at Gizmodo.com quotes some experts discussing Madhusudhan's paper.  Planetary chemist Oliver Shorttle says "I do not believe the report of DMS in the spectrum of K2-18 b moves the astrobiological needle." He states this:

"There is presently no requirement from the data that this planet hosts liquid water oceans and a climate amenable to life. In fact, based on the data there is every reason to believe the climate will be far too hot for liquid water oceans, with the deep atmosphere potentially being underlain by oceans of magma, not liquid water. For this reason, even if 1 and 2 return a DMS detection, our expectation should be that this [molecule] has emerged in a lifeless, hot, sulfur and hydrogen rich atmosphere and ask ourselves what the atmospheric chemistry is that would have enabled this. Believing instead that this is DMS of biological origin would require overturning our every expectation as to the climate of this planet, without any other reason to do this from the data."

In the same article, astrophysicist Ignas Snellen says this:

"The whole thing is completely blown out of proportions.... The research team finds bumps in their spectrum. It is not clear whether these are real, and if so, what they could be caused by. There could be dozens of molecules (if real), or even cloud features. What do the authors do? They just look whether DMS [dimethyl sulfide] could cause this (and add DMDS). They ignore the dozens of other species [i.e. non-biological sources of molecules] that could cause this bump and call it a day. If I had been the referee, I would have stopped this publication right there. There is no reason to invoke astrobiology, let alone call it the biggest breakthrough or whatever....In the long run this will hurt astronomy when nobody will take us seriously anymore."

An NPR story says that  a scientist has analyzed the most recent data from K2-18 b, and has found it has no signal of any kind. We read this:

"The results he got suggested that there's too much noise in the data to draw any conclusions. Rather than seeing a bump or a wiggle that indicated a signal, 'the data is consistent with a flat line,' says Taylor, adding that more observations from the telescope are needed to know what can be reliably said about this planet's atmosphere."

Madhusudhan's overenthusiasm and pareidolia reminds me of the overenthusiasm and pareidolia of another astronomer, Avi Loeb. Harvard astronomer Avi Loeb somehow got the idea that a  2014 meteor (the CNEOS 2014-01-08 meteor) may have been an interstellar spacecraft that blew up high in the sky. Loeb ran a million-dollar oceanic expedition looking for what he hoped would be remnants of a crashed extraterrestrial spaceship, an expedition he organized.  He found no sign of anything looking like a spaceship or any of its parts. Loeb claims to have found tiny round specks only about a millimeter in size. All that he recovered were some tiny metal specks. The metal specks he found are just like metal sea specks found all over the world.  But  Loeb tried to suggest that he may have discovered smithereens of an exploded interstellar spacecraft. 

There was nothing special about the specks Loeb and his team gathered (as I discuss here), and there is nothing special about the data Madhusudhan got from K2-18 b.

We have above three examples of glory-seeking astronomers or astrobiologists making grand announcements of having observed something that they did not really observe, apparently as an effect of pareidolia, in which a person guilty of wishful thinking examines some hard-to-interpret, borderline data at the limit of observations, and claims that he has found something of grand significance.  Such a thing occurs not only in the world of astronomy, but also in the world of neuroscience.  Nowadays pareidolia is occurring very massively in experimental neuroscience. Eagerly hoping to find evidence of things they fervently believe in, neuroscientists are again and again claiming to have observed things they did not really observe. 

What we should never forget is that when you're a scientist, there are always a hundred ways for you to get some illusory evidence that is just a false alarm.  This doesn't require or typically involve any outright deception. It merely requires for a scientist to use some method that isn't quite right. 

One extremely common way for a scientist to conjure up a phantasm is to use too small a sample size or too small a study group size, which tends to result in false alarms. Scientists know how they can avoid this sin: by doing a sample size calculation to determine the minimum study group size needed to produce a moderately persuasive result, and to only use study groups with such a size. But a large fraction of scientific studies (particularly animal neuroscience studies) fail to include such a calculation, and fail to have adequate study group sizes. 

Another way for a scientist to conjure up a phantasm is to prune or filter his data until the desired thing seems to appear. It might be that when considering his full data set, there will seem to be no evidence of some thing (call it X) that the scientist wants the data to show evidence for.  But the scientist can prune the data at its beginning or end, until some evidence of X seems to show up. For example, if there were 4 weeks of data collection, the scientist can just get rid of the week 1 data or the week 4 data, or maybe both weeks of data. Or, the scientist can apply some "data filter" which gets rid of certain data points, until some evidence of X seems to show up.  The decision to apply such a data filter can often be rationalized in various ways, to make it sound like some "quality filter" excluding "bad data" or "outlier data."

Another way for a scientist to conjure up a phantasm is to collect data in a biased way that will maximize the chance that the desired result will appear. I will give a hypothetical example. Let us imagine that you are a scientist who wants to show that rainy days in New York City cause a higher chance of 300-point drops in a stock market indicator such as the Dow Jones Industrial Average.  You might begin recording daily stock market results on a rainy day in which there was a 300-point drop in the stock market.  You might then continue to record daily results, and conveniently end your data collection on a rainy day in which there was more than a 300-point drop in the stock market.  Given such convenient start and stop points of your data collection, you may well be able to write up a "statistically significant" correlation between rainy days in New York and 300-point drops in the stock market. But if you had resolved beforehand to start collecting data on some day 14 days in the future, and continue collecting data for exactly 100 days, then the desired result would probably not show up. 

Once so-called "raw data" has been collected, there are 1001 ways to "massage" the data before it is analyzed, some of which may make sense and some of which are dubious. A scientist can produce all kinds of rationalizations for particular data exclusions and data inclusions and use of data averages or "data smoothing" and use of "weighted averages" that may have been used, which can have a huge effect on whether some illusory phantasm shows up. 

Moving from the topic of data collection to data analysis, there are innumerable ways in which a scientist can conjure up phantasms by some kind of data analysis that isn't quite right. Don't be reassured when some science paper claims that it uses some kind of "standard software" for data analysis. There is almost always no such thing as a "standard analysis" of data. There are standard software tools used for data analysis, but such tools can be used in a million valid ways, and a million dubious ways. An example is Microsoft Excel, the leading spreadsheet program.  There are a million bad ways to use it, as well as a million good ways. 

When neuroscientists attempt to judge whether a rodent recalled by using the faulty "freezing behavior" method and manual inspection of how many seconds the rodent was immobile (within some arbitrarily chosen interval that can be anything between 30 seconds and five minutes), it's pretty much the perfect recipe for "see whatever you want to see" analysis. And when such analysis is done by people who are not blind to whether the observed rodents are in the experimental group or the control group, you have a likelihood of "see whatever you are hoping to see." Blinding fails to occur in most cognitive neuroscience studies, and almost never do we get a detailed statement giving us confidence that an effective protocol for blinding occurred. As for pre-registration (which can reduce some of the problems discussed in this post), a neuroscientist recently confessed it is rare in neuroscience. 

A science paper may try to reassure you that it used some standard software for doing some type of data analysis (such as measuring brain scan data or trying to measure "freezing behavior" in mice).  But there are always countless different ways to use such software, some good and some bad.  Every software program has program settings or startup options or menu options that allow you to customize how the program is used. Software programmers usually ask "how can I give the user the freedom to do exactly what he wants," and almost never ask "how I can make it so that there's no way to use the software in a stupid way." 

Nowadays neuroscientists often use "roll your own" customized computer programming to operate on gathered data in a way we can describe as "keep torturing the data until it confesses."  AI programs make it easier than ever to create such software, which is often of low quality.

Another way a scientist can conjure up phantasms is by failing to do a preregistered study that announces an experiment will be testing one very specific hypothesis, and going on a kind of "fishing expedition" within his analytic activity. For example, let's imagine the scientist does brain scans looking for some correlation between some behavior (or some aspect of thinking) and activity in some tiny brain region, some particular hundredth of a brain. By failing to limit himself to checking one small specific part of the brain corresponding to a previously declared hypothesis, and giving himself the freedom to check any of 100 small parts of the brain, he will have a good chance of finding some tiny region that weakly correlates with the behavior or aspect of mentality.  That's simply because given 100 parameters that show random variations, and the freedom to check any of them, it's easy to get something that looks like a slight correlation, even if only chance and not causation is involved.  The name sometimes given for this procedural sin is HARKing, which stands for Hypothesizing After Results are Known. 

Scientists have a hundred ways to conjure up illusory phantasms, and once a phantasm has been conjured up, there are many tricks by which the phantasm may be made to seem like something real, such as the use of complex charts and thick jargon which make the problematic presentation seem very scientific. All in all, we may say that the power of scientists to give you an impression of the reality of something illusory is comparable to the similar power of Hollywood's CGI special effects wizards. 

Very much of the more interesting-sounding neuroscience research results are examples of "seeing what was not there" pareidolia, in which some false alarm is conjured up using one of the techniques discussed above. There is no actual evidence for non-genetic representations in the brain. But neuroscientists often claim to have found faint traces of such things. Such neuroscientists are doing work similar to the false alarm generation work of Greaves, Madhusudhan and Loeb. 

In considering matters such as these, I like to remember a particular rule:

The rule of well-funded and highly motivated research communities: almost any large well-funded research community eagerly desiring to prove some particular claim can be expected to  occasionally produce superficially persuasive evidence in support of such a claim, even if the claim is untrue.  

We can consider an example of this rule, one involving astrology, the claim that the stars and planets exert a mysterious occult influence on the destiny of humans. Let us imagine that instead of there being merely a handful of poorly funded astrology researchers in the United States, there were instead 10,000 or more very well-funded astrology researchers, with billions of dollars in research grants to use to try to support their belief in astrology, by doing things like crunching statistics in various ways with computers.  It would then occur that we would occasionally read in the press stories presenting superficially persuasive evidence for astrology.  Such evidence probably would not stand up well to very close scrutiny, but it would be sufficient to give some talking points to astrology supporters. 

Similarly, if there was a large community of 10,000 ardent fairy researchers who were funded with billions of dollars, we would probably occasionally see superficially persuasive papers offering evidence for fairies. For example, with such an army of researchers, and so much money to spend, there might be occasional infrared heat signature studies suggesting anomalous little blobs of heat floating about that might be interpreted as fairies.  The researchers would be helped by the research rule that says, "Torture the data sufficiently, and it will confess to almost anything." 

And so it is for the 10,000 or more US neuroscientists funded with billions of dollars of research money (more than 5 billion dollars each year, according to this site).  Such scientists are able to occasionally produce studies providing superficially persuasive evidence for the dogmas the neuroscientists want to believe in, such as the idea that there is a physical hallmark of conceptual learning in the brain. Such evidence does not hold up well to very close scrutiny, but it is at least sufficient to provide some talking points for the neuroscientists.  Such evidence is actually no greater than the evidence we would expect to be produced for an untrue claim, given the "rule of well-funded and highly motivated research communities" cited above. 

An example of the schlock being typically produced by today's cognitive neuroscientists is the very low-quality paper being promoted in today's science news, the paper "Creating true and false memories from forgotten information in Drosophila." We have scientists experimenting with fruit flies, but the sample sizes used are way-too-small, almost always less than 15 (with study group sizes as low as 8 or 9). What excuse could someone have for not using a decent study group size such as 20 when experimenting with fruit flies? The authors confess, "No statistical methods were used to determine the sample size." A sample size calculation would have revealed how inadequate the study group sizes were. We have the confession, "Investigators were not blinded to group allocation during data collection." I doubt that anyone has devised a reliable method for measuring whether a fruit fly remembered something; and anything a neuroscientist claims about memory performance of fruit flies should be received with the greatest suspicion. The title of the paper is a groundless boast.