Tuesday, June 18, 2024

Cognitive Neuroscience Is Floundering, So the Kavli 2024 Neuroscience Prize Went to Low-Quality Research

If you look up the phrase "cognitive neuroscience" you will typically get a description that does not correspond to any scientifically established reality. You will typically get a description saying something like "the branch of science that investigates the neural causes of cognition," a description that presupposes the incorrect claim that cognition has a neural basis. Cognitive neuroscience is getting nowhere, mainly because it is based on the false assumption that the brain is the cause of the mind. Honest and properly designed studies will never produce evidence showing a neural basis for cognition. But there are 1001 ways to do poorly designed Questionable Research Practices studies that may fool someone into thinking that a little progress has been made in showing a neural basis of minds. 

The Kavli Foundation is a foundation founded by millions of dollars in grants from the late Fred Kavli. The foundation issues science prizes and science grants. One of its semi-annual prizes is in neuroscience. The 1 million dollar Kavli Prize in neuroscience was recently announced when a  false claim was made by the Kavli Foundation. The official citation appearing in the award announcement says that the 2024 prize was announced "for the discovery of a highly localized and specialized system for representation of faces in human and non-human primate neocortex." No such thing was ever discovered.  There are no representations of faces anywhere in the brain.  What we have in the 2024 Kavli neuroscience prize announcement is a bogus achievement legend, a claim that something big was done, when no such thing was ever done. 

Claims by neuroscientists that they have found "representations" in the brain (other than genetic representations) are examples of what very abundantly exists in biology: groundless achievement legends. There is no robust evidence for any such representations. 

Excluding the genetic information stored in DNA and its genes, there are simply no physical signs of learned information stored in a brain in any kind of organized format that resembles some kind of system of representation. If learned information were stored in a brain, it would tend to have an easily detected hallmark: the hallmark of token repetition.  There would be some system of tokens, each of which would represent something, perhaps a sound or a color pixel or a letter. There would be very many repetitions of different types of symbolic tokens.   Some examples of tokens are given below. Other examples of tokens include nucleotide base pairs (which in particular combinations of 3 base pairs represent particular amino acids), and also coins and bills (some particular combination of coins and bills can represent some particular amount of wealth). 

symbolic tokens

Other than the nucleotide base pair triple combinations that represent mere low-level chemical information such as amino acids, something found in neurons and many other types of cells outside of the brain, there is no sign at all of any repetition of symbolic tokens in the brain. Except for genetic information which is merely low-level chemical information, we can find none of the hallmarks of symbolic information (the repetition of symbolic tokens) inside the brain. No one has ever found anything that looks like traces or remnants of learned information by studying brain tissue. If you cut off some piece of brain tissue when someone dies, and place it under the most powerful electron microscope, you will never find any evidence that such tissue stored information learned during a lifetime, and you will never be able to figure out what a person learned from studying such tissue.  This is one reason why scientists and law enforcement officials never bother to preserve the brains of dead people in hopes of learning something about what such people experienced during their lives, or what they thought or believed, or what deeds they committed.    

But despite their complete failure to find any robust evidence of non-genetic representations in the brain, neuroscientists often make groundless boasts of having discovered representations. What is going on is pareidolia, people reporting seeing something that is not there, after wishfully analyzing large amounts of ambiguous and hazy data. It's like someone eagerly analyzing his toast every day for years, looking for something that looks like the face of Jesus, and eventually reporting he saw something that looked to him like the face of Jesus.  It's also like someone walking in many different forests, eagerly looking for faces on trees, and occasionally reporting a success. Read here for why claims of "place cells" in the hippocampus are unfounded claims based on low-quality research guilty of pareidolia. 

pareidolia

 The official citation page makes these claims:

"The present laureates used functional magnetic resonance imaging (fMRI) to localize different areas in neocortex specialized for face processing. Nancy Kanwisher pioneered the establishment of the functional region of interest (fROI) approach to localize the fusiform face area (FFA) in humans using fMRI. Kanwisher was the first to develop and employ a paradigm to identify a region sensitive to faces in each person. This finding strongly supported the idea of modular localization of cognitive function in the neocortex." 

The claims above are not accurate. No robust evidence has ever been provided that there are "different areas in neocortex specialized for face processing." One paper gets a result of only about 1% percent signal change when testing face recognition in different brain areas, getting only about a 1% signal difference for this FFA region.  Two other papers (this paper  and this paper) also find less than a 1% signal difference in this FFA region when testing facial recognition.  Another paper finds only a half of 1% signal change in the FFA during face recognition. Another paper using a larger sample size of 26 people reports a signal change of much less than 1% (only a small fraction of one percent) when testing this FFA region with face recognition. 

Such tiny percent signal changes do nothing to establish any reading of information from brains when visual recognition occurs. For one thing, since the sample sizes are mostly small (around 15 people per study), you could easily get a 1% or 2% signal variation by chance (just as you can easily get 55% of your coin flips being "heads" if you only flip 20 or 40 times).  If there is some tiny little signal change in one region of the brain when a face is recognized, that might be something that has nothing to do with reading memory information from brains. For example, it might be a little of an alert effect or an "aha" emotional boost effect caused by the mere fact of a successful recognition. 

The citation page of the 2024 Kavli neuroscience prize has a link to two papers by Nancy Kanwisher, neither one of which should create any confidence that she achieved what the citation page claims she did. The first paper is the 1997 paper by Kanwisher and others entitled "The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception." The paper is a bad example of a Questionable Research Practices study. A well-designed experimental neuroscience study will have all study group sizes being at least 15 or 20 subjects. That wasn't done in this study. The scientists started out with 15 subjects, and then started doing experiments on subsets of those subjects, those who performed in a way that the scientists found most encouraging. In the resulting study we have way-too-small study group sizes such as only five subjects per study group.  The study used no control subjects, and also completely failed to use any blinding protocol (a necessity for any study like this to be taken seriously). A sample size calculation would have revealed how inadequate the study group sizes were, but the authors failed to do any such calculation (or at least they do not have claim to have done such a calculation). The study was not a pre-registered study, and we get a strong feeling of the authors making up their methods as they gathered data, in violation of sound scientific procedure. 

It is amazing that a study this poorly designed is now being cited  on a prize page as an example of laudatory scientific work. The study would be more properly mentioned on some kind of "Hall of Lame"  page giving examples of poor-quality neuroscience research. Most ridiculously the citation page of the 2024 Kavli neuroscience prize includes a graph showing a 3% percent signal change in the brain of only a single subject. Citing this as evidence for a general effect in brains is as silly as claiming that male heartbeats increase when a male sees a picture of Taylor Swift, and using as your evidence a graph showing a very slight 3% heart rate increase in a single male. 

The only other paper by Nancy Kanwisher mentioned on the citation page of the 2024 Kavli neuroscience prize is her 2017 paper "The Quest for the FFA and Where It Led." That paper is a strange affair that diverges from the conventions of sound scientific research papers. It's a kind of "my glorious quest" paper that seemed to be telling us the wonderful story of how  Nancy Kanwisher progressed against her critics who complained about how weak her evidence was. The paper fails to provide any convincing evidence for Kanwisher's claim that she found a region of the brain specialized for recognizing faces. 

The paper has as its Figure 1 a groundless-looking "schematic map" depicting supposed brain regions specialized for particular tasks. No source is given for this map, which looks like one of those groundless phrenology maps that long appeared in scientific publications. No justification is given for any of the colors that appear in the map. Who made this map, and what data caused them to color the brain regions in these particular ways? The paper does not tell us. For all we know, it's just Nancy's wild guesses.  This extremely dubious Figure 1 is reproduced on thcitation page of the 2024 Kavli neuroscience prize as if it was some kind of scientific data, which it is not. 

The citation page of the 2024 Kavli neuroscience prize discusses two other scientists who shared the prize money.  We read these incorrect triumphal claims on the page:

"Winrich Freiwald and Doris Tsao together used fMRI to localize similar face patches in macaque monkeys. Having localized these, they recorded from single neurons in each patch. They showed that the overwhelming majority of visually responsive neurons in the largest such region were face-selective. They proceeded to outline a system of multiple face patches, detailing their interconnections and functional specialization. Face recognition in the earliest patches was dependent on viewpoint, but later became viewpoint-independent through a series of processing stages. Winrich Freiwald in further work characterized populations of cells selectively responsive to faces familiar to the viewer. Doris Tsao identified different features of the face that make up a code enabling single cells to identify faces."

There are no literally "face-selective" neurons in any brain, and the very concept is obscure and implausible. There is no evidence that brains or any part of brains recognize faces. There is merely evidence that people and monkeys recognize faces. 

Let's look at the papers the  citation page of the 2024 Kavli neuroscience prize cite to try to back up the claims quoted above. The first is a 2008 paper by Winrich Freiwald, Doris Tsao and another researcher, one entitled "Comparing face patch systems in macaques and humans." This is a low-quality paper guilty of the same Questionable Research Practices so predominant in today's neuroscience, such as a total failure to follow any blinding protocol. The paper shows us brain scans of some monkeys, but only a too-small study group size of only 9 monkeys. The percent signal changes are shown in the paper's supplemental information document, and my guess is that they were buried there so that there would be a minimum chance of a reader discovering how unimpressive they were:  changes of only about 1% (Figure S8). Such changes are very unimpressive as evidence of monkey brain regions responding differently when they see faces. Given the too-small study group sizes, we have here no robust evidence to back up the paper's claim that there are "face-selective regions in monkeys." (A similar study also finds brain percent signal changes of only about 1% in the same type of monkeys when exposed to faces, and finds similar responses when the monkeys were exposed to faces and non-face objects.) 

The authors have used the typical misleading practice of showing a brain visual with some region colored in a bright color (such as yellow), to fool us into thinking that there was some big difference in some region, when the difference was only about 1%. Instead of giving us Figure S8 (which would let us know the difference was only about 1%), the   citation page of the 2024 Kavli neuroscience prize has reproduced one of those misleading diagrams. 

misleading neuroscience visuals

The other scientific paper of Winrich Freiwald and Doris Tsao referred to by the  citation page of the 2024 Kavli neuroscience prize is the 2009 paper "A face feature space in the macaque temporal lobe." That's a paper as low-quality as the 2008 paper I refer to above. In the 2009 paper the study group size used is a mere three subjects. Also, the paper failed to use any blinding protocol, something necessary for any paper of this type to be taken seriously.  The method described is a rather ridiculous one, in which the authors select particular cells for study, in an arbitrary manner, without any assurance that these were randomly selected cells, and not cells cherry-picked for some characteristic the authors were hoping to find. Based on arbitrary criteria chosen by the authors, the authors make the claim that an individual cell in the brain can be "face selective." What exactly was the criteria used to judge whether a cell was "face selective"?  The paper gives us no answer other than giving us a link to some external page that gives no answer. 

In another study by Tsao we seem to be given the idea that she regards a "face selective" cell as one that responds more when a monkey is shown a face, as opposed to some image that is not a face. A proper term for that would be "face responsive" not "face selective."  We can explain such a thing without any belief that brain cells are producing recognition. When shown a face (as opposed to a neutral sight such as a geometric shape), a monkey may become slightly more alert or attentive. Some cells may therefore "perk up" a bit. But that isn't evidence that cells in the brain are involved in face recognition. Similarly, your eyebrow may raise slightly when a pretty lady walks by, but that does nothing to show that your eyebrows produce recognition of pretty ladies. And a man's penis may start to slightly enlarge when a buxom lady walks by in a bikini, but the penis can't see anything and isn't recognizing anything.  It's the person who is recognizing something, not some part of his body. 

The citation page of the 2024 Kavli neuroscience prize also refers to a low-quality neuroscience paper by Doris Tsao and Le Chang entitled "The Code for Facial Identity in the Primate Brain." This is an example of what can be called parlor-trick neuroscience. They have  used  brain scan data from only two monkeys, and an arbitrary selection of particular cells from such monkeys. The authors have also used a database of 200 face images and some extremely elaborate statistical analysis and computer programming  they did on such images and their sparse brain data, with it all having a sound of "keep playing with the data until we get something publishable."  They claimed to have predicted something, but it's one of those deals where the predictive model (a product of abundant statistical and programmatic fiddling) is leveraging data outside of the brain scans, data found in the external database. We get no robust evidence of anything important, and the title is a misleading one. No evidence has been produced of any such thing as a code for facial identity in the primate brain.  We have mainly what sounds like scientists fiddling with data in arbitrary ways for a long time until they finally ended up with something that pleased them, after conjuring up a "spaghetti code" analysis pathway. Misleading studies like this are getting more and more common in the world of neuroscience. They involve combining data from external databases with data from brain scans or brain EEG readings, mixed with lots of arbitrary computer programming and extremely complex or convoluted make-things-up-as-you-go-along analytics, in some way that everything is so confusingly  entangled that the authors may get away with grand claims that are not justified. Typically what goes on is that the authors give you an impression "we got this from brain scans" when the reality is really "we got this from a very confusing mashup of brain scans, arbitrary computer programming and statistics fiddling,  and data pulled from outside of brain scans," with the mashup being so complex and entangled that you have no good evidence of something about brains alone. The paper critically depends on much computer programming, but the authors failed to follow good practice by publishing a link allowing anyone to inspect their code, and see whether it followed good practices. 

spaghetti code

Because of an abundance of neural noise all over the place in brains (a strong reason for thinking brains cannot explain thought and memory recall that can occur with 100% accuracy), reliability of spike readings is a massive problem in some of the study types mentioned above, there being a huge problem of false positives in which cells are incorrectly identified as responding to some stimulus they did not actually respond to. A paper on the topic of single-unit neuron recording tells us this:

"Next, we examined a recording of 283,469 spikes recorded from mPFC [a brain region] in detail (Fig. 6b). Approximately 95% of the spikes recorded were considered noise as a result of low-amplitude spikes on several channels (Fig. 6b, left, gray points)."

Alas, the 2024 Kavli Neuroscience Prize has been awarded for low-quality research work which has not been well-replicated. The fact that this was done tells us something important about how cognitive neuroscience is getting nowhere these days. Cognitive neuroscience is based on false claims such as the claim that the brain is the source of the mind and the claim that the brain is the storage place of memories. It is no surprise that the scientists trying to back up these claims are floundering, running around in circles and getting nowhere. So when the committee of the Kavli Foundation searched for neuroscientists to be given a million dollar prize, all they could find is some low-quality research that was never well-replicated. 

For a scientific paper giving insight into the type of poor research that won the 2024 Kavli Neuroscience Prize, read the 2022 paper "The Failure of Blobology: fMRI Misinterpretation, Maleficience and Muddle" by brain imaging expert Stephen Jose Hanson.  The paper has this quote about the Fusiform Face Area (FFA) and the research of 2024 Kavli Neuroscience prize winner Nancy Kanwisher (I have underlined and boldfaced a few lines):

"Although the absurdity of the discovery of a 'face area' [an area of the fusiform gyrus that was somehow primarily responsible for face processing—fusiform face area (FFA) (Kanwisher et al., 1997)] wasn't sufficient reason to abandon the blobology program, subsequent years showed how bankrupt this phrenology (modularity) program had become. Because functional localization like the 'face area' has no actual anatomical anchors in the brain, but was based on a procedure, using another set or all the existing face stimuli in the experiment to, unfortunately, create circular kind of evidence for where the face area was to be found in the fusiform gyrus ('localizer logic', see Friston et al., 2010 who illustrates the dangers of localizers).... There is not sufficient space here to cover all the other FFA counter- evidence2. But suffice it to say, based on classifier evidence there is no '... fusiform face area, a blueberry-sized region on the bottom surface of the posterior right hemisphere that responds significantly more strongly when people look at faces than when they look at any other stimulus class.' (Kanwisher, 2006) Mathematically, this simply cannot be concluded from the GLM (unsupervised regression) or related tests."

The "Failure of Blobology" paper is an unflattering portrait of brain imaging neuroscientists gone astray, playing misleading games of "keep torturing the data until it confesses." 

science news hype
Typical content from a cognitive neuroscientist

What goes on in the awarding of prizes such as the Kavli Neuroscience Prize is that there is a strong "help myself" element, in which judges may be doing themselves favors by awarding an award to a particular type of researcher. Researchers who have done a particular type of dubious research may be more inclined to award a prize to other researchers doing the same type of dubious research, because this makes people more likely to hold in high regard the type of dubious research such judges have done. Check the judge list awarding this year's Kavli Neuroscience Prize, and you will find one or two whose careers have been centered around the same type of dubious "blobology" that was awarded with this year's prize. 

Postscript: The paper "Prevalence of Mixed-methods Sampling Designs in Social Science Research" has a Table 2 giving recommendations for minimum study group sizes for different types of research. The minimum subjects for an experimental study are 21 subjects per study group. Some of the studies mentioned above are experimental studies that used less than this minimum number.  The "case study" type mentioned below is a different type of study in which you merely document one or a few occurrences of some condition or situation, without trying to show a cause. 

minimum sample sizes

No comments:

Post a Comment