Principles of a Post-Materialist Science

What we can call the Age of Materialist Science has given us a scientific academia landscape with the following dysfunctional features:

• People trained as scientists are coerced into accepting or paying lip-service to doubtful belief tenets such as the nonexistence of souls and spirits, the neural origin of all mental effects, and the accidental origin of all biological innovations, contrary to a vast number of facts and observations.

• Much of what is called science consists of the belief dogmas of a belief community, often ideas contradicted by much observational evidence, such as beliefs that human mental phenomena are purely brain effects.

• Scientists act like conformist members of a belief community, afraid to challenge belief dogmas that have become kind of sacred tenets in their community.

• Scientists are effectively encouraged to ignore a vast body of relevant observations of the paranormal, observations conflicting with the materialist belief tenets of the scientist's belief community.

• Much of the activity of scientists consists of doing poorly designed experiments trying to provide evidence for various beliefs prevailing in the scientist belief community.

• A system of peer review exists by which anomalous observations and contrarian analysis and heterodox viewpoints can be prevented from being published.

• Smug achievement legends are constantly repeated, even when they make no sense, such as the claim that a nineteenth century scientist did something to explain protein molecules and super-complex cells that he knew nothing about. 

• Overconfidence, hubris and knowledge overestimation is systematically encouraged, along with absurd claims that scientists “pretty much understand” things that are a thousand miles over their head.

The diagram below contrasts principles of good science with the current tendencies of scientists in universities:

The diagram below gives us a rough sketch of what we have in the Age of Materialist Science:

Someday this very dysfunctional system may be replaced by something better. Below are some very rough thoughts about some of the principles that a reformed, post-materialist science might follow.

• Do not put any previous scientist on any kind of pedestal, or reverently attach special value to his thoughts and theories.

• Recognize the strong possibility of an observer getting novel and currently inexplicable observational results, and instead of ignoring such results, direct funding and attention to follow up on them.

• Make federally funded scientific research freely available to all, rather than hiding scientific work behind paywalls or in expensive journals that make the taxpayer pay again for research he already funded.

• Value all observations by previous careful observers, not dismissing such observations with excuses such as saying people believed the wrong things when the observations were made.

• Discourage inflexible and one-sided printed science textbooks, replacing them with electronic works that allow readers to add links and comments that draw attention to errors and omissions in the works, and draw attention to conflicting evidence and contrary viewpoints.

• Recognize that humans do not understand the deep mysteries of the origin of life, the origin of large organisms and the origin of minds, and be less generous in funding scientists spending most of their careers trying to bolster previous boasts claiming explanations for  such things, while encouraging a critical analysis of their work.

• Do not ignore or dismiss repeatedly reported observational phenomena with a claim that the thing cannot be happening because it is impossible.

• Recognize that science is only one of quite a few important ways of reaching truth, acknowledging the equal importance of other paths such as logic, mathematics, history, scholarship and direct personal experience. 

• Revise psychology textbooks and biology textbooks so that they discuss at length hard-to-explain or inexplicable human observations and anomalous experiences, rather than censoring and suppressing such observational reports. 

• Follow the principle that when reliable observers frequently report specific kinds of observations of the  inexplicable and anomalous, such observations should be given more attention rather than less attention.

• Recognize the high tendency of social structures such as universities and colleges to create conformist belief communities that may have a negative impact on scientific progress, and give rise to speech customs and belief traditions that masquerade as well-established science.

• Create a credential system where anyone who passes a very hard 3-hour test on some scientific subject can be certified as an expert on that topic, even if he has not passed through the conformity-creating system of colleges and universities.

• Create and fund alternative structures and organizations for learning and research separate from universities and colleges that have been so infected by conformist belief traditions, not as a replacement but as a rival and an alternative.

• Respect the observations of people who are not professional scientists, rather than having some snobbish elitist attitude that observations count for little unless they are made by professors or near-professors.

• Revamp the current university and college science instruction system, which suffers from an authoritarian teacher/follower model not varying much from the teaching model of two thousand years ago, replacing it with some model encouraging more dissent and discussion of alternative viewpoints and evidence conflicting with predominant assumptions.

• Revamp or rethink the "one-to-many" model of collegiate classroom instruction which tends too strongly to produce a meek acceptance of ideology or dubious claims passed on by authorities.

• Replace in many university departments the current habit of giving someone with a master's degree a PhD based only on some very narrow research on one tiny topic, and make extremely broad multi-subject cross-discipline study the thing that gets you the PhD. 

• Federally fund independent scientists with worthy research proposals, even those working outside of the university system.

• Have a large fraction of all federally funded scientific research approved by persons outside of academia, to help prevent "echo chamber maintenance" effects whereby some scientist belief community with "purse string" control keeps funding research designed to support its cherished beliefs.

• Create a system in which public comments can appear at the end of every online science paper, allowing the public to dispute poorly designed papers, discuss conflicting evidence, and dispute papers making claims not justified by anything in the paper.

• To discourage studies in which researchers feel free to slice and dice data in innumerable ways until they find something "statistically significant," create a pre-registration system for experimental studies, in which papers must only report on whether the results supported a previously announced hypothesis and whether the data collection and data analysis procedures followed matched a detailed experimental plan published before any data was collected. Also have a red-flag system whereby the reader is warned of the failure of an experimental paper to follow such a standard.

• Create sample-size calculation conventions and study-group-size standards that limit the proliferation of misleading experimental studies in which false alarms (easily explainable by chance effects) are heralded as scientific discoveries.  

• Stop acting as if unintuitive principles not suggested by common experience (such as "everything must be explained by matter and energy" or "there cannot be mysterious unseen influences") should be preferred over intuitive principles suggested by common experience (such as "accidents don't produce complex inventions" and "where there's lot of smoke, there's probably fire," which is a good rule-of-thumb in dealing with whether  observational reports suggest some underlying reality).  

• Stop promoting scientists based on the number of papers they have published, and encourage alternate promotion criteria such as the number of times a scientist has published a paper judged to follow a "Best Practices" standard. 

• End the current secretive peer-review system that acts as an ideological filter preventing dissenting viewpoints, reports of conflicting evidence, and novel observational reports, replacing it with a “let scientists stumble but flag their stumbles” system that will encourage public comments about any mistakes in a scientific paper, along with also a quality grading system whereby inferior papers can be low-graded.

• Stop making dubious claims of a scientific consensus that are not established by secret ballots of scientists, and create some system for secret balloting of scientists that will clarify how much they agree on opinions, a system that always offers a variety of belief answers including “I don't know.”

• Make a large fraction of all scientific funding go to studies that will be guaranteed publication studies in which publication is assured even if a null result is found.

• Make a small fraction of all scientific funding go to groups trying to disprove or falsify prevailing ideas and assumptions.

• Reform the speech and writing habits of scientists, to discourage the continuation of misleading speech practices and misleading visuals that are shockingly common in scientific literature.

• Stop referring to speculative unobserved things such as dark matter, dark energy and accidental macroevolution as "science," and accurately refer to them as "scientist speculations." 

• Reform current profit structures that reward bad science and bad scholarship that ignores important relevant evidence, and create novel profit structures that reward best-practices science and scholarship that takes into account all relevant evidence.

The current profit structure is extremely dysfunctional

Why the "A Memory Is Stored Throughout the Brain" Idea Makes Things Much Worse

No matter what form such an idea takes, the idea that the human brain stores memories creates the most gigantic difficulties, difficulties so bad that we should reject all claims that memories are stored in brains.  Let us look at two different forms of such an idea, and look at some of the difficulties that each form creates. 

The most common form of the idea that brains store memories is the idea that a memory is stored in one particular little spot of the brain, with each memory being stored in a different tiny spot.  Below are some (but not all) of the huge problems that such an idea creates:

(1) The spot selection problem. While computers have operating system algorithms for choosing a random storage spot,  a brain would seem to have no method or capability of choosing one small little storage spot for a memory to be placed. So if, for example,  we imagine that a brain placed a memory in storage spot number 263,432 out of 250,000 storage spots, we have the problem: why would that particular spot have been used to store the memory, and not some other spot? 

(2) The writing and encoding problem.  Once some spot had been selected for a memory to be written, something learned would have to be translated into neural states or synapse states and then written. There is no credible theory of how learned information or episodic memories could be translated into neural states or synapse states. There is no known mechanism in the brain for writing information. A computer has an operating system with formally designed encoding protocols such as the ASCII protocol and a protocol for converting decimal numbers into binary numbers. A brain has no such thing. A computer has a read-write head for writing information. The brain has no such thing. 

(3) The navigation problem.  Humans routinely display the ability to instantly recall learned information, given a name, date or image. So, for example, if you say "death of Lincoln," I will instantly be able to recite various facts about the death of Abraham Lincoln, such as that it occurred because John Wilkes Booth shot Lincoln through the back of his head at Ford's Theater in April, 1865.  If we believe that a memory is stored in some tiny little spot in the brain, such as storage spot 186,395 out of 250,000, then we have the problem: how was the brain able to instantly find that exact tiny spot where the memory was formed? This difficulty is a "show stopper" for all claims that a memory is stored in one exact spot of a brain, an insuperable difficulty.  We cannot get around such a difficulty by imagining that a brain uses the type of things that a book or a computer use to allow instant retrieval.  Books and computers use information addressing and indexes to allow instant access of a particular data item.  The brain has neither addressing nor indexes.  Unlike houses that have street addresses, neurons don't have neuron numbers or any other addressing system. Storing a memory in a brain would be like throwing a little 3" by 5" card into a giant swimming pool filled to the top with a million little 3" by 5" cards.  Just as it should take you ages to find a specific piece of information stored in such a swimming pool, it would take you ages to find in the brain some particular piece of learned information, if it was stored in one tiny spot, like a book stored in one spot on the shelves of a huge library.  

 

(4) The reading and decoding problem.  If a memory was stored in one particular spot, there would be the problem of how a memory could be read from that exact tiny spot. The brain seems to have nothing like a read mechanism.  Nor is there any known mechanism by which information that had been stored as neural states or synapse states could be translated into a thought that would appear in your mind. 

But there is another form of the idea that brains store memories.  There is the idea that the brain stores a memory throughout the brain, rather than writing the memory only in one little spot.  However, the difficulties in this idea are even worse than in the idea that the brain stores each memory in one specific spot. The idea that a memory is stored throughout the brain has the following difficulties:

(1) A greatly worsened writing and encoding problem. The idea that a memory is stored throughout the brain has the same writing and encoding problems mentioned above, except that now the problem is much worse. This is because now rather than just imagining that a memory is written in one tiny spot by a brain without any known writing mechanism, we must now imagine that such a brain manages to write all over itself each time that a memory is stored.  This would take much more time than writing to a single spot in the brain. Humans routinely show the ability to instantly form new memories,  an ability that neuroscientists cannot credibly explain. You only make that problem worse if you imagine that each time a memory is formed, the brain is writing to many different places rather than one. 

(2) A memory disassembly problem. The idea that a memory is stored throughout the brain creates a gigantic new problem that did not exist if you assume a memory is written to only one tiny spot: a memory disassembly problem. If you imagine that a memory is broken up into tiny pieces and stored throughout the brain,  then you have the problem that such a disassembly process would require additional time, making it all the more impossible to explain the wonder of instant memory formation. Similarly, it only takes a second for me to store a piece of paper by opening a book in my library and sticking the page inside the book; but if I have to cut up the page into twenty pieces and store the pieces in twenty books, that takes much longer. 

(3)  A memory reassembly problem. The idea that a memory is stored throughout the brain creates a gigantic new problem related to memory recall: a problem of reassembling the memory that had been stored in scattered pieces throughout the brain. If we imagine a brain with only one memory, such a thing does not seem so hard (the brain could just read throughout itself looking for memory pieces, and read them all up). But if we imagine many, many thousands of memories that had each been stored by storing pieces of individual memories throughout the brain, then such an assembly seems impossible to occur, no matter how long it would take.  

I can give an analogy. Suppose I am storing 1000 family photos through a scattered storage method. I take each of the thousand photos, cut them up into little pieces, and store each by putting them in different pages in the books that make up my large library. Now, suppose my wife comes and asks, "Please get me a picture of our trip to Los Angeles."  Retrieving that photo would be a nightmare.  I couldn't just get all the photo pieces by shaking each book in my library.  That's because the pieces of each photo would be mixed up with all the pieces of 1000 other photos.  Similarly, we can imagine no way in which a brain that has scattered pieces of each memory throughout itself could ever reassemble such pieces to produce a good recall of a particular memory.  And if it ever could do such a thing, such a recall would take very long lengths of time, and a recollection could never occur instantly.   

(4) A greatly worsened reading and decoding problem. The idea that a memory is stored throughout the brain has the same reading and decoding problems mentioned above, except that now the problem is much worse. This is because now rather than just imagining that a memory is read from one tiny spot by a brain without any known writing mechanism, we must now imagine that such a brain manages to read from all over itself each time that a memory is retrieved.  This would take much more time that reading from a single spot in the brain. Humans routinely show the ability to instantly retrieve new memories,  an ability that neuroscientists cannot credibly explain. You only make that problem worse if you imagine that each time a memory is recalled, the brain is reading from many different places rather than one.  

There was recently in the news an MIT press release story making the utterly unfounded claim that some research had shown that "a single memory is stored across many connected brain regions." What we have is another misleading claim about engrams from MIT, which for many years has been a notorious source of unfounded claims about neural memory storage.  In the 2018 post here I took a long look at how MIT memory researchers had repeatedly made grandiose but unfounded claims about memory research.  I showed that MIT researchers had again and again made grandiose claims based on shoddy poorly-designed rodent studies guilty of using way too small sample sizes.  The results proclaimed by such researchers are mainly false alarms, the type of false alarms that are very easy for a researcher to get when he uses fewer than 20 subjects per study group. 

The latest memory research announcement by MIT discusses research guilty of the same old shoddy research practices that MIT memory researchers have been guilty of for so many years.  Once again, when we read the scientific paper (which can be read here) we find that the researchers used way-too-small study group sizes, such as one group of only 7 mice, another group of another 9 mice, and another group of only 10 mice. If the scientists had acted like good experimental scientists and had done what is called a sample size calculation, they would have found out that such tiny study group sizes are utterly inadequate to produce a reliable result. But they did no such calculation. They confess in their paper, "No statistical methods were used to predetermine sample sizes."

The scientists fear-conditioned mice by electrically shocking them (this typically involves getting mice to learn there is one little area of a cage where the shocking will occur). The scientists then measured something in lots of different regions in the brain of a very small number of mice, and the scientists have somehow got the idea that some regions were involved in memory storage.  To test such suspicions they "optogenetically stimulated" mice to try to artificially create fear in the mice, zapping the little regions they thought were involved in storing a memory.  This "optogenetic stimulation" is a method of using light to zap the brain of a mouse. 

The thinking behind such strange zappings of mouse brains is that by zapping some little part of a mouse's brain, you can get a mouse to remember some fear memory formed when a mouse was zapped by stepping on an electrical plate.  The underlying theoretical assumption was wildly implausible. It was the idea that if a mouse has a particular memory stored in many brain regions, then you can get the mouse to re-experience that memory by stimulating only one of those regions.  Such an idea makes no sense. It's kind of like thinking that I would get Tom Brady to throw a pass by sticking a sewing needle in his arm, stimulating one of the many muscles he uses in throwing a pass.  

Conclusions about whether the fear memory was recalled were based on a poor low-reliability technique that neuroscientists have long used: a judgment about whether so-called "freezing behavior" occurred (such behavior being defined as mere inactivity). The underlying assumption is that mice freeze when afraid, and that you can judge if a mouse is recalling a fear memory by looking for an instant of non-movement in which a mouse may be "freezing in fear." Given the start-stop, helter-skelter way in which mice move, any judgment about whether a mouse froze is going to be a subjective, unreliable judgment. So there is too much of a possibility of observational bias here, one in which an observer subjectively reports the effect he is hoping to find. Similarly, you might subjectively report that your goldfish in a goldfish bowl tends to move towards you when you are looking into the bowl, but that would probably tell us more about your desire to see something than about the goldfish. The idea that mice freeze when terrified isn't even a very sound one.  I have seen  dozens of mice flee when scared by a human, but I never once seen a mouse freeze when suddenly scared by the presence of a human. 

There is a very reliable way to measure fear in mice: you measure the mouse's heart rate, which undergoes a very sharp spike in mice when they are afraid. Our neuoroscientists senselessly continue to use unreliable subjective judgments about "freezing behavior" to try to measure fear in rodents, rather than sensibly using reliable measurements of heart rate spikes in rodents.  Being guilty of this flaw, the new MIT study has provided no reliable evidence about whether or not the mice remembered fear when parts of their brains were zapped. 

Moreover, when "freezing" (simple non-movement) occurred in the mice, the "freezing effect" could have been produced not by a recall of fearful memories, but by the very fact the energy was being transmitted into the brain of the mice. Imagine you are running along, and suddenly a scientist switches on some weird thing that causes some energy to pour into your brain. This all by itself might cause you to stop, even if it didn't cause you to recall some memory that caused you to stop. What could have been going on in the mice was just a kind of pausing effect caused by a novel stimulus rather than a recalled fear effect. A science paper says that it is possible to induce freezing in rodents by stimulating a wide variety of regions. It says, "It is possible to induce freezing by activating a variety of brain areas and projections, including the hippocampus (Liu et al., 2012), lateral, basal and central amygdala (Ciocchi et al., 2010); Johansen et al., 2010;  Gore et al., 2015a), periaqueductal gray (Tovote et al., 2016), motor and primary sensory cortices (Kass et al., 2013), prefrontal projections (Rajasethupathy et al., 2015) and retrosplenial cortex (Cowansage et al., 2014).”

Neither the paper nor its supplementary information contains any  mention of a blinding protocol, other than the bare statement that "all behavior experiments were collected and analyzed blind to experimental group."  Unless a paper has a detailed discussion of  how an effective blinding protocol was implemented (one that really achieves a blinding effect to prevent observer bias), we should assume that no effective blinding protocol was implemented.  For example, if you had one group of 7 mice with optogenetic wires attached to their brains, and another group of control mice with no such wires, anyone would be able to tell which group was the group that was hoped would show more "freezing" behavior (even if those judging how much the mice froze were not specifically told which group was which).  So some method that can technically be referred to as "blind" may not be blind at all because of a lack of an effective protocol. Whenever any paper claims a blinding protocol but fails to specify how an effective protocol was achieved,  we should assume that no effective methods of blinding were used (a severe defect in an experiment). 

Being guilty of quite a few serious methodological flaws (primarily the use of way-too-small study group sizes), the new MIT study has produced no robust evidence that memories are stored in the brains of mice, and no robust evidence that a memory is stored in many different regions of the brains rather than in some particular spot. According to the paper here, "Quality research practice requires both testing statistical significance and reporting effect size." But the new MIT paper reports no effect size. That is what goes on when shoddy experimental research practices have been followed, such as using way-too-small study group sizes. 

In this paper here we have a discussion of the absurd technique most commonly used to measure fear in rodents:

"In mice, freezing is a common and easily measured response used as an index of fear conditioning (Blanchard and Blanchard 1988; Graef 1994). Fanselow (1990) and Paylor et al. (1994) define freezing as the absence of any movement except for respiratory-related movements. Freezing behavior is measured by direct observation, scoring an animal as either freezing or active per interval of time, usually every 5–10 sec (Fanselow 1990; Paylor et al. 1994) or measuring freezing duration with a stopwatch (Phillips and Le Doux 1992)."

The technique discussed above measures only mouse inactivity, which will vary randomly. There is no sound basis for calling such a measurement a measurement of "freezing behavior." If I take 10 snapshots of a mouse per minute, that show the mouse not moving in three of those snapshots, that is no reason for thinking that the mouse was afraid when three of those ten snapshots were taken.  What is occurring these days among cognitive neuroscientists is deceptive labeling of mouse inactivity measurement. Graphs that should be labeled "mouse inactivity (%") are being misleadingly labeled "mouse freezing (%)."  The term "freezing" should never be used unless a sudden stopping of traversal was observed. 

Big Study Finds Brain Gray Matter and Cortical Thickness Peak at Age 6 or Earlier, Contradicting Brain Dogmas

A new study published in Nature (with very many listed authors)  has produced a result very relevant to claims that the human mind is produced by the brain.  Entitled "Brain Charts for the Human Lifespan," the paper says, "We aggregated 123,984 MRI scans, across more than 100 primary studies, from 101,457 human participants between 115 days post-conception to 100 years of age."  MRI scans are a type of scan that allow you to see the physical structure of the brain. 

Human experience is that intelligence roughly peaks around age 20, with no major decline before age 40.  The lack of intellectual decline before age 40 is partially why nations generally elect leaders that are 40 years old or older, and it is partially why major corporations generally have as their Chief Executive Officer someone who is age 40 or older.    The claim has often been made that gray matter in the brain is some type of neural matter particularly associated with intelligence. It has often been claimed that you think with the gray matter of your brain.  Given human intelligence peaking around age 20, and not declining much before age 40, such claims predict that gray matter in the brain should peak at around age 20, without much decline before age 40. 

But this is not at all what the "Brain Charts for the Human Lifespan" study found. It found that gray matter in the brain peaks at around age 6, with about a 12% decline by age 20, and about a 20% decline by age 40.  This is shown in a chart from the paper:

From the "Brain Charts for the Human Lifespan" paper

According to this chart:

• Gray matter volume peaks around age 6;
• gray matter volume declines by about 12% by age 20;
• gray matter volume declines by about 20% by age 40;
• cortical thickness peaks by about age 2 or 3;
• cortical thickness declines by about 10% by age 20;
• cortical thickness declines by about 15% by age 40;
• white matter volume peaks at about age 30.

The chart above is a bit hard to read, but at a web site set up by the paper authors, the gray matter volume trend by age is graphed in the easy-to-read graph below:

 Source: https://brainchart.shinyapps.io/brainchart/

The data presented in the new study simply does not match human experience regarding intelligence and age.  I cannot cite any numbers showing a growth of IQ by age, because IQ tests are designed to test differences in either children of one age or adults.  But the following are simple facts of human experience

• Children age 6 have an intelligence that only seems to be 50% or smaller than the intelligence of adults (despite the gray matter volume peaking around age 6);
• adults of age 40 do not have an intelligence noticeably less than those of adults at age 20, and have an intelligence much greater than those age 6 or younger (despite such 40-year-olds having 20% less gray matter volume than those age 6, about 15% less cortical thickness than those age 2, and roughly 10%  less gray matter volume than those age  20);
• adults of age 20 have an intelligence much higher than children of age 6 (despite such 20-year-olds having about 12% less gray matter volume than those age 6, and about 10% less cortical thickness than those age 2 or 3);
• adults of age 30 do not seem any smarter than adults age 20 (despite such 30-year-olds having white matter volume peaking at their age);
• children with an age of about 2 or 3 have an intelligence that only seems to be a small fraction of the intelligence of adults (despite their cortical thickness peaking around this age). 

Once again, the "brains make minds" dogma gigantically flunks an empirical test. But you won't hear about this failure in the mainstream media, which tends to keep scientists and the public in a "filter bubble" that allows them to keep thinking that their cherished dogmas are holding up well, no matter how miserably such dogmas are failing empirical tests. So, for example, a Nature article on the "Brain Charts for the Human Lifespan" study completely fails to mention how dramatically the study's data conflicts with human experience about how intelligence changes with age. 

"Brains Make Minds" Idea Flunks an Audit of a Large Brain Scan Database

For many years neuroscientists have been claiming important results about brains and minds, after doing brain imaging experiments using small sample sizes.  Typically such claims will be based on way-too-small sample sizes smaller than 15.  A new press release from the University of Minnesota Twin Cities announces results which indicate that such small-sample correlation-seeking imaging experiments are utterly unreliable.  The headline of the press release is "Brain studies show thousands of participants are needed for accurate results."

There is a technique to measure the reliability of brain scans when used to make claims about supposed neural signs of cognitive activity.  The technique involves measuring what is called the test-retest reliability of brain scans.  The technique involves trying to determine to what extent some claimed neural sign of cognitive activity shows up both times when two different brain scans are taken of the same person. 

So, to imagine a hypothetical example, suppose some claim is made that the hippocampus of some subject activated more strongly when the subject recalled something. A check can be made as to whether the same thing was seen when the same subject had his brain scanned a second time, doing the same recall task.  If no such increased activation is seen on the second brain scan, we have a good reason for thinking that the claim about the first scan is unwarranted, and that the first scan has simply given a false alarm, a result of random brain fluctuations.  

Conveniently "covering their tracks," the vast majority of neuroscientists fail to do a retest of subjects when doing brain scanning experiments. However, there are some large databases of brain scans that include scanning retests of many subjects. It is therefore possible to judge how well claimed neural correlations of cognitive activity tend to replicate when a second test is done of the same subject. 

One such brain imaging database is the Adolescent Brain Cognitive Development Database. The database includes scans of thousands of subjects doing particular tasks such as a Monetary Incentive Delay task. a Stop Signal task and an n-back or nBack task (as described here). The database includes brain scans of more than 10,000 adolescents, and for more than 7000 of these adolescents a second set of scans were taken two years later, with the subjects performing the same tasks as in the first scan.  Such a database provides an excellent platform to test whether correlations between brain states and mental activity tend to repeat when the same subjects were scanned two years later.  

Such an examination is reported in the scientific paper entitled "Reliability and stability challenges in ABCD task fMRI data" by James T. Kennedy and others, which you can read here or here.  The study used a measure of retest reliability called the intraclass correlation. An intraclass correlation of less than .4 is generally regarded as "poor." In the wikipedia.org article on the intraclass correlation we read the following:

"Cicchetti (1994) gives the following often quoted guidelines for interpretation for kappa or ICC inter-rater agreement measures:

• Less than 0.40—poor.
• Between 0.40 and 0.59—fair.
• Between 0.60 and 0.74—good.
• Between 0.75 and 1.00—excellent.

A different guideline is given by Koo and Li (2016):

• below 0.50: poor
• between 0.50 and 0.75: moderate
• between 0.75 and 0.90: good
• above 0.90: excellent"

The results reported in the scientific paper entitled "Reliability and stability challenges in ABCD task fMRI data" by James T. Kennedy and others were devastatingly negative.  In the paper's abstract we read this: 

"Reliability and stability [quantified via an intraclass correlation (ICC) that focuses on rank consistency] was poor in virtually all brain regions, with an average ICC of .078 and .054 for short (within-session) and long-term (between-session) ICCs, respectively, in regions of interest (ROIs) historically-recruited by the tasks. ICC values in ROIs did not exceed the ‘poor’ cut-off of .4, and in fact rarely exceeded .2 (only 5.9%).... Poor reliability and stability of task-fMRI, particularly in children, diminishes potential utility of fMRI data due to a drastic reduction of effect sizes and, consequently, statistical power for the detection of brain-behavior associations."

What this means is that there was extremely low level of repetition of effects between one scan on a subject and a later scan on the same subject. As mentioned above, an intraclass correlation of less than .4 or .5 is commonly described as "poor." The very low intraclass correlations reported (only .078 and .054) can be described as extremely poor or appallingly poor.  In the quote below, the authors of the study describe their results as a "particularly disappointing outcome," and wonder what factors contributed to so poor an outcome. We read the following: 

"Our main finding was that within-session reliability and longitudinal stability of individual differences in task-related brain activation was consistently poor for all three ABCD tasks. Data cleaning approaches like outlier removal, movement regression, and rank normalization significantly increased reliability and stability, but by a small, seemingly inconsequential amount (average change of less than .025). While the finding of poor within-session reliability and longitudinal stability in the ABCD task fMRI data did not come as a surprise, given the mounting evidence for generally lackluster reliability of task-fMRI in mostly adult samples (Elliott et al., 2020; Herting et al., 2018; Noble et al, 2021), the present estimates are far below the .397 average reliability of task-fMRI activation estimated in the meta-analysis by Elliott et al. (2020). The question then arises, what factors could contribute to this particularly disappointing outcome? "

These results are what we would expect under the idea that the brain is not the source or cause of human mental activity, and not the storage place of memories.  In such a case we would expect that when scientists claimed some correlation between brain activity and mental activity after brain scanning some subjects, they would almost always be finding mere false alarms that would strongly tend to disappear when a second brain scan was made of the same subjects. 

Why the Academia Cyberspace Profit Complex Keeps Giving Misleading Brain Research Reports

Why do so many untrue and misleading stories about brains and minds appear in the press? The answer is largely a financial one: because various parties profit from such misleading stories. Using the famous "follow the money" slogan in the main movie about the Watergate affair (All the President's Men), let us "follow the money" and see how various parties profit from misleading stories about brains and minds in the press. 

The interesting diagram below illustrates a profit complex that links academia and cyberspace (a word that means the same as the Internet). 

To understand this profit complex, you must first understand how modern scientists are judged by their peers and superiors in academia. There are two numbers by which scientists are judged: (1) the number of scientific articles that the scientist has written or co-written, called the paper count; (2) the number of other papers that have mentioned or cited one of the papers the scientist has written or co-written, called the citation count. If you are a scientist hoping for a promotion such as tenure or a higher salary, you very much want these numbers to be as high as possible. 

The desire to raise such numbers (for the benefit of a scientist) is very much a factor when a scientist designs an experiment. Given the choice between some "quick and dirty" experimental design that will be likely to produce some result that is either a quick and easy result or a positive result or a result that can be claimed as some important result, and some other design that involves some more stringent research method that is longer, harder, or less likely to result in a positive result or a result that can be claimed as important, a scientist who is very interested in increasing his paper count and his citation count will be more likely to choose the "quick and dirty" design.  Such "quick and dirty" designs will very often involve way-too-small sample sizes, in which fewer than 15 subjects are studied (often for studies in which many dozens, hundreds or thousands of subjects would be needed if you wanted to get a reliable result).  A scientific study found that research papers that failed to replicate were on average 153 times more likely to be cited than papers describing research that replicated, stating this: "papers that replicate are cited 153 times less, on average, than papers that do not." Such failing-to-replicate studies typically involve shoddy "quick and dirty" experimental designs. 

Nowadays science journals have a tendency called "publication bias," which is a tendency to publish papers reporting positive results and reject papers reporting null or negative results.  When a scientist does an experiment that produces a null or negative result, and is not able to get a journal to publish the paper, the scientist's paper count is not increased, and the effort does nothing to advance the scientist's career. So scientists will avoid very careful and stringent designs less likely to result in a paper reporting a positive result, and will be more likely to create "quick and dirty" designs more likely to result in a positive result and more likely to result in a positive result that can be produced more quickly.  The quicker the experiment can be done, the more quickly can the scientist's paper count be increased. 

After creating this design, some observations are produced. Desiring to report some positive result and ideally some important-sounding result, scientists will tend to filter or segregate the observations to produce some subset that is more favorable to the reporting of a positive and interesting-sounding result. Sometimes this process can be described as cherry-picking, and other times this process is something rather along the lines of "keep slicing and dicing the data until it gives what is wanted" or "keep torturing the data until it confesses."  There are 101 reasons that can be given for excluding some data points and keeping other data points. There are also hundreds of statistical methods that can be used to massage and filter data until you are left with more favorable results.  In this analysis of data, scientists will have a motivation not to use blind analysis techniques that minimize the chance of biased analysis in which scientists report seeing what they want to see. 

After such analysis is completed, there comes the writing of a scientific paper.  When writing up a scientific paper, scientists are very motivated to describe the research as showing some positive result, even if the research has mainly or entirely produced a negative or null result. This is because scientists want to increase their paper count (the number of published papers they have authored or co-authored); and given publication bias in which journals tend to reject papers reporting only negative results, a paper reporting a negative result may be unlikely to be published. Scientists will also be very motivated to report getting some important result. The more that a scientist tends to claim that some important result was produced by the research, the more likely will be the publication of the paper. Also, the more important the result that is claimed, the more likely the paper will be to be cited by other papers. Such citations are extremely important to scientists, as scientists are judged not just by their paper count (the number of papers they have written), but also by their citation count (the number of times such papers have been cited). 

It very often happens that in writing up papers describing their research, scientists make claims that are misleading, exaggerated or just plain false. At a blog entitled "Survival Blog for Scientists" and subtitled "How to Become a Leading Scientist," a blog that tells us  "contributors are scientists in various stages of their career," we have an explanation of why so many science papers have inaccurate titles:

"Scientists need citations for their papers....If the content of your paper is a dull, solid investigation and your title announces this heavy reading, it is clear you will not reach your citation target, as your department head will tell you in your evaluation interview. So to survive – and to impress editors and reviewers of high-impact journals,  you will have to hype up your title. And embellish your abstract. And perhaps deliberately confuse the reader about the content."

Is this how scientists are trained?

A neuroscientist makes this confession:

"This system comes with big problems. Chief among them is the issue of publication bias: reviewers and editors are more likely to give a scientific paper a good write-up and publish it in their journal if it reports positive or exciting results. So scientists go to great lengths to hype up their studies, lean on their analyses so they produce 'better' results, and sometimes even commit fraud in order to impress those all-important gatekeepers."

After a scientific paper has been written up and published, it is announced with a press release issued by the main academic institution involved in the research. Nowadays the press releases of universities and colleges are notorious for making sensationalized claims that are not warranted by anything discovered in the research being discovered. Often a tentative claim made in a scientific paper (basically a "perhaps" or a "maybe") will be stated as if it is was simply a discovery of a definite fact.  Other times a university press release will make some important-sounding claim that was never made in the scientific paper writing up the research.  An example was that when  there appeared a scientific paper merely claiming that "Regional synapse gain and loss accompany memory formation in larval zebrafish," there appeared a great number of press stories repeating the headline of a press release claiming that the formation of a memory had been observed (a claim not made in the paper).  We have every reason to believe that synapse gains and losses occur  continually in the human body, regardless of whether some new memory is forming. 

Authorship anonymity is a large factor that facilitates the appearance of misleading university and college press releases.  Nowadays university and college press releases typically appear without any person listed as the author. So when a lie occurs (as it very often does), you can never point the figure and identify one particular person who was lying.  When PR men at universities are thinking to themselves "no one will blame me specifically if the press release has an error," they will feel more free to say misleading and untrue things that make unimpressive research sound important.  We should always hold every single scientist involved in a scientific paper responsible and accountable for every untruth that appears in a scientific paper they co-authored and also ever untruth that appears in the university press release announcing the paper, unless that scientist has publicly protested the misstatement. 

 

Misleading press releases produce an indirect financial benefit for the colleges and universities that release them.  When there occurs untrue announcements of important research results, such press releases make the college and university sound like some place where important research is being done. The more such press releases appear, the more people will think that the college or university is worth the very high tuition fees it charges. 

Judging from the quote below, it seems that science journalists often look down on the writers of university and college press releases, even though such science journalists very often uncritically parrot the claims of such people.  In an Undark.org article we read this:

"Still, there are young science journalists who say they would rather be poor than write a press release. Kristin Hugo, for example, a 26-year-old graduate of Boston University’s science journalism program, refuses to step into a communications role with an institution, nonprofit or government agency.  'I’ve been lucky enough that I haven’t had to compromise my integrity. I really believe in being non-biased and non-partisan,' she says. 'I really, really, really want to continue that. I wouldn’t necessarily begrudge someone for going into [public relations] because there’s money in that, but I’d really like to stay out of it.' "

Misleading press releases also help to sustain cyberspace profit systems outside of a college or university. Such press releases are repeated (often with further exaggerations and misstatements) by a host of web sites offering clickbait headlines leading to web pages containing ads. The more people click on these clickbait headlines, the more page views there are for pages containing ads. The more people view those pages, the more advertising revenue the web sites get. 

So web sites giving science news stories have a very large financial incentive to produce exaggerated or untrue headlines that users will be more likely to click on.  If the headline on a web page truthfully says, "Another Junk Science Brain-Scanning Result," almost no one will click on the headline to go to the page with the story containing ads. But if the headline untruthfully says, "Breakthrough Study Reveals the Secret of Memory," then thousands of people may click on the headline, producing many pages views of the story the link leads to, and much more advertising revenue. 

The web sites are one profit center benefiting from poor and misleading science journalism that exaggerates or misrepresents unimpressive research. Another profit center is the science journalists themselves. Most science journalist do not work on some salary basis in which they are paid the same annual salary regardless of what they write. Instead most science journalists work on a per-article basis, earning about $1 per word for an article in a print magazine such as Discover Magazine, or about $300 per article for an online article. Such journalists tend to pitch their stories to editors. The more sensational sounding the story, and the more exciting the claims made, the more likely the story will be to get published.  An article that applies critical scrutiny to some impressive-sounding press release claim will be unlikely to be published.  By uncritically parroting unfounded but exciting-sounding claims in university and college press releases,  science journalists help to fatten their own wallets.  Often science journalists will imaginatively add their own unwarranted claims and unjustified spin about some research, hoping to further increase their chances of receiving fees for writing exciting-sounding news stories.  In general, science journalists getting paid by word or by article are often very unreliable sources of information.  

To "follow the money" all the way, we must go back to the scientists who originally chose "quick and dirty" designs, and who may have misstated the implications and findings coming from their research. What is the result when "quick and dirty" experiment designs are chosen? The result is that the paper count (the number of published papers) of a scientist will increase more quickly. What is the result when scientists misstate or exaggerate what their observations show or imply, making their research sound important when it is not? The result is a greater number of citations of their papers by other scientists. The very important "citation count" of a scientist will increase.  What is the financial result when a scientist has piled up a high paper count and a high citation count? That scientist will be more likely to get promoted, more likely to get the tenure that gives him a lifetime job, more likely to get a higher salary, more likely to get a lucrative book deal with a major publisher, and so forth. 

What we have is an infrastructure that all over the place incentivizes bad agents who mislead and misinform, as long as such persons mislead and misinform in some way that produces exciting-sounding results that fit in with popular academia belief systems.  Given such an infrastructure, you should not be surprised to hear that today's cognitive neuroscience is a house of cards that mostly rests on an illusory foundation. Most of the things that neuroscientists claim have been established by cognitive neuroscientists have not actually been established by them at all. Most of the more important-sounding claims made in the neuroscience news stories of recent years are claims lacking any solid foundation in observations. Junk science flourishes, because there are so many people in so many different places who profit from junk science. 

"Thousands of Participants Are Needed for Accurate Results," But Most Brain Scan Studies Don't Even Use Dozens

For many years neuroscientists have been claiming important results about brains and minds, after doing brain imaging experiments using very small sample sizes.  For example, we may read headlines saying that some particular region of the brain is more active during some type of mental event,  and the total number of subjects who had their brains scanned will usually be smaller than 15. A new press release from the University of Minnesota Twin Cities announces results which indicate that such small-sample correlation-seeking brain imaging experiments are utterly unreliable.  The headline of the press release is "Brain studies show thousands of participants are needed for accurate results."

We read this:

"Scientists rely on brain-wide association studies to measure brain structure and function—using MRI brain scans—and link them to complex characteristics such as personality, behavior, cognition, neurological conditions and mental illness. New research published March 16, 2022 in Nature from the University of Minnesota and Washington University School of Medicine in St. Louis...shows that most published brain-wide association studies are performed with too few participants to yield reliable findings."

The abstract of the paper in the science journal Nature can be read here. The paper is entitled, "Reproducible brain-wide association studies require thousands of individuals." 

The press release tells us this:

"The study used publicly available data sets—involving a total of nearly 50,000 participants—to analyze a range of sample sizes and found:

• Brain-wide association studies need thousands of individuals to achieve higher reproducibility. Typical brain-wide association studies enroll just a few dozen people.
• So-called 'underpowered' studies are susceptible to uncovering strong but misleading associations by chance while missing real but weaker associations. 
• Routinely underpowered brain-wide association studies result in a surplus of strong yet irreproducible findings."

The claim that a typical brain scanning experimental study uses "a few dozen" people is probably an overestimate. Brain imaging studies touted in the press seem to typically involve fewer than 15 subjects. 

The press release tells us that the conclusions above are based on some very heavy number crunching using databases that store brain scans of a large number of people, including in many cases data on what they were doing or thinking while being scanned, what kind of mental characteristics or health history the people had, and what kind of genes the people had.  The largest such database was the UK Biobank, which according to page 5 of the document here includes "resting-state functional MRI measures changes in blood oxygenation associated with intrinsic brain activity (i.e., in the absence of an explicit task or sensory stimulus)," as well as "task-functional MRI" which "uses the same measurement technique as resting-state fMRI, while the subject performs a particular task or experiences a sensory stimulus." (The task was mainly something called the Hariri faces/shapes “emotion” task.)  Another large database used was a Human Connectome Project database including "task-evoked fMRI" brain scans of people while they were doing things involving working memory, gambling, language, social cognition, relational processing and emotional processing (as mentioned on page 36 of the document here).  Another large database used was an Adolescent Brain Cognitive Development (ABCD) database that included fMRI scans while subjects performed tasks such as a Monetary Incentive Delay task. a Stop Signal task and an "n-back" or "nBack" task (as described here). 

In the press release we read this:

"To identify problems with brain-wide association studies, the research team began by accessing the three largest neuroimaging data sets: the Adolescent Brain Cognitive Development Study (11,874 participants), the Human Connectome Project (1,200 participants) and the UK Biobank (35,375 participants). Then, they analyzed the data sets for correlations between brain features and a range of demographic, cognitive, mental health and behavioral measures, using subsets of various sizes. Using separate subsets, they attempted to replicate any identified correlations. In total, they ran billions of analyses, supported by the MIDB Informatics Group and the powerful computing resources of the Minnesota Supercomputing Institute. The researchers found that brain-behavior correlations identified using a sample size of 25—the median sample size in published papers—usually failed to replicate in a separate sample.  As the sample size grew into the thousands, correlations became more likely to be reproduced. Robust reproducibility is critical for today’s clinical research. Senior author Nico Dosenbach, MD, PhD, an associate professor of neurology at Washington University, says the findings reflect a systemic, structural problem with studies that are designed to find correlations between two complex things, such as the brain and behavior."

What this study very strongly indicates is that the vast majority of brain imaging studies trying to correlate brains and mental states or mental activity have misled us by producing false alarms. The study indicates that such brain imaging studies have not merely been guilty of some slight shortfall, but have been guilty of a hundred-fold shortfall (the difference between about 20 and "thousands" being a difference of a hundred times).  It's as bad as if someone told you he produced a score of 1000 on his SAT test, but really only produced a score of 10. 

The study described above was led by neuroscientist Scott Marek. An article on the study in the journal Nature says this:

“ 'There’s a lot of investigators who have committed their careers to doing the kind of science that this paper says is basically junk,' says Russell Poldrack, a cognitive neuroscientist at Stanford University in California, who was one of the paper’s peer reviewers. 'It really forces a rethink.' ”

The New Scientist article on the Marek study is behind a paywall, but at least I can show its headline:

For many years we have been scammed and the US federal government has been scammed by neuroscientists doing ridiculously low-powered brain imaging studies looking for correlations between brains and minds.  For many years our experimental neuroscientists doing small-sample brain imaging studies (looking for correlations between brain states and mental states) have been playing a game of "sham, scam, thank you Sam," the Sam being Uncle Sam who provided the dollars for such worthless studies producing only false alarms. This is a racket, but since it is a nice little source of dishonest income and easy work for professors, the racket will probably long continue. 

The US government seems to be incredibly poor at recognizing bad performance by biology authorities.  In the New York Times there was recently an opinion article with the headline "How Millions of Lives Might Have Been Saved from COVID-19." Without naming any names of the bumbling officials guilty of the bungled US response to COVID-19, we get some startling comparisons between competent responses in small countries and incompetent responses in the US. For example, we are told that Taiwan has suffered only 853 COVID-19 deaths, and that "if the United States had suffered a similar death rate, we would have lost about 12,000 people, instead of nearly a million."  Because the US government seems to be extremely poor at recognizing bad performance by biology authorities. we will probably continue to see the "sham, scam, thank you Sam" researchers bilking the government by doing worthless federally-funded small-sample brain imaging studies producing only  false alarms. 

One of the quotes above tells us that correlations reported with a sample size of 25 "usually failed to replicate in a separate sample," but that "as the sample size grew into the thousands, correlations became more likely to be reproduced." Does this mean that strong correlations were found between brains and cognitive activity or cognitive states as long as you used samples of thousands? No. The Nature article on the Marek study tells us this:

"Researchers measure correlation strength using a metric called r, for which a value of 1 means a perfect correlation and 0 none at all. The strongest reliable correlations Marek and Dosenbach’s team found had an r of 0.16, and the median was 0.01."  

So even when data on thousands of subjects was used, no strong or medium correlations were found, and the median correlation was a negligible 0.01.  A medium-strength correlation has an r of about .5, and a strong correlation has an r of about .7.  The results discussed above are consistent with the idea that the brain is not the source of the human mind, and is not the storage place of human memories.  Under such an idea, we would expect there to be no strong correlations between brain states and unemotional mental activity such as calm thinking or calm recall.