The authors then confess to us about the miserably bad statistical power of their results. The "rule of thumb" used in experimental studies is that to reach a statistical power considered "good," you need to get what is called a statistical power of at least 80%. But the authors confess their results have a statistical power of only 10%.
Wednesday, February 21, 2024
Universities Continue to Boast About Pitifully Weak Neuroscience Results
The authors then confess to us about the miserably bad statistical power of their results. The "rule of thumb" used in experimental studies is that to reach a statistical power considered "good," you need to get what is called a statistical power of at least 80%. But the authors confess their results have a statistical power of only 10%.
Tuesday, March 29, 2022
Why the Academia Cyberspace Profit Complex Keeps Giving Misleading Brain Research Reports
"Scientists need citations for their papers....If the content of your paper is a dull, solid investigation and your title announces this heavy reading, it is clear you will not reach your citation target, as your department head will tell you in your evaluation interview. So to survive – and to impress editors and reviewers of high-impact journals, you will have to hype up your title. And embellish your abstract. And perhaps deliberately confuse the reader about the content."
Friday, October 30, 2020
Inaccurate Titles and Misleading Citations Are Common in Science Papers
I have discussed at some length on this blog problems in science literature such as poor study design, insufficient study group size, occasional fraud, misleading visuals and unreliable techniques for fear measurement. Such things are only some of the many problems to be found in neuroscience papers. Two other very common problems are:
(1) Scientific papers often have inaccurate titles, making some claim that is not actually proven or substantiated by the research discussed in the paper.
(2) Scientific papers often make misleading citations to papers that did nothing to show the claim being made.
Regarding the first of these problems, scientists often write inaccurate titles to try to get more citations for their papers. For the modern scientist, the number of citations for papers he or she wrote is a supremely important statistic, regarded as a kind of numerical "measure of worth" as important as the batting average or RBI statistic is for a baseball hitter. At a blog entitled "Survival Blog for Scientists" and subtitled "How to Become a Leading Scientist," a blog that tells us "contributors are scientists in various stages of their career," we have an explanation of why so many science papers have inaccurate titles:
"Scientists need citations for their papers....If the content of your paper is a dull, solid investigation and your title announces this heavy reading, it is clear you will not reach your citation target, as your department head will tell you in your evaluation interview. So to survive – and to impress editors and reviewers of high-impact journals, you will have to hype up your title. And embellish your abstract. And perhaps deliberately confuse the reader about the content."
A study of inaccuracy in the titles of scientific papers states, "23.4 % of the titles contain inaccuracies of some kind."
The concept of a misleading citation is best explained with an imaginary example. In a scientific paper we may see some line such as this:
Research has shown that the XYZ protein is essential for memory.34
Here the number 34 refers to some scientific paper listed at the end of the scientific paper. Now, if the paper listed as paper #34 actually is a scientific paper showing the claim in question, that this XYZ protein is essential for memory, then we have a sound citation. But imagine if the paper does not show any such thing. Then we have a misleading citation. We have been given the wrong impression that something was established by some other science paper.
A recent scientific paper entitled "Quotation errors in general science journals" tried to figure out how common such misleading citations are in science papers. It found that such erroneous citations are not at all rare. Examining 250 randomly selected citations, the paper found an error rate of 25%. We read the following:
"Throughout all the journals, 75% of the citations were Fully Substantiated. The remaining 25% of the citations contained errors. The least common type of error was Partial Substantiation, making up 14.5% of all errors. Citations that were completely Unsubstantiated made up a more substantial 33.9% of the total errors. However, most of the errors fell into the Impossible to Substantiate category."
When we multiply the 25% figure by 33.9%, we find that according to the study, 8% of citations in science papers are completely unsubstantiated. That is a stunning degree of error. We would perhaps expect such an error rate from careless high-school students, but not from careful scientists.
This 25% citation error rate found by the study is consistent with other studies on this topic. In the study we read this:
"In a sampling of 21 similar studies across many fields, total quotation error rates varied from 7.8% to 38.2% (with a mean of 22.4%) ...Furthermore, a meta-analysis of 28 quotation error studies in medical literature found an overall quotation error rate of 25.4% [1]. Therefore, the 25% overall quotation error rate of this study is consistent with the other studies."
In the paper we also read the following: "It has been argued through analysis of misprints that only about 20% of authors citing a paper have actually read the original." If this is true, we can get a better understanding of why so much misinformation is floating around in neuroscience papers. We repeatedly have paper authors spreading legends of scientific achievement, which are abetted by incorrect paper citations often made by authors who have not even read the papers they are citing.
A recent article at Vox.com suggests that scientists are just as likely to make citations to bad research that can't be replicated as they are to make citations to good research. We read the following:
"The researchers find that studies have about the same number of citations regardless of whether they replicated. If scientists are pretty good at predicting whether a paper replicates, how can it be the case that they are as likely to cite a bad paper as a good one? Menard theorizes that many scientists don’t thoroughly check — or even read — papers once published, expecting that if they’re peer-reviewed, they’re fine. Bad papers are published by a peer-review process that is not adequate to catch them — and once they’re published, they are not penalized for being bad papers."
We also read the following troubling comment:
"Blatantly shoddy work is still being published in peer-reviewed journals despite errors that a layperson can see. In many cases, journals effectively aren’t held accountable for bad papers — many, like The Lancet, have retained their prestige even after a long string of embarrassing public incidents where they published research that turned out fraudulent or nonsensical...Even outright frauds often take a very long time to be repudiated, with some universities and journals dragging their feet and declining to investigate widespread misconduct."
Tuesday, April 21, 2020
A Diagram of Explanatory Dysfunction in Academia
An achievement legend is a story that is repeatedly told in the classrooms of colleges and universities, a story claiming without proof that some wonderful progress was made by scientists. Such legends include the ones below and many others:
- The story that the
origin of species and the origin of humanity were successfully
explained by the nineteenth century biologist Charles Darwin.
- The story that
neuroscientists have been able to figure out where the human mind
comes from and how human memory works, by studying the human brain.
- The story that the “book
of life” was discovered in the middle of the twentieth century,
when scientists found some molecule that contained a blueprint or
recipe for making human beings.
- The story that
scientists have been been able to figure out important truths about
the evolution of the universe, by coming up with ideas such as dark
matter and primordial exponential cosmic inflation (not to be
confused with ordinary expansion).
All of these stories meet the definition of legend, which is “a traditional story sometimes regarded as historical but unauthenticated.” None of these claims of achievement has been proven. There are very substantial reasons for rejecting each one of them. But these stories of great achievements keep being told again and again by our professors. One of the hardest things to dispel is a dubious achievement legend once it has spread. Such legends provide prestige boosts to various people in academia, and once a person has been hooked on an intoxicating conceit, it becomes incredibly hard for such a person to give it up and adopt a more realistic viewpoint about his relatively modest state of knowledge.
These achievement legends are pillars of a worldview that is predominant at colleges and universities. The worldview is based on the idea that the most important realities such as life and mind are explained by matter, and that such realities arose from blind accidental processes. Believing in the smug achievement legends, those who hold this worldview believe that scientists are making excellent progress in coming up with purely material explanations for life and mental phenomena.
But such a worldview is contradicted by a gigantic number of observational facts. Such facts and observations can be divided into two different categories: the paranormal and the not-at-all-spooky. In the paranormal category are a host of observations that mainstream professors may dismiss as “impossible,” even though they are massively reported. These include things such as apparition sightings, deathbed visions, near-death experiences, anecdotal accounts of clairvoyance, extremely high scores on tests of extrasensory perception (such as we would never expect any human to get by chance), photos of dramatic recurring patterns in mysterious orbs, and so forth. But there is also a huge number of "not at all spooky" observations that stand in opposition to the prevailing academic worldview. Among these are observations such as these:
- Low-level observations
indicating that DNA does not have and cannot have a blueprint or
recipe for making a human.
- Innumerable observations
of incredibly organized biological complexity vastly more fine-tuned
than anything that can reasonably be explained as a result of
chance.
- Observations proving
that brain tissue does not have the long-term information storage
capability it would need to have if prevailing ideas about brains
and minds are correct.
- Observations proving
that massive damage to brains often has only slight effects on mind
and memory, contrary to what we would expect under the dogma that
the mind is merely the product of the brain, and the dogma that
memories are stored in brains.
- Observations
establishing that the universe has many types of physical fine-tuning such as we would not expect under prevailing academic
assumptions.
The evidence I just discussed gives rise to contrarian viewpoints that differ from the prevailing worldview held by academic professors. But there is little or no fair discussion of such contrarian viewpoints in the regimented literature and classroom presentations of academia. If there is any discussion of such viewpoints, an academic authority will typically make sure to label the contrarian thinkers with various defamatory or deprecatory labels designed to prevent anyone from taking their opinions seriously.
Academia produces a huge flow of research and literature. But the research and literature is largely what I call dogma-driven. Dogma-driven research is research designed to reassure professors that they are on the right track, and that their grand explanatory pretensions are justified. Examples include the following:
- Innumerable papers
presenting different variations of Guth's theory of primordial
cosmic inflation, most of which are designed to bolster our
confidence in such a theory.
- Innumerable papers
speculating about dark matter and dark energy, two things that have
never been observed.
- Countless papers trying
to shore up the doctrine of common descent, by presenting 1001
different scenarios describing inheritance paths that life could
have taken to progress through Darwinian evolution.
- Thousands of brain
scanning studies, designed to provide evidence for claims that the
brain is the source of mental phenomena or the storage place of
memories.
- Thousands of papers
presenting variations of the unverified theory of supersymmetry.
Theory-biased analysis occurs both in journals and in textbooks. The observational facts are all passed through the prism of existing theories, which are often based on smug achievement legends. There will be either no discussion of a vast number of observational facts conflicting with such theories, or the discussion will be very skimpy and jaundiced. Reasonable alternative explanations will not be discussed, or will be discussed in some skimpy deprecatory manner.
The result of such research is all too often unimpressive. A significant fraction of all research findings reported in scientific journals cannot be reproduced. The problem was highlighted in a widely cited 2005 paper by John Ioannidis entitled, “Why Most Published Research Studies Are False.”
Almost every day in the science news you can find examples of hyped-up headlines that are inaccurate. The glaring example I find on today's ScienceDaily.com is a story with the extremely false headline "Strongest evidence yet that neutrinos explain how the universe exists." The long-standing matter-antimatter asymmetry mystery is that the Big Bang should have produced equal amounts of matter (protons and electrons) and antimatter (antiprotons and positrons), in incredible densities denser than a neutron star. Since matter and antimatter destroy each other upon contact, forming into only photons of energy, the Big Bang should have left nothing but photons of energy. This problem cannot at all be solved by any possible finding about neutrinos, mere "ghost particles" that are millions of times less massive than electrons, which are 1836 times less massive than protons. When you read the story you will find zero evidence justifying the headline.







