"Consciousness Theory Showdown" Shows Mainly Shady Neuroscientist Sleight-of-Hand

When talking about the problem of explaining human minds, those in academia love to use the term "problem of consciousness." But is is a huge fallacy to think there is merely some "problem of consciousness" when there is a trillion times bigger "problem of explaining human minds, human mental capabilities and human mental phenomena."  Once you realize this, you may realize that presenting some "theory of consciousness" can never do much to solve the explanatory problems in the philosophy of mind, which are huge and "all over the place."  

The trick of posing a mere "problem of consciousness" is a ridiculous ruse. A human being is not merely "some consciousness." A human being is an enormously complex reality, and the mental reality is as complex as the physical reality.  You dehumanize and degrade human beings when you refer to their minds as mere "consciousness." The problem of human mentality is the problem of credibly explaining the forty or fifty most interesting types of human mental experiences, human mental characteristics and human mental capabilities.

It is always a silly, stupid trick when someone tries to reduce so complex a reality to try and make it sound like the faintest shadow of what it is, by speaking as if there is a mere "problem of consciousness," and talking as if humans are just "some consciousness" that needs to be explained.  Such a shabby, pathetic trick (which can be called consciousness shadow-speaking) is as silly as ignoring the vast complexity of the organization of the human body, and speaking as if explaining the origin of human bodies is just a task of explaining how there might occur "some carbon concentrations." 

The person attempting so pathetic a trick is acting as silly as a person who stands at the seashore, fills a glass with seawater, and says, "Oceans are easy to explain -- they're just water."  Just as the ocean includes trillions of deep, baffling complexities such as all of the organization and biochemistry of sea creatures -- something infinitely more complex than mere water -- the human mind and human mental experiences involve trillions of complexities, and such a reality is something almost infinitely more complex than mere "consciousness."  The reductionist who engages in consciousness shadow-speaking is someone engaging in a trick as misleading as someone who says, "Mathematics is real simple -- it's just counting." 

The majority of people who try to reduce the mountain-sized problem of explaining human minds and human mental experiences in all their variety into the mouse-sized problem of explaining some mere dry abstraction of "consciousness"  are people who were too lazy to very deeply study minds and brains, and who used this stupid trick of consciousness shadow-speaking to try to make their explanation job a million times easier.  People who lack credible explanations for very complex realities (whether physical or mental) love to use poorly descriptive language in which they try to make the complex realities sound a million times simpler than they are. 

The dialog below illustrates the stupidity of trying to explain human minds by describing a human mind as mere "consciousness" and then trying to create a "theory of consciousness" that applies to everything conscious. 

James: John, I've made great progress in explaining how the human body arises during a mother's pregnancy. John: Great, tell me about it. James: I call my explanation a “theory of solidity.” John: A theory of solidity? James: Yes, because that's the essential nature of human bodies, that they are solid. So my theory attempts to explain how solidity arises. John: I think you've gone in the wrong direction, and made a big mistake. James: Why? John: Because a human body is something gigantically greater than mere “solidity.” A human body is a state of vast hierarchical organization, with a oceanic level of functional complexity. For example, in our bodies are 20,000 different types of protein inventions, most very special arrangements of many thousands of atoms. And we have 200 types of cells, each so complex they are compared to factories. You would do nothing to explain so impressive a reality of physical organization by merely explaining “solidity.” Your body is something gigantically more than mere “solidity.”

James: John, I've made great progress in explaining how the human mind arises. John: Great, tell me about it. James: I call my explanation a “theory of consciousness.” John: A theory of consciousness? James: Yes, because that's the essential nature of human minds, that they are conscious. So my theory attempts to explain how consciousness arises. John: I think you've gone in the wrong direction, and made a big mistake. James: Why? John: Because a human mind is something gigantically greater than mere “consciousness.” You and I are not merely “some consciousness.” We are thinking, believing, seeing, reading, hearing, loving imagining minds with insight, emotions, viewpoints, and a great variety of mental powers such as instant learning ability, the ability to hold memories for decades, and the ability to instantly recall knowledge when only hearing a word or seeing a face. Human minds and human mental experiences are a reality of oceanic depth, so much more than mere “consciousness.”

A recent study attempted to do a "showdown" between two different theories called "theories of consciousness," in an attempt to reveal a winner and a loser. The experimental study should be regarded with the greatest of suspicion, because of all the suspicions we should have about any thing at all calling itself a "theory of consciousness." An experimental showdown between two different theories calling themselves a "theory of consciousness" is rather like trying to do an experimental showdown between the theory of palm-reading and the theory of astrology. The paper is entitled "Adversarial testing of global neuronal workspace and integrated information theories of consciousness."

The paper makes quite a few dubious claims about predictions made by one or the other of these theories.  We should treat with suspicion claims made about what is predicted by either of these theories (the global workspace theory and the integrated information theory). Scientists often make unwarranted claims that this or that theory predicts something. Often such claims are made to try to achieve some aura of predictive success for some theory. It works like this:

(1) A scientist claims that some theory he favors predicts the observation of X. 

(2) The scientist then tries to show that X was observed. 

(3) The scientist then says we should have confidence in the theory because it made a successful prediction. 

Very often this is misleading in one way or another. The claim that the theory predicted the observation of X may be untrue. The claim that X was observed may be untrue. And just because some theory predicts something does not mean the theory is true or likely to be true. There are all kinds of false theories that may predict 1001 things, and some of those things may be true. 

We read in the paper claims about predictions of two rival theories of consciousness:

"We tested three preregistered, peer-reviewed predictions of IIT and GNWT for how the brain enables conscious experience (Fig. 1a). Prediction 1 addresses the cortical areas holding information about different aspects of conscious content. IIT predicts that conscious content is maximal in posterior brain areas, whereas GNWT predicts a necessary role for PFC. Prediction 2 pertains to the maintenance of conscious percepts over time. IIT predicts that conscious content is actively maintained by neural activity in the posterior ‘hot zone’ throughout the duration of a conscious experience, whereas GNWT predicts ignition events in PFC at stimulus onset and offset, updating the global workspace, with activity-silent information maintenance in between. Prediction 3 examines interareal connectivity during conscious perception. IIT predicts sustained short-range connectivity within the posterior cortex, linking low-level sensory (V1/V2) with high-level category-selective areas (for example, fusiform face area and lateral occipital cortex), whereas GNWT predicts long-range connectivity between high-level category-selective areas and PFC."

We should treat with skepticism all of these claims that such statements are actually predictions of such theories, and we should note that none of the claimed "predictions" qualify as precise predictions or exact numerical predictions. The claimed "predictions" are wooly kind of statements that are vague enough to be claimed as true no matter what is observed. Also the claimed "predictions" are not clearly at odds with each other, meaning you don't actually have a situation which is suitable for doing observations and announcing that one of the theories is the winner and the other the loser. 

To perform this dubious "showdown" of these two theories of consciousness, a large number of subjects had their brains scanned in fMRI machines, and another group had their eyes scanned while their brain waves were read using invasive brain-implanted electrodes.  Different images were shown to these observers, with each sight appearing for only about a second. We read this:

"To test critical predictions of the theories, five experimental manipulations were included in the experimental design: (1) four stimulus categories (faces, objects, letters and false fonts), (2) 20 stimulus identities (20 different exemplars per stimulus category), (3) three stimulus orientations (front, left and right view), (4) three stimulus durations (0.5 s, 1.0 s and 1.5 s), and (5) task relevance (relevant targets, relevant non-targets and irrelevant)."

So apparently subjects were shown pictures for a tiny instant, ranging from between half a second and 1.5 seconds. The pictures might have been a picture of a face, an object, a letter such as A or B, or a "false font."  The authors claim to have done "decoding of conscious content" from analyzing data obtained from these subjects: fMRI brain scan data, EEG brain wave data, eye movement data using an eye movement tracker, and magnetoencephalography  brain scan data. The claim is misleading. No robust evidence of any "decoding of conscious content" has occurred. 

To try to back up this claim of "decoding of conscious content," we have a Figure 2 that shows us some result obtained by an AI-type pattern recognizer after analyzing both EEG brain wave data and eye movement data gathered using an eye tracker device (the Eyelink 1000 Plus system shown in a photo below).  We see a "decoding accuracy" graph in which accuracy above 50% completely dies off after 1 second of someone seeing the visual stimulus. This is for 29 subjects who had intracranial electrodes inserted into their brains. The analytics are black-box analytics, and it is hard to unravel what flaws or tricks may have gone on to get these results. A look at the programming code used shows a byzantine maze of spaghetti code. No evidence is provided of being able to predict from brain data alone what a person is thinking or imagining. All we have is some attempt to show that by analyzing brain wave data and eye movement data taken at the instant someone was seeing something, you can predict the category of what the person was seeing. 

It is well known that EEG readings are extremely sensitive to muscle movements, which cause blips in the lines picked up electrodes. So imagine some experiment in which you get EEG readings while someone is shown a picture that may be a recognizable face, a picture of a cute or scary animal, or something neutral like the letter "X" or "Y."  A person might more often make muscle movements when seeing certain types of images. He might give a smile of recognition or appreciation when seeing a celebrity's face or a kitten, or he might squint when seeing some puzzling image, or he might raise his eyebrow when seeing some scary image; or he might grimace when seeing an offensive image. No such muscle movements might occur when the person sees something like the letter X or the letter Y. From such muscle movements alone, an AI pattern classifier might be able to guess higher than 50% what the category was of the thing the person saw. But that would not be "decoding of conscious content." 

Then there's the fact that the eye movement data gathered by some hi-tech eye movement device could have tended to pick up eye movement differences when different categories of images were shown.  Show a human a picture of face, and his eye will tend to focus on the face, or his eye may widen if he is surprised. Show a human a picture of a mere character (such as X) or some meaningless symbol, and the person's eye will not tend to focus as strongly, and it will not widen. 

So the evidence presented for "decoding of conscious content" in this paper is not robust evidence of being able to detect the type of thing someone is thinking about or seeing by analyzing brain data. The evidence the paper presents is all data based on "the moment of perception," when different type of muscle movements or eye movements may have occurred when different type of things are seen. 

Figure 3 in the paper is very similar to Figure 2. We are shown a line graph which seems to indicate some above-average predictive success coming from analyzing iEEG brain wave data (and also eye movement data) coming from 31 patients with implanted electrodes. The claimed success is purely in predicting the category of a type of image someone saw.  But the predictive success only occurs at the half-second mark, vanishing at the one second mark.  The result is consistent with the idea that the claimed predictive success comes purely from picking up different types of muscle movements (such as eye movements) which occur differently when a person has different types  of facial expressions in reacting to things he sees. 

The paper is one of many neuroscience papers which makes false claims about neural representations. There is no evidence that the brain contains any representations of anything anyone learns, recalls or sees. But neuroscientists love to claim that this or that thing they see in the brain is a "representation" of something. In this case the authors again and again refer to "representations" in the brain, without producing any good evidence for any such thing. 

An example of the paper's misrepresentations about representations is its statement "In posterior cortex ROIs, cross-temporal RSA revealed sustained face–object categorical representation." The evidence given for this claim is Figure 3D, which shows no sign of anything beyond the 1.5 second mark after someone saw something.  Whatever is being graphed is some momentary response to a stimulus, and it is  misleading to refer to that as either "sustained" or a "representation."  A similar misstatement would occur if I showed you a picture of something disgusting, and then claimed that your momentary facial expression was a representation of what I showed you. Momentary responses are not representations. 

The authors of this study have failed to produce any robust evidence for either the global workspace theory or the integrated information theory, and the authors give a kind of "it's a draw" verdict about their results, without saying that either theory was the winner. The global workspace theory is not a credible theory of consciousness, for reasons discussed here. The integrated information theory theory is not a credible theory of consciousness, for reasons discussed here and here. 

We have in this study a classic example of how neuroscientists resort to "something else" kind of cheats. Here's how it works:

(1) A neuroscientist will produce a study claiming to have determined something or predicted something based on brain data. 

(2) Sneaked into the study design will be some other source of data other than brain data.  That "something else" may be some software facility that the study is using, such as a database that has text annotations corresponding to images subjects were shown. Or the "something else" may be an eye-tracking system, which allows the study to make predictions not merely on brain data, but on how a person's eyes are behaving. Or the "something else" may be any number of other things, such as some AI system that predicts words someone is about to state, based on historical tendencies of people to say one word after saying a previous word. 

(3) Misleadingly it will be claimed or insinuated (in either the paper itself or the paper's press release) that the study predicted successfully based only on brain data, when any predictive success was crucially dependent on something other than just brain data. 

In this study the sleazy "something else" was eye movement data gathered by some high-tech eye tracker in addition to the EEG data  being taken to detect brain waves. The paper tells us that the Eye Link 1000 Plus system was used. Below is how that system looks (from a page promoting that system). 

Figure 4 of the paper shows the same defects as Figure 3 and Figure 2, as no predictive success beyond the 1.5 second mark is shown, and there is the same reliance on a combination of EEG brain wave data and eye-scanning data, which cannot be called a prediction from brain states alone. Figure 4 is even less reliable as evidence that Figure 2 and Figure 3, because the sample size used is much less than 31. 

In the Supplementary Notes, we read about this funny business going on:

"In the preregistration document, it is stated that iEEG patients with poor behavioral performance, defined as <70% hits or >30% FAs, were to be excluded (Data quality checks and exclusion of subjects, page 15). This threshold was considered based on a target recruitment of 50 patients. However, due to the coronavirus pandemic and despite our best efforts, only 34 patients were collected at the time of manuscript completion. To weigh the pros and cons of data inclusion and to increase sample size and coverage to better test the theories, it was decided to include in the analysis three iEEG patients whose behavior fell marginally short of the predefined behavioral criteria (i.e., hits < 70%, FA > 30%) to compensate for the lower number of participants."

So the authors set a standard for subjects that would be included, and found that they did not have enough subjects if that standard were to be followed. So the standard was then lowered.  But even with that  bit of malfeasance, was the study group size adequate for a good statistical power? We don't know, because no sample size calculation was done. 

The results in this study were highly dependent upon the patients with implanted electrodes, referred to in the paper as iEEG subjects. These were very sick patients with treatment-resistant epilepsy, who were being evaluated for surgery, through a method in which electrodes were implanted to try and find suitable spots for surgery. We are told, "A total of 4,057 electrodes (892 grids, 346 strips and 2,819 depths) were implanted across 32 patients with drug-resistant focal epilepsy undergoing clinically motivated invasive monitoring."  So each of these patients had an average of about 126 electrodes implanted in their brains. Most of the times people have electrodes implanted for epilepsy surgery evaluation, it is a much smaller number of electrodes such as only 20.  

A key question is: were all these electrode implantations medically necessary? Or was there only a medical need to implant a much smaller number of electrodes?  Were many of the risky electrode implants into the brains of these sick patients done purely for the sake of this poorly designed study? We do not know the answer to these questions, because the authors have not told us. They have not made any claim that all of the electrode implants were medically necessary. 

The abuse of very sick epilepsy patients is one of the most appalling scandals of modern experimental neuroscience. Neuroscientists hungry for brain data are luring very sick epilepsy patients into agreeing to implants inside their brains of more electrodes than are needed for surgical evaluation. When this happens, the patient undergoes very serious risks that are not medically necessary, for the sake of the research needs of the neuroscientist and not the needs of the patient. A paper tells us this:

"A recent meta-analysis reviewed complication rates and types of complications in patients undergoing subdural grid implantation for seizure mapping [41]. The most common complication which was reported was intracranial haemorrhage with a mean rate of 4% closely followed by other complications such as neurologic infections, superficial infections and elevated intracranial pressure. They also found that an increased number of electrodes (>67 electrodes) was independently associated with complications."

Another paper tells us this:

"There are definite medical risks associated with the use of intracranial electrodes. The complication rate of subdural electrodes has been reported to range between 6% and 26%.19–23) Relatively common adverse events associated with subdural electrodes are fever, headache, and nausea.24) Another group reported transient cerebrospinal fluid (CSF) leakage (13–31%), infection (6–8%), intracranial bleeding (8%), and cerebral edema in addition to an intracranial mass effect.25) Nair et al. reported that complications included (in the order of their frequency) infection, transient neurological deficit, epidural hematoma, increased intracranial pressure, and infarction.2) An increase in the complication rate was associated with (a) a greater number of grids/electrodes, (b) longer duration of monitoring, (c) older age of the patient, (d) left-sided grid insertion, (e) the use of burr holes in addition to craniotomy, and (f) an earlier year of monitoring (most likely a reflection of the aforementioned surgeon’s experience)."

The authors of any paper that reports on readings of electrodes implanted in the brains of epilepsy patients  have a duty to fully inform us about whether epilepsy patients were endangered by the implantation of additional electrodes that were not medically necessary, and which were implanted mainly for the research purposes of the paper authors.  Any such paper authors that fail to do that are authors we should tend to distrust. 

Postscript: A paper on pre-registration in psychology research finds some dismal results. We read this:

"In this paper, we have investigated adherence and  disclosure of deviations for all articles published with the Preregistered badge in Psychological Science between February 2015 and November 2017 and shared our findings with the corresponding authors for feedback. Two out of 27 preregistered studies contained no deviations from the

preregistration plan. In one study, all deviations were disclosed.

Nine studies disclosed none of the deviations."

What is supposed to go with a pre-registered paper is that scientists publish an exact plan for how they will gather and analyze data, before gathering data, and that they adhere to such a plan. The paper suggests that almost all pre-registered studies actually deviate from their pre-registered plan, and that a large fraction (maybe one third) of studies that deviate from their pre-registered plan fail to report that they deviated from their plan. 

The Brains of 60 Subjects Seemed to Look the Same During Eyes Closed Mental Rest, Recall and Math Activity

The EEG is a device that can detect electrical activity from parts of the brain. When an EEG device is used, electrodes are placed next to different parts of the skull. The device will pick up a dozen or more different lines that show electrical activity in different parts of the brain. 

Brains have a great deal of signal noise, and the abundance of such noise is one of several major reasons for disbelieving that the brain is the source of human thinking and recall which can occur with incredible accuracy, such as when people perfectly recall very large bodies of text and perfectly perform extremely difficult math calculations without using tools such as computers, pencils or paper. The analysis of brain waves obtained by EEG devices is an area of science where bad methods, pareidolia and junk analysis is very abundant.  There is an abundance of people trying to use fancy statistical methods to try to extract identifiable "signals" or "signs" from data that is very noisy and polluted. Muscle movements abundantly contaminate EEG readings. 

A widely used publicly available dataset of EEG data is available on a site called Physionet. On a page entitled "EEG During Mental Arithmetic Tasks" it is possible to download EEG data for 36 subjects. The data includes EEG readings taken during "rest activity" and EEG readings taken when the subjects were told to perform mathematical operations.  The paper here ("Electroencephalograms during Mental Arithmetic Task Performance") describes how the data was gathered.  The data set is sometimes called the "EEG During Mental Arithmetic Tasks" or it may be called something like the "Physionet EEG mental arithmetic task dataset."

I don't recommend trying to download this data, because it uses some file format that your spreadsheet or text editor will not be able to understand.  But at the page here, we have some comments by a person who downloaded this data, and also downloaded a utility program that allows him to see the data represented as particular wavy lines. 

After showing us a picture showing one subject whose brain wave lines looked different when he was doing the math tasks, the writer states, "Other participants didn’t see much change at all while doing their tasks." By this he means that when he looks at the brain waves of such participants, they don't look different when the subjects were doing the math tasks (compared to when they were resting). The writer also states, "In fact, some data looked like the brain had more activity while doing nothing at all."  We see one visual with brain wave lines showing "baseline" activity for Subject 15, and another visual showing brain wave lines during that subject's performance of math tasks. The first visual shows wavy lines that are a lot wavier that the second visual, contrary to the idea that mental activity would involve more active brain waves. 

You can read some scientific papers written by scientists that create algorithms or models that analyze data sets such as this, algorithms or models trying to detect whether a particular set of EEG readings was or was not taken when a patient was engaging in heavy thinking. A typical paper of this type will discuss several different algorithms or models the scientists tested. We may be told that the most successful algorithm had something like a 75% success rate in predicting whether a set of EEG readings were produced rest activity or thinking.  

Such a thing is unimpressive when you consider that the data set being used for testing is usually small. In many cases half of the patient data will be used to "train" the model, and the other half will be used to test the model. So maybe the data for only 8 or 10 patients will be used to test the model.  The odds of accidental success on guessing whether the person's mind was active or not (even if the model is worthless) are something like this (I used the StatTrek binomial probability calculator to calculate some of the odds):

Eight patients:

Chance of 8 guesses all correct = 2 to 8th power = 1/256.

Chance of 7 out of 8 guesses correct = .035

Chance of 6 out of 8 guesses correct = .014

Ten patients:

Chance of 10 guesses all correct = 2 to 10th power = 1/1024

Chance of 9 out of 10 guesses correct = .01

Chance of 8 out of 10 guesses correct = .05

Chance of 7 out of 10 guesses correct = .17

Now, with odds like these it means very little if some scientific paper says that it tried several different predictive models, and found that one of the models had a 70% predictive accuracy. You might rather easily get that level of success by pure chance, even if the model is worthless or if the "mental activity" scans have no identifying characteristics.  We must also remember here factors such as what is called publication bias and what is called the file drawer effect. Publication bias is that scientific journals tend to reject negative results, and accept for publication only papers reporting positive results. The file drawer effect is that scientists are free to try different things without publishing their failures, and without submitting failed attempts for publication. So a scientist who produces a slightly successful predictive model analyzing EEG data may have in his file drawers 40 failed attempts involving unsuccessful predictive models. Getting maybe a "70% successful" predictive model on the 20th or 30th try does not mean that the EEG data actually shows a difference when people are thinking versus when their minds are resting. 

Then there is the fact that the gathering of EEG data must be done very carefully for any data set that compares intensive mental activity with rest activity. Visual activity, muscle activity and stress can produce traces in EEG data.  So, for example, it might be easy to detect the difference between rest activity and mental activity if the subject is motionless and closes his eyes during rest activity, and the subject uses a keyboard to type answers during the mental activity.  In that case the difference would come from the fact that during the rest activity there is no use of the eyes and muscles, and during the mental task there is use of the eyes and muscles. 

The paper here ("Electroencephalograms during Mental Arithmetic Task Performance") describes some poor methods of gathering rest data and mental arithmetic data used to create the "EEG During Mental Arithmetic Tasks" data set that has been the basis of quite a few scientific papers.  We are told this:

"Mental arithmetic performance is considered as a standardized stress-inducing experimental protocol. Serial subtraction during 15 min is considered to be a psychosocial stress. In this way, our study design required intensive cognitive activity from the subjects. Intensive mental load is accompanied by a change in the emotional background when the subject makes additional effort to resolve tasks, so one can talk about evoked emotions in this case.. During EEG recording, the participants sat in a dark soundproof chamber, comfortably reclined in an armchair. Prior to the experiment, participants were instructed to try to relax during the rest state and were informed about the arithmetic task—participants were asked to count mentally without speaking or using finger movements, accurately and quickly, in the rhythm they had determined. After 3 min of adaptation to experimental conditions, EEG registration of the rest state with closed eyes was made (over the next 3 min). Then the participants performed a mental arithmetic task—serial subtraction—for 4 min."

We are also told that the scientists kept only a subset of the original data gathered, throwing out about half of the data:

"Based on EEG visual inspection by a qualified electroneurophysiologist, 30 of the 66 initial participants were excluded from the database due to poor EEG quality (excessive number of oculographic and myographic artifacts), so the final sample size is 36 subjects."

It is easy to see how that could have gone wrong. The desire to get a set of EEGs with mental activity brain scan data looking during different from rest state brain scan data might have come into play, creating a bias in so subjective a selection of which subjects to keep. 

There's much gone wrong here. We have no description of a rest state which is a clear description of a lack of mental activity. Were the subjects hearing something told them during the rest state? That isn't a rest state. Did any of the subjects move during the rest state? That isn't a rest state. Were the subjects counting during the rest state? We can't even tell from the wording above. Did the subjects have their eyes closed when they were doing the mental subtractions? We don't know. Were the subjects disqualified if they violated the instructions by softly speaking as they counted backwards? Apparently not. The subjects were told to follow a rhythm during counting, an instruction which might have tended to produce sounds or motions such as tapping. The subjects were not told to be motionless, but merely told not to use their fingers (an instruction that would not exclude arm movements or foot tapping movements or a rocking motion in their reclining armchair). Also, the subjects were asked about what was the final number after their mental subtractions. That might have created a possible element of anxiety, in which people would be worried about whether the final number (after their mental subtractions) would be a correct one. Such anxiety might have shown up in the EEG readings, which might have shown signs of anxiety that were not signs of mental effort.  Also, based on subjective whims of a human judge only about half of the data collected has been put in the public data set. The mental activity requested (serial subtraction) is a mental activity that almost seems designed to create distress and frustration in subjects, which may show up as EEG blips that are not signs of thinking. 

Data like this has no value unless there is a crystal-clear description of the exact procedure used during the rest state and the mental activity state. That description should include a precise detailing of whether the subjects had their eyes opened, an exact quotation of what they were told, an exact description of whether the subjects moved or spoke, a description of what (if any) methods were used to prevent the subjects from moving, and so forth.  Comparing mental rest states and mental activity states (from EEG data) cannot be done effectively unless the mental activity states occur under the exact sensory conditions and movement conditions of the rest state, and it would seem the only good method would be for patients to have eyes closed (without any sounds) both in the rest state and the mental activity state, without any possible source of mental anxiety in either state.  All papers based on the data set described (the "EEG During Mental Arithmetic Tasks" data set) would seem to have little value because of the failure (in the paper describing how the data was gathered) to describe an effective, well-documented protocol for distinguishing between real rest activity and sightless, soundless, motionless mental activity without any element of potential anxiety. 

I can tell you how a valid data set of EEG data might be created for the comparison of rest data and mental activity data.  People would be blindfolded in a dark silent room. They would be told that when they first hear a first electronic beep, they should remain motionless for two minutes and think of absolutely nothing other than the blackness of outer space. They would be told that when they hear the second beep, they should remain motionless and start some arithmetic activity such as adding the number 7 continually, continuing for two minutes until they hear the third beep, at which point the EEG readings will stop. The people would also be told to remain motionless and without any expression throughout the whole four minutes of testing.  They would also be told that no one will ask them what the final number was in their minds, so that there is no reason for any anxiety. They would be told, "Don't worry at all if you think one of your numbers is wrong -- just keep adding 7 to whatever was your last number was." A variety of sensitive motion detectors could be used to exclude any subjects who moved significantly. The number of subjects in the final data set would be at least 60, requiring an original pool of test subjects much greater.  Exclusion of subjects would be based on an objective criteria such as motion detector activation, rather than some arbitrary exclusion based on subjective human exclusions. Heart rate data would be gathered, and any subjects showing signs of increased heart rate during the mental activity phase (a sign of stress) would be excluded from the data set. Sensitive sound detectors would also listen for people who softly counted the numbers, excluding such subjects. Ideally, the subjects would wear mouth devices preventing any soft counting. 

Papers based on an analysis of data gathered in such a way (with a sufficient study group size) would fail to show any analysis method correctly predicting whether the rest state or the mental activity state occurred, tending to confirm the idea that thinking is not actually produced by the brain.  The accuracy of any such method over multiple tests would never be some high percentage such as 80%. 

In neuroscience papers attempting to do EEG analysis to find neural correlates of mental activity,  we tend to see some of the same problems found in papers attempting to do fMRI analysis to find neural correlates of mental activity.  The biggest problem is insufficient study group sizes.  Claims are made such that if you analyze some EEG data in such-and-such a way, you will be able to tell (with such-and-such an accuracy) whether or not mental activity occurred.  The claims are made on the basis of tiny data sets such as 8 or 10 or 12 patients. Such claims should never convince unless they are done on large data sets involving more than 50 subjects, and unless the data sets are fully documented by a discussion of a sound procedure used to gather the data sets.  Almost always what is being picked up is not signs of mental activity but signs of muscle activity, speech, vision or emotional states. 

Here are some examples of papers that we should not be taking seriously because of defects I will mention. All of these are examples of "how not to do an EEG study looking for brain wave correlates of mental activity." 

• "What does delta band tell us about cognitive processes: A mental calculation study" (link). The study got data on only 18 subjects. The mental calculation activity required muscle movement, and the rest activity did not. So the EEG data was not gathered so that pure mental activity was compared to pure mind resting, and "neural correlates of thinking" claims are invalid. 
• "Real-Time Mental Arithmetic Task Recognition From EEG Signals"  (link).  Data was not gathered in a way to exclude physical differences between rest states and activity and  not gathered in a way to exclude emotional differences between rest states and mental activity.  We are told, "In the relax task, subjects were asked to open their eyes and try to be relaxed. There was no mental arithmetic task to fulfill in this session. Subjects were required to breathe deeply and focus on their breath."  Then we are told in the mental activity state "subjects were required to complete arithmetic calculations as quick as possible." Any differences detected may have been due purely to differences in stress, differences in muscle activity and differences in breathing.  
• "EEG activation patterns during the performance of tasks involving  different components of mental calculation" (link). We have no description of a data gathering method that excluded muscle activity or caused identical levels of muscle activity during the rest period and the mental calculation period.  Any differences detected may have been due purely to differences in muscle activity. 
• "EEG microstate features according to performance on a mental arithmetic task" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "Automated Classification of Mental Arithmetic Tasks Using Recurrent Neural Network and Entropy Features Obtained from Multi-Channel EEG Signals" (link). This paper has little  value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "Impact of mental arithmetic task on the electrical activity of the human brain" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is  defective for reasons I have explained above. 
• "Mental arithmetic task detection using geometric features extraction of EEG signal based on machine learning" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is  defective for reasons I have explained above. 

• "Do specific EEG frequencies indicate different processes during mental calculation? (link). The EEG data was gathered from only ten subjects, and the "rest" state involved no real rest, but looking at a visual and saying, "Nothing." The math calculation involved hard problems such as "a complex arithmetic task, e.g. (24 + 39)/9 = , to which the subject had to give the solution verbally immediately after a warning response signal was presented,"  We have no description of a data gathering method that excluded muscle activity or caused identical levels of muscle activity during the rest period and the mental calculation period.  Any differences detected may have been due purely to differences in muscle activity or differences in stress between the easy task of saying nothing and the stressful task of having to answer the hard math problem "immediately." 
• "Mental Arithmetic Task Recognition Using Effective Connectivity and Hierarchical Feature Selection From EEG Signals" (link). EEG data was gathered from 29 subjects who we are told alternated between a short period of "mental arithmetic" and "rest." We have no indication of whether this "mental arithmetic" was silent or involved speech or muscular activity.  So we can't tell whether muscular activity was the same during the rest period and the mental activity period. 
• "Mental arithmetic task classification with convolutional neural network based on spectral-temporal features from EEG" (link). This study used a too-small dataset made from only 12 subjects.
• "Electroencephalographic Study of Real-Time Arithmetic Task Recognition" (link). There were only eight subjects, and a professional EEG equipment was not even used, but only a cheap consumer device.  There was also no rest state for comparison. 
• "EEG Based Mental Arithmetic Task Classification Using a Stacked Long Short Term Memory Network for Brain-Computer Interfacing" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "A Modified Multivariable Complexity Measure Algorithm and Its Application for Identifying Mental Arithmetic Task" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 

The paper "Investigating neural efficiency of elite karate athletes during a mental arithmetic task using EEG" discusses a relatively good protocol for gathering data during rest and mental activity. We are told that during the rest stage subjects were told to keep their eyes closed and do nothing, and during the activity stage subjects kept their eyes closed and silently counted backward from 600, subtracting 3 each time (e.g. 597, 594, 591, and so forth).  But there were only ten subjects, and the paper does not report any great success in distinguishing rest states and activity states, with the investigators concentrating on other things.  

A scientific paper ("A test-retest resting, and cognitive state EEG dataset during multiple subject-driven states" by Yulin Wang and others) laments, "Given the various advantages of EEG including non-invasive, high temporal resolution, easy-to-operate, and cheap as a neuroimaging technique, it is surprising that there exist relatively fewer high-quality, open-access, big EEG datasets when compared to magnetic resonance imaging (MRI) datasets to enable the investigation of the brain function." Correct. In general, neuroscientists involved in EEG analysis have not done their job correctly, and have failed to create large publicly available brain wave EEG data sets using very careful methods like those I describe above, which would minimize the confounding factors of signal artifacts created by muscle movement and emotional states. 

The paper tries to help this situation by creating an EEG public dataset. The effort has some good elements,  but some shortcomings.  Data was gathered for 60 subjects during an eyes open rest state, an eyes closed rest state, and some mental activity states. We read this:

"During resting-state EEG recording, participants were instructed to view a fixation point for five minutes (Eyes Open) and then close eyes for another five minutes (Eyes Closed). They needed to keep still, quiet, and relaxed as much as they can, and try to avoid blinking for Eyes Open (EO) session and stay awake for Eyes Closed (EC) session. EEG cognitive state:  The present experiment consisted of three subject-driven cognitive states: retrieval of recent episodic memories, serial subtractions, and (silent) singing of music lyrics."

Alas, we are not told whether there was any method to exclude subjects who did not follow the instructions to "keep still, quiet and relaxed as much as they can" (methods such as motion detectors), and we do not know whether subjects failing to follow such instructions were excluded. Also, we are not told that the same instructions to "keep still, quiet and relaxed as much as they can" were given to the subjects while they were performing the cognitive tasks. So we don't know whether the levels of motion were the same when the subjects rested and when they did the cognitive tasks. But on the plus side, the number of subjects used (60) is pretty good, and there is also a good "test/retest" feature in which each subject was tested on multiple days. 

Figure 6 of the paper gives us this very interesting visual showing something called the "averaged power spectrum" for all of the 60 subjects. We have five colored lines, two of which (light blue and yellow) represent the rest states, and the other representing the mental activity states. It is interesting that all of the lines are the same, except that for the "eyes open" rest state, part of the line looks a little different. Referring to an "eyes-closed" that was a state of mental inactivity, the paper tells us "the spectrum of the four states of eyes-closed, subtraction, music, and memory are particularly similar." 

This is what we would expect under a "your brain does not make your mind" assumption. There is no significant brain signal difference between someone resting his mind with his eyes closed, and someone doing mental activities.  We see something similar in Figure 7 of the paper, which shows us something called the "power distribution of alpha rhythm." The Eyes Closed rest state (EC, in which people's minds were supposed to be inactive) looks the same as when the people were doing mental activity and mental recall. The last four columns on this chart all look the same, and the second column is the Eyes Closed rest state (the last two columns being memory activity and math activity). 

I find the two visuals above to be quite consistent with the claim that your brain is not the source of your mind and not the storage place of your memories.