The Brains of 60 Subjects Seemed to Look the Same During Eyes Closed Mental Rest, Recall and Math Activity

The EEG is a device that can detect electrical activity from parts of the brain. When an EEG device is used, electrodes are placed next to different parts of the skull. The device will pick up a dozen or more different lines that show electrical activity in different parts of the brain. 

Brains have a great deal of signal noise, and the abundance of such noise is one of several major reasons for disbelieving that the brain is the source of human thinking and recall which can occur with incredible accuracy, such as when people perfectly recall very large bodies of text and perfectly perform extremely difficult math calculations without using tools such as computers, pencils or paper. The analysis of brain waves obtained by EEG devices is an area of science where bad methods, pareidolia and junk analysis is very abundant.  There is an abundance of people trying to use fancy statistical methods to try to extract identifiable "signals" or "signs" from data that is very noisy and polluted. Muscle movements abundantly contaminate EEG readings. 

A widely used publicly available dataset of EEG data is available on a site called Physionet. On a page entitled "EEG During Mental Arithmetic Tasks" it is possible to download EEG data for 36 subjects. The data includes EEG readings taken during "rest activity" and EEG readings taken when the subjects were told to perform mathematical operations.  The paper here ("Electroencephalograms during Mental Arithmetic Task Performance") describes how the data was gathered.  The data set is sometimes called the "EEG During Mental Arithmetic Tasks" or it may be called something like the "Physionet EEG mental arithmetic task dataset."

I don't recommend trying to download this data, because it uses some file format that your spreadsheet or text editor will not be able to understand.  But at the page here, we have some comments by a person who downloaded this data, and also downloaded a utility program that allows him to see the data represented as particular wavy lines. 

After showing us a picture showing one subject whose brain wave lines looked different when he was doing the math tasks, the writer states, "Other participants didn’t see much change at all while doing their tasks." By this he means that when he looks at the brain waves of such participants, they don't look different when the subjects were doing the math tasks (compared to when they were resting). The writer also states, "In fact, some data looked like the brain had more activity while doing nothing at all."  We see one visual with brain wave lines showing "baseline" activity for Subject 15, and another visual showing brain wave lines during that subject's performance of math tasks. The first visual shows wavy lines that are a lot wavier that the second visual, contrary to the idea that mental activity would involve more active brain waves. 

You can read some scientific papers written by scientists that create algorithms or models that analyze data sets such as this, algorithms or models trying to detect whether a particular set of EEG readings was or was not taken when a patient was engaging in heavy thinking. A typical paper of this type will discuss several different algorithms or models the scientists tested. We may be told that the most successful algorithm had something like a 75% success rate in predicting whether a set of EEG readings were produced rest activity or thinking.  

Such a thing is unimpressive when you consider that the data set being used for testing is usually small. In many cases half of the patient data will be used to "train" the model, and the other half will be used to test the model. So maybe the data for only 8 or 10 patients will be used to test the model.  The odds of accidental success on guessing whether the person's mind was active or not (even if the model is worthless) are something like this (I used the StatTrek binomial probability calculator to calculate some of the odds):

Eight patients:

Chance of 8 guesses all correct = 2 to 8th power = 1/256.

Chance of 7 out of 8 guesses correct = .035

Chance of 6 out of 8 guesses correct = .014

Ten patients:

Chance of 10 guesses all correct = 2 to 10th power = 1/1024

Chance of 9 out of 10 guesses correct = .01

Chance of 8 out of 10 guesses correct = .05

Chance of 7 out of 10 guesses correct = .17

Now, with odds like these it means very little if some scientific paper says that it tried several different predictive models, and found that one of the models had a 70% predictive accuracy. You might rather easily get that level of success by pure chance, even if the model is worthless or if the "mental activity" scans have no identifying characteristics.  We must also remember here factors such as what is called publication bias and what is called the file drawer effect. Publication bias is that scientific journals tend to reject negative results, and accept for publication only papers reporting positive results. The file drawer effect is that scientists are free to try different things without publishing their failures, and without submitting failed attempts for publication. So a scientist who produces a slightly successful predictive model analyzing EEG data may have in his file drawers 40 failed attempts involving unsuccessful predictive models. Getting maybe a "70% successful" predictive model on the 20th or 30th try does not mean that the EEG data actually shows a difference when people are thinking versus when their minds are resting. 

Then there is the fact that the gathering of EEG data must be done very carefully for any data set that compares intensive mental activity with rest activity. Visual activity, muscle activity and stress can produce traces in EEG data.  So, for example, it might be easy to detect the difference between rest activity and mental activity if the subject is motionless and closes his eyes during rest activity, and the subject uses a keyboard to type answers during the mental activity.  In that case the difference would come from the fact that during the rest activity there is no use of the eyes and muscles, and during the mental task there is use of the eyes and muscles. 

The paper here ("Electroencephalograms during Mental Arithmetic Task Performance") describes some poor methods of gathering rest data and mental arithmetic data used to create the "EEG During Mental Arithmetic Tasks" data set that has been the basis of quite a few scientific papers.  We are told this:

"Mental arithmetic performance is considered as a standardized stress-inducing experimental protocol. Serial subtraction during 15 min is considered to be a psychosocial stress. In this way, our study design required intensive cognitive activity from the subjects. Intensive mental load is accompanied by a change in the emotional background when the subject makes additional effort to resolve tasks, so one can talk about evoked emotions in this case.. During EEG recording, the participants sat in a dark soundproof chamber, comfortably reclined in an armchair. Prior to the experiment, participants were instructed to try to relax during the rest state and were informed about the arithmetic task—participants were asked to count mentally without speaking or using finger movements, accurately and quickly, in the rhythm they had determined. After 3 min of adaptation to experimental conditions, EEG registration of the rest state with closed eyes was made (over the next 3 min). Then the participants performed a mental arithmetic task—serial subtraction—for 4 min."

We are also told that the scientists kept only a subset of the original data gathered, throwing out about half of the data:

"Based on EEG visual inspection by a qualified electroneurophysiologist, 30 of the 66 initial participants were excluded from the database due to poor EEG quality (excessive number of oculographic and myographic artifacts), so the final sample size is 36 subjects."

It is easy to see how that could have gone wrong. The desire to get a set of EEGs with mental activity brain scan data looking during different from rest state brain scan data might have come into play, creating a bias in so subjective a selection of which subjects to keep. 

There's much gone wrong here. We have no description of a rest state which is a clear description of a lack of mental activity. Were the subjects hearing something told them during the rest state? That isn't a rest state. Did any of the subjects move during the rest state? That isn't a rest state. Were the subjects counting during the rest state? We can't even tell from the wording above. Did the subjects have their eyes closed when they were doing the mental subtractions? We don't know. Were the subjects disqualified if they violated the instructions by softly speaking as they counted backwards? Apparently not. The subjects were told to follow a rhythm during counting, an instruction which might have tended to produce sounds or motions such as tapping. The subjects were not told to be motionless, but merely told not to use their fingers (an instruction that would not exclude arm movements or foot tapping movements or a rocking motion in their reclining armchair). Also, the subjects were asked about what was the final number after their mental subtractions. That might have created a possible element of anxiety, in which people would be worried about whether the final number (after their mental subtractions) would be a correct one. Such anxiety might have shown up in the EEG readings, which might have shown signs of anxiety that were not signs of mental effort.  Also, based on subjective whims of a human judge only about half of the data collected has been put in the public data set. The mental activity requested (serial subtraction) is a mental activity that almost seems designed to create distress and frustration in subjects, which may show up as EEG blips that are not signs of thinking. 

Data like this has no value unless there is a crystal-clear description of the exact procedure used during the rest state and the mental activity state. That description should include a precise detailing of whether the subjects had their eyes opened, an exact quotation of what they were told, an exact description of whether the subjects moved or spoke, a description of what (if any) methods were used to prevent the subjects from moving, and so forth.  Comparing mental rest states and mental activity states (from EEG data) cannot be done effectively unless the mental activity states occur under the exact sensory conditions and movement conditions of the rest state, and it would seem the only good method would be for patients to have eyes closed (without any sounds) both in the rest state and the mental activity state, without any possible source of mental anxiety in either state.  All papers based on the data set described (the "EEG During Mental Arithmetic Tasks" data set) would seem to have little value because of the failure (in the paper describing how the data was gathered) to describe an effective, well-documented protocol for distinguishing between real rest activity and sightless, soundless, motionless mental activity without any element of potential anxiety. 

I can tell you how a valid data set of EEG data might be created for the comparison of rest data and mental activity data.  People would be blindfolded in a dark silent room. They would be told that when they first hear a first electronic beep, they should remain motionless for two minutes and think of absolutely nothing other than the blackness of outer space. They would be told that when they hear the second beep, they should remain motionless and start some arithmetic activity such as adding the number 7 continually, continuing for two minutes until they hear the third beep, at which point the EEG readings will stop. The people would also be told to remain motionless and without any expression throughout the whole four minutes of testing.  They would also be told that no one will ask them what the final number was in their minds, so that there is no reason for any anxiety. They would be told, "Don't worry at all if you think one of your numbers is wrong -- just keep adding 7 to whatever was your last number was." A variety of sensitive motion detectors could be used to exclude any subjects who moved significantly. The number of subjects in the final data set would be at least 60, requiring an original pool of test subjects much greater.  Exclusion of subjects would be based on an objective criteria such as motion detector activation, rather than some arbitrary exclusion based on subjective human exclusions. Heart rate data would be gathered, and any subjects showing signs of increased heart rate during the mental activity phase (a sign of stress) would be excluded from the data set. Sensitive sound detectors would also listen for people who softly counted the numbers, excluding such subjects. Ideally, the subjects would wear mouth devices preventing any soft counting. 

Papers based on an analysis of data gathered in such a way (with a sufficient study group size) would fail to show any analysis method correctly predicting whether the rest state or the mental activity state occurred, tending to confirm the idea that thinking is not actually produced by the brain.  The accuracy of any such method over multiple tests would never be some high percentage such as 80%. 

In neuroscience papers attempting to do EEG analysis to find neural correlates of mental activity,  we tend to see some of the same problems found in papers attempting to do fMRI analysis to find neural correlates of mental activity.  The biggest problem is insufficient study group sizes.  Claims are made such that if you analyze some EEG data in such-and-such a way, you will be able to tell (with such-and-such an accuracy) whether or not mental activity occurred.  The claims are made on the basis of tiny data sets such as 8 or 10 or 12 patients. Such claims should never convince unless they are done on large data sets involving more than 50 subjects, and unless the data sets are fully documented by a discussion of a sound procedure used to gather the data sets.  Almost always what is being picked up is not signs of mental activity but signs of muscle activity, speech, vision or emotional states. 

Here are some examples of papers that we should not be taking seriously because of defects I will mention. All of these are examples of "how not to do an EEG study looking for brain wave correlates of mental activity." 

• "What does delta band tell us about cognitive processes: A mental calculation study" (link). The study got data on only 18 subjects. The mental calculation activity required muscle movement, and the rest activity did not. So the EEG data was not gathered so that pure mental activity was compared to pure mind resting, and "neural correlates of thinking" claims are invalid. 
• "Real-Time Mental Arithmetic Task Recognition From EEG Signals"  (link).  Data was not gathered in a way to exclude physical differences between rest states and activity and  not gathered in a way to exclude emotional differences between rest states and mental activity.  We are told, "In the relax task, subjects were asked to open their eyes and try to be relaxed. There was no mental arithmetic task to fulfill in this session. Subjects were required to breathe deeply and focus on their breath."  Then we are told in the mental activity state "subjects were required to complete arithmetic calculations as quick as possible." Any differences detected may have been due purely to differences in stress, differences in muscle activity and differences in breathing.  
• "EEG activation patterns during the performance of tasks involving  different components of mental calculation" (link). We have no description of a data gathering method that excluded muscle activity or caused identical levels of muscle activity during the rest period and the mental calculation period.  Any differences detected may have been due purely to differences in muscle activity. 
• "EEG microstate features according to performance on a mental arithmetic task" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "Automated Classification of Mental Arithmetic Tasks Using Recurrent Neural Network and Entropy Features Obtained from Multi-Channel EEG Signals" (link). This paper has little  value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "Impact of mental arithmetic task on the electrical activity of the human brain" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is  defective for reasons I have explained above. 
• "Mental arithmetic task detection using geometric features extraction of EEG signal based on machine learning" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is  defective for reasons I have explained above. 

• "Do specific EEG frequencies indicate different processes during mental calculation? (link). The EEG data was gathered from only ten subjects, and the "rest" state involved no real rest, but looking at a visual and saying, "Nothing." The math calculation involved hard problems such as "a complex arithmetic task, e.g. (24 + 39)/9 = , to which the subject had to give the solution verbally immediately after a warning response signal was presented,"  We have no description of a data gathering method that excluded muscle activity or caused identical levels of muscle activity during the rest period and the mental calculation period.  Any differences detected may have been due purely to differences in muscle activity or differences in stress between the easy task of saying nothing and the stressful task of having to answer the hard math problem "immediately." 
• "Mental Arithmetic Task Recognition Using Effective Connectivity and Hierarchical Feature Selection From EEG Signals" (link). EEG data was gathered from 29 subjects who we are told alternated between a short period of "mental arithmetic" and "rest." We have no indication of whether this "mental arithmetic" was silent or involved speech or muscular activity.  So we can't tell whether muscular activity was the same during the rest period and the mental activity period. 
• "Mental arithmetic task classification with convolutional neural network based on spectral-temporal features from EEG" (link). This study used a too-small dataset made from only 12 subjects.
• "Electroencephalographic Study of Real-Time Arithmetic Task Recognition" (link). There were only eight subjects, and a professional EEG equipment was not even used, but only a cheap consumer device.  There was also no rest state for comparison. 
• "EEG Based Mental Arithmetic Task Classification Using a Stacked Long Short Term Memory Network for Brain-Computer Interfacing" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 
• "A Modified Multivariable Complexity Measure Algorithm and Its Application for Identifying Mental Arithmetic Task" (link). This paper has little value because it used the "EEG During Mental Arithmetic Tasks" data set which is defective for reasons I have explained above. 

The paper "Investigating neural efficiency of elite karate athletes during a mental arithmetic task using EEG" discusses a relatively good protocol for gathering data during rest and mental activity. We are told that during the rest stage subjects were told to keep their eyes closed and do nothing, and during the activity stage subjects kept their eyes closed and silently counted backward from 600, subtracting 3 each time (e.g. 597, 594, 591, and so forth).  But there were only ten subjects, and the paper does not report any great success in distinguishing rest states and activity states, with the investigators concentrating on other things.  

A scientific paper ("A test-retest resting, and cognitive state EEG dataset during multiple subject-driven states" by Yulin Wang and others) laments, "Given the various advantages of EEG including non-invasive, high temporal resolution, easy-to-operate, and cheap as a neuroimaging technique, it is surprising that there exist relatively fewer high-quality, open-access, big EEG datasets when compared to magnetic resonance imaging (MRI) datasets to enable the investigation of the brain function." Correct. In general, neuroscientists involved in EEG analysis have not done their job correctly, and have failed to create large publicly available brain wave EEG data sets using very careful methods like those I describe above, which would minimize the confounding factors of signal artifacts created by muscle movement and emotional states. 

The paper tries to help this situation by creating an EEG public dataset. The effort has some good elements,  but some shortcomings.  Data was gathered for 60 subjects during an eyes open rest state, an eyes closed rest state, and some mental activity states. We read this:

"During resting-state EEG recording, participants were instructed to view a fixation point for five minutes (Eyes Open) and then close eyes for another five minutes (Eyes Closed). They needed to keep still, quiet, and relaxed as much as they can, and try to avoid blinking for Eyes Open (EO) session and stay awake for Eyes Closed (EC) session. EEG cognitive state:  The present experiment consisted of three subject-driven cognitive states: retrieval of recent episodic memories, serial subtractions, and (silent) singing of music lyrics."

Alas, we are not told whether there was any method to exclude subjects who did not follow the instructions to "keep still, quiet and relaxed as much as they can" (methods such as motion detectors), and we do not know whether subjects failing to follow such instructions were excluded. Also, we are not told that the same instructions to "keep still, quiet and relaxed as much as they can" were given to the subjects while they were performing the cognitive tasks. So we don't know whether the levels of motion were the same when the subjects rested and when they did the cognitive tasks. But on the plus side, the number of subjects used (60) is pretty good, and there is also a good "test/retest" feature in which each subject was tested on multiple days. 

Figure 6 of the paper gives us this very interesting visual showing something called the "averaged power spectrum" for all of the 60 subjects. We have five colored lines, two of which (light blue and yellow) represent the rest states, and the other representing the mental activity states. It is interesting that all of the lines are the same, except that for the "eyes open" rest state, part of the line looks a little different. Referring to an "eyes-closed" that was a state of mental inactivity, the paper tells us "the spectrum of the four states of eyes-closed, subtraction, music, and memory are particularly similar." 

This is what we would expect under a "your brain does not make your mind" assumption. There is no significant brain signal difference between someone resting his mind with his eyes closed, and someone doing mental activities.  We see something similar in Figure 7 of the paper, which shows us something called the "power distribution of alpha rhythm." The Eyes Closed rest state (EC, in which people's minds were supposed to be inactive) looks the same as when the people were doing mental activity and mental recall. The last four columns on this chart all look the same, and the second column is the Eyes Closed rest state (the last two columns being memory activity and math activity). 

I find the two visuals above to be quite consistent with the claim that your brain is not the source of your mind and not the storage place of your memories.