ParticipantsTwenty-two healthy, right-handed participants (13 f/9 m) were recruited for the study. The average age of the study population was 26.4 ± 2.6 years as the inclusion criteria comprised a narrow age-group from 20 to 40 years to minimize the effect of changes in EEG power due to age28. Participants had a usual (work-) day routine of going to bed before 11 p.m. and waking up between 5 and 7 a.m. None of the included participants had worked night shifts for a year or traveled to a different time zone during the 2 weeks prior to participation. Volunteers consuming drugs or nicotine were excluded from the study. During the experiment, caffeine consumption was prohibited, hence participants stating to feel a physical or psychological effect of caffeine withdrawal after 24 h of caffeine absence were not considered in the study. The study was approved by the ethical review board of Graz University of Technology and conducted in accordance with the Declaration of Helsinki. Prior to the study, each participant gave a written informed consent. The study was conducted at the laboratory of the Institute of Neural Engineering of Graz University of Technology.Study procedureThe study procedure was designed in such a way as to mimic a usual working day including demanding tasks after the first and second measurement sessions. These tasks required active engagement from participants in both geometric and linguistic games. After the third recording session, dinner was served that was standardized for all participants and consisted of a pizza of choice. During the last two breaks between sessions, participants were given tasks to induce some sense of tiredness, as encountered in real-life work scenarios. These tasks involved watching a documentary as well as listening to music. Figure 1a shows the timetable of a study day that started at 12 p.m. and finished at 1.30 a.m. after midnight. The experiment consisted of six consecutive measurement sessions performed every 2 h at 2 p.m., 4 p.m., 6 p.m., 8 p.m., 10 p.m. and 12 a.m. (midnight).Fig. 1Study and session procedure. (a) Study process containing six recording sessions performed every 2 h starting at 2 p.m. until 12 a.m. (midnight). During the breaks between sessions, participants were instructed with specific tasks designed to mimic a usual working day. At 7 p.m. the participants were served a standardized dinner. The total duration of the study was 13.5 h. (b) Session procedure composed of seven blocks including an initial impedance check, a PVT, three questionnaires, a 2-min resting EEG measurement, a 6-min eye run for later eye artifact subtraction, the gesture EEG paradigm and another 2-min resting EEG measurement. The session lasted approximately 1 h.Each session lasted approximately 1 h and was composed of several tasks (see Fig. 1b): (i) the EEG electrode check (to visually inspect the alignment of the cap) and impedance check, (ii) a psychomotor vigilance task (PVT)29, (iii) three questionnaires targeting emotions using the Positive and Negative Affect Schedule (PANAS)30, (iv) hunger with the Hunger Level Scale developed by the NEMO (Nutrition Education Materials Online) Management Group of Queensland Government, (v) tiredness by making use of the Tiredness Symptoms Scale (TSS)31, (vi) a 2 min resting EEG recording, (vii) one 6 min eye run to simultaneously record the participant’s EEG and electrooculography (EOG) while performing blinks and eye movements for later eye artifact attenuation (viii) the experimental gesture EEG paradigm as described below and (ix) another 2 min resting EEG recording.In total, 20 complete sets of six measurement sessions were obtained. Due to technical incidents, one measurement session had to be discarded for two participants.Gesture EEG paradigmThe study investigated four different right-hand gestures. The selection of the number of classes was made with the intention of later assessing the efficacy of a classifier designed for a four-direction decoder as part of the INTRECOM project. Each participant was comfortably seated in front of a computer screen positioned at a distance of 50–60 cm while placing the right hand on a table inside a wooden box. The participant was requested to follow the instructions shown on the computer screen. In addition, the participant was asked to avoid eye blinking and swallowing during the duration of a trial period. The gesture EEG paradigm was designed based on work of Ofner et al.32. At the beginning of each trial, the class cue was presented for 1 s corresponding to one of four classes: fist, pistol, pincer grasp and Y-gesture of the American sign language (see Fig. 2). The detailed timing of one trial can be seen in Fig. 3. Here, a fixation cross in front of a filled green circle with a smaller inner white circle was displayed after the cue for a random time interval of 0.5–1 s. The participant was asked to focus their gaze firmly on the cross to avoid eye movements. The presentation of the fixation cross was followed by the preparation period of 2–3 s in which the green circle started to shrink with random speed towards the size of the inner white circle. When the filled green circle touched the inner white circle, the participant was instructed to perform the gesture corresponding to the previously shown class cue. The participant was then asked to hold the indicated gesture for approximately 3 s until the screen went black indicating the end of a trial. Between trials, a break of 1.5 s allowed the participant to go back to the initial position and prepare for the next trial. The total duration of a trial period varied between 8 and 9.5 s.Fig. 2Gestures performed by the participant during the EEG recording of the gesture paradigm. The following gestures are depicted: (a) pincer grasp, (b) pistol, (c) fist, and (d) “Y”-gesture of the American sign language. During the paradigm, execution of the gestures was indicated by a visual cue presented on a screen. The participants were instructed to hold the indicated gesture for 3 s until the appearance of a blank screen.Fig. 3Duration and timing of one trial of the gesture EEG paradigm. The gesture to be executed was indicated by a photograph at the beginning of a trial. After 1 s, the “Gesture cue” disappeared and a filled green circle in addition to a white inner circle and a fixation cross representing the “Ready cue” took its place for a random time interval of 0.5–1 s. During the preparation period, the filled green circle in the background shrunk at random speed for 2–3 s. At the time of the “Go cue”, the filled green circle had hit the inner white circle indicating the start of movement execution. The position of the gesture was kept for 3 s after which a blank screen appeared signaling the end of a trial. One trial duration varied randomly between 8 and 9.5 s.The gesture EEG paradigm consisted of eight movement runs of approximately 5 min with 30 s of break in between. Each run was composed of 32 trials (random cues, balanced), resulting in 256 recorded trials per session and per participant. During the 30 s breaks the participants were instructed to fill out a visual analogue scale (VAS) to assess their level of fatigue.During each session and before the gesture EEG paradigm, an eye run was performed to simultaneously collect the participant’s EEG and EOG activity while at rest with eyes open, capturing eye movements such as blinks, vertical and horizontal eye movements33. The collected data was then used to train an eye artifact attenuation algorithm as described by Kobler et al.33,34.QuestionnairesDuring each of the sessions prior to the gesture EEG paradigm, participants were asked to complete questionnaires evaluating their state in terms of emotion, hunger, tiredness, and fatigue. These questionnaires were either given in English or translated to German depending on the participant’s preferences. To assess the participant’s emotions, the Positive and Negative Affect Schedule (PANAS)30 was used, consisting of twenty different emotional states (e.g., interested, excited, upset) rated on a scale from 1 to 5 (1 being low and 5 being high). The PANAS score was divided into a positive and negative affect score by calculating the average of the ten positive and ten negative emotions, respectively. A non-standardized questionnaire introduced by the NEMO Management Group of Queensland Government was used to evaluate the participants’ level of hunger. The questionnaire listed several physical sensations of hunger, and participants were asked to pick the most suitable one on a scale from 1 (indicating starvation) to 10 (indicating a full, uncomfortable feeling in the stomach). Furthermore, various symptoms of tiredness were evaluated using the Tiredness Symptoms Scale (TSS)31 listing fourteen symptoms rated from 0 to 14 (with 0 indicating no impact and 14 indicating a significant impact). In addition, during each of the seven 30 s breaks between paradigm runs, the participants were instructed to rate their level of fatigue using a visual analogue scale (VAS) in the range of 0 (not at all fatigued) to 10 (extremely fatigued).RecordingFor the EEG measurements, 60 active, gel-based electrodes (actiCAP Brain Products GmbH, Germany) were placed on the scalp according to the 10–10 international electrode standard setup covering frontal, central, parietal, occipital and temporal areas. Four additional active electrodes were used to measure the electrical activity induced by the dipolar behavior of the eyes and were positioned at the outer canthi of the eyes as well as on the inferior and superior of the left eye. The reference and ground electrode were placed on the right mastoid and the forehead at the position of FPz, respectively. The EEG and EOG signals were recorded at 500 Hz using biosignal amplifiers (BrainAmp, Brain Products GmbH, Germany).A video camera was placed above the participant’s right hand inside a box, with a green marker attached to the nail of the right index finger to enable movement tracking by a motion capture system developed at the institute. The sampling frequency of the camera was 30 Hz.The EEG cap was worn continuously throughout the entire study without being removed. Since we opted for a gel-based setup, cap shifting due to head movements was minimized. Participants were instructed to avoid sudden movements or scratching their heads throughout the study. In the case of an itchy scalp, participants were permitted to use the blunt needles used for gel application in a gentle and cautious manner for relief.PreprocessingThe recorded signals were processed using MATLAB R2022b (MathWorks. Massachusetts, USA) and the open source toolbox EEGLAB35. At first, the raw EEG signals were inspected visually, channels contaminated by noise were identified and interpolated by the four closest electrodes weighted by their inverse distance to the bad channel. Further, the 50 Hz line noise and its first harmonic at 100 Hz were removed using a Butterworth bandstop filter of 2nd order. A Butterworth highpass filter of 5th order with a cutoff frequency of 0.3 Hz was employed to avoid stationary artifacts such as drifts. High frequency noise above 70 Hz was attenuated using a Butterworth lowpass filter (of order 70). In order to remove EOG artifacts such as blinks or saccades, an eye artifact attenuation model was trained on the pre-recorded and processed eye run data of each participant using the SGEYESUB algorithm, as described by Kobler et al.34. The most frontal row of electrodes (AF) was not included in following processing steps to minimize the impact of residual artifacts related to eye movements. Pops and drifts in the EEG signals were attenuated using the HEAR algorithm36. Subsequently, temporal electrodes (FT7 and FT8, T7 and T8, TP7 and TP8) were excluded due to their higher noise levels compared to the other channels and their lack of relevant information for subsequent analysis in both sensor and source-space.Movement-related cortical potentialsMRCPs were extracted using a lowpass Butterworth filter (4th order) with a cut-off frequency of 3 Hz. The data were segmented into epochs of 5.5 s, ranging from 2.5 s before to 3 s after movement onset. The movement onset was obtained from simultaneous recordings of the movement of the right index finger using a motion capture system that extracted the x- and y-coordinates of the green marker. By calculating the speed and applying thresholding, the onset of the movement was detected. In order to remove bad trials to reduce the impact of transient artifacts, trial rejection based on a threshold of ± 100 µV was performed on the broad-band data. These trials were then removed from the lowpass-filtered epochs. On average, approximately 7 of the original 256 recorded trials were removed per participant and session. Additionally, 15 trials had to be excluded from further analysis as the motion capture system was incapable of tracking the movement. After downsampling the epochs to 9 Hz, the data was re-referenced to a common average reference.Analysis of MRCPsIn order to examine temporal variations in the dynamics of MRCPs, we first assessed whether MRCPs associated with a particular gesture exhibited significantly greater deviations over time compared to those associated with other gestures. Therefore, we averaged epochs of a session and a specific gesture to obtain the MRCPs at each channel for every participant. The negative peak of the motor potential and the positive peak of the post-motor potential were then extracted. Their relative values were calculated by comparing them to those obtained during the initial measurement session at 2 p.m. In order to investigate variations among gestures and across different sessions, a Kruskal–Wallis test37 was conducted with a statistical significance level of 5%. To address the issue of multiple comparisons, p-values were adjusted following the Benjamini–Hochberg procedure38.Amplitude of motor and post-motor potentialTo examine potential deviations in MRCPs over time regardless of the executed gesture, the amplitudes of both the motor and post-motor potentials were isolated from the average MRCPs for each channel across all four gestures for each measurement session. A Wilcoxon signed rank test39 was then employed for every channel to assess statistical significance between every pair of sessions, using a significance level of p = 0.05. As before, the method of Benjamini–Hochberg38 was applied to correct the p-values for multiple comparisons regarding the number of channels and sessions.To explore the correlation between fatigue, tiredness, and the amplitudes of the MRCPs, we utilized linear mixed-effect (LME) models. LME models represent statistical models that are useful in the context of studies with repeated measurements to account for correlations between multiple observations. These models incorporate fixed effects, which are the ones of interest, and random effects that account for variations between subjects or experimental conditions40. Here, we implemented LME models with participant-dependent intercepts to investigate the influence of fatigue and tiredness on MRCP amplitudes. Specifically, we employed univariate LME models for each channel, using VAS fatigue assessments (averaged across the seven results obtained during the breaks between measurement runs of the gesture EEG paradigm) or TSS scores as independent variables. The response variables in these models were the amplitudes of either the motor or post-motor potentials within each channel. The LME p-values for each channel and analysis were corrected in terms of multiple comparisons by using the procedure of Benjamini–Hochberg38.Temporal evolution of MRCP dynamicsFor further examination of temporal changes in MRCP dynamics, a Wilcoxon rank sum test39 was employed to compare MRCP patterns between every pair of sessions. Therefore, epochs of all four gestures and all participants were combined and statistical testing was performed for each channel and at each timepoint of the movement epoch. A sampling frequency of 9 Hz was used. To control for the false discovery rate introduced by multiple testing of sessions, channels and timepoints, as previously, Benjamini–Hochberg procedure38 was implemented.Source localizationTo obtain the brain sources and induced power changes associated with movement, we made use of the Brainstorm toolbox provided by Tadel et al.41. We computed the head models by employing boundary element methods in accordance with the MNI/Colin27 anatomy template included in Brainstorm and the general EEG electrode positions embedded in EEGLAB. For the estimation of the noise covariance matrix, we estimated the average covariance of 1 s intervals from the first preprocessed resting EEG recording of each session across conditions. As input, we used epochs within the time interval from − 2.5 to 3 s around the movement onset and within the frequency range of 0.3 to 70 Hz sampled at 500 Hz. These epochs were averaged across conditions and participants. For in-depth analysis of specific frequency bands, we bandpass filtered the signals with a stopband attenuation of 60 dB in the frequency range of delta (0.5–3.5 Hz), theta (4–7.5 Hz), alpha (8–12.5 Hz), beta (13–29.5 Hz) and gamma (30–70 Hz) using Brainstorm. Subsequently, we used the standardized low-resolution brain electromagnetic tomography (sLORETA) to estimate the 15,002 brain sources with dipole orientations perpendicular to the cortical surface27. As described in Pereira et al.19, we retrieved the significantly different voxels with respect to a baseline, from − 1.75 to 1.5 s relative to the movement onset, by implementing a parametric t-test. To address the issue of multiple comparisons, Benjamini–Hochberg correction38 was applied for the number of voxels, timepoints, measurement sessions and frequency bands to adjust the p-values.ClassificationFor the classification of the four gestures, a multiclass shrinkage linear discriminant analysis (sLDA)42,43 classifier was applied to the extracted epochs. As input, causal windows of 1 s were shifted along the epochs in steps of 1/9th of a second. Feature extraction windows of 1 s had previously yielded promising results in Ofner et al.32. Classification was performed offline and individually on each participant. To investigate whether variations in EEG during movement tasks over time influence the decoding capability of a classifier, five decoding strategies were employed.To assess the general decoding capability of a classifier being trained on data collected at various timepoints throughout the day and night, two approaches were investigated. Strategy 1 made use of a fivefold cross-validation by combining four folds of each session as a training set. The trained classifier was then applied to the remaining fold of each measurement session. Strategy 2 was based on a leave-one-out approach by using five sessions as training set and the remaining session as testing set. This strategy was applied to each one of the six sessions by training on all sessions except for the corresponding one used as a test set. Two of the participants did not have a complete set of measurements, therefore only twenty participants were considered for this approach.To assess the performance of a classifier in the context of continuous data inflow, three training schemes were employed. Strategy 3 considered a fivefold cross-validation within each session individually. Strategy 4 used an incremental number of sessions as training set and subsequently applied the model on the next measurement session in chronological order (i.e., train on session 1 and test on session 2, then train on session 1 and 2 and test on session 3 etc.). Strategy 5 combined the first two sessions as a training set and evaluated the model on the remaining four sessions. As stated previously, for the fourth and fifth training scheme two participants had to be excluded as their datasets were incomplete.
