Proposed future therapy approachOur proposed approach is intended to support (mainly strongly affected) stroke patients in therapy sessions supported by an active exoskeleton, where the unaffected arm of the patient will control the affected arm with the help of the exoskeleton in a so-called mirror mode. The exoskeleton does execute mirrored movements to enable bilateral movement support. This means the movement of the unaffected arm is mirrored to the affected arm while the task for the patient is still to move both arms to induce bilateral motor activity in both hemispheres. This also means that these executed bilateral movements are already part of the therapy so it is not considered as an additional calibration session. However, the exoskeleton is not able to support the affected arm based on the movement intention of the patient. Due to this, we want to predict the movement intention of the affected arm by analysis and decoding of the patient’s EEG. For training a classifier for this task, we use the EEG recorded during mirror mode therapy as explained before. By this cross-task transfer, we completely avoid calibration sessions for using the BCI.Figure 1Comparison between the proposed classifier transfer approach for stroke rehabilitation (see a)) and a standard MI scenario used in rehabilitation (see b)). In (a) a patient, exercising in the exoskeleton’s mirror mode is shown while EEG data is recorded. The classifier is transferred after training to predict the movement intentions of the affected arm. In (b) a standard MI training scenario with an evaluation of the classifier for stroke rehabilitation is shown. The images are used from a supporting video (source: https://www.youtube.com/watch?v=dCn1ktzbpZ8).Proposed classifier training methodTo train an EEG classifier to only predict movement intentions for the affected arm of stroke patients in the future rehabilitation sessions, a two-step concept was developed. It has to be noted that in this work with healthy subjects, we assumed the right arm of the subjects to be the affected one and the left arm to be the unaffected one. In the first step, the EEG classifier is trained during the execution of bilateral arm movements (in a mirror mode rehabilitation session). The onsets of the bilateral movements are inferred from the unaffected arm to generate reliable movement onset labels to train the classifier. In the second step, the classifier is transferred to predict unilateral movements of the affected arm. The transfer approach consists of training on EEG data derived from bilateral movement executions and applying a custom EEG-channel selection to improve the transferability of the classifier by data adaptation. Since the LRP (Lateralized Readiness Potential)55,56, which is associated with movement planning, can be observed from EEG-channels of the motor cortex side contralateral to the moved upper limb57, we focus on the differences and similarities in the EEG data between bilateral and unilateral movement intentions. Therefore, we customized our selection of EEG channels for the data processing, that is related to the planning of unilateral movements with the affected limb. Hence, the abilities of the unaffected limb are used to generate reliable training labels and the classifier can be custom-trained on the provided EEG data, containing information about the movement intentions of the affected arm. The proposed method was evaluated by conducting experiments, involving bilateral and unilateral movement tasks, executed by healthy subjects.The training and testing conditions of the evaluation are illustrated in Fig. 2a): train-test conditions: (A) unilateral-unilateral (no transfer), (B) bilateral-bilateral (no transfer), and (C) bilateral-unilateral (cross-task transfer). Condition C is our target condition, i.e., cross-task transfer takes place. The classifier is trained on EEG evoked by bilateral arm movements and transferred to infer intended unilateral arm movements. Condition A (training and testing on EEG evoked by movements of the supposed to be affected arm) and Condition B are only used to compare how well a classifier performs without transfer compared to the cross-task transfer (Condition C). Moreover, for our envisioned application, the most relevant performance differences are between condition C (cross-task transfer from bilateral to unilateral) and condition A (unilateral-unilateral, no transfer), where training and testing were evaluated on EEG activity recorded from the hemisphere contralateral to the moved arm.Figure 2Concept for the evaluation of the classifier and the results of the ERP analysis (topography and grand average ERPs). Three train-test conditions (A, B, C) were selected for the evaluation of the classifier and shown in (a). The ERP analysis shown in (b) and (c) was performed on the data of the unilateral and bilateral movement tasks. For the ERP analysis, the data was band-pass filtered (\(0.1-4\,\hbox {Hz}\)), eye blinks were removed by manual exclusion of ICA components, and segmented into epochs from \(-1.5\,\hbox {s}\) to \(0\,\hbox {s}\) based on the movement onset. Before averaging epochs, a baseline correction of \(-1.5\,\hbox {s}\) to \(-1\,\hbox {s}\) was applied based on the movement onset. A total of 960 epochs across all subjects were used to calculate the grand average of ERPs for both movement tasks. The grand average ERPs and topography were visualized for the two movement tasks: unilateral in (b) and bilateral in (c).Experimental setup and procedureEight healthy subjects (4 male, 4 female) at the age of \(25.5 \pm 4.0\) years participated in our study. The study was approved by the ethics committee of the University of Bielefeld according to the guidelines of the German Society for Psychology and the Professional Association of German Psychologists and all subjects gave their written informed consent to participate in the study. Only healthy right-handed subjects with no history of neurological or muscular diseases were recruited for the experiments. All subjects were advised to be well-rested for the experiment. The experiment setup is illustrated in Fig. 3. The subjects were seated in a comfortable chair inside a shielded cabin. In front of the subjects a custom-built board, including hand-switches and a button were placed on a table. The subjects were asked to perform a reaching task, by pressing the button with their thumb. The button was placed at a height of approximately \(25\,\hbox {cm}\) and at a distance of \(30\,\hbox {cm}\) away from the resting position. The resting position was defined by the hand switches, where the subjects were asked to place their hands during the resting period. The position of the button was adjusted to the arm length of the subjects. The start and endpoint of the movements were standardized by ensuring a 90-degree forearm-upper arm angle at rest and 0 degree when pressing the button.Two types of movement tasks were conducted in the experiment (see Fig. 2b, c): 1. unilateral reaching movements and 2. bilateral reaching movements. The sequence of the two task types was varied between subjects (counterbalanced) to neutralize possible learning effects. For the unilateral task, only the dominant right arm was moved whereas in the bilateral task, a synchronous movement of both arms (both thumbs pressing the button) was executed. Each task included 3 sets of 40 self-initiated movements. Therefore, each subject performed a total of 120 trials for each task. Each trial consisted of a resting period of at least 5 s followed by a self-initiated and self-paced reaching movement. Trials with a resting period under 5 s were excluded from the evaluation and an error symbol was presented on a monitor for a duration of \(200\,\hbox {ms}\). The error symbol consisted of a fixation cross that turned from a green to a red background color. During the whole experiment, a fixation cross with a green background was continuously shown on the monitor. After each set, the subjects were asked to relax for at least 5 min to avoid any fatigue. The whole experiment was designed and controlled by using the Presentation software [Neurobehavioral Systems, Inc., Albany, USA].Figure 3Experimental setup of the study. In (a) a subject is shown sitting in front of a screen wearing an EEG cap with 64 electrodes. In (b) the custom build experimental board including hand-switches (orange) and a button (blue) as well as the placed EMG-sensors (yellow) and motion tracking marker (green) are illustrated.Data acquisitionEEG data was recorded using a LiveAmp64 amplifier and an actiCap montage with 64 active electrodes [Brain Products GmbH, Munich, Germany]. The electrodes were located according to the extended 10–20 system with FCz as a reference electrode. All impedances were kept below a threshold of \(5\,\hbox {k}\Omega\) and were controlled after each measurement set. The data was acquired at a sampling rate of \(500\,\hbox {Hz}\) and prefiltered by the measurement device to a bandwidth of \(0.1-131\,\hbox {Hz}\). To avoid possible artifacts during the recording, the subjects were asked to avoid head and eye movements as far as possible.EMG signals were recorded bipolar (Ag/AgCl electrodes) by a WavePlus wireless system and picoEMG sensors by Cometa [Cometa srl., Barregio, Italy]. The EMG was sampled at \(2000\,\hbox {Hz}\) and reduced to a bandwidth of 10–\(500\,\hbox {Hz}\) by filters of the measurement device. The signals were recorded from 8 muscles for the right and left side of the body which were: M. biceps brachii medial, M. triceps brachii medial M. triceps brachii lateral, M. deltoideus lateral, M. deltoideus anterior, M. deltoideus posterior, M. trapezius pars descendens (upper trapezius) and M. flexor carpi radialis. The skin was prepared with alcohol and electrodes were placed according to anatomical landmarks58.To mark the physical movement onsets, an infrared motion tracking system [Qualisys AB, Gothenburg, Sweden] was used in addition to the mechanical hand-switches. In total, 4 motion tracking cameras (Oqus \(300+\)) were placed in the shielded cabin to record motion data. To track the motions, 3 reflecting markers were placed on the back of the hand, the elbow (next to the lateral epicondyle) and the deltoideus (muscle belly) on each side of the body. The motion tracking data was acquired at a sampling rate of \(500\,\hbox {Hz}\).All events during the experiments, such as pressing/releasing the hand switches and the button, as well as invalid trials (shown error symbols), were tracked by the EEG system. Additionally, the start and stop of the recordings of each measurement system were recorded by the trigger channels of the EEG system to synchronize all the data in the offline analysis.Estimation of physical movement onsetFor estimating the physical (ground truth) movement onset, the position data tracked by the motion capture system were analyzed and processed in an offline evaluation. Since the executed reaching tasks consisted of moving the hand from a resting position towards the button, the data from the reflective marker of the moved hand was used for the estimation. Note that in a later rehabilitation session, movement onsets will be detected by the exoskeleton25,59. Since only healthy subjects participated in the study, the right arm was assumed to be the affected arm, while the left arm was assumed to be unaffected as stated above. Therefore, for bilateral movements only the position data of the left hand was selected for estimating the ground truth movement onset.In the first processing step, the EEG and motion capture data were synchronized. Afterward, the position data was re-initialized to the resting position by subtracting the mean position data, calculated from the first second (resting period) of each experiment. In the next step, the absolute distance to the resting position was calculated by computing the euclidean distance from the three-dimensional position data. Additionally, the velocity of the hand was calculated for each timepoint by taking the difference between two consecutive samples of the euclidean distance. The velocity was filtered by a lowpass filter with a cutoff frequency of \(4\,\hbox {Hz}\) (butterworth, 4. order) and normalized to the maximum value of the current trial. The distance and velocity were then combined by multiplication in order to provide an exact estimate of the movement onset. This procedure was chosen to calculate the onset, independently of small position fluctuations (producing high-speed values) or slight variations of the resting position between trials.Starting from the movement period towards the resting period, it was searched backward for a data point with a magnitude below a defined threshold. The search started at the time when the mechanical hand switch was released since this is assumed to be the movement onset plus the mechanical delay of the device. The threshold was set to \(0.6\,\hbox {mm}\) and specified concerning the resolution of the motion tracking system after calibration. The movement onsets were marked in the EEG data.Channel selection and reductionIn order to provide proper transferability of the classifier, we custom selected EEG channels by means of the knowledge about the surface distribution of relevant EEG activity with respect to individual EEG channels. Since the LRP can be observed in the hemisphere contralateral to the side of the moved limb (right arm), we custom-selected channels for the classifications that were located on the left hemisphere, especially in the area of the motor cortex. With this approach, we aim to enhance the transferability of the classifier, which is trained on evoked EEG potentials from bilateral movement planning to predict unilateral movement intentions by selecting EEG channels related to right arm movements.Besides enhancing the performance of the transferred classifier, we also aim to reduce the number of channels to provide an approach that is feasible to be used with persons suffering from stroke. Therefore, we systematically reduced the number of EEG channels used for the prediction of movement intentions in order to reduce the preparation effort in a real rehabilitation session. Due to this, we evaluated the use of 32, 21, 16, 8, and 4 custom-selected channels to predict movement intentions. For the different numbers of channels, the selection was made considering the C1 channel as a center of EEG activity related to movement planning, with the other channels located around it. Therefore, by reducing the number of channels the area covered by the electrodes around this center was further reduced in size. The specified EEG channels for the custom selection are illustrated in Fig. 4.Figure 4Custom selected channels from the left hemisphere. Channels used for the study are marked by a red circle. The first 32 channels of the 64-channel cap layout are marked in Green, whereas the second 32 channels are marked in Yellow (combined to a total of 64 EEG channels).In order to evaluate the relevance of custom channel selection, we further compared the custom selection to standard electrode constellations based on the extended 10–20 system. Since such a standard constellation comprises at least 16 EEG channels, we evaluated and compared 32, 21, and 16 channels for the standard constellation as a baseline for our custom channel selection. The standard channel constellations for the different numbers of channels are illustrated in Fig. 5.Figure 5Standard channel constellation based on the extended 10–20 system. The channels used for this study are marked in green.EEG processing and classificationFor the processing and classification of the EEG signals, the signal processing and classification platform pySPACE60 was used. A previously developed machine learning pipeline20, specialized to detect the LRP, was adopted.Preprocessing and windowingThe EEG signals were processed window-wise by cutting out overlapping windows with a length of \(1\,\hbox {s}\) and a stepsize of \(0.05\,\hbox {s}\). For each trial, a total of 81 windows, starting from window \([-5.00, -4.00]\,\hbox {s}\) to \([-1.00, 0.00]\,\hbox {s}\) were cut out with respect to the labeled physical movement onset at \(0\,\hbox {s}\).First, a subset of EEG channels corresponding to the evaluated channel selection methods was included in the next processing steps (see Channel Selection and Reduction). Subsequently, the data were standardized channel-wise (zero mean, SD of one) and decimated to \(20\,\hbox {Hz}\). Next, a FFT bandpass filter with a passband of 0.1–\(4.0\,\hbox {Hz}\) was applied.Feature extraction and classificationThe channel dimension was reduced by applying an xDAWN spatial filter61 with 4 remaining pseudo-channels, which was designed to enhance event-related potentials. Afterwards, the last 4 samples of each window, that correspond to the last \(0.2\,\hbox {s}\), were extracted as time domain features. Therefore, a total of 16 features were extracted for each window. The features were then normalized by applying a Gaussian feature normalization (zero mean, variance one). After feature extraction, an SVM with L1-norm regularization was trained for a binary classification task. The class labels were NoLRP (resting) and LRP (movement intention). To train the classifier, 2 out of 3 recorded measurement sets (80 movement trials) were used as training data and the remaining set (40 movement trials) was separated for testing (for more details see section Performance Evaluation and Metrics). The complexity parameter (hyperparameter) of the SVM was optimized by applying a grid search with 7 equal-spaced values in a range of \(10^{-6}\)–\(10^{0}\) and using a five-fold cross validation on the training dataset to obtain the optimal hyperparameter. After obtaining the optimal complexity value, the SVM was trained on the whole training dataset. The class weights of the SVM were set to a ratio of 1:2 (NoLRP:LRP). The windows \([-1.10, -0.10]\,\hbox {s}\) and \([-1.00, 0.00]\,\hbox {s}\) were used as training instances of the LRP class and the windows \([-3.05, -2.05]\,\hbox {s}\), \([-3.25, -2.25]\,\hbox {s}\) and \([-3.50, -2.50]\,\hbox {s}\) were selected as training instances of the NoLRP class. After training, the classifier was used to predict all windows of a separate test set. This was done to simulate a real online application scenario, where a classifier continuously determines whether EEG windows correspond to a resting period or a period of movement intention. The SVM scores were then transformed into a probability by using Platts sigmoid function62. A probability greater than 0.5 corresponded to a detected intention to move (LRP class); otherwise, a rest period (NoLRP class) was detected.Performance evaluation and metricsSince the class ratios (NoLRP:LRP) are unbalanced for the continuous detection of movement intentions (longer resting periods than movement planning) the balanced accuracy (BA) was used as a performance metric. The balanced accuracy calculates the performance concerning the individual class rates for both classes and is defined as the mean of the true negative rate (TNR) and the true positive rate (TPR). During the evaluation, care was taken that the TNR and TPR for each classification result were not imbalanced to avoid a disbalance or bias between the prediction of the NoLRP and LRP classes.To emulate an online application scenario, the classifier was evaluated by creating set-wise train and test pairs. As mentioned above, for each condition, 2 measurement sets were used for training and the remaining set was used as a test set to evaluate the performance (leave one set out validation). Therefore, for each condition, a total of 24 performance results were produced due to 3 train/test permutations for all 8 subjects.To evaluate the performance results with respect to the characteristics of the LRP, a relabelling technique was applied to the classification outcome in order to generate ground truth labels for performance evaluation. Since the individual planning and execution of a movement, for example, depending on the waiting time, is affecting the temporal characteristics of the LRP63, a variability between single trials must be considered. Since the actual start of the movement planning remains unknown, ground truth labels of the windows were computed based on the classification outcome for each individual trial considering an online application. In the following, the procedure is described in detail.First, a change point of classes (class boundaries) was computed, which gives an estimate of a started movement planning phase after the resting period and therefore the starting point of the LRP class in time. This change point was defined as a point between two consecutive windows that lies within an interval between window \([-2.00, -1.00]\,\hbox {s}\) to window \([-1.00, 0.00]\,\hbox {s}\). This is the range where movement planning is to be expected when continuously classifying windows in an online application scenario. The windows at the boundary of the defined interval correspond to windows where the true label is known with high certainty for the NoLRP (\([-2.00, -1.00]\,\hbox {s}\)) and LRP (\([-1.00, 0.00]\,\hbox {s}\)) class. The choice of the class boundaries was also discussed in our previous work in20. If three consecutive NoLRP windows were counted backward in time (starting from window \([-1.00, 0.00]\,\hbox {s}\) backward) within this range, the label change point was detected and the labels of all windows before this point were set to the NoLRP class and past this point to the LRP class. In case no change point was found inside this range, all windows within this range were defined as instances of the LRP class corresponding to a detected long movement planning phase. However, for windows where the true label is known (from the experimental design), the class label remained fixed for each movement trial. Therefore, all windows before window \([-2.00, -1.00]\,\hbox {s}\) were always instances of the NoLRP class while window \([-1.00, 0.00]\,\hbox {s}\) was always an instance of the LRP class. In conclusion, this technique was used to provide ground truth labels for each sliding window based on the nature of the LRP under predefined constraints where the detection of a movement intention was allowed. To illustrate the procedure, the applied method is shown in Fig. 6.Figure 6Illustration of the relabeling technique. In (a) the determination of the label change point between two consecutive windows is shown while in (b) the ground truth label after applying the method is illustrated.Statistical analysisThe classification performances were analyzed by two-way repeated measures ANOVA with number of channels and train-test condition (Fig. 2a) as within-subjects factors to investigate the effect of cross-task transfer depending on the number of channels: transfer vs. no transfer (see Fig. 7). Additionally, we performed two-way repeated measures ANOVA with channel constellation and train-test condition (Fig. 2a) as within-subjects factors to compare both a standard constellation and custom channel selection for each train-test condition (see Fig. 8).Ethical approvalThe conducted study was examined and found to be harmless by the University of Bielefeld according to the ethical guidelines of the German Society for Psychology and the Professional Association of German Psychologists.
