Experimental paradigms and data processing algorithms design in real time cognitive neuroimaging тема диссертации и автореферата по ВАК РФ 19.00.02, кандидат наук Волкова Ксения Владимировна

• Волкова Ксения Владимировна
• кандидат наук
• 2021

Оглавление диссертации кандидат наук Волкова Ксения Владимировна

Contents

Introduction

Chapter 1. Literature review

1.1 Characteristics of BCI systems

1.1.1 Recording techniques

1.1.2 Experimental paradigm

1.1.3 Neural phenomena

1.1.4 Signal processing

1.2 Decoding movement from ECoG

1.2.1 ECoG methodology

1.2.2 Motor paradigms

1.2.3 Decoding algorithms

1.2.4 Conclusions

Chapter 2. Advanced solutions for non-invasive motor imagery BCI

2.1 Motor imagery EEG BCI

2.2 Physiologically relevant CSP topographies selection

2.3 Latent variable method for detection of background states

Chapter 3. Experimental setups for ECoG research

3.1 Synchronous recording of continuous movement and ECoG signal

3.2 Realtime movement decoding

3.3 Digitizing tablet input

3.4 Passive functional mapping

3.4.1 Motor mapping

3.4.2 Tactile stimulation

3.4.3 Intraoperative speech mapping

3.5 Cortical stimulation mapping

Chapter 4. ECoG signal processing and data analysis methods

4.1 Preprocessing and denoising

4.1.1 Ocular artifacts removal

4.1.2 Removal of interictal activity

4.2 Decoding movement parameters using classical and deep learning

methods

4.2.1 Classical decoding methods

4.2.2 Deep neural network architectures for ECoG decoding

4.2.3 Electromyography as a proxy for ECoG based experiments

Chapter 5. Decoding movement from ECoG

5.1 Offline decoding of finger movement

5.1.1 Experiment design

5.1.2 Data analysis

5.2 Acquisition of online control of ECoG BCI

5.2.1 Experiment design

5.2.2 Data analysis

5.2.3 Discussion of the results

Chapter 6. Mapping of the eloquent cortex

6.1 Stimulation mapping of the sensorimotor cortex

6.2 Stimulation-free mapping

6.2.1 Comparative analysis of decoding algorithms

6.2.2 Passive speech mapping

Conclusions

References

List of abbreviations

Appendix A. Patient selection procedure for ECoG studies

Appendix B. Processing subject anatomy and extracting sensor

locations

Appendix C. Cortical stimulation responses

Appendix D. Implementation of the probabilistic model for latent

variable method

Введение диссертации (часть автореферата) на тему «Experimental paradigms and data processing algorithms design in real time cognitive neuroimaging»

Introduction

Dominantly, psychophysiological and social aspects of human behaviour (language, consciousness, social interaction, learning) are implemented by means of a motor system required to produce speech, handwriting and express oneself with gestures. Therefore, motor system impairment often leads to the inability of a human to communicate, express oneself, form memories etc. This is especially so in case of severe injuries resulting in the forced loss of communication and social interaction leading to depression and other undesired symptoms. In some cases motor function loss can be ameliorated by establishing a direct communication channel between the brain and a technical device capable of performing the lost function, e.g. a robotic arm or a speech generator operating under the real-time control directly derived from the brain signals.

Systems that process electrophysiological measurements of brain activity in real time, in particular, brain-computer interfaces (BCIs), are used in a number of fields for various research and practical purposes (Abdulkader et al., 2015). Since a BCI provides an additional channel that allows information exchange with the environment that is independent of conventional pathways such as peripheral nerves and muscles, its use is primarily relevant for restoring movement and communication capabilities in people who have lost their mobility due to illness or injury (Chaudhary et al., 2016).

In Bernsein's words, "the most accurate definition of movement coordination is solution of redundancy in degrees of freedom of the effector, i.e. transformation of the latter into a controlled system. This task is solved by the principle of sensory corrections that are carried out conjointly by various afferentation systems, structured according to the formula of reflectory loop" (Bernstein, 2009). In the loop existing naturally in the organism, the organ being moved is a part of the body, sensory corrections are ensured by the afferentation of this organ, and the neural signalling in the human brain reflects the state of this system. In the case of the loop artificially created by implementation of a BCI, neural signalling in the brain results in the movement of an external device such as an artificial hand. Similar to the natural case, the state represented in the brain reduces the degrees of freedom of this effector, and the afferentation providing sensory corrections partially relies

on visual information. However, to provide haptic and proprioceptive information that is part of the afferentation in the natural movement, special means need to be implemented.

However, other than a clinical necessity, BCI is also an emerging tool in modern cognitive neuroscience. The prospects of using neural interfaces for this purpose were highlighted in the roadmap of brain/neural-computer interaction (Horizon, 2020) that suggested using BCI to perform real-time analysis of neural signals to investigate and understand brain and cognitive functions. This direction was identified as one of the prominent non-clinical BCI use scenarios to be developed.

The key characteristic of BCI that makes it a powerful research tool is operation in a closed-loop paradigm where user can volitionaly modulate their neural activity towards a certain objective and receive direct feedback on the state of this activity (Batista, 2020). In this setting, many parameters regarding the activity, objective and feedback can be shaped by researchers. Thus, BCI provides a method to causally manipulate neural activity, compelling the user to generate patterns with specific properties and observing the neural and behavioural consequences. These patterns are shaped by tasks and driven by the subject volitional control, creating a more controlled and flexible alternative to traditional perturbation techniques such as lesions or stimulation. This approach makes it possible to study the neural processes underlying various cognitive tasks such as learning, decision making, attention, mental imagery, perception and, of course, motor control (Jensen et al., 2011).

Challenges of understanding the inner workings of motor system and importance of modelling the whole loop (including inner representations of the action, expectations, perception, and action modification) that were first articulated in the seminal research of motor control mechanisms (Bernstein, 1966; Anohin, 1980) still persist. Modelling the mechanisms of motor learning, internal representations and resolving redundancy (Bernstein, 1966; Batista, 2020) are essential research problems that can be approached with the use of BCI technology. Our research contributes to these topics, not only employing the principles of motor control to implement the realtime interface, but also creating a setting in which where these principles can be further researched.

The fact that BCI allows to research brain function but also requires knowledge of it during the building and tuning of the system constitutes another important

topic related to this field: the man-machine relationship and characterization of BCI performance in terms of usability, feasibility, workload, satisfaction, impact on quality of life, and consideration of psychological factors (Kogel et al., 2019). Although these details are rarely the focus of BCI development, they are important and proved to be necessary to enable the results reported in this work. In our research, we discuss them along other factors, considerations and motivation of the system elements that were used, such as the choice of experimental paradigm, decoding algorithms, and, importantly in case of this work, the recording technique.

Brain-computer interfaces, including motor imagery BCIs (which detect states related to the execution of real or imaginary movement) often employ non-invasive neuroimaging techniques, in particular, EEG (Machado et al., 2010; Padfield et al., 2019). As a method of quantifying bioelectric activity of the brain, EEG has a number of advantages (such as accessibility, ergonomics, safety) that justify its widespread use. However, effective bandwidth of the control channel in an EEG BCI is bound by the properties of EEG signal that include unsteadiness, high noise levels, the presence of myographic and oculographic artifacts, and the low spatial resolution that limits the number of possible recognizable states (Mak and Wolpaw, 2009; Waldert, 2016).

To improve BCI characteristics such as robustness and decoding accuracy, various methods can be employed, including those drawing upon the use of additional a priori information about the physiology, electrical phenomena used to control the interface, or features of the experimental paradigm (Jayaram et al., 2016; Padfield et al., 2019).

However, since the limitations of non-invasive interfaces are primarily related to the physical properties of the method used to measure the electrical activity of the brain, significant enhancement of the information content of the recorded signal and, consequently, the bandwidth of the information channel can be achieved only by invasive neuroimaging (Waldert, 2016). Invasive interfaces that require cortical implantation of microelectrode arrays (Kim et al., 2018) allow to decode various movement parameters and implement control of devices with a large number of degrees of freedom (Hochberg et al., 2012; Collinger et al., 2013; Miranda et al., 2015). However, the implementation of such interfaces poses risks associated with surgery (Kohler et al., 2017) and, for this reason, is limited to individual patients for whom specialized systems have been developed within the clinical environment

(Miranda et al., 2015) as well as animal studies (Carmena et al., 2003; Velliste et al., 2008).

At the same time, a currently emerging area of BCI development comprises the use of electrocorticography (ECoG), in which electrodes are placed on top the brain surface, either subdurally or epidurally (under or on top of the dura mater), without disrupting the integrity of the cortex (Schalk and Leuthardt, 2011a). Electrocorticography is considered to be a safer method than microelectrode implantation and is routinely used in clinical practice for the purpose of seizure focus localization, identification of tumor boundaries and mapping of the eloquent cortex (Hill et al., 2012).

The properties of ECoG (high spatial resolution, low noise levels, near absence of oculographic and myographic artifacts, proximity to the signal sources) and its relative accessibility in comparison to other invasive recording methods (patients implanted with ECoG electrodes for clinical purposes can participate in research while undergoing monitoring) have led to growing amount of studies employing this technology. ECoG signal provides multiple features that allow to quantify local cortical activations as well as interaction between different areas. The dynamic of these parameters allows to characterize the changing state of the network and study the representation of various external and internal parameters in it. In this way, ECoG-based research is advancing understanding of the mechanisms of cognitive processes, including working and episodic memory, language, and spatial cognition, and providing new insights to modelling their function.

As applied to motor control, ECoG signal features enable accurate detection of the movement onset, distinguishing the movement of individual fingers, decoding parameters such as exerted force, movement speed and direction, and utilization of the interface for control of a complex prosthetic arm (Ball et al., 2009; Kubanek et al., 2009; Yanagisawa et al., 2011; Chestek et al., 2013; Hotson et al., 2016). Thus, multiple movement parameters are represented in ECoG along with features representing other cognitive processes, providing an opportunity to study and model patterns underlying movement planning and execution.

In addition, the electrodes used to record ECoG signal can also be used for cortical stimulation, which in some cases is part of the mapping procedure (Ritaccio et al., 2018; Kramer et al., 2019). In this way, ECoG provides opportunities for research and development of methods that, with the advancement

of implantation technologies, can provide the foundation for the development of complex bidirectional brain-computer interfaces. The design of such systems will require the development of experimental paradigms and signal processing methods that enable real-time decoding of motion parameters and cortical stimulation.

Thus, current trends in the field of brain-computer interfacing include development of data analysis methods that improve the performance of non-invasive interfaces as well as the design of experimental paradigms and signal processing algorithms for the interfaces based on invasive technologies. The work described further in this thesis belongs to the field of real-time cognitive paradigms, in particular, the paradigms used for the implementation of brain-computer interfaces. The scope of our studies includes experimental paradigms and data analysis methods that allow decoding movement parameters from electrophysiological activity of the brain.

Current state of research

The history of non-invasive BCIs development has shown that the use of modern methods of signal processing and data analysis, as well as additional information from the field of neurophysiology and ergonomics allows the implementation of neural interfaces that can be used to restore movement and communication capabilities, as well as for other clinical and research purposes (Abdulkader et al., 2015). However, due to the fundamental limitation of control signal bandwidth that can be achieved with non-invasive neuroimaging techniques (Waldert, 2016), the opportunities for the further enhancement of the capabilities of neural interfaces, including motor imagery BCIs, lie in the area of invasive neuroimaging techniques such as intracortical implantation of microelectrode arrays (Miranda et al., 2015) and electrocorticography (Schalk and Leuthardt, 2011a).

Furthermore, ECoG is a particularly promising method for BCI implementation due to higher long-term stability of the signal relative to intracortically implanted electrodes (Shokoueinejad et al., 2019), low noise levels and high spatial resolution, the availability of quantifying frequency band power measurements in high gamma range that reflect local neuronal interactions in the cortex (Schalk and Leuthardt, 2011a), and the relatively large number of patients subject to monitoring that requires ECoG electrode implantation for clinical

purposes, who do not need to be exposed to additional risks of surgery to take part in research.

For this reason, a large number of works that implement movement parameter decoding from ECoG, with different electrode positions, experimental paradigms and signal processing algorithms have been completed in the last two decades. These studies have shown that it is possible to detect from ECoG signal the movement of hand (Pistohl et al., 2012; Bleichner et al., 2016), individual fingers (Kubanek et al., 2009; Hotson et al., 2016), tongue and lips (Graimann et al., 2003; Miller et al., 2007), and legs (Satow et al., 2003). The movements are usually performed on the body side contralateral to the electrodes location on the cortex, but the possibility of decoding movements performed on the ipsilateral side of the body has also been considered (Hotson et al., 2014). Several systems have implemented continuous pointer control based on voluntary modulation of ECoG signal by the user achieved by attempting movements detected by the interface (Leuthardt et al., 2004; Schalk et al., 2008; Wang et al., 2013). A number of works were dedicated to the task of discerning discrete hand positions and reproducing them with the prosthesis (Yanagisawa et al., 2011; Chestek et al., 2013; Hotson et al., 2016).

However, the problem of real time decoding of continuous movement parameters currently remains relevant, since reliable prediction of multiple movement parameters such as speed, acceleration of trajectory would enable the implementation of complex biomimetic interfaces. In addition, the possibility of creating sensation of the prosthesis through micro-stimulation of sensory cortex areas and sensory substitution methods is currently being considered (Johnson et al., 2013; Hiremath et al., 2017; Lee et al., 2018; Kramer et al., 2019). As sensory feedback plays a major role in planning and executing the motion (Cronin et al., 2016), such capability will help to provide more natural and coordinated control of the device.

Research aims and objectives

The purpose of this work is the development of methods that improve decoding characteristics of a brain-computer interface (in particular, the accuracy of real time movement parameters decoding), and the design of experimental paradigms that enable registration of electrophysiological signals informative of movement parameters as well as facilitate learning the skill of BCI control. The distinctive feature of this work lies in development of a series of improvements to

the BCI methodology based on the psycho- and neurophysiological principles of motor control and motor learning.

Research objectives:

1. Using psychophysiological properties of motor function implementation and additional a priori information on the properties of the experimental paradigm, develop methods for improving the characteristics of non-invasive brain-computer interface based on electroencephalogram (EEG).

2. To develop and implement experimental paradigms and signal processing techniques for the invasive brain-computer interface based on electrocorticogram (ECoG), including the paradigms that enable learning in BCIs with continuous decoding.

3. To implement real time decoding of movement parameters (finger trajectory), for the invasive brain-computer interface based on electrocorticogram (ECoG), including the methods for learning to use the ECoG based BCI.

4. To gain additional knowledge regarding the artificial means for closing the motor-control loop by implementing cortical mapping techniques and comparing the mapping results obtained through electrical stimulation and passive functional mapping of the eloquent cortex.

Methodology

The studies described in this work were conducted using EEG and ECoG as neuroimaging techniques. Functional cortical mapping was performed by means of electrical cortical stimulation (ECS), the gold standard method for that task, as well as high gamma functional mapping, a method that is based on processing ECoG modulations and does not require stimulation.

Subject selection criteria differed for invasive and non-invasive parts of research. In studies using non-invasive neuroimaging (EEG), the subject sample comprised healthy adults, 21-25 year old, men and women. The research was conducted in the laboratories of the Center for Cognition and Decision Making at the National Research University Higher School of Economics. The subjects selected for the studies using invasive neuroimaging (ECoG) were cognitively preserved

patients with epilepsy or neocortex tumors, over 20 years of age, undergoing either implantation of ECoG electrodes or intraoperative monitoring for the purpose of localization of epileptic activity/tumor boundaries and mapping of the eloquent cortex. All subjects signed informed consent to participate in the research studies. This part of research was conducted at the Medical Center of Moscow State University of Medicine and Dentistry, which is a clinical partner of the Center for Bioelectric Interfaces of the National Research University Higher School of Economics.

The research was carried out in the following stages:

1. Advanced non-invasive motor imagery BCI (MI BCI)

(a) Development of a method for motor states classification in the EEG-based MI BCI using an automated procedure for selecting physiologically plausible spatial components of the EEG data.

(b) Development of a method for motor states classification in the EEG-based MI BCI taking into account signal variability that can be caused by latent background state changes.

(c) Demonstration of the EEG-based MI BCI performance in the developed applications.

2. Experimental paradigms and methods for decoding continuous movement kinematics from ECoG signals

(a) Design and assembly of the experimental setup for stimulus presentation and synchronized recording of continuous movement kinematics and ECoG in clinical environment.

(b) Development of experimental paradigms and signal processing methods for decoding continuous movement kinematics from ECoG.

(c) Creating one-directional ECoG-based interface based on decoding the kinematics of continuous finger movement from the ECoG signal.

(d) Development of training paradigms to enable rapid subject + machine adaptation and implementation real-time continuous finger movement decoding.

3. ECoG-based functional cortex mapping

(a) Development and implementation of a paradigm for active (stimulation-based) functional mapping of the eloquent cortex (sensory, motor, language-related).

(b) Functional mapping by electrical stimulation through chronically implanted ECoG grid electrodes with characterization of sensory and motor responses.

(c) Development of a passive-mapping pipeline and signal processing methods for intraoperative localization speech-related motor areas.

(d) Validation of the passive-speech mapping accuracy.

Research outcomes

1. Advanced non-invasive motor imagery BCI (MI BCI)

(a) A novel method for automatic selection of spatial components based on the dipolarity index that reduces over-fitting when building an MI BCI classifier.

(b) A Bayesian framework for explicit tracking of hidden transient brain states during MI BCI decoding.

2. Experimental paradigms and methods for decoding continuous movement kinematics from ECoG signals

(a) Clinically tested experimental setup for bidirectional ECoG based BCI research. The setup enables synchronized recording of multiple multi-channel data streams including continuous movement kinematics and ECoG in clinical environment.

(b) Implemented self-paced and cue-based finger movement as well well as Center out and Center our + rotations paradigms.

(c) Classical and deep learning based algorithms for decoding of kinematics from ECoG data.

(d) ECoG-based interface based on decoding the kinematics of continuous finger movement from the ECoG signal.

(e) Calibration technique developed to enable rapid subject + machine adaptation, allowing implementation of real-time continuous finger movement decoding.

3. ECoG-based functional cortex mapping

(a) Implemented paradigm of active (stimulation-based) functional mapping of the eloquent cortex (sensory, motor, speech-related).

(b) Assessment of the perspectives for implementation of artificial sensation with ECoG through analysis of patients' motor and sensory responses to cortical stimulation.

(c) A passive-mapping pipeline and signal processing methods for intraoperative localization speech-related motor areas.

(d) Results of validation of the developed approach on two subjects by comparing against the gold-standard stimulation based approach.

The main results

1. Methods utilizing a priori information have been developed to improve the performance of non-invasive BCI. The results show increase in decoding accuracy when using the developed methods in comparison with the basic algorithm. The developed methods are rooted in cortical mechanisms of motor control and also implement brain state tracking machinery that allows for accommodating changes associated, for example, with fatigue - a common psychophysiological phenomenon in BCI users.

2. Experimental setups and paradigms for decoding movement parameters as well as functional mapping of the eloquent cortex have been created. The developed setups have been implemented in the research carried out at the Centre for Bioelectric Interfaces, HSE.

3. Finger movement decoding from ECoG has been implemented. An iterative calibration data recording technique has been introduced, enabling realtime decoding of the finger trajectory after less than 0.5 hour of subject training.

4. Passive functional mapping procedure of motor speech areas has been implemented. The results obtained by applying the proposed technique are consistent with the results of cortical stimulation.

Overview of the structure

The thesis consists of the introduction, six chapters, conclusion and four appendices. The first chapter presents an overview of the parameters that define and determine the operation of a brain-computer interface, followed by a more detailed review of the implementations of movement parameter decoding from ECoG signal. In the second chapter, the developed EEG BCI is described along with the proposed methods of enhancing the decoding capabilities in non-invasive motor imagery based interfaces (MI BCI). The following chapters are dedicated to the development of methods of invasive brain-computer interfacing, using ECoG as a neuroimaging technique. The third chapter described the designed experimental setups implemented for ECoG decoding and stimulation studies. The fourth chapter addresses signal processing and data analysis methods developed and implemented for the ECoG processing. Chapter five contains analysis of the obtained ECoG-based finger kinematics decoding results as well as the description of the novel approaches developed to achieve rapid human-machine adaptation in order to obtain accurate real-time decoding after a short period of subject training. Chapter six is dedicated to the implemented methods for intraoperative functional mapping including the results of validation of the passive motor cortex speech-mapping. In Conclusion we critically review the obtained results, highlight the limitations of the conducted experiments and the developed methods and outline the directions for future research.

Список литературы диссертационного исследования кандидат наук Волкова Ксения Владимировна, 2021 год

Spüler, M., Walter, A., Ramos-Murguialday, A., Naros, G., Birbaumer, N., Gharabaghi, A., Rosenstiel, W. and Bogdan, M. (2014), 'Decoding of motor intentions from epidural ECoG recordings in severely paralyzed chronic stroke patients', Journal of neural engineering 11(6), 066008.

Stapornchaisit, S., Kim, Y., Takagi, A., Yoshimura, N. and Koike, Y. (2019), 'Finger angle estimation from array emg system using linear regression model with independent component analysis', Frontiers in Neurorobotics 13.

Sturm, I., Blankertz, B., Potes, C., Schalk, G. and Curio, G. (2014), 'ECoG high gamma activity reveals distinct cortical representations of lyrics passages, harmonic and timbre-related changes in a rock song', Frontiers in human neuroscience 8, 798.

Sugiyama, M., Krauledat, M. and MAZller, K.-R. (2007), 'Covariate shift adaptation by importance weighted cross validation', Journal of Machine Learning Research 8(May), 985-1005.

Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C. and Liu, C. (2018), A survey on deep transfer learning, in 'International conference on artificial neural networks', Springer, pp. 270-279.

Taplin, A. M., de Pesters, A., Brunner, P., Hermes, D., Dalfino, J. C., Adamo, M. A., Ritaccio, A. L. and Schalk, G. (2016), 'Intraoperative mapping of expressive language cortex using passive real-time electrocorticography', Epilepsy & Behavior Case Reports 5, 46 - 51.

URL: http://www.sciencedirect.com/science/article/pii/S2213323216300093

Taylor, D. M., Tillery, S. I. H. and Schwartz, A. B. (2002), 'Direct cortical control of 3d neuroprosthetic devices', Science 296(5574), 1829-1832.

Tilz, C., Hummel, C., Kettenmann, B. and Stefan, H. (2002), 'Ictal onset localization of epileptic seizures by magnetoencephalography', Acta neurologica scandinavica 106(4), 190-195.

Todaro, C., Marzetti, L., Sosa, P. A. V., Valdes-Hernandez, P. A. and Pizzella, V. (2018), 'Mapping brain activity with electrocorticography: Resolution properties and robustness of inverse solutions', Brain Topography pp. 1-16.

Tompson, J., Goroshin, R., Jain, A., LeCun, Y. and Bregler, C. (2015), Efficient object localization using convolutional networks, in 'Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition', pp. 648-656.

Toro, C., Cox, C., Friehs, G., Ojakangas, C., Maxwell, R., Gates, J. R., Gumnit, R. J. and Ebner, T. J. (1994), '8-12 hz rhythmic oscillations in human motor cortex during two-dimensional arm movements: evidence for

representation of kinematic parameters', Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 93(5), 390-403.

Toro, C., Deuschl, G., Thatcher, R., Sato, S., Kufta, C. and Hallett, M. (1994), 'Event-related desynchronization and movement-related cortical potentials on the ECoG and EEG', Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 93(5), 380-389.

Tsubokawa, T., Katayama, Y., Yamamoto, T., Hirayama, T. and Koyama, S. (1993), 'Chronic motor cortex stimulation in patients with thalamic pain', Journal of neurosurgery 78(3), 393-401.

van Vugt, M. K., Sederberg, P. B. and Kahana, M. J. (2007), 'Comparison of spectral analysis methods for characterizing brain oscillations', Journal of neuroscience methods 162(1-2), 49-63.

Vansteensel, M. J., Pels, E. G., Bleichner, M. G., Branco, M. P., Denison, T., Freudenburg, Z. V., Gosselaar, P., Leinders, S., Ottens, T. H., Van Den Boom, M. A. et al. (2016), 'Fully implanted brain-computer interface in a locked-in patient with als', New England Journal of Medicine 375(21), 2060-2066.

Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S. and Schwartz, A. B. (2008), 'Cortical control of a prosthetic arm for self-feeding', Nature 453(7198), 1098.

Vidaurre, C., Sannelli, C., Müller, K.-R. and Blankertz, B. (2011), 'Machine-learning-based coadaptive calibration for brain-computer interfaces', Neural computation 23(3), 791-816.

Viventi, J., Kim, D.-H., Vigeland, L., Frechette, E. S., Blanco, J. A., Kim, Y.-S., Avrin, A. E., Tiruvadi, V. R., Hwang, S.-W., Vanleer, A. C. et al. (2011), 'Flexible, foldable, actively multiplexed, high-density electrode array for mapping brain activity in vivo', Nature neuroscience 14(12), 1599.

Volkova, K., Lebedev, M. A., Kaplan, A. and Ossadtchi, A. (2019), 'Decoding movement from electrocorticographic activity: A review', Frontiers in Neuroinformatics 13.

Waldert, S. (2016), 'Invasive vs. non-invasive neuronal signals for brain-machine interfaces: will one prevail?', Frontiers in neuroscience 10, 295.

Walker, I., Deisenroth, M. and Faisal, A. (2015), 'Deep convolutional neural networks for brain computer interface using motor imagery', Imperial college of science, technology and medicine department of computing p. 68.

Wander, J. D., Blakely, T., Miller, K. J., Weaver, K. E., Johnson, L. A., Olson, J. D., Fetz, E. E., Rao, R. P. and Ojemann, J. G. (2013), 'Distributed cortical adaptation during learning of a brain-computer interface task', Proceedings of the National Academy of Sciences 110(26), 10818-10823.

Wang, N. X., Farhadi, A., Rao, R. P. and Brunton, B. W. (2018), Ajile movement prediction: Multimodal deep learning for natural human neural recordings and video, in 'Thirty-Second AAAI Conference on Artificial Intelligence'.

Wang, P. T., King, C. E., McCrimmon, C. M., Lin, J. J., Sazgar, M., Hsu, F. P., Shaw, S. J., Millet, D. E., Chui, L. A., Liu, C. Y. et al. (2016), 'Comparison of decoding resolution of standard and high-density electrocorticogram electrodes', Journal of neural engineering 13(2), 026016.

Wang, P. T., King, C. E., McCrimmon, C. M., Shaw, S. J., Millett, D. E., Liu, C. Y., Chui, L. A., Nenadic, Z. and Do, A. H. (2014), Electrocorticogram encoding of upper extremity movement duration, in 'Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE', IEEE, pp. 1243-1246.

Wang, W., Collinger, J. L., Degenhart, A. D., Tyler-Kabara, E. C., Schwartz, A. B., Moran, D. W., Weber, D. J., Wodlinger, B., Vinjamuri, R. K., Ashmore, R. C. et al. (2013), 'An electrocorticographic brain interface in an individual with tetraplegia', PloS one 8(2), e55344.

Wang, W., Degenhart, A. D., Collinger, J. L., Vinjamuri, R., Sudre, G. P., Adelson, P. D., Holder, D. L., Leuthardt, E. C., Moran, D. W., Boninger, M. L. et al. (2009), Human motor cortical activity recorded with micro-ECoG electrodes, during individual finger movements, in 'Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE', IEEE, pp. 586-589.

Wang, Y. and Wu, D. (2017), Real-time fMRI-based brain computer interface: A review, in 'International Conference on Neural Information Processing', Springer, pp. 833-842.

Wang, Z., Gunduz, A., Brunner, P., Ritaccio, A. L., Ji, Q. and Schalk, G. (2012), 'Decoding onset and direction of movements using electrocorticographic (ECoG) signals in humans', Frontiers in neuroengineering 5, 15.

Wang, Z., Miller, K. J., Schalk, G. et al. (2011), 'Prior knowledge improves decoding of finger flexion from electrocorticographic signals', Frontiers in Neuroscience 5, 127.

Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M., Chapin, J. K., Kim, J., Biggs, S. J., Srinivasan, M. A. and Nicolelis, M. A. (2000), 'Realtime prediction of hand trajectory by ensembles of cortical neurons in primates', Nature 408(6810), 361.

Wilson, J. A., Mellinger, J., Schalk, G. and Williams, J. (2010), 'A procedure for measuring latencies in brain-computer interfaces', IEEE transactions on biomedical engineering 57(7), 1785-1797.

Wilson, J. A., Walton, L. M., Tyler, M. and Williams, J. (2012), 'Lingual electrotactile stimulation as an alternative sensory feedback pathway for brain-computer interface applications', Journal of neural engineering 9(4), 045007.

Wold, S., Esbensen, K. and Geladi, P. (1987), 'Principal component analysis', Chemometrics and intelligent laboratory systems 2(1-3), 37-52.

Wolpert, D. M. and Ghahramani, Z. (2000), 'Computational principles of movement neuroscience', Nature neuroscience 3(11), 1212-1217.

Wu, J., Shuman, B. R., Brunton, B. W., Steele, K. M., Olson, J. D., Rao, R. P. and Ojemann, J. G. (2016), Multistep model for predicting upper-limb 3d isometric force application from pre-movement electrocorticographic features, in 'Engineering in Medicine and Biology Society (EMBC), 2016 IEEE 38th Annual International Conference of the', IEEE, pp. 1564-1567.

Wu, Z., Xie, T., Yao, L., Zhang, D., Sheng, X., Farina, D., Chen, L., Mao, Y. and Zhu, X. (2017), 'Electrocorticographic temporal alteration mapping: A clinical technique for mapping the motor cortex with movement-related cortical potentials', Frontiers in neuroscience 11, 326.

Wulf, G. and Prinz, W. (2001), 'Directing attention to movement effects enhances learning: A review', Psychonomic bulletin & review 8(4), 648-660.

Xie, T., Zhang, D., Wu, Z., Chen, L. and Zhu, X. (2015), 'Classifying multiple types of hand motions using electrocorticography during intraoperative awake craniotomy and seizure monitoring processes—case studies', Frontiers in neuroscience 9, 353.

Xie, Z., Schwartz, O. and Prasad, A. (2017), 'Decoding of finger trajectory from ECoG using deep learning', Journal of neural engineering .

Xie, Z., Schwartz, O. and Prasad, A. (2018), 'Decoding of finger trajectory from ECoG using deep learning', Journal of neural engineering 15(3), 036009.

Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., Yokoi, H., Kamitani, Y. and Yoshimine, T. (2011), 'Real-time control of a prosthetic hand using human electrocorticography signals', Journal of neurosurgery 114(6), 1715-1722.

Yanagisawa, T., Hirata, M., Saitoh, Y., Kato, A., Shibuya, D., Kamitani, Y. and Yoshimine, T. (2009), 'Neural decoding using gyral and intrasulcal electrocorticograms', Neuroimage 45(4), 1099-1106.

Yanagisawa, T., Hirata, M., Saitoh, Y., Kishima, H., Matsushita, K., Goto, T., Fukuma, R., Yokoi, H., Kamitani, Y. and Yoshimine, T. (2012), 'Electrocorticographic control of a prosthetic arm in paralyzed patients', Annals of neurology 71(3), 353-361.

Yu, X., Chum, P. and Sim, K.-B. (2014), 'Analysis the effect of pca for feature reduction in non-stationary EEG based motor imagery of BCI system', Optik 125(3), 1498-1502.

Yuan, H. and He, B. (2014), 'Brain-computer interfaces using sensorimotor rhythms: current state and future perspectives', IEEE Transactions on Biomedical Engineering 61(5), 1425-1435.

Zentgraf, K., Lorey, B., Bischoff, M., Zimmermann, K., Stark, R. and Munzert, J. (2009), 'Neural correlates of attentional focusing during finger movements: A fMRI study', Journal of Motor Behavior 41(6), 535-541.

Zhang, X., Yao, L., Wang, X., Monaghan, J. and Mcalpine, D. (2019), 'A survey on deep learning based brain computer interface: Recent advances and new frontiers', arXiv preprint arXiv:1905.04149 .

Zhang, Y., van Drongelen, W., Kohrman, M. and He, B. (2008), 'Three-dimensional brain current source reconstruction from intra-cranial ECoG recordings', Neuroimage 42, 683-695.

Zhao, H.-b., Wang, H., Li, C.-s. and Li, Y.-G. (2009), Brain-computer interface design based on slow cortical potentials using matlab/simulink, in '2009 International Conference on Mechatronics and Automation', IEEE, pp. 1044-1048.