Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces

Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R. & Donoghue, J. P. Instant neural control of a movement signal. Nature 416, 141–142 (2002).Article 
CAS 
PubMed 

Google Scholar 
Hochberg, L. R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006).Article 
CAS 
PubMed 

Google Scholar 
Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Schwartz, A. B., Cui, X. T., Weber, D. J. & Moran, D. W. Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron 52, 205–220 (2006).Article 
CAS 
PubMed 

Google Scholar 
Simeral, J. D., Kim, S.-P., Black, M. J., Donoghue, J. P. & Hochberg, L. R. Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8, 025027 (2011).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Collinger, J. L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381, 557–564 (2013).Article 
PubMed 
PubMed Central 

Google Scholar 
Gilja, V. et al. Clinical translation of a high-performance neural prosthesis. Nat. Med. 21, 1142–1145 (2015).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Kim, S.-P. et al. Point-and-click cursor control with an intracortical neural interface system by humans with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 19, 193–203 (2011).Article 
PubMed 
PubMed Central 

Google Scholar 
Bacher, D. et al. Neural point-and-click communication by a person with incomplete locked-in syndrome. Neurorehabil. Neural Repair 29, 462–471 (2015).Article 
PubMed 

Google Scholar 
Brandman, D. M. et al. Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. J. Neural Eng. 15, 026007 (2018).Article 
PubMed 
PubMed Central 

Google Scholar 
Pandarinath, C. et al. High performance communication by people with paralysis using an intracortical brain-computer interface. Elife 6, e18554 (2017).Article 
PubMed 
PubMed Central 

Google Scholar 
Jarosiewicz, B. et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci. Transl. Med. 7, 313ra179 (2015).Article 
PubMed 
PubMed Central 

Google Scholar 
Nuyujukian, P. et al. Cortical control of a tablet computer by people with paralysis. PLoS ONE 13, e0204566 (2018).Article 
PubMed 
PubMed Central 

Google Scholar 
Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature 620, 1031–1036 (2023).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Taylor, D. M., Tillery, S. I. H. & Schwartz, A. B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).Article 
CAS 
PubMed 

Google Scholar 
Andersen, R. A., Snyder, L. H., Bradley, D. C. & Xing, J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annu. Rev. Neurosci. 20, 303–330 (1997).Article 
CAS 
PubMed 

Google Scholar 
Homer, M. L., Nurmikko, A. V., Donoghue, J. P. & Hochberg, L. R. Sensors and decoding for intracortical brain computer interfaces. Annu. Rev. Biomed. Eng. 15, 383–405 (2013).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Downey, J. E., Schwed, N., Chase, S. M., Schwartz, A. B. & Collinger, J. L. Intracortical recording stability in human brain–computer interface users. J. neural Eng. 15, 046016 (2018).Article 
PubMed 

Google Scholar 
Perge, J. A. et al. Intra-day signal instabilities affect decoding performance in an intracortical neural interface system. J. Neural Eng. 10, 036004 (2013).Article 
PubMed 
PubMed Central 

Google Scholar 
Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I. & Shenoy, K. V. Making brain-machine interfaces robust to future neural variability. Nat. Commun. 7, 13749 (2016).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Degenhart, A. D. et al. Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nat. Biomed. Eng. 4, 672–685 (2020).Article 
PubMed 
PubMed Central 

Google Scholar 
Nuyujukian, P. et al. Performance sustaining intracortical neural prostheses. J. Neural Eng. 11, 066003 (2014).Article 
PubMed 

Google Scholar 
Davis, S. E. et al. A nonparametric updating method to correct clinical prediction model drift. J. Am. Med. Inform. Assoc. 26, 1448–1457 (2019).Article 
PubMed 
PubMed Central 

Google Scholar 
Tsymbal, A. The Problem of Concept Drift: Definitions and Related Work. (Tech. Rep. Department of Computer Science, Trinity College, 2004).
Google Scholar 
Gama, J., Medas, P., Castillo, G. & Rodrigues, P. in Advances in Artificial Intelligence—SBIA 2004. 286–295 (Springer Berlin Heidelberg, 2004).Fan, C. et al. Plug-and-play stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication. Conference on Neural Information Processing Systems 37 (2023).Wilson, G. H. et al. Long-term unsupervised recalibration of cursor BCIs. bioRxiv https://doi.org/10.1101/2023.02.03.527022 (2023).Kim, S.-P., Wood, F., Fellows, M., Donoghue, J. P. & Black, M. J. Statistical Analysis of the Non-stationarity of Neural Population Codes. In: The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006. 811–816 (IEEE, 2006).Ahmadipour, P., Yang, Y., Chang, E. F. & Shanechi, M. M. Adaptive tracking of human ECoG network dynamics. J. Neural Eng. 18, 016011 (2021).PubMed 

Google Scholar 
Driscoll, L. N., Duncker, L. & Harvey, C. D. Representational drift: emerging theories for continual learning and experimental future directions. Curr. Opin. Neurobiol. 76, 102609 (2022).Article 
CAS 
PubMed 

Google Scholar 
Chestek, C. A. et al. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural Eng. 8, 045005 (2011).Article 
PubMed 
PubMed Central 

Google Scholar 
Gallego, J. A., Perich, M. G., Chowdhury, R. H., Solla, S. A. & Miller, L. E. Long-term stability of cortical population dynamics underlying consistent behavior. Nat. Neurosci. 23, 260–270 (2020).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Chestek, C. A. et al. Single-neuron stability during repeated reaching in macaque premotor cortex. J. Neurosci. 27, 10742–10750 (2007).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Carmena, J. M., Lebedev, M. A., Henriquez, C. S. & Nicolelis, M. A. L. Stable ensemble performance with single-neuron variability during reaching movements in primates. J. Neurosci. 25, 10712–10716 (2005).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysiol. 106, 764–774 (2011).Article 
PubMed 
PubMed Central 

Google Scholar 
McGinley, M. J. et al. Waking state: rapid variations modulate neural and behavioral responses. Neuron 87, 1143–1161 (2015).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Vinck, M., Batista-Brito, R., Knoblich, U. & Cardin, J. A. Arousal and locomotion make distinct contributions to cortical activity patterns and visual encoding. Neuron 86, 740–754 (2015).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Hennig, J. A. et al. Learning is shaped by abrupt changes in neural engagement. Nat. Neurosci. 24, 727–736 (2021).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Barrese, J. C. et al. Failure mode analysis of silicon-based intracortical microelectrode arrays in non-human primates. J. Neural Eng. 10, 066014 (2013).Article 
PubMed 
PubMed Central 

Google Scholar 
Woeppel, K. et al. Explant analysis of utah electrode arrays implanted in human cortex for brain-computer-interfaces. Front. Bioeng. Biotechnol. 9, 759711 (2021).Article 
PubMed 
PubMed Central 

Google Scholar 
Santhanam, G. et al. HermesB: a continuous neural recording system for freely behaving primates. IEEE Trans. Biomed. Eng. 54, 2037–2050 (2007).Article 
PubMed 

Google Scholar 
Gilja, V. et al. A high-performance neural prosthesis enabled by control algorithm design. Nat. Neurosci. 15, 1752–1757 (2012).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Bishop, W. et al. Self-recalibrating classifiers for intracortical brain-computer interfaces. J. Neural Eng. 11, 026001 (2014).Article 
PubMed 
PubMed Central 

Google Scholar 
Jarosiewicz, B. et al. Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain-computer interfaces. J. Physiol. Paris 110, 382–391 (2016).Article 
PubMed 

Google Scholar 
Brandman, D. M. et al. Robust closed-loop control of a cursor in a person with tetraplegia using Gaussian process regression. Neural Comput. 30, 2986–3008 (2018).Farshchian, A. et al. Adversarial domain adaptation for stable brain-machine interfaces. International Conference on Learning Representations (ICLR, 2019).Burkhart, M. C., Brandman, D. M., Franco, B., Hochberg, L. R. & Harrison, M. T. The discriminative Kalman filter For Bayesian filtering with nonlinear and nongaussian observation models. Neural Comput. 32, 969–1017 (2020).Article 
PubMed 
PubMed Central 

Google Scholar 
Ma, X. et al. Using adversarial networks to extend brain computer interface decoding accuracy over time. Elife 12, e84296 (2023).Karpowicz, B. M. et al. Stabilizing brain-computer interfaces through alignment of latent dynamics. bioRxiv https://doi.org/10.1101/2022.04.06.487388 (2022).Jude, J., Perich, M. G., Miller, L. E. & Hennig, M. H. Capturing cross-session neural population variability through self-supervised identification of consistent neuron ensembles. In: Proceedings of the 1st NeurIPS Workshop on Symmetry and Geometry in Neural Representations (eds. Sanborn, S., Shewmake, C., Azeglio, S., Di Bernardo, A. & Miolane, N.) vol. 197 234–257 (PMLR, 2023).Hosman, T., Pun, T. K., Kapitonava, A., Simeral, J. D. & Hochberg, L. R. Months-long high-performance fixed LSTM decoder for cursor control in human intracortical brain-computer interfaces. In: 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER) (IEEE, 2023).Homer, M. L. et al. Adaptive offset correction for intracortical brain–computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 239–248 (2014).Article 
PubMed 
PubMed Central 

Google Scholar 
Kim, S.-P., Simeral, J. D., Hochberg, L. R., Donoghue, J. P. & Black, M. J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476 (2008).Article 
PubMed 
PubMed Central 

Google Scholar 
Rokni, U., Richardson, A. G., Bizzi, E. & Seung, H. S. Motor learning with unstable neural representations. Neuron 54, 653–666 (2007).Article 
CAS 
PubMed 

Google Scholar 
Truccolo, W., Friehs, G. M., Donoghue, J. P. & Hochberg, L. R. Primary motor cortex tuning to intended movement kinematics in humans with tetraplegia. J. Neurosci. 28, 1163–1178 (2008).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Ganguly, K. & Carmena, J. M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).Article 
PubMed 
PubMed Central 

Google Scholar 
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).Article 
CAS 
PubMed 

Google Scholar 
Kobak, D. et al. Demixed principal component analysis of neural population data. Elife 5, e10989 (2016).Article 
PubMed 
PubMed Central 

Google Scholar 
Downey, J. E. et al. Motor cortical activity changes during neuroprosthetic-controlled object interaction. Sci. Rep. 7, 16947 (2017).Article 
PubMed 
PubMed Central 

Google Scholar 
Gallego, J. A. et al. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nat. Commun. 9, 4233 (2018).Article 
PubMed 
PubMed Central 

Google Scholar 
Fraser, G. W., Chase, S. M., Whitford, A. & Schwartz, A. B. Control of a brain–computer interface without spike sorting. J. Neural Eng. 6, 055004 (2009).Article 
PubMed 

Google Scholar 
Masse, N. Y. et al. Non-causal spike filtering improves decoding of movement intention for intracortical BCIs. J. Neurosci. Methods 236, 58–67 (2014).Article 
PubMed 
PubMed Central 

Google Scholar 
Todorova, S., Sadtler, P., Batista, A., Chase, S. & Ventura, V. To sort or not to sort: the impact of spike-sorting on neural decoding performance. J. Neural Eng. 11, 056005 (2014).Article 
PubMed 
PubMed Central 

Google Scholar 
Christie, B. P. et al. Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. J. Neural Eng. 12, 016009 (2015).Article 
PubMed 

Google Scholar 
Jarosiewicz, B. et al. Functional network reorganization during learning in a brain-computer interface paradigm. Proc. Natl. Acad. Sci. USA 105, 19486–19491 (2008).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Golub, M. D. et al. Learning by neural reassociation. Nat. Neurosci. 21, 607–616 (2018).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Oby, E. R. et al. New neural activity patterns emerge with long-term learning. Proc. Natl. Acad. Sci. USA 116, 15210–15215 (2019).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Jarosiewicz, B. et al. Advantages of closed-loop calibration in intracortical brain-computer interfaces for people with tetraplegia. J. Neural Eng. 10, 046012 (2013).Article 
PubMed 
PubMed Central 

Google Scholar 
Shanechi, M. M. et al. A real-time brain-machine interface combining motor target and trajectory intent using an optimal feedback control design. PLoS ONE 8, e59049 (2013).Article 
CAS 
PubMed 
PubMed Central 

Google Scholar 
Simeral, J. D. et al. Home use of a percutaneous wireless intracortical brain-computer interface by individuals with tetraplegia. IEEE Trans. Biomed. Eng. 68, 2313–2325 (2021).Article 
PubMed 
PubMed Central 

Google Scholar 
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).Article 
CAS 
PubMed 

Google Scholar 
Tseng, P.-H., Urpi, N. A., Lebedev, M. & Nicolelis, M. Decoding movements from cortical ensemble activity using a long short-term memory recurrent network. Neural Comput. 31, 1085–1113 (2019).Article 
PubMed 

Google Scholar 
Premchand, B. et al. Decoding movement direction from cortical microelectrode recordings using an LSTM-based neural network. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC) 3007–3010 (IEEE, 2020).Hosman, T. et al. BCI decoder performance comparison of an LSTM recurrent neural network and a Kalman filter in retrospective simulation. 9th International IEEE/EMBS Conference on Neural Engineering (NER) 1066–1071 (IEEE, 2019).Berens, P. CircStat: a MATLAB toolbox for circular statistics. J. Stat. Softw. 31, 1–21 (2009).Article 

Google Scholar