Self-Calibration Enhances BrainGate Brain Computer Interface Ease, Reliability
Decoder innovations to the investigational BrainGate brain-computer interface (BCI) being developed at Brown University in Providence, Rhode Island, now allow the system to recalibrate itself. This enhancement enables users to work with BrainGate for longer sessions without interruptions for recalibration by technical staff.
A research article published in the 11 November 2015 edition of the journal Science Translational Medicine entitled “Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface” (Science Translational Medicine 11 Nov 2015: Vol. 7, Issue 313, pp. 313ra179 DOI: 10.1126/scitranslmed.aac7328) describes three software innovations that substantially improved the user experience and performance of the BrainGate brain computer interface. Researchers said the gains are a significant advance in their ongoing work to develop and test a practical BCI assistive technology that people with paralysis could use easily, reliably, independently, and on demand to regain control over external devices.
The diverse and collaborative BrainGate research team creates and tests the devices designed to usher in a new era of transformative neurotechnologies, and includes leading neurologists, neuroscientists, engineers, computer scientists, neurosurgeons, mathematicians, and other researchers, all focused on developing technologies to restore abilities of communication, mobility, and independence of people with neurologic disease, injury, or limb loss. The Science Translational Medicine paper’s coauthors are Beata Jarosiewicz, Anish A. Sarma, Daniel Bacher, Nicolas Y. Masse, John D. Simeral, Brittany Sorice, Erin M. Oakley, Christine Blabe, Chethan Pandarinath, Vikash Gilja, Sydney S. Cash, Emad N. Eskandar, Gerhard Friehs, Jaimie M. Henderson, Krishna V. Shenoy, John P. Donoghue, and Leigh R. Hochberg, variously representing the Brown University Department of Neuroscience, the Department of Veterans Affairs Medical Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service at Providence, the Brown Institute for Brain Science, The Brown School of Engineering, the Department of Neurology at Massachusetts General Hospital in Boston, MA 02114, USA, the Stanford University Department of Neurosurgery in Stanford, California, the Stanford Neurosciences Institute, Department of Neurobiology, Department of Bioengineering, and Department of Electrical Engineering, the Harvard Medical School Department of Neurology in Boston, the Rhode Island Hospital Neurosurgery department at Providence, Case Western Reserve University, and the Howard Hughes Medical Institute at Stanford.
The researchers observe that enabling the human brain to be hooked up to a computer to allow paralyzed individuals to type is a technological feat in itself. However, they note that these ‘brain-computer interface technologies’ can also be tiring and burdensome for users, requiring frequent disruptions for recalibration when the decoded neural signals change. The study’s corresponding author, assistant professor (research) of neuroscience at Brown University and the Brown Institute for Brain Science and investigator at the Providence Veterans Affairs Medical Center (PVAMC) Beata Jarosiewicz, Beata Jarosiewicz and her colleagues combined three calibration methods retrospective target interference, velocity bias correction, and adaptive tracking of neural features, for seamless typing and stable neural control. This combination allowed two individuals with tetraplegia and with cortical microelectrode arrays to compose long texts at their own respective paces, with no need to interrupt typing for recalibration.
Image caption: Typing with BrainGate BrainGate2 clinical trial participant T7 used the brain-computer interface to control a specially designed radial keyboard. Innovations in the system’s decoder keep the system calibrated automatically. Image courtesy Braingate.org
Using a baby aspirin-sized array of electrodes implanted into the brain, early research from the BrainGate team has shown that the neural signals associated with intent of limb movement can be decoded by a computer in real-time and used to operate external devices. This investigational system, BrainGate (Note: BrainGate is an Investigational Device. Limited by Federal Law to Investigational Use) has allowed people with spinal cord injury, brainstem stroke, and ALS to control a computer cursor simply by thinking about the movement of their own paralyzed hand.
The scientists note that current research is focused not only on improving the ability to operate a computer, but also on providing people with ALS, spinal cord injury, and stroke with reliable, constant control over their environment — development of technology that may ultimately give natural control over advanced prosthetic limbs, provide people with paralysis easy control over powerful assistive movement and communication devices, and, eventually, enable naturally-controlled movements of paralyzed limbs. In addition, the research team are developing a new generation of wireless medical technologies that will be able to record and monitor neural activity to assist in the diagnosis and management of neurologic disease.
The researchers explain in a Brown release that Intracortical BCIs such as BrainGate use a tiny array of implanted electrodes to pick up the electrical activity of neurons in the motor cortex of the brain. Computers then translate those signals into digital commands that have allowed users to control electronic devices such as computers and robotic arms by simply intending to move their own arm or hand. The translation relies on a decoder — an algorithm that infers the movement intentions of the user from the patterns of their neural activity.
However, they observe that a major challenge associated with decoding movement intention from intracortical electrodes is that the signals change over time. Consequently, intracortical BCIs have to this point required the above-noted frequent interruptions for decoder recalibration. Neural signal patterns can change when the electrodes move even slightly; a neuron whose signal was not previously detected can end up joining the recorded ensemble while another might become excluded. As the neural signals shift and drift, the performance of the BCI the ability of users to move a cursor or robot by thinking about the move will decline until the decoder can be recalibrated. This recalibration is typically performed using a special task in which the participant tries to move the cursor to prescribed targets so that movement intentions can be mapped to the new neural activity patterns.
The essential advance described in the new study is a set of decoder upgrades that allow the algorithm to recalibrate itself during practical BCI use without making the user stop for calibration task every time the signals change. Results of the research reported in the paper show that the new decoder preserved BCI performance much longer than before, and even contributed to improving users accurate typing speed on an on-screen keyboard. Rather than frequent pauses for recalibration, users could type for hours, pausing only when they wanted to and without need for technicians to intervene.
“Eliminating the need to run a calibration task whenever the recorded signals change will make a clinical BCI more user-friendly and easy to use,” says Dr. Jarosiewicz.
The release notes that in a particularly dramatic demonstration at the Stanford site of the pilot clinical trial, a woman identified as T6, who is diagnosed with amyotrophic lateral sclerosis, was able to use BrainGate for six sessions of a few hours each over the course of 42 days without any interruptions for explicit recalibration after the decoder was initialized on the first day. T6 was able to type paragraphs, pausing and unpausing the BCI on her own, while the decoder calibrated itself.
“Watching our participants use this more robust system to type on a computer screen highlights the progress being made toward a clinically useful system,” says Dr. Leigh Hochberg, a professor of engineering at Brown, director of the Center for Neurorestoration and Neurotechnology at PVAMC, director of the Neurotechnology Trials Unit at Massachusetts General Hospital Neurology, and senior author of the Science Translational Medicine paper.
The self-calibration innovations take advantage of information that accrues as trial participants use the system. In one innovation, dubbed “retrospective target inference,” the decoder analyzes the user’s recent target selections to update its interpretation of the underlying neural signals that produced it. “For example, if a participant is moving the cursor around to type B-R-A-I-N-G-A-T-E,” Dr. Jarosiewicz explains, “the decoder has definitive new information about the users movement intentions during the trajectory between each letter. For example, when the cursor was moving from B to R, the persons movement intentions can be inferred to have been directly toward the R.”
“We can use that information to update the map between the neural signals and the decoders understanding of the persons movement intentions,” Dr. Jarosiewicz observes.
Meanwhile, the decoder now also keeps track of the baseline levels of neural activity in the motor cortex during moments when participants have chosen to pause. Because of the changes in neural signals over time, the resting firing patterns of neurons in the motor cortex might have changed since the last pause. Keeping track of this baseline rate during pauses helps the system to start with better calibration when the user decides to turn the system back on.
A third innovation developed by the investigational team is keeping track, while participants use the system, of emerging biases in the velocity of cursor movement and subtracting them out of the decoded movements. Keeping a better balance of speeds in different directions leads to more accuracy and a more intuitive user experience.
The team performed several research sessions to confirm that the new innovations were making a significant and positive difference for participants, including asking participants to type for an hour or two in some sessions when the self-calibration innovations were engaged and some when they were turned off. They found that performance remained stable in sessions when the features were on, but degraded significantly when the features were off. In one research session with the innovations turned off, the researchers found that turning them back on allowed the system to rescue performance after it had been degrading for lack of the features.
In principle, the improved decoder can be applied to other BCI tasks such as three-dimensional control of a robot arm or even a persons own electronically reanimated arm or hand. “There are unique challenges to doing that,” acknowledges Dr. Jarosiewicz, “but it’s a problem the team is tackling next.”
The goal remains to develop BrainGate into a system that would be available to people who need it, whenever they need it.
“There is still a lot of research to do,” Dr. Hochberg cautions, but affirms that “With continued clinical research, we will learn how our findings extend to more participants. We want to make the system faster, easier, smaller, fully implanted, more portable, less requiring of an expert researcher or caregiver, and more nimble in its ability to provide control of external devices. In these studies, we are making steps toward robust and flexible communication systems for people with severely limited movement, including limited or no speech. We are also dedicated not only to enabling control over computers or robotic assistive devices, but for people with spinal cord injury or stroke working toward the goal of reconnecting brain to limb, allowing the powerful intracortical signals to activate fully implanted functional electrical stimulation devices, and re-enabling intuitive movement of ones own arm and hand.”
Dr. Hochberg emphasizes that “Our extraordinary research participants are true pioneers. They are participating in the trial not because they hope to gain any personal benefit, but because they want to help us to develop and test a system that will help other people with paralysis in the future.”
The research was supported by the Department of Veterans Affairs (Rehabilitation Research & Development Service awards: B6310W, B6453R, B6459L, A6779I); the National Institutes of Health (NIDCD grants R01DC009899, R01DC014034, NICHD-NCMRR grants N01HD10018, RC1HD063931, N01HD53403, and NINDS grant R01NS066311-S1); the Doris Duke Charitable Foundation; the MGH-Deane Institute; the Joseph Martin Prize for Basic Research; the Katie Samson Foundation; the Craig H. Neilsen Foundation; Stanford BioX-NeuroVentures; and the Garlick Family.
Sources:
Brown University
BrainGate project
Science Translational Medicine
