BCI technology may give voice to Mandarin speakers with paralysis
Scientists create new brain-computer interface system for tonal language
Scientists have developed a brain-computer interface (BCI) capable of instantly translating a person’s thoughts into written Mandarin Chinese.
The technology expands the reach of BCIs, devices that capture brain signals to help people communicate, beyond its traditional focus on English speakers. It opens up the potential for millions more people worldwide, particularly those living with paralyzing conditions like amyotrophic lateral sclerosis (ALS), to interact with the world again.
The work “opens [BCIs] up to huge populations of potential patients,” Matthew Leonard, PhD, a neurolinguistics expert at the University of California San Francisco who was not involved in the research, said in a press release.
The findings were published in the journal Science Advances in a paper, titled “Real-time decoding of full-spectrum Chinese using brain-computer interface.” The devices used in the study were provided by the neurotechnology company Neuroxess. Several of the scientists who led the study are employees at Neuroxess.
Understanding BCI technology and its current scope
BCIs are experimental devices that use sensors implanted in the brain to capture and interpret neural activity, allowing people who are unable to communicate, including some people in more advanced stages of ALS, to interact with the world around them.
Several studies have shown that it is possible to decode speech using BCIs, allowing individuals who cannot physically move to communicate, either with text written on a screen or with a computer-generated voice. However, most of the research in this area has focused on people who speak English.
To make this technology usable to more people, scientists designed a BCI system that could decode Mandarin. Unlike English, Mandarin has tonal inflections that alter the meaning of words, so the researchers needed to create new systems for the computer to identify both the syllables and tonal inflections that distinguish Mandarin words.
For the study, the researchers used brain sensors that had been placed in a 43-year-old Chinese woman to help guide treatment for epilepsy. The woman’s speech was unaffected, so over the course of about two weeks in the hospital, she spent hours saying specific sounds that are found in Mandarin, and the sensors in her brain detected the corresponding electrical activity.
The researchers could then use advanced computer algorithms to decode what the woman was saying based only on her brain signals.
The project was ultimately successful. In late 2024, the woman was able to use the BCI to spell out brief New Year’s greetings in real time in Chinese characters. Jinsong Wu, MD, PhD, co-author of the study at China’s Fudan University, said this moment was “incredibly exciting.”
Although the system showed promise, it’s far from perfect. The overall accuracy was about 70%, and the system’s speed is about five times slower than a typical person speaking Chinese.
More work will be needed to improve the technology and advance its use for people with paralyzing conditions such as ALS. Still, the researchers said this study is an important step toward making BCIs available to people who speak tonal languages.
“These findings demonstrate the feasibility of syllable-based decoding for Mandarin and provide a scalable strategy for real-time speech [BCI] in tonal languages … across diverse linguistic populations,” the scientists concluded.
About the Author
