Researchers have developed a new brain-computer interface (BCI) that can let paralysed people write by mentally writing letters by hand.
Working with a participant with paralysis who has sensors implanted in his brain, the team used an algorithm to identify letters in real time as he attempted to write them, putting the results on a screen.
This technology could be further developed to allow people with paralysis type rapidly without using their hands, said study coauthor Krishna Shenoy, a Howard Hughes Medical Institute Investigator at Stanford University who jointly supervised the work with Jaimie Henderson, a Stanford neurosurgeon.
By attempting handwriting, the study participant was able to ‘type’ 90 characters per minute — more than double the previous record for typing with such a brain-computer interface.
Thought-powered communication
Even if injury or disease the ability to move, the brain’s neural activity for being able to do so remains. By making use of this activity, researchers can help people with paralysis or amputations regain lost abilities.
In recent years, Shenoy’s team has decoded the neural activity associated with speech in the hopes of reproducing it. Patients with implanted sensors mentally pointed at and clicked on letters on a screen to type at about 40 characters per minute, the previous speed record for typing with a BCI.
Wanting to try something new and different, Frank Willett, a neuroscientist in Shenoy’s group, wondered if it might be possible to harness the brain signals evoked by writing by hand “We want to find new ways of letting people communicate faster,” he said.
The team worked with a participant enrolled in a clinical trial involving BCIs. Henderson implanted two tiny sensors into the part of the brain that controls the hand and arm, making it possible for the person to, for example, move a robotic arm or a cursor on a screen by attempting to move their own paralysed arm.
The participant, who was 65 years old at the time of the research, had a spinal cord injury that left him paralysed from the neck down. A machine learning algorithm recognised the patterns his brain produced when he attempted to write each letter.
With this system, the man could copy sentences and answer questions at a rate similar to that of someone his age typing on a smartphone. The reason why this so-called “Brain-to-Text” BCI is so fast is because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, Willett explained.
A new system
Shenoy’s team envisions using attempted handwriting for text entry as part of a more comprehensive system that also includes point-and-click navigation, much like that used on current smartphones, and even attempted speech decoding. “Having those two or three modes and switching between them is something we naturally do,” he said.
The team intends to next work with a participant who cannot speak, such as a person with amyotrophic lateral sclerosis, a degenerative neurological disorder leading to loss of movement and speech.
The new system could potentially help those suffering from paralysis caused by a number of conditions, Henderson added. Those include brain stem stroke, which afflicted Jean-Dominique Bauby, the author of the book The Diving Bell and the Butterfly. “He was able to write this moving and beautiful book by selecting characters painstakingly, one at a time, using eye movement,” Henderson said. “Imagine what he could have done with Frank’s handwriting interface!”
Source: Howard Hughes Medical Institute
