A new brain-computer interface (BCI) can turn ‘mental handwriting’ – the act of imagining yourself writing on a piece of paper – into text on a screen in real time.
The breakthrough system, developed by US researchers, uses a computer to decode attempted handwriting movements from signals in the brain.
The signals are detected by tiny square electrode arrays roughly the size of a baby aspirin pill, which have been inserted into the brain.
Researchers say the system could allow people with paralysis who don’t have use of their hands or fingers to rapidly type messages.
In a clinical trial, a paralysed man used the BCI to write text on a computer screen just by thinking about the movements involved in handwriting.
The man was able to copy sentences and answer questions at a rate similar to that of someone his age typing on a smartphone.
Researchers are using tiny electrode arrays (pictured) to record signals from the motor cortex of the brain. Those signals can then be used to control robotic prostheses, computers or other devices. The hope is that such a system may one help restore communication and movement in people with paralysis due to injury or illness
Researchers worked with a 65-year-old man (pictured) who had been paralysed from the neck down. The BrainGate BCI allowed him to identify letters as he attempted to write them in his mind. Then, the system displayed the text on a screen – in real time
The new study is the latest phase of a clinical trial called BrainGate, directed by Dr Leigh Hochberg, a professor of neurology at Brown University in Providence, Rhode Island.
Frank Willett, a research scientist at Stanford University and the Howard Hughes Medical Institute (HHMI), led the study.
‘We want to find new ways of letting people communicate faster,’ Willett said.
‘This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text.’
When an injury or disease robs a person of the ability to move, the brain’s neural activity for walking, speaking and grabbing objects remain intact.
Researchers can therefore tap into the brain activity of people with paralysis or amputations to regain some of their lost abilities.
For their study, the researchers worked with a 65-year-old man who had a spinal cord injury in 2007 that left him paralysed from the neck down.
Scientists implanted two of the tiny electrode arrays into the part of his brain that controls the hand and arm – the motor cortex.
Each of the electrode arrays is covered with 100 electrodes that pick up signals from neurons firing in the motor cortex.
The authors instructed the man to attempt to write sentences as if his hand were not paralysed, by imagining he was holding a pen on a piece of ruled paper.
During the exercise, a neural network (a type of machine learning) was able to translate attempted handwriting movements from neural activity into text that appeared on a screen in real time.
In a clinical trial, a paralysed man used the BCI to write text on a computer screen just by thinking about the movements involved in handwriting.
Two tiny arrays of implanted electrodes relayed information from the brain area that controls the hands and arms to an algorithm, which translated it into letters that appeared on a screen
Using this ‘brain-to-text’ system, the man answered questions by attempting to write each letter of his response.
Jaimie Henderson, professor of neurosurgery, said: ‘This approach allowed a person with paralysis to compose sentences at speeds nearly comparable to those of able-bodied adults of the same age typing on a smartphone.
‘The goal is to restore the ability to communicate by text.’
Study co-author Krishna Shenoy said: ‘Just think about how much of your day is spent on a computer or communicating with another person.
‘Restoring the ability of people who have lost their independence to interact with computers and others is extremely important, and that is what is bringing projects like this one front and centre.’
A previous system from the BrainGate team demonstrated back in 2015, also using sensors implanted on the motor cortex, gave paralysed people the ability to select letters with a thought-controlled cursor.
Pointing at and clicking on letters in this way let people type about 40 characters per minute, the previous speed record for typing with a brain computer interface (BCI).
In these new trials, the man was able to type more than twice as fast – 90 characters per minute with 94.1 per cent accuracy, which sets a new record.
This is also comparable to typical smartphone typing speeds in people of the same age group as the study participant (115 characters per minute).
This BCI is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another.
In future, researchers envision using attempted handwriting for text entry as part of a more comprehensive system that also includes point-and-click navigation, much like that used on current smartphones, and even speech decoding.
The team intends to work with a participant with amyotrophic lateral sclerosis, a degenerative neurological disorder that results in the loss of both movement and speech.
According to the BrainGate team, further demonstrations of the system’s ‘longevity, safety and efficacy’ will be required before it can be put to widespread clinical use.
The research has been published today in the journal Nature.