Woman Left Paralyzed After Stroke Can Now Communicate Through Digital Avatar Powered by AI, Brain Implants

Man with a headache - Stroke -3D Illustration. Peterschreiber.media/ Shutterstock

A woman who was left severely paralyzed after suffering a stroke has been able to communicate again thanks to ground-breaking technology using artificial intelligence (AI) developed by researchers at the University of California, San Francisco (UCSF), and the University of California (UC), Berkeley.

Ann Johnson, a 47-year-old former high school math teacher in Canada, has been severely paralyzed since suffering a brainstem stroke at the age of 30.

Doctors still do not know what prompted the stroke, which left her with a condition known as locked-in syndrome, a rare neurological disorder resulting in complete paralysis of all voluntary muscles.

Although years of physical therapy allowed Ms. Johnson to move her facial muscles enough to laugh or cry, she has still never regained the ability to speak.

Now, 18 years later, Ms. Johnson can communicate through a digital avatar powered by artificial intelligence (AI), thanks to researchers at UCSF and UC Berkeley.

Led by UCSF neurosurgeon Edward Chang, the researchers developed implantable technology—known as a brain-computer interface, which translates brain signals into modulated speech and facial expressions—allowing Ms. Johnson to speak and convey emotion via a talking digital avatar that used a copy of her voice—a recording made during a 15-minute toast she gave at her wedding 20 years ago, The New York Times reported.

Sophia, a robot integrating the latest technologies and artificial intelligence developed by Hanson Robotics is pictured during a presentation at the "AI for Good" Global Summit at the International Telecommunication Union (ITU) in Geneva, Switzerland June 7, 2017. Reuters/Denis Balibouse

AI, Digital Avatar Aids Communication

The BCI uses paper-thin electrodes implanted on the surface of the brain to detect electrical activity in the part of the brain that is critical for speech and facial movements.

Those electrodes intercept the brain signals that would typically help Ms. Johnson move the muscles in her lips, tongue, jaw, and larynx, as well as her face.

A cable plugged into a port installed in her head connects the electrodes to a bank of computers, which use AI algorithms to recognize unique brain signals for speech and translate them into sentences that are spoken through a digital avatar on a nearby screen.

Ms. Johnson worked with researchers to train the system’s AI algorithm, which involved repeating different phrases from a 1,024-word conversational vocabulary constantly until the computer recognized the brain activity patterns associated with all the basic sounds of speech.

The system can also decode those signals into text at nearly 80 words per minute, which is roughly half of the natural rate of speech, at around 150 to 200 words per minute, according to researchers, and has a median accuracy of around 75 percent.

The digital avatar even uses animated facial expressions, including opening and closing the jaw and smiling, that make it all the more realistic.

Ms. Johnson, who is a mother of two, hopes the latest technology will help other patients with similar conditions “know their lives are not over now.”

‘Disabilities Don’t Need to Stop Us’

“I want to show them that disabilities don’t need to stop us or slow us down,” she said. “When I was at the rehab hospital, the speech therapist didn’t know what to do with me. Being a part of this study has given me a sense of purpose, I feel like I am contributing to society. It feels like I have a job again. It’s amazing I have lived this long; this study has allowed me to really live while I’m still alive!”

The researchers also hope the technology will help provide a solution to other patients who are unable to communicate.

Their findings were published in the journal Nature on Aug. 23.

“Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others,” said Mr. Chang, who led the study at UCSF. ”These advancements bring us much closer to making this a real solution for patients,” he added.

The latest medical breakthrough comes just months after a paralyzed man was able to walk again for the first time in years thanks to an implant fitted in his brain and spinal cord allowing him to voluntarily move his legs and feet just by thinking about it.

The 40-year-old man, who was paralyzed in his legs and partially paralyzed in his arms after a cycling accident 12 years ago damaged his spinal cord, was told he would never walk again.

However, thanks to the device, known as a “brain-spine interface,” he can now stand, climb stairs, and even traverse complex terrains with the help of a walking aid, according to researchers.