اخبار عربية How brain implants turn thoughts into speech

اليكم الان How brain implants turn thoughts into speech والان إلى التفاصيل من المصدر بتوقيت بيروت_ beiruttime:

Casey Harrell lost his voice to ALS or amyotrophic lateral sclerosis. As a volunteer in a clinical trial, he’s helping scientists test a brain-computer interface. Electrode arrays that are implanted in Casey’s brain send signals to nearby computers, where those signals are translated into words he intended to say. 

Casey uses this system every day to talk with his wife, daughter, friends and work colleagues. The moment he realized that it worked was “pure joy,” Casey says. Even though the researchers scripted his words for that first test, “it was a profound moment,” he says. “The possibilities for the near future for me and the rest of humanity were overwhelming to realize.”

This episode of The Deep End: Talk to Me follows Casey and the fast-moving neuroscience that enables him to speak.

Episode 4 transcript

Patients paralyzed from the neck down can manipulate robotic arms with their thoughts. The woman in this video is controlling a robot with her mind. The so-called BrainGate system relies on a sensor of electrodes implanted in the motor cortex of the brain. Researchers hope by implanting electrodes in the brain it can read the brain signals meant to move the muscles involved in speaking, and translate the person’s intention to speak into comprehensive speech produced by a computer. 

Laura Sanders: Controlling things with your mind. It sounds like science fiction, but it’s definitely real. Today, we’re going to explore what it’s like to communicate with a brain implant that decodes your thoughts and turns them into words.

(voices) Hey mom, can I tell you something? The voice is absolutely an integral part of your identity. Hello? That added layer of it being you. Do you want to talk? Your voice. Hi. Your voice is you. 

Sanders: From Science News, I’m Laura Sanders, and this is The Deep End, Talk to Me. Okay, so what is BrainGate? It’s an experimental system that connects electrodes that are implanted in a person’s brain to a nearby computer. It’s called a brain-computer interface, or BCI. The goal here is big: to help people with paralysis or brain injuries or missing limbs regain control over their world.

Leigh Hochberg: BrainGate clinical trials have now been going on for nearly 22 years. It’s 21 point something at the moment. 

Sanders: Leigh Hochberg is the director of the BrainGate Consortium and Clinical Trials. He also has a job at Massachusetts General Hospital, Brown University and the VA.  

Hochberg: People who may have had a brainstem stroke and suddenly lost the ability both to move and to speak, or for somebody with ALS who is progressively losing the ability to move or to speak, there’s long been a hope that particularly implantable brain-computer interfaces could restore communication and could restore mobility.

Sanders: In 2004, a BrainGate device was implanted into a person’s brain for the very first time. That person was a man named Matthew Nagle, and he had been paralyzed in an accident. With BrainGate, he could control a computer cursor with his thoughts. Here’s a 2008 report from IT World. 

Technology gave him hope. He was one of four testing BrainGate, a new technology where a chip is implanted in the brain that picks up electrical impulses. A computer then interprets those impulses as actions. 

Hochberg: Those early studies have been really encouraging, and I think speaks to the potential for the field. 

Sanders: Over the years, there’s been amazing examples of people using BrainGate. People have used it to click around on the internet, play video games, one woman who was paralyzed was even able to drink a cinnamon latte with a robotic arm. 

Hochberg: And she was able to think about using her own arm and reach out with a robotic arm and pick up that thermos and bring it to her mouth, and tip it towards her mouth, and then put the straw in her mouth and then to take that sip. And while it wasn’t the first time in nearly 15 years that she had had something to drink, it was the first time that she was really able to do so solely of her own volition. I remember the smile on her face when she did that. 

Sanders: Over the decades, BrainGate has had a lot of research success in a lot of ways. But what about talking? Hochberg says that when it comes to decoding speech, the science here is still very, very young. 

Hochberg: We are at early days, just to be super clear. This is the very beginning of this field. There are only a small handful of people in the world who have had any type of implantable BCI that has been decoding speech. 

Sanders: Where does the science stand when it comes to decoding the intention to talk? In the past few episodes, we’ve heard the stories of people who have lost their ability to speak. One of those people was Casey Harrell, and Casey also happens to have a BrainGate device implanted in his brain. 

Casey Harrell: My name is Casey Harrell and I am 47 years old. 

Sanders: Casey wakes up in the morning, and plugs in. 

Casey: The first thing I will usually do after getting out of bed is to plug into the Matrix, so to speak. 

Sanders: By that, he means attaching two wires to two pedestals that come out of the top of his head. These two black wires then snake down to nearby computers. Collectively, these electrodes in Casey’s brain, wires from his head, and computers are called the BrainGate 2 neural interface system. So Casey gets it all fired up and then he’s off. 

Casey: Unless I have a medical appointment or something that will move me out of the house, I will sit in this position for most of the day, with specifically small breaks in between. 

Sanders: All the while, Casey is generating brain signals as he thinks about saying words. These are small, precise electrical pulses made by Casey’s neurons, specifically the cells in his speech motor cortex. Remember how we learned there are over 100 muscles that are involved in speech? These are the parts of the brain that control those muscles. 

Casey: I am using a decoder for about ten to 14 hours a day, seven days a week, for the past almost two years, making me the world’s biggest power user of the brain-to-computer interface known to man. 

Sanders: Like Jules, Casey has ALS. The disease has slowly taken away his ability to move and to talk. 

Casey: I began to lose my ability to talk shortly after my diagnosis in 2020. Then it became a slow motion car crash of multiple years before almost no one would be able to understand me. 

By the time of my surgery, only my assistant and my wife, Levana, could really understand me fully. A few other friends who were around often would be able to understand some of what I was saying. I relied upon my assistant and my wife to basically serve as my translators to other people. I did not go anywhere, even a video call without them, either one or both.

Sanders: In July of 2023, Casey volunteered for a clinical trial called BrainGate 2. A UC Davis neurosurgeon put four small electrical arrays onto the left side of Casey’s brain. I wanted to know more about how these arrays worked, so I talked to the neurosurgeon who put Casey’s devices into his brain. 

David Brandman: My name is David Brandman, and I am the co-director of the UC Davis neuroprosthetics lab, developing next-generation neurotechnology to help people with paralysis. A brain-computer interface is a device that records brain signals from people that are paralyzed and then translates those brain signals to allow people to control objects in their environment using their minds. 

Sanders: Casey’s implant is made of four arrays. Each of them are square and they’re about as wide as a peppercorn. Each array has a bunch of pointy ends that go down into his brain. They look a little like those metal cat brushes. Altogether, the four arrays contain 256 electrodes. These electrodes are all listening to Casey’s brain activity when he thinks about saying a word. 

Brandman: The way this device works is by recording the electrical activity of the brain as he tries to speak. 

Sanders: The wires from Casey’s head carry those neural signals to nearby computers. That’s where machine learning algorithms attempt to do a very complex sort of matching. The algorithms match Casey’s brain activity patterns to the patterns that predictably occur with certain sounds called phonemes. 

Brandman: We translate his brain signals into one of 39 different English phonemes. Now, a phoneme is a building block of language. So if I say the word hello, the word hello can be broken down into ha, uh, uhl, and oh. Hello.

Sanders: Next up, those algorithms assembles these phonemes into words by making its best guess about what Casey wants to say.

Brandman: So I translate this brain activity into a phoneme and then, using artificial intelligence, turn phonemes into words, words into sentences, and sentences into paragraphs.

Sanders: So that’s the process, and it allows Casey to type messages on Slack, text his friends, and send emails. Sometimes in his emails to me, he includes a tagline at the bottom: “Sent from my brain,” with that cute little pink brain emoji. His setup also lets him convert these typed messages into spoken words. These words show up on a screen for him where Casey proofreads them. If everything looks good, Casey gives the go-ahead and the system speaks the words out loud. Brandman tells me that when Casey tries to speak naturally, he can say about six understandable words per minute. That’s pretty slow.

Brandman: Using this prosthesis, he’s in the ballpark of 50 words per minute. Now, as I’m speaking to you right now, I’m speaking on the order of 120 to 150 words per minute, depending on how excited I am, right? So we’ve got him back to about a third of how quickly he spoke as an able-bodied person.

Sanders: I had emailed Casey some questions a few days before our talk. But he didn’t answer them by email. He used his brain-computer interface to answer my questions in real time just by thinking about the words he wanted to say to me. As we introduced ourselves, I saw how the timescale had stretched. 

Casey: It’s a challenge because we can have this conversation, but I am slower than the average Joe, even where we grew up.

Sanders: Casey and I both grew up in the south, Casey in the Atlanta area and me in middle Tennessee. But he’s lived in the Bay Area for over 20 years now. And like me, his accent had faded. 

Casey: And yes, I do not have much of a Southern accent anymore. But I love my roots, and I would love to show my daughter where and how I grew up. But it is exceedingly difficult to travel when you cannot move. 

Sanders: Before his mobility was limited, Casey did a lot of cool things.

Casey: I spent most of my professional career as a climate campaigner and have went all over the globe, fighting companies who are destroying the Earth and trying to bring some justice to our planet and the people who inhabit it. I am very fortunate because I have been able to do something that I love and also develop some deep friendships and relations with other phenomenal people. 

Sanders: My interview with Casey was unlike any I’ve ever had. I’m used to being flexible for interviews. There’s time zone problems, language barriers, mathematicians who sometimes answer my questions in symbols and formulas. But when I logged on to Zoom to talk with Casey, I quickly realized this conversation would be different. I was self-conscious at the beginning of our talk. I fumbled around too much. I’d ask Casey a question and then another one before he had a chance to answer. I accidentally interrupted him a few times. Sometimes I wasn’t sure if he was composing an answer or just waiting for me to say something else. This poor performance on my part wasn’t unusual for Casey at all. 

Casey: It is hard for people in your position to stay focused because you were not used to the cadence of a conversation this slowly. 

Sanders: He’s right. I was not used to it. And timing, in conversations and in life, is everything. Here’s Greg Bryant of UCLA.

Bryant: Turns, conversational turns, happen at the level of hundreds of milliseconds at the longest. Two hundred milliseconds is a long turn. That’s a fifth of a second. I mean, that’s really fast. So if you are slow at all, it’s noticeable. We have to coordinate our talk so our terms make sense. I can’t take too long or be too quick to respond to your questions. I have to know when to stop and start. We get into this flow, kind of a dynamic, and that is hugely consequential for how we feel about the conversation, the other person, ourselves. You know, it’s all timing. 

Sanders: Let’s turn to Jules. His words start in his brain too, but then they take a different route. He types them by looking at letters on a screen, then his voice clone says them. It’s also a slow process and one that takes a lot of mental power, too. At one point in our conversation, MA describes the effort this requires of Jules. 

MA: And in his brain, you know, you can imagine he’s doing all these mental gymnastics, to, right now, he’s listening to us. He’s writing something with his eyes, like letter by letter to contribute to the conversation. Plus, he has the answers that he pre-wrote, of the questions that you asked. So it really is like, there’s a whole lot happening behind the scenes that of course anybody who doesn’t have to think about their voice takes for granted. So I’ll pause to let him complete what he’s saying.

Sanders: MA and I wait here for Jules to weigh in. So far you’ve heard ten seconds of the pause. The full length of the pause was a minute and 18 seconds. 

Jules: It is a challenge to communicate with people who don’t interact with me on a daily basis. I have a major bottleneck from my thoughts to actually speaking. 

Sandes: And as with Casey, these pauses can introduce difficulties. Here’s speech pathologist Jessica Pietrowski with more on that. 

Pietrowski: Humans are not very comfortable with silence, especially communication. And when we’re talking to one another, we fill in the silences, it’s less awkward. And so, someone who’s communicating with a device is probably very used to those latency periods and some more silence and they’re comfortable with it. Whereas the person they’re communicating with is like, let’s keep it going, like this is, I want to help, I want to fill in the blanks, I don’t want this to feel awkward, but in turn, might make it a little more awkward than they think. 

Sanders: Jules hates when people read his screen before he’s ready to say what he wants. This particular hate inspired a joke in his stand-up routine. 

(Jules audio): People read off my screen while I’m typing and try to finish what I am saying. It’s like having to constantly deal with an over-competitive charades partner. 

Sanders: Jules and Casey encounter other people’s impatience constantly. Not that it isn’t well-intentioned but still, it does not help. Even worse, they face other people’s misconceptions about what they’re capable of. Casey is an interesting, funny and compelling conversationalist. That might not be apparent to everyone who doesn’t slow down to hear him. And that’s an assumption that really bothers him. 

Casey: It is incredibly undignified and a terrible consequence of the health care system to treat me like this very quickly. 

Sanders: Brandman has seen this dismissal, this mistreatment too. It happens with some of the volunteers he works with on the BrainGate 2 project I mentioned earlier. That’s the same trial Casey’s in.

Brandman: I think as healthcare professionals, we conflate the inability to communicate with the lack of understanding. And I’ve witnessed this firsthand as I’ve watched doctors and nurses interact with my BrainGate participants and then treat them as if they are invalid. Or they’re not there because they’re unable to express themselves. And in fact, I go out of my way in the intensive care unit on the wards and dealing with anyone who’s dealing with my participants to say, look, you know, they’re inside. They’re there. They can understand everything that’s happening. They just happen to not be able to speak. And so please treat them with the respect and dignity that they deserve. And it’s a constant challenge for the participants that I work with.

And it’s the reality of which we face and it’s the bias that we have against people that can’t speak for themselves.

Sanders: Casey and the other people with ALS are not alone. Many people who have trouble talking or who just need more time can be judged this way. Here’s Karina Nagin from Bridging Voice. 

Nagin: It’s how people see you. And so when you have ALS and you cannot move and you don’t even have facial expressions and you’re hooked up to machines, people assume you’re not there. And we see this every day where people either don’t talk to the person with ALS, assuming that they can’t respond. They speak really loudly and slow. Their mind is completely working. 

Sanders: For now, Casey’s words may be slow. But despite all the drawbacks, they are still an absolute marvel. Remember Hochberg from the start of this episode? He’s one of the neuroscientists who works with Casey, and he told me a story from the first few days of testing after Casey’s BCI was implanted.

Hochberg: We gave him a 50-word vocabulary on day one. And even on day one, he was able to say some key sentences from those 50 high-value words. 

Sanders: At this point in the testing, Casey was asked to say some pre-specified sentences so researchers could test their system out.

Casey: Most of the sentences were what we called closed loops.

Sanders: That means that the researchers already knew what brain activity patterns should go with the words Casey thought about saying. It was a test to see if the system was working. 

Casey: The decoder would be able to verify whether I was right or wrong, giving me short preloaded sentences to say. I was asked to say, “What good is that?” 

Sanders: “What good is that?” is not the most poignant sentence ever uttered, but it had a big effect both in the room and for researchers joining remotely by video.

Casey: “What good is that?” Everyone is crying after I am successful.

Sanders: Later on, there was time for Casey to actually express himself.

Casey: It was only after the first few sessions that I had any real time to say what I wanted for a good amount of time in relative privacy. That said, it was pure joy, even though I wasn’t getting to say what I wanted to say in that moment. It was a profound moment for me. The possibilities for the near future, for me and the rest of humanity were overwhelming to realize.

Sanders: Things only got better from there. To set the scene here, during this session, Hochberg was in his office and Casey was at his home, sitting in his living room. Hochberg was watching Casey on a screen. 

Hochberg: On the second session, came back and gave him the full 125,000 plus word English dictionary to be able to select from. That is, the decoders and the AI and the machine learning language models would be able to select from basically a full dictionary. And the first thing that he said was something along the lines of, “Is this thing on?” Kind of a funny statement. 

The next thing that he said on the second day of using brain-computer interface, or the second thing that we decoded was, “I’m looking for a cheetah.” And it’s the second day. We know there’s all kinds of opportunities for decoding errors. And when I saw that, I said, you know, there’s no way that the second sentence on the second day of using an implanted brain-computer interface is going to say anything about a cheetah. You know, that’s obviously a decoding error and that’s fine. 

Sanders: Hochberg could see Casey on the screen, but he couldn’t see something really important that had just happened in the room. 

Hochberg: His four-year-old daughter had just walked into the room dressed as a cheetah. 

Sanders: Here’s Casey remembering the same moment.

Casey: Yes, I remember this time very well. It was the second session of talking that I experienced early in August 2024. My daughter was incredibly cute in her cheetah outfit. This was one of her favorite animals. They were going through a phase where everything they did was super fast, running from thing to thing. Everyone was thinking that it must have been a mistake that I could not use this language. It was so random. This was only from the people like Leigh who were not in the room with us. The researchers who were there could validate that, yes, I said this. I meant it because of the unique context. 

Sanders: It’s one of those moments that many parents have. A kid does something cute, a parent plays along. But in this moment for Casey and his child, it was anything but ordinary. 

Hochberg: And she was born around the time that he was diagnosed with ALS. And one could imagine that because the voice that was coming out of the computer was a version of his own voice, and then the rest of the interaction was between dad and daughter. And it was between parent and child. 

Sanders: The BCI had given Casey something priceless, a minute to play with his sweet little cheetah. 

Casey: I was quietly confident that it would work the way it ended up working for me. I had done a lot of research and the thing was that the technology was on the cusp of a breakthrough. And why not me? I asked myself. 

Sanders: Early on in the research project, Casey and his wife had been told not to expect it to work. This was an experiment and the outcome was far from certain. Casey, however, was optimistic.

Hochberg: So to me, seeing it work that quickly and that powerfully really, really emphasized that these are technologies that are going to be able to restore not only activities of daily living, not only key kind of instrumental activities of daily living, but they’re going to restore communication between parent and child. They’re going to restore communication between friends and family and care partners and health care providers. 

Sanders: Because this is a podcast about the human voice, I’ve been asking people I interview what their voice means to them. Here’s Casey’s answer.

Casey: It means so much. I think it is the most important thing that I have lost in this whole disease.

Sanders: Getting it back, even in a limited form, is everything.

Casey: It was so amazing to be able to be understood right from the get-go and to be able to talk and respond, respond to my daughter, especially because that was lacking completely from my life up to that point.

Sanders: Casey’s restored voice changed his life.

Casey: It is the ability to parent my daughter. It is the ability to connect with my wife. It is the ability to make a wry joke with my friends. Timing is everything, and that is why having nothing, this is completely revolutionary. And to look back on my previous answer, it is something that really can still improve. 

Sanders: At the beginning of the experiment, the research team had told Casey and his wife that they’d be able to try using the BCI for casual chats just a few hours a week. Those hours felt like a huge gift at the time, considering that they hadn’t been able to talk meaningfully for a long time. Here’s Brandman. 

Brandman: As we kept coming to his house two or three times a week, things kept getting better and better. And it got to a point where Casey wanted to use the system to say all the things that he wanted to say.

Sanders: Early on, only the researchers could set Casey up with the system, plugging him into the Matrix, as he puts it.

Brandman: And so our research technologist would start coming to his house three, four, five times a week to don and doff the system to enable him to communicate so that he could go about his day. And so at a certain point, we saw, well, he really wants to use the system. Can we streamline this for him? And so we petitioned the FDA to actually allow us to change the clinical trial protocol slightly so that we could train his care partners to don and doff the system without us being there. And so since then, he’s been a near daily user of the system.

Sanders: That’s a far cry from a few hours a week. I’m going to read you some numbers from a research paper Brandman and his colleagues published about Casey’s BCI in Nature Neuroscience in June of 2026.

“Over nearly two years, the participant, that’s Casey, used the BCI for more than 3,800 hours at home with no researchers present to maintain rich, interpersonal communication with his family and friends, independently control his personal computer, and sustain full-time employment, despite being paralyzed. He communicated 183,060 sentences, totaling 1,960,163 words, at an average rate of 56 words per minute. He labeled 92 percent of sentences as being decoded at least mostly correctly.”

Those numbers in this very bare-bones science kind of way are dry, if we’re being honest. But when you stop to think about it, each sentence, each of those nearly two million words, is a tether that connects Casey to his life. They’re messages to his daughter, jokes with his partner, texts to his friends. They’re games of Wordle with his wife and kid. Here’s Casey and his family in a video shot by UC Davis Health. 

Okay, so no A, but we got an E. M-I. O. M-I. Good job, team. We are almost there.

Sanders: We’ve talked a lot about words here and the fundamental ways that Casey’s BCI has restored his ability to say them. But Casey’s BCI goes a little bit farther than that. It gives him the ability to add some extras. He’s got some bells and whistles up his sleeve.

Casey: It is really a communication device, first and foremost, and less of a tool that I use to make funny remarks. But I do have that as well in my repertoire. For example, I have the ability to add sound effects to what I am saying. 

Sanders: For instance, Casey has a sad trombone sound. 

(sad trombone sound)

Casey: So if I find something funny and want to say, LOL, the theme from the TV show, Seinfeld, will play. 

(theme song)

Sanders: This is what he means. These jokes make me think of Jules, which made me wonder if Casey, like Jules, could also swear with his system.

Casey: To answer your question of whether I can curse with this device, I most certainly (beep) can.

Sanders: So for all your inquiring minds, there you have it. Looking ahead a bit, there’s a lot happening on the horizon. The science here is moving fast and many companies are working on similar systems. We’re going to cover that in a few episodes. But for now, Casey is alone. He is an N of 1, as the scientists say – the only person using a brain-computer interface for communication this heavily. 

Casey: I hope to have this changed very soon because I feel like everyone who wants this should have the opportunity to have it, especially those who are with my disease. 

Sanders: Hochberg tells me that this drive to help others is common among people who volunteer for these sorts of clinical trials. 

Hochberg: Each participant in the trial has been just so generous and so clear that they recognize that it’s early days in the research, but they’ve told us that they want to help other people with paralysis. They want to be able to allow people with brainstem stroke or cervical spinal cord injury or ALS or other paralyzing illnesses or injuries to have improved function, to be able to regain control over that computer or regain the fluent speech and communication that they had previously. So it’s that generosity of spirit that has really defined the participants in our BrainGate studies. 

Sanders: Casey’s generosity, his drive to help other people, really shines through as we talk. But I want to get back to him. I ask Casey what it’s been like to have this technology in his home, in his brain. He tells me how precious our voices are, and how easy it is for many of us to forget that.

Casey: We take them for granted. The ability to scream when you are running and want to get out some stress. The ability to say sweet nothings into your partner’s ear. These are the things that life is all about. And when you are robbed of those times, you want them back at all costs. And this is why this is good enough for now, and we should have accelerated approval for devices like this.

Sanders: Casey’s BCI is slow compared to the pace of a normal conversation. And it gets things wrong occasionally.

Casey: Sorry, I am trying to answer you. And it is changing what I want to say. And I am being stubborn instead of trying to say it another way, which I will do now.

(Sanders audio) I’m not in any rush. You take as much stubborn time as you want. 

Sanders: And it needs an internet connection and power. And let’s not forget that Casey has an adapter coming out of the top of his head. But these are solvable problems, one that scientists are actively working on. So not perfect, but as Casey puts it, good enough for now. 

Scientific efforts to restore voices have been transformative for Jules, Casey, and others. But the ability to separate a voice from a body also raises new possibilities for our societies and for our lives. 

MA: His voice will live on in whatever videos or things we’ve captured, whatever he’s left us intentionally, and that’s as far as that goes. 

Sanders: On the next episode, we’re stepping out of the lab and into a strange new digital world where voices might not be what they seem. 

Hany Farid: And so we are living in, I think, in equal parts an exciting time and a terrifying time. And somehow those are coexisting. 

Sanders: What changes are coming in a world where voices can be so easily cloned?

Farid: Today, with nothing more than a keyboard and an internet connection, you can type and you can get an image, a voice, a video of anything, anybody doing or saying anything.

Sanders: The Deep End is a production of Science News. It’s based on original reporting by me, Laura Sanders. This episode was produced and mixed by Ella Rowen. Our project manager is Ashley Yeager. Nancy Shute is our editor in chief. Our music is by Blue Dot Sessions. The podcast is made possible in part by the Burroughs Wellcome Fund with support from PRX.

This podcast was produced with support from the Burroughs Wellcome Fund and in partnership with PRX. The podcast logo was created by Neil Webb.

المزيد من التفاصيل من المصدر - (اضغط هنا)

كانت هذه تفاصيل How brain implants turn thoughts into speech نرجوا بأن نكون قد وفقنا بإعطائك التفاصيل والمعلومات الكامله .

و ننوه بأن الموضوع الأصلي قد تم نشرة ومتواجد على بتوقيت بيروت_ beiruttime و قام فريق العمل فيبرسبي نيوز بالتاكد منه او ربما تم التعديل علية او قد يكون تم نقله بالكامل اوالاقتباس ويمكنك قراءة الموضوع من مصدره الاساسي.