A Scientific Breakthrough in Mind Reading: Inventing Gadgets Everyone Can Buy

2024 Author: Seth Attwood | [email protected]. Last modified: 2023-12-16 15:55

Engineers from the Institute for the Study of Higher Nervous Activity have been able to create a system that translates thoughts into intelligible and recognizable speech. By controlling someone's brain activity, technology reconstructs thoughts into words.

This scientific breakthrough, coupled with the power of AI and speech synthesizers, opens a new era in the interaction between the computer and the brain. Of course, this also opens up new opportunities for people who, for one reason or another, have lost the ability to speak.

“Our voices help keep in touch with our friends, family and the world around us, so loss of voice power due to injury or illness is devastating for people. However, with today's stock of knowledge, we have a potential way to restore this power. We have shown that with the right technology, people's thoughts can be deciphered and understood by any listener,”says Nima Mesgarani, Ph. D. and one of the authors of a study conducted at the Mortimer B. Zuckerman Institute of Mental Behavior at Columbia University.

Decades of research have shown that when people speak, or even imagine that they are uttering words, characteristic patterns of activity appear in their brains. A clear and recognizable signaling pattern also arises when we listen to someone speaking, or imagine that we are listening. Experts have tried to decipher these patterns for decades, but only now a future has opened up before them, in which thoughts can no longer be hidden inside the brain, but instead can be translated into spoken language at will.

But accomplishing this feat was not easy. Early attempts to decipher brain signals from Dr. Mesgarani and others focused on simple computer models that analyzed spectrograms, which are visual representations of sound frequencies.

But due to the fact that this approach did not produce anything close to intelligible speech, Dr. Mesgarani's team instead turned to a vocoder, a computer algorithm that can synthesize speech after being trained to record people's conversations.

“This is the same technology that Amazon Echo and Apple Siri use to verbally answer our questions,” said Dr. Mesgarani, who is also an associate professor of electrical engineering at the Fu Foundation's Columbia School of Engineering and Applied Science.

To teach the vocoder to interpret brain activity, Dr. Mesgarani teamed up with Ashesh Dinesh Mehta, M. D., PhD, neurosurgeon at Northwell Health Physician Partners' Neuroscience Institute and co-author of today's article. Dr. Mehta treats patients with epilepsy, some of whom must undergo regular surgeries.

“Working with Dr. Mehta, we asked epileptic patients who had already had brain surgery to listen to suggestions from different people while we measured patterns in their brain activity,” says Dr. Mesgarani. "These neural patterns trained the vocoder."

The researchers then asked the same patients to listen to speakers pronouncing the numbers 0 through 9 while recording brain signals that could then be passed through a vocoder. The sound produced by the vocoder in response to these signals was analyzed and refined using neural networks such as artificial intelligence that mimic the structure of neurons in a biological brain.

The end result was a robotic voice repeating a sequence of numbers. To verify the accuracy of the recording, Dr. Mesgarani and her team instructed people to listen to the recording and report what they heard.

“We found that humans can understand and repeat sounds about 75% of the time, which is far superior to any previous attempts,” says Dr. Mesgarani. The improvement in intelligibility was especially evident when comparing the new recordings with earlier attempts based on the spectrogram. "A sensitive vocoder and powerful neural networks represented the sounds that patients originally heard with amazing accuracy."

Dr. Mesgarani and her team now plan to test more difficult words and sentences. They also intend to perform the same tests on the brain signals emitted when a person speaks or imagines speech. Ultimately, they hope their system can be part of an implant, similar to those worn by some epileptic patients, which translate the wearer's thoughts directly into words.

“In this scenario, if the owner of the chip thinks, 'I need a glass of water,' our system can take in the brain signals generated by that thought and convert them into synthesized verbal speech,” says Dr. Mesgarani. “It's a game changer and for anyone who has lost the ability to speak due to injury or illness, technology gives a new chance to communicate with the world around them.”

Editorial comment of The Big The One:Since some of our employees have a certain relationship to neurophysiology, we can absolutely unequivocally state that solving the problem of reading thoughts and translating these thoughts into words is not a problem that some doctor of philosophy can solve together with an intelligent neurosurgeon. This is a task for the research institute, which will solve it for a hundred, two hundred or more years. Moreover, it is not at all a fact that the scientific research institute will solve this problem - even if all NASA supercomputers are brought there, on which a crowd of engineers will begin to simulate neural networks. However, an article in a scientific journal will not lie and the fact of recognition of thoughts is definitely there. How, then, can these two facts be brought together?

Very simple. In the last 20-30 years, a huge number of completely incredible and very complex technologies such as microprocessors and hard drives have appeared in the world. And every day more and more new inventions appear, which are written down to some "talented students" who, sitting in the garage, have collected a piece from scrap metal, over which research institutes have been struggling for 50 years. And in this case we have an invention of a similar genesis. That is, a development that was made (or handed over to people) for many years by some unknown person, but which was recorded on an aunt and an uncle from Colombia.

In fact, there is absolutely no difference at all who Wikipedia will write down as the "inventors" of the technology. The main thing is that the technology was presented to the world using a scientific journal. Further, some Chinese company will begin to produce such "gadgets" that will accustom people to the very idea that their head has now become completely transparent. Finally, the third stage will be the same Orwellian "1984" when even the thoughts of any person can be freely read from a distance. For example, by placing special sensors in apartments or integrating a special chip that responds to brain impulses into the body. Moreover.

As a rule, each technology has a reverse application. For example, if there is a technology for converting nerve impulses into words, then using the same principle, you can turn any words into someone's thoughts. In this situation, the presidents of different countries have a grandiose prospect of being supported by absolutely the entire electorate, since now there is a technical opportunity to make every voter even think like him - it is enough to put a correctly modulated signal into the socket.

In general, what to say - the prospects for the world are opening up the most wonderful and we are following the development of events with interest.