Lip-reading involves tracking facial movements to determine what someone is saying. Many lip-reading devices point a camera at the user's face. Others rely on sensors stuck around the speaker's mouth. Neither approach is suitable for daily use, says Ruidong Zhang of Cornell University in Ithaca, NY. His team built a new lip-reading tech on a pair of eyeglasses, which uses acoustics—sound—to recognize silent speech. Zhang presented this work on April 19, 2023 at the ACM Conference in Hamburg, Germany.
"Imagine the sonar system whales use, " says Zhang. They send a sound into their environment and listen for echoes (回声). From those echoes, they locate objects in their surroundings. "Our approach is similar, but not exactly the same," Zhang explains. "We're not just interested in locating something. Instead, we're trying to track moving patterns."
Zhang calls the new tech EchoSpeech, consisting of two small speakers under one lens of a pair of glasses, two small microphones under the other lens, and a circuit board attached to one of the side arms. When EchoSpeech is switched on, its speakers play high-pitched sounds. People can't hear them. The sound waves move in every direction. Some travel around the user's lips and mouth. While speaking, the user's facial movements change the paths of those sound waves. That, in turn, changes the echo patterns picked up by the microphones. These patterns are sent to the wearer's smartphone over Bluetooth. Using AI, an EchoSpeech app then interprets the echo patterns. It matches each pattern to commands the smartphone then follows.
EchoSpeech currently recognizes 31 voice commands and recognizes numbers that are three to six digits long. According to Zhang, future versions could recognize a much larger vocabulary. If so, users could write personal text messages via silent speech. In a noisy restaurant, they could use that approach to send messages to friends who're hard of hearing.
People who've had their vocal cords (声带) removed have been contacting Zhang's team. They want to know if this interface could read their lips and then speak out loud for them. Zhang is now exploring whether EchoSpeech could do this in a person's own voice.
【小题1】Why does Zhang mention the sonar system used by whales in paragraph 2?| A．To make their new tech more convincing. |
| B．To highlight the advantages of their new tech. |
| C．To explain the inspiration for their new tech. |
| D．To make their new tech more understandable. |
| A．Sound travels in a straight line. |
| B．One's facial movements are unique. |
| C．Sound waves are reflected from an object. |
| D．Microphones are sensitive to loud sound. |
| A．People can hear sounds created by EchoSpeech. |
| B．EchoSpeech will be put into use before long. |
| C．EchoSpeech can enhance interpersonal communication. |
| D．Users can use EchoSpeech to write personal messages. |
| A．New glasses read your lips and tell your phone. |
| B．Lip-reading devices can understand voice commands. |
| C．Lip-reading techniques are springing up worldwide. |
| D．Recognizing silent speech has already become possible. |
