Surf the internet with your voice

Doctoral student Brandi House uses Vocal Joystick to control the movement of a robotic arm. The screen on the lower right shows how the software analyzes her vocalizations to create instructions for the arm's movement. (Credit: Image courtesy of University of Washington)


From Science Daily:

Vocal Joystick detects sounds 100 times a second and instantaneously turns that sound into movement on the screen. Different vowel sounds dictate the direction:
"ah," "ee," "aw" and "oo" and other sounds move the cursor one of eight directions. Users can transition smoothly from one vowel to another, and louder sounds make the cursor move faster. The sounds "k" and "ch" simulate clicking and releasing the mouse buttons.

Versions of Vocal Joystick exist for browsing the Web, drawing on a screen, controlling a cursor and playing a video game. A version also exists for operating a robotic arm, and Bilmes believes the technology could be used to control an electronic wheelchair.

It's a great idea, using the musical/sounds quality of vowels rather than of actual words. I wonder if someday speech therapists could use this for kids as well?