RESEARCH
The project involves basic research needed to make possible a brain-computer interface for decoding thought and communicating it to an intended target. Applications are to situations in which it is either impossible or inappropriate to communicate using visual means or by audible speech; the long-term aim is to provide a significant advance in Army communication capabilities in such situations. Non-invasive brain-imaging technologies like electroencephalography (EEG) offer a potential way for dispersed team members to communicate their thoughts. A Soldier thinks a message to be transmitted. A system for automatic imagined speech recognition decodes EEG recordings of brain activity during the thought message. A second system infers simultaneously the intended target of the communication from EEG signals. Message and target information are then combined to communicate the message as intended.
Overview
In 1967, Dewan published a paper in Nature in which was first described a method for communicating linguistic information by brain waves measured using EEG. He trained himself and several others to modulate their brains' alpha rhythms: to turn these rhythms on and off at will. Alpha rhythms reflect synchronous neuronal activity, at or about a frequency of 10Hz, concerning not only whether the eyes are open but also one's state of attention. Mental activity and attention abolish these rhythms, which are normally strongest in a state of mental relaxation. Dewan was able to signal letters of the alphabet using Morse code by voluntarily turning these rhythms on and off, with eyes closed. Signalling such letters, one by one, provides the words and phrases that the communicator has in mind.
In 1988, Farwell and Donchin described a second method for transmitting linguistic information. This method is based on the P300 response, again measured using EEG. The P300 is evoked when a person is presented a stimulus that matches what it is they are looking for: a target. Farwell and Donchin display to the thinker the letters of the alphabet, one by one, and eventually display the letter that he or she has in mind. The P300 potential would be evoked, for that target letter, so signalling the thinker's desire to communicate that letter. Again, thinkers can communicate words by signalling the word's letters one by one. Much work as been devoted to brain computer interface in recent years.
Our basic scientific research is geared towards determining whether brain waves that are more directly linked to speech production can be used to communicate linguistic information. Speech is a natural method for communicating linguistic information. Were one able to use EEG to measure directly the activity of brain speech networks, one could potentially develop an easier and faster method for communicating linguistic information using EEG. Our work on imagined speech production pursues this idea.
The research aims also to determine, from brain waves, where the linguistic information should be sent: sent in a particular direction, sent to a particular person, etc. The question is not so much how the message should be sent but where or to whom. Work on the relationship between alpha rhythms and attention has, since Dewan's time, revealed that the pattern of alpha rhythm activity in the two hemispheres of the brain provides information on where a person is focusing attention. For example, paying attention to an area in the left half of one's visual field causes the alpha rhythm activity in the right hemisphere of the brain to desynchronize (and so diminish in intensity), and vice versa. These shifts in brain activity are thought to be helpful in directing more sensory and cognitive resources to the area being attended. We are studying alpha rhythms, evoked responses and steady-state evoked potentials measured using EEG to help develop a brain-computer interface that helps the thinker communicate to where or to whom a message should be sent.
Finally, we aim to learn more about activity in brain networks when two or more tasks are carried out simultaneously. Many studies in cognitive neuroscience involve brain-imaging measurements taken during the performance of a single task (e.g., visual detection, language processing, decision-making). Imagined speech production and direction intention are likely to use differing brain resources. Can these differences be used by a brain-computer interface to infer both a communicator's message and the recipient?
The funded research is of a basic scientific nature and focuses on neuroscience and signal processing. Work on this project develops neither devices nor applications. A functioning brain-computer interface for communicating thought and the intended recipient like that described above is years away. Yet one can identify several areas of future application. These include the development of a silent communications system for dispersed ground forces, of a speech-based means of communication for locked-in individuals, and of commercial communications devices based on brain-wave decoding.
Publications
Brigham, K. & Vijaya Kumar, B.V.K. (2010). Imagined speech classification with EEG signals for silent communication: a preliminary investigation into synthetic telepathy. Fourth International Conference on Bioinformatics and Biomedical Engineering, 1-4 [PDF]
Brigham, K. & Vijaya Kumar, B.V.K. (2010). Subject identification from electroencephalogram (EEG) signals during imagined speech. Fourth IEEE International Conference on Biometrics Theory, Applications and Systems, 1-8 [PDF]
Chi, X., Hagedorn, J.B., Schoonover, D. & D'Zmura, M. (2011). EEG-based discrimination of imagined speech phonemes. International Journal of Bioelectromagnetism 13(4) 201-206 [PDF]
Deng, S. & Srinivasan, R. (2011). Semantic and acoustic analysis of speech by functional networks with distinct time scales. Brain Research 1346 132-144 [PDF]
Deng, S., Srinivasan, R., Lappas, T. & D'Zmura, M. (2010). EEG classification of imagined syllable rhythm using Hilbert spectrum methods. Journal of Neural Engineering 7 1-13 [PDF]
D'Zmura, M., Deng, S., Lappas, T., Thorpe, S. & Srinivasan, R. (2009). Toward EEG sensing of imagined speech. Jacko, J.A. (Ed.), Human-Computer Interaction, Part I, HCII 2009, LNCS 5610 (Berlin: Springer) 40-48 [PDF]
Horton, C., D'Zmura, M. & Srinivasan, R. (2011). EEG reveals divergent paths for speech envelopes during selective attention. International Journal of Bioelectromagnetism 13(4) 217-222 [PDF]
Horton, C., D'Zmura, M. & Srinivasan, R. (2013). Suppression of competing speech through entrainment of cortical oscillations. Journal of Neurophysiology March 20, 2013, doi: 10.1152/jn.01026.2012. [PDF]
Srinivasan, R., Thorpe, S., Deng, S., Lappas, T. & D'Zmura, M. (2009). Decoding attentional orientation from EEG spectra. Jacko, J.A. (Ed.), Human-Computer Interaction, Part I, HCII 2009, LNCS 5610 (Berlin: Springer) 176-183 [PDF]
Thorpe, S., D'Zmura, M. & Srinivasan, R. (2011). Lateralization of frequency-specific networks for covert spatial attention to auditory stimuli. Brain Topography, doi:10.1007/s10548-011-0186, 1-16 [PDF]
Tian, X. & Poeppel, D. (2011). Mental imagery of speech and movement implicates the dynamics of internal forward models. Frontiers in Psychology: Auditory Cognitive Neuroscience, doi:10.3389/fpsyg.2010.00166, 1-23 [PDF]
Tian, X. & Poeppel, D. (2012). Mental imagery of speech: linking motor and perceptual systems through internal simulation and estimation. Frontiers in Human Neuroscience 2012; 6:314. doi: 10:3389/fnhum.2012.00314 [PDF]
Tian, X. & Poeppel, D. (2013). The effect of imagination on stimulation: the functional specificity of efference copies in speech processing. Journal of Cognitive Neuroscience 2013 Mar 7 [PDF]
Tian, X., Poeppel, D. & Huber, D.E. (2011). TopoToolbox: Using sensory topography to calculate psychologically meaningful measures from event-related EEG/MEG. Computational Intelligence and Neuroscience, doi:10.1155/2011/674605, 1-8 [PDF]
References
Dewan, Edmond M. (1967). Occipital alpha rhythm eye position and lens accomodation. Nature 214, 975-977.
Farwell, L.A. & Donchin, D. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70, 510-523.