How the brain uses sight and sound to process speech
Natural audiovisual speech encoding in the early stages of the human cortical hierarchy
This project looks at how watching a speaker’s face and hearing their voice work together to help people understand speech, especially in noisy situations.
Quick facts
| Grant type | R01 grant |
|---|---|
| Study type | NIH-funded research |
| Funding institution | University of Rochester NIH-funded |
| Lab location | 1 site (Rochester, United States) |
| Project ID | NIH-11238493 on NIH RePORTER |
What this research studies
You would watch and listen to natural speech while researchers record brain activity to see how visual cues like lip movements and the sound of speech combine over time. The team compares early acoustic-level processing with later language-level processing to map when and where integration happens in the cortex. Experiments use noninvasive brain recordings and advanced signal analyses on continuous, natural speech rather than isolated syllables. The goal is to build a detailed timeline of how sight and sound jointly shape speech perception.
Who could benefit from this research
Good fit: Ideal participants are adults who can attend in-person lab sessions and can watch a speaker’s face and listen to speech, including people with normal hearing and those with hearing or speech-perception difficulties.
Not a fit: People who cannot tolerate or attend lab-based recordings, or who have severe cognitive impairments that prevent following task instructions, may not benefit or be eligible.
Why it matters
Potential benefit: If successful, this work could inform better hearing aids, communication strategies, or therapies that use visual cues to improve speech understanding for people with hearing or language difficulties.
How similar studies have performed: Prior research shows that seeing a speaker’s face helps comprehension and that audio-visual interactions occur, but this project uses natural speech and detailed brain recordings to provide more precise timing and mechanistic detail.
Where this research is happening
Rochester, United States
- University of Rochester — Rochester, United States (Active)
Researchers
- Principal investigator: Lalor, Edmund — University of Rochester
- Study coordinator: Lalor, Edmund
About this research
- This is an active NIH-funded research project — typically early-stage science, not a clinical trial accepting patient enrollment.
- Some NIH-funded labs run parallel clinical studies or seek volunteers for related work. To check, contact the principal investigator or institution listed above.
- For full project details, budget, and progress reports, visit the official NIH RePORTER page below.