A 'Point and Listen' Tool for People with Vision Loss
Official title: Point and Listen: Augmented Reality Interfaces for the Visually Impaired
Funded at Smith-Kettlewell Eye Research Institute · NIH-11046532 · Data updated 2026-11-12
This project is creating a new smartphone and wearable app that helps people who are blind or visually impaired 'hear' information about objects they point to.
Quick facts
| Grant type | R01 grant |
|---|---|
| Study type | NIH-funded research |
| Funding institution | Smith-Kettlewell Eye Research Institute NIH-funded |
| Lab location | 1 site (San Francisco, United States) |
| Project start | March 10, 2016 |
| Project end | February 28, 2027 |
| Funding year | 2025 |
| Award (this year) | $438,051 |
| NIH project number | 5R01EY025332-08 |
| Project ID | NIH-11046532 on NIH RePORTER |
Who could benefit from this research
✓Could help
Ideal candidates for this technology would be individuals who are blind or visually impaired and are interested in using smartphone or wearable camera technology to interact with their environment.
✕Less likely to help
Patients who are not comfortable with or unable to use smartphone or wearable camera devices may not receive direct benefit from this specific technology.
What this research studies
This project is developing an app called CamIO, which uses augmented reality to help people with vision loss understand physical objects around them. By simply pointing your smartphone or a special wearable camera at an object like a document or a map, the app will speak aloud information about what you are pointing to. This technology allows you to explore objects naturally with your fingers and hear details, without needing special braille labels or modified items. The goal is to make everyday items more accessible and easier to interact with for those who are blind or have low vision.
Why it matters
Potential benefit: If successful, this technology could significantly improve daily independence and access to information for people who are blind or visually impaired.
How similar studies have gone: While augmented reality for vision assistance is an evolving field, this project builds on existing prototype app work and aims to introduce novel enhancements like finger tracking and wearable camera support.
Where this research is happening
-
Smith-Kettlewell Eye Research Institute
San Francisco, United States Active
Researchers
- Principal investigator: Coughlan, James M — Smith-Kettlewell Eye Research Institute
- Study coordinator: Coughlan, James M
Related research
Other NIH-funded projects on the same topics.
-
How our eyes move together and apart to see in 3-D
A study at a leading eye research center is exploring how each eye moves on its own and how they work together to guide vision, which could improve understanding and treatment of eye alignment problems like strabismus.
-
Bilirubin-related vision difficulties in children after neonatal jaundice
This project studies how newborn jaundice with high bilirubin may impact vision and brain areas involved in seeing later in childhood.
-
How people with central vision loss aim and keep things in view
This project looks at how people with central vision loss from conditions like age-related macular degeneration move their eyes and head to find and keep things in view during everyday tasks.
-
Maximizing Visual Potential for People with Age-Related Macular Degeneration
This grant investigates how people with age-related macular degeneration adapt their eye movements and attention to use the best remaining vision to perform everyday tasks.
-
How attention can change eye suppression in amblyopia (lazy eye)
This project tests whether directing attention to the weaker eye can reduce visual suppression for adults with amblyopia (lazy eye).
-
How the brain coordinates both eyes
Looking at whether separate brain areas let each eye move on its own or move together, which could help people with eye alignment problems like strabismus.
About this research
- This is an active NIH-funded research project — typically early-stage science, not a clinical trial accepting patient enrollment.
- Some NIH-funded labs run parallel clinical studies or seek volunteers for related work. To check, contact the principal investigator or institution listed above.
- For full project details, budget, and progress reports, visit the official NIH RePORTER page.