March 6th, 2025

Students from Skyline High School in Oakland, CA visited Smith-Kettlewell to learn about applications and limitations of Artificial Intelligence (AI) for assistive technology. Students learned how to get a local multimodal AI running on their devices and use/extend its capabilities for assistive technology.

The day started with a short greeting and introduction by Dr. James Coughlan, the director of SKERI’s Rehabilitation Engineering Research Center on Blindness and Low Vision. Dr. Coughlan was followed by Dr. Nasif Zaman, who spoke more broadly about the need for universal and inclusive design that takes the needs of the users into account. He showed examples of features like optical character recognition, use of alternative text, and applications like Envision AI that allow blind users to interact with a broader range of features in the modern world.

Dr. Sile O’Mhodrain joined the girls virtually to give a keynote presentation to discuss her journey from the BBC to academia and the need for universal design in all aspects of technological advancement.

These talks were followed by a hands-on portion of the visit where with the help of SKERI researchers, students interacted with vision language models to learn about the capabilities and limitations of current assistive AI implementations like Microsoft’s Seeing AI. Students had fun asking the AI model to draw and then describe a variety of objects. Using external camera capabilities for object detection, students identified the limitations of detection under varying conditions.

The event concluded with a short panel facilitated by Dr. Catherine Agathos about the joys, trials, and tribulations of the academic research career path and different ways SKERI scientists have joined this important enterprise.

You can read more about the event and students’ feedback from the event on IGNITE Worldwide’s website.