Scientific

Digital drawing has historically been a significant challenge for blind individuals, with previous solutions requiring extensive imagination and technical skills. However, the Coughlan lab has been developing a groundbreaking “what you hear is what you get” drawing tool, revolutionizing this field. This innovative tool, powered by Audiom, enables blind users to create and edit complex shapes, maps, and art through auditory feedback. Users navigate a canvas using arrow keys, dropping points to form shapes and lines, which are then audibly represented, allowing for an intuitive and accessible drawing experience. Additionally, the tool supports the addition of sounds to objects, enhancing the creative process. This advancement not only facilitates artistic expression among the blind community but also offers educational applications, such as geometry assignments. With the ability to instantly share or export creations, this shape editor represents a significant leap forward in making digital drawing accessible to blind individuals.  Smith-Kettlewell Eye Research Institute is Listening to the Future of Navigation – Meet Engineer, Brandon Biggs (blindabilities.com)

I will present a system based on an iPhone app developed in the Coughlan Lab. This system is a novel “Point-and-Tap” interface that enables people who are blind or visually impaired (BVI) to easily acquire multiple levels of information about tactile graphics and 3D models. The interface uses an iPhone’s depth and color cameras to track the user’s hands while they interact with a model. To get basic information about a feature of interest on the model read aloud, the user points to the feature with their index finger. For additional information, the user lifts their index finger and taps the feature again. This process can be repeated multiple times to access additional levels of information. No audio labels are triggered unless the user makes a pointing gesture, which allows the user to explore the model freely with one or both hands. In addition, multiple taps can be issued in rapid succession to skip through to the desired information (an utterance in progress is halted whenever the fingertip is lifted off the feature), which is much faster than having to listen to all levels of information being played aloud in succession to reach the desired level. Experiments with BVI participants demonstrate that the approach is practical, easy to learn, and effective.

Movement is ubiquitous in everyday life, and accounting for our physical position as we move through the world is a constant process. Over the lifespan, experience in estimating one’s position accumulates, and the nervous system’s representation of this prior experience is thought to inform current perception of spatial orientation. Broadly, spatial orientation perception is a multimodal sensory process. The nervous system rapidly monitors, interprets, and integrates sensory information from various sources in an efficient, statistically optimal manner to estimate an organism’s position in its environment. In humans, key information in this process comes from the visual and vestibular systems, which use head-based sense organs. While statistics of natural visual and vestibular stimuli have been characterized, unconstrained head movement and position, which may drive correlated dynamics across these head-based senses in the real world, have not. Furthermore, head-based sensory cues essential to human spatial orientation perception, like estimation of one’s head orientation relative to gravity and heading (the direction of linear-velocity in a head-based coordinate system), have not been robustly measured in unconstrained, natural behaviors. Measurement of these head-based sensory cues in naturalistic settings, even if incomplete, will likely comprise a portion of the behaviors that make up ones’ total prior experience, and the quantitative characteristics of these behaviors may explain previously observed patterns of bias in verticality and heading perception.  In this brown bag, I will discuss methods of motion tracking in and out of the lab, my previous work to characterize natural statistics of head orientation and heading over 50 hours of human activity using these methods, work to use these natural statistics to constrain Bayesian models of sensory processing, and future research and applications that might leverage these data and approaches. Christian Sinnott | Smith-Kettlewell (ski.org) 

Unilateral amblyopia, a leading cause of childhood vision loss, demands early intervention for optimal treatment outcomes. Unfortunately, accurate screening tools specifically designed for preschoolers remain elusive. This talk explores the potential of binocular rivalry dynamics as a novel biomarker for amblyopia detection, bridging the gap between a promising pilot study and a future R21 proposal. Pilot Study: Our initial investigation revealed distinct binocular rivalry patterns in amblyopic adults compared to controls. Amblyopic individuals exhibited fewer reversals between rivalrous images presented to each eye, with a higher prevalence of incomplete reversals suggesting disrupted binocular processing. These findings highlight the potential of rivalry dynamics as a diagnostic tool. R21 Proposal: Building upon these initial findings, our R21 proposal aims to: Validate and refine rivalry-based diagnostics: We will expand the study population to include more amblyopia subtypes and controls, exploring both natural viewing conditions and balanced interocular contrast presentations. Additionally, we will optimize the minimum data collection duration for accurate classification, aiming for a sub-5-minute test.Develop an objective eye-movement measure: We will introduce optokinetic nystagmus (OKN) as an objective indicator of rivalry states, potentially eliminating the need for subjective reports, a challenge in preschoolers. By investigating signatures of abnormal rivalry dynamics through both subjective perception and objective OKN measures, we seek to establish a reliable, non-invasive method for amblyopia detection.This combined approach holds significant promise for developing a rapid, objective vision screening tool specifically designed for preschoolers. Early detection of amblyopia may pave the way for timely intervention, ultimately improving visual outcomes and enhancing children’s quality of life. https://yua.jul.mybluehost.me/directory/adrien-chopin

Independent wayfinding is a major challenge for blind and visually impaired (BVI) travelers. I describe recent work on improvements to real-time object recognition algorithms that will be used to support an accessible navigation app for BVI travelers in indoor environments. Recognition of key visual landmarks (such as Exit signs and artwork) in an environment provides information about the user’s current location, and improvements to the recognition algorithms will enhance the localization process used in the navigation app.