Scientific

Scientific Program Coordinator Charity Pitcher-Cooper

YouDescribeX the Human-in-the-loop AI interface

YouDescribeX (the X is for eXtra eXperimental). In this revised audio description tool for YouTube viewers can request AI described videos in addition to putting videos on the wishlist, the wishlist has been revitalized propting describers to make AD for those videos first, describers will have the option of using the freestyle version (watch the video, make a script, and record their own voice) or to have an AI supported version that auto captures all the text on screen, automatically chooses description track insertion sites by finding the gaps in the dialog, suggests possible audio description copy for correction and improvement by the describer, and then a synthetic voice reads the descriptions. Describers can choose to put their AD into a community editing process where their script and track placement is improved by other describes.  I cannot wait to show you the good (wow is the text on screen capture incredibly good), the bad (the text on screen capture is so good it sometimes will pick up text on background things- like trashcans) and the ugly (descriptions for still images is still relatively poor, dynamic images are even more difficult) but our describers can correct the AI, and as the tool grows, their suggestions will lead to better audio description. If you have an interest in the YouDescribe classic data (10 years of audio description data)! It can be found here: https://github.com/youdescribe-sfsu/You-Described-We-Archived GitHub – youdescribe-sfsu/You-Described-We-Archived: This is the public repository to download from the You Described, We Archived dataset. This is the public repository to download from the You Described, We Archived dataset. – GitHub – youdescribe-sfsu/You-Described-We-Archived: This is the public repository to download from the You…github.com. In the future we will have three sets of data: auto described by AI, human edited from AI, and freestyle audio description.

Dr. Jenny Read, Vision Research

Stereoscopic vision in the praying mantis

Stereopsis – deriving information about distance by comparing views from two eyes – is widespread in vertebrates but so far known in only class of invertebrates, the praying mantids. Understanding stereopsis which has evolved independently in such a different nervous system promises to shed light on the constraints governing any stereo system. Behavioral experiments indicate that insect stereopsis is functionally very different from that studied in vertebrates. Vertebrate stereopsis depends on matching up the pattern of contrast in the two eyes; it works in static scenes, and may have evolved in order to break camouflage rather than to detect distances. Insect stereopsis matches up regions of the image where the luminance is changing; it is insensitive to the detailed pattern of contrast and operates to detect the distance to a moving target. Work from my lab has revealed a network of neurons within the mantis brain which are tuned to binocular disparity, including some that project to early visual areas. This is in contrast to previous theories which postulated that disparity was computed only at a single, late stage, where visual information is passed down to motor neurons. Thus, despite their very different properties, the underlying neural mechanisms supporting vertebrate and insect stereopsis may be computationally more similar than has been assumed.

Yingzi Xiong, Assistant Professor of Ophthalmology, John Hopkins Medicine

Dual Sensory Impairment: Spatial Localization with Combined Vision and Hearing impairment

Dual sensory impairment (DSI) refers to combined vision and hearing impairment. DSI concerns a large population and its prevalence increases drastically with age. In the US, it is estimated that 40% of patients seeking vision rehabilitation also have hearing loss. The majority of people with DSI still have functional vision and hearing and would therefore benefit from rehabilitation to maximize the use of their residual senses. However, vision and hearing rehabilitation traditionally have been two separate systems that are not readily addressing the unique challenges faced by people with DSI. The long-term goal of our research on DSI is to understand the interaction between impaired vision and impaired hearing in important everyday activities, develop novel assessment instruments to assess vision and hearing functions in a unified framework, and establish new rehabilitation strategies to maximize the use of residual vision and hearing in this population.   The specific topic I will present focuses on spatial localization, which refers to the ability to determine the direction and distance of people and objects around us. Spatial localization is critical for safe navigation and effective social interaction. For people with vision impairment, the challenges in spatial localization are often addressed by Orientation and Mobility training and assistive devices, both of which emphasize the use of hearing. For people with DSI, it is critical that we understand the interaction between residual vision and hearing before making rehab recommendations. Using a combination of screening tools, laboratory spatial localization tasks, and questionnaires, we asked how impaired vision and impaired hearing affect an individuals’ spatial localization ability, both independently and in combination. I will comment on the implications of our findings for DSI rehabilitation, and also share the challenges in conducting DSI research and some future directions. https://www.hopkinsmedicine.org/profiles/details/yingzi-xiong

Marcello Maniglia, PhD University of California Riverside, US

Visual perception, eye movements and visual field awareness after central vision loss: Evidence from patients with Macular degeneration, simulated scotoma, and visual training

Macular Degeneration (MD) represents the leading cause of visual impairment in the Western World. Late-stage MD eventually leads to the development of a central region of blindness (scotoma), which compromises their ability to perform everyday life activities like reading and recognizing faces. Patients spontaneously cope with this challenge by adopting compensatory oculomotor strategies, including the development of a new fixation region in their healthy peripheral retina, called Preferred Retinal Locus (PRL), that they use as a functional replacement of the fovea. Given the role of the fovea as a perceptual and oculomotor reference in the healthy visual system, consequences of central vision loss are far-reaching. Moreover, visual deficits in MD can be extensive yet subjectively subtle, often manifesting as distortions of the visual field or being masked by perceptual filling-in mechanisms. Patients are often unaware of characteristics of their visual deficits. Studies with simulated central vision loss in healthy participants suggest that increased awareness of these deficits could be beneficial for visual rehabilitation. Hence, it becomes apparent that perception, eye movements and visual field awareness are all altered by loss of central vision. Therefore, characterization of MD and designing of rehabilitative interventions should address the multifaceted nature of this condition. Here I will present a series of studies investigating perceptual, oculomotor and attentional characteristics of central vision loss, at baseline and after different training interventions. These interventions aimed at improving visual abilities, eye movements and scotoma awareness, in both patients with MD and healthy participants tested with gaze-contingent, simulated scotoma. Taken together, results suggest that a multidimensional approach to low vision might hold the keys for a better understanding of the effects of central vision loss on perceptual, attentional and oculomotor systems. Furthermore, such approaches might help develop successful interventions for patients with MD. 

Fatema Ghasi is an Associate Professor at Cole Eye Institute Cleveland Clinic

Fixational Eye Movements in Strabismus and Amblyopia: Implications for visual function deficits and treatment outcomes

We have developed robust and objective measures that do not depend upon the young patient’s cooperation or provider’s assessments of visual acuity and strabismus angle and quantifies the entire spectrum of visual function deficits in a fast, reliable, and pediatric-friendly way. The systematic analysis of fixation eye movement traces obtained in the lab in patients with binocular vision disorders has revealed several features that can be utilized to detect the presence and severity of amblyopia and angle and control of strabismus. We have found that fixation eye movement abnormalities correlate with reduced light sensitivities and depth perception and extent of suppression of vision experienced by these patients. We have also found that assessment of fixation eye movement characteristics can be a useful tool to predict functional improvement post amblyopia and strabismus repair. https://fescenter.org/team/investigators/ghasia-fatema-md/

ANTHONY NORCIA, PROFESSOR (RESEARCH) OF PSYCHOLOGY, STANFORD

Identifying neural sub-systems underlying human stereopsis

The psychophysical and oculomotor literatures have long proposed that stereoscopic vision is served by multiple underlying subsystems. The subsystems are usually construed in terms of dichotomies, e.g  patent vs qualitative, local vs global,  first-order vs second-order, transient vs sustained, absolute vs relative, etc.  Whether these multiple sub-systems are all independent is a matter of debate and moreover, there is very little known about their neural basis.  Here I will describe recent work in which we attempt to unify two of the more prominent dichotomies — one spatial — absolute vs relative disparity and the other temporal — transient vs sustained.  In a series of high-density EEG experiments using dynamic random dot stereograms that alternate between a flat plane and a stereo grating at 2 Hz, we find that odd harmonic components of the response reflect relative disparity extraction and that even harmonic components reflect absolute disparity extraction.   Moreover, the odd harmonics reflect a sustained temporal mechanism, while the even harmonics reflect a transient mechanism. Taken together our results indicate that relative disparities are processed by sustained mechanisms while absolute disparity processing is transient in nature. https://profiles.stanford.edu/anthony-norcia

Professor Janine D. Mendola, Ph.D., Visual Neuroscience Lab

Neuroimaging Studies of Binocular Rivalry: How the brain changes its mind

I will discuss human fMRI and MEG studies of binocular rivalry, including some recent work on individual differences. Alternations between bistable percepts provide a powerful platform to study visual awareness and suppression. The frequency tagging technique is used to record whole brain SSVEPs with analyses of power, coherence, and connectivity.  In untagged data, we explore the role of alpha band signals in perceptual stabilization. One theme is comparison of well matched binocular states, such as different types of rivalry. Another theme is the non-invasive measurement of interocular inhibition, which may be a partially heritable trait. There is also the possibility to discuss potential relationships to binocular vision disorders such as amblyopia.  http://mvr.mcgill.ca/Janine/research.html

Dr. August Colenbrander

My Perspective on Vision and Vision Rehabilitation

“Vision is the most important sense for guiding our interaction with the environment.” In my career, I have been involved in clinical ophthalmology, in visual science and in vision rehabilitation. Those domains all involve the eyes, but there are significant differences in how they approach vision. This talk is based on a presentation I gave in June, when receiving a Lifelong Achievement Award from the International Society for Low Vision Research and Rehabilitation. In it, I want to share some of the insights I have gained over more than forty years in vision rehabilitation. I want to particularly stress aspects that may be overlooked.

Senior Scientist Preeti Verghese

Residual stereopsis in anisometropia and small-angle strabismus

We have developed a custom test to measure stereopsis at locations across the visual field to yield a  “stereo map”— visual field locations that are capable of stereopsis. Our study shows that for individuals with anisometropia and small-angle strabismus, residual stereopsis is typically present in the periphery, but is absent in the fovea. This finding is consistent with our hypothesis that residual stereopsis in amblyopia and strabismus is mediated by peripheral loci with larger receptive fields at farther eccentricities that are less sensitive to blur and to small misalignments. As disparity drives fusional vergence responses, we hypothesized that if stereopsis in the central foveal region is impaired, then the fusional vergence response to a small stimulus confined to the near foveal region will be reduced. Our measurements indicate that individuals with anisometropia and micro-strabismus have low vergence gain for small stimuli confined to the foveal region, but near normal gain for larger stimuli that extend into the periphery. Taken together, our stereo mapping and vergence studies indicate that stereo-deficiency in the central retina is associated with poor fusional drive.

Brain-Based Visual Dysfunction (Cerebral Visual Impairment)

CVI is now the commonest cause of bilateral visual functional loss in children with a visual impairment and the prevalence is increasing worldwide. CVI is a brain-based condition resulting usually from adverse events at or around birth and sometimes because of late-acquired cerebral injury. It manifests as a spectrum of higher visual function deficits (HVFDs) affecting visual function in everyday life in the absence of visible eye pathology and often in the presence of normal visual acuity, which usually negates further assessment and investigations. HVFDs manifest as difficulties with visually guided behavior, such navigation, often within a cluttered and crowded environment; route finding; inaccurate reaching for objects and perceptual difficulties, such as relative motion perception, recognizing faces, and reading difficulties. Underlying these challenges is an altered cerebral visual processing network and oculomotor anomalies, yet to be understood fully. A lack of specific assessment tools has made it difficult to map the spectrum and severity of this complex, multi-dimensional condition. Consequently, designing (re)habilitative measures has been challenging. We will describe our collaborative CVI research programme. The development of the HVFQI App – an interactive web-based Higher Visual Function assessment tool and its application to our research study. We will present our study, data, analysis, and results. We will discuss the relevance of our work for CVI and the future direction of this research programme. We hope to engender discussion and feedback on our ongoing work.