SocialCueSwitch: Towards Customizable Accessibility by Representing Social Cues in Multiple Senses

Abstract

In virtual environments, many social cues (e.g. gestures, eye contact, and proximity) are currently conveyed visually or auditorily. Indicating social cues in other modalities, such as haptic cues to complement visual or audio signals, will help to increase VR’s accessibility and take advantage of the platform’s inherent flexibility. However, accessibility implementations in social VR are often siloed by single sensory modalities. To broaden the accessibility of social virtual reality beyond replacing one sensory modality with another, we identified a subset of social cues and built tools to enhance them allowing users to switch between modalities to choose how these cues are represented. Because consumer VR uses primarily visual and auditory stimuli, we started with social cues that were not accessible for blind and low vision (BLV) and d/Deaf and hard of hearing (DHH) people, and expanded how they could be represented to accommodate a number of needs. We describe how these tools were designed around the principle of social cue switching, and a standard distribution method to amplify reach.

Highlights

• Identifies social VR cues — gesture, eye contact, proximity, who is speaking to whom — that are typically only conveyed visually or aurally.
• Introduces 'social cue switching': letting users choose whether a cue is rendered as visual, audio, or haptic feedback.
• Ships a sample toolkit and a standard distribution approach so accessibility tools built across labs can reach a common audience.

Citation

@inproceedings{segal2024socialcueswitch,
  title={SocialCueSwitch: Towards Customizable Accessibility by Representing Social Cues in Multiple Senses},
  author={Segal, Jonathan Isaac and Rodriguez, Samuel and Raghavan, Akshaya and Baez, Heysil and Jung, Crescentia and Collins, Jazmin and Azenkot, Shiri and Won, Andrea Stevenson},
  booktitle={Extended Abstracts of the CHI Conference on Human Factors in Computing Systems},
  series={CHI EA '24},
  articleno={295},
  pages={1--7},
  year={2024},
  publisher={ACM},
  doi={10.1145/3613905.3651109}
}