Healthcare & teamwork

Code Black: Desktop-Mediated Co-Design of AR-HMD Microinteractions for Emergency Department Teamwork

A speculative, desktop-mediated co-design study with 12 healthcare workers using an editable 3D design probe to shape AR-HMD microinteractions for emergency department teamwork.

Jonathan Segal¹, Jalynn Nicoly², Francisco Ortega², Angelique Taylor¹

¹Cornell University, New York, NY, USA²Colorado State University, Fort Collins, CO, USA

ACM Symposium on Spatial User Interaction (SUI '26)September 3, 2026

Best Paper (Honorable Mention)

DOI arXiv PDF

Role-based interface concepts in the desktop emergency department co-design probe. — Role-based AR interface concepts explored through the desktop design probe. Animation from the Code Black SUI talk.

The research at a glance

How should AR interfaces appear, change, and disappear during emergency department teamwork?

Approach

Twelve healthcare workers explored and edited a desktop-based Unity design probe. Five returning participants and 26 additional healthcare workers provided follow-up feedback on the resulting interface alternatives.

Takeaway

Role visibility, task-linked timers, and dosage verification need to fit clinical workflows. The work contributes a speculative co-design framework and a visual catalog of possible interfaces.

Formative, speculative co-design; this study did not evaluate a functional AR-HMD system or clinical team performance.

Abstract

Emergency Department (ED) teams coordinate shifting roles, medication decisions, and time-critical interventions under uncertainty. Augmented reality head-mounted displays (AR-HMDs) have shown potential to spatially anchor information during care, creating opportunities to examine how spatial interfaces might support teamwork. We conducted a speculative co-design study with 12 healthcare workers (HCWs) using an editable, desktop-mediated Unity-based 3D design probe to visualize and refine work-as-imagined AR-HMD interfaces for role-based notifications, task-specific timers, and dosage verification. Guided by microinteraction rules, participants identified future spatial user interfaces (SUI) requirements such as how they appear, update, or are dismissed in relation to clinical practice, safety concerns, and existing tools. Five returning participants and 26 additional HCWs subsequently provided follow-up feedback on derived visual interface alternatives. Findings show that desktop-mediated spatial co-design elicited formative specifications for role visibility, task-linked timing, and verification-oriented dosage assistance, while revealing tensions involving clutter, shared awareness, communication, privacy, and reliability. Rather than evaluating a functional AR-HMD system or team-based clinical performance, this study contributes the Speculative Co-Design Framework for AR-HMD Teamwork (SCF-HMD) and a visual design catalog for translating expert critique of work-as-imagined (WAI) concepts into situated goals for future AR-HMD systems.

Contributions

A speculative co-design study with 12 healthcare workers using an editable, desktop-mediated Unity-based 3D design probe.
Elicits microinteraction requirements for role-based notifications, task-specific timers, and dosage verification in AR-HMDs.
Contributes the Speculative Co-Design Framework for AR-HMD Teamwork (SCF-HMD) and a visual design catalog for future systems.

co-designAR-HMDsemergency medicinemicrointeractionsspatial UI

Citation

@inproceedings{segal2026codeblack,
  title={Code Black: Desktop-Mediated Co-Design of AR-HMD Microinteractions for Emergency Department Teamwork},
  author={Segal, Jonathan and Nicoly, Jalynn and Ortega, Francisco and Taylor, Angelique},
  booktitle={Proceedings of the ACM Symposium on Spatial User Interaction},
  series={SUI '26},
  year={2026},
  publisher={ACM},
  doi={10.1145/3822518.3830045}
}