Use cases
The XR-SENSE technological framework will be validated across three high-impact, demanding pilot domains: Connected and Autonomous Vehicles, Remote Collaborative Surgery, and Bidirectional XR Broadcasting. These pilots serve as real-world testbeds to demonstrate how the project’s unified visuo-haptic medium, real-time edge processing, and human-centric AI streaming handle extreme network latencies, complex data coregistration, and critical safety requirements. By deploying these diverse use cases, the project aims to transition XR from traditional, passive audio-visual experiences into deeply immersive, socially inclusive, and highly interactive bi-directional ecosystems.
Connected & Autonomous Vehicles
This pilot focuses on enhancing vehicular safety and remote social telepresence in complex transit environments by integrating real-time infrastructure data directly into the driver’s field of view. It utilizes low-latency edge computing and wearable haptic feedback to lower cognitive load and prevent spatial disconnect.
Occlusion-Aware Overlays: Processes real-time point clouds and radiance fields from vehicle sensors to generate “see-through” safety displays that reveal hidden road hazards.
Remote XR Co-Pilots: Enables a distant user to enter the vehicle’s spatial environment as a high-fidelity avatar for real-time collaborative navigation.
Tactile Situational Awareness: Deploys haptic steering wheel vibrations and skin-stretch cues to deliver intuitive safety alerts without visually distracting the driver.
Remote Collaborative Surgery
Operating in a high-stakes medical environment, this pilot tests the limits of real-time visuo-haptic synchronization during complex medical procedures. It allows remote experts to mentor local practitioners in a shared virtual space while monitoring the surgical team’s physiological stress levels.
Deformable 3D Tomographies: Projects and co-registers real-time CT or MRI overlays directly onto the patient’s physical anatomy.
High-Fidelity Haptic Feedback: Employs specialized wearable thimbles and armbands to simulate physical sensations like tissue resistance and material stiffness.
Affective XR Spaces: Translates qualitative human factors into machine-readable priors, while providing isolated patients with tactile, emotionally reassuring connections to remote family members.
Bidirectional XR Broadcasting
This pilot redefines traditional mass media by transforming passive one-way streaming into an interactive, multi-user, and affect-aware virtual studio. Integrating Brainstorm’s EdisonPro engine, it enables presenters and remote audiences to co-exist and interact as dynamic 3D assets.
Serverless Spatial Synchronization: Uses Atomic Distributed Shared Memory (ADSM) to seamlessly sync the spatial positions and interactions of numerous concurrent participants.
AI-Driven Streaming Optimization: Features an adaptive AI agent that uses perceptual and spatial saliency priors to dynamically prioritize bandwidth for critical visuo-haptic data.
Multisensory Audience Engagement: Facilitates real-time bidirectional interactions, allowing remote viewers to handle virtual props or participate in shared tactile feedback.