NVIDIA announces public beta of NVIDIA XR AI, a developer library for building multimodal AI agents for AR glasses and XR devices already in use by Siemens
AI agents need to be able to work alongside people, helping them understand their environment, build knowledge and take action in real time, but building agentic systems can be challenging.
NVIDIA XR AI is a developer library that helps developers build agentic applications by connecting the input from AR glasses and XR devices with AI models, enterprise data, tools and accelerated computing; agents can perceive, reason and act within the workflow.
The developer library provides a foundation for developers to build or connect skills and tools for enterprise XR applications, simplifying the integration of multimodal perception, enterprise retrieval, reasoning models and agent orchestration.
These capabilities will make it easier to build spatially aware, multimodal AI agents that deliver low-latency, context-aware assistance in AR and XR experiences.
Siemens is exploring, in a research context, how NVIDIA XR AI and DGX Spark can help factory engineers find maintenance information, troubleshoot issues, verify work, and capture what happened on the shop floor.
An engineer wearing lightweight glasses can ask an AI agent about a programmable logic controller issue and receive real-time guidance, connecting industrial systems, digital twins and automation workflows.
The platform brings together core capabilities. It ingests real-world signals from AR and XR devices, including video, audio, depth, pose and sensor data.
The platform connects agents to specialised tools and services, including NVIDIA Metropolis and the NVIDIA Metropolis for video search and summarisation (VSS) for visual AI and video understanding, and NVIDIA NeMo Retriever for enterprise knowledge retrieval and retrieval-augmented generation.
It also supports a broad ecosystem of AI models, including NVIDIA Nemotron reasoning models, NVIDIA Cosmos Reason and other compatible foundation models and integrates agent orchestration and accelerated runtime services to help developers move from prototype to production.
Rana, an AutoBio company building AI systems for scientific research, is introducing its LabOS system on NVIDIA XR AI to bring spatial intelligence directly into scientific workflows, which is compatible with smart glasses from Meta, Rokid and VITURE.
The LabOS co-scientist provides real-time, hands-free guidance for complex experimental workflows, starting with stem cell therapy and gene-editing research.
AI agents, delivered through AR glasses and powered by NVIDIA GPUs, serve as a next-generation interface for AI-assisted science. This keeps researchers focused on complex procedures while receiving contextual guidance in real time.
