Vrgineers and ManoMotion have announced they are introducing an AI hand-tracking algorithm, which was developed to enhance the immersive experience of pilot training in virtual and mixed reality environments.
This innovation, showcased at the AWE USA 2023 event, is exclusive to XTAL Mixed Reality headsets and is optimised for cockpit settings.
The unique aspect of the solution is its integration of pass-through RGB cameras and an advanced AI hand-tracking algorithm. By combining these elements, the hand visualisation algorithm should effectively remove the background and focus on displaying the hands within virtual environments.
ManoMotion utilises the RGB camera stream to visualise the hands in both VR and MR simulations, which are specifically tailored to Vrgineers’ pilot cockpit tracking requirements.
The two companies said that the goal of the mixed reality cockpit hand tracking project is to create an AI-driven hand visualisation functionality for pilot cockpit training.
By leveraging the real-time RGB camera stream, the system detects and isolates the pilot’s hands, overlaying them onto the virtual reality scene seamlessly.
Vrgineers and ManoMotion said they collaborated on gathering training data through video feeds capturing users interacting with aircraft cockpit instrument panels.
They said the he real-time data is then processed using a pre-trained hand-tracking model to separate the hands from the live video stream and render them visually.
The integration of AI hand-tracking technology aims to offers an immersive experience with enhanced realism and interactivity, bridging the gap between virtual simulations and real-world applications.
“As a virtual and mixed reality simulation specialist, we always try to find new innovative ways to increase the immersion of our solutions,” said Vrgineers CEO Marek Polcak.
“Vrgineers was the first company to fully integrate infrared hand-tracking into a virtual reality headset. And now, together with ManoMotion AI recognition using visible light, we are doing the same for the mixed reality.
“By supporting both technologies, we can combine their strengths to deliver the very best existing hand-tracking available in any headset on the market.”
