Unity – Having proved popular with global automakers, the Unity game engine is now being tuned to appeal to the wider manufacturing industry
Always on the front foot, the automotive industry was one of the first to embrace game engine technology. From collaborative design review and immersive training to product marketing and point of sale configurators, applications built upon game engines, also known as 3D engines or 3D development platforms, are prevalent among the major automakers.
Showcasing Unity’s forthcoming real-time ray tracing capability on the 2019 BMW 8 Series Coupe
Unity and Unreal Engine by Epic Games are the two major forces. Unity’s automaker customer list is impressive, including Volkswagen, Audi, BMW, Volvo, Lexus, Lincoln, Toyota and others. The main attraction of the engine is the ability to create very high-quality visualisations of complex models in real-time.
Designers can make changes in CAD, bring the model into Unity, render it instantly with realistic materials, and then explore it interactively and immersively on many different devices. When using Unity’s High-Definition Render Pipeline (HDRP) the quality is so good that creative agency Team One is now using the engine as a virtual studio to create photorealistic marketing assets for Lexus.
Cars that have not yet hit the production line can be placed in any global location. No more shipping top-secret vehicles around the world or waiting patiently for the perfect light of the golden hour. The creative process has been transformed.
Unity’s presence in the automotive sector kind of happened by accident. The company has traditionally been laser focused on the 3D games market. But a few years ago, a review of the customer base found huge global firms were using the engine – not just in automotive, but aerospace and AEC (architecture, engineering and construction) firms too, including Boeing, Raytheon, Airbus, AECOM, Atkins and Mortensen.
To cater to the needs of these customers, Unity set up a dedicated engineering business unit. The team continues to grow significantly. There are now 200 employees, with more recruits starting every week.
Tim McDonough, GM Engineering, Unity, admits that while this is not a huge number, the business unit is able to punch above its weight because it can harness the advanced technology in the multi-purpose engine. “We can go back to our core organisation and say ‘hey, we want to do real-time ray racing, or we want to do this’ and we get the benefit of all their R&D, plus the ability to get help with features that are just massive.”
It’s only in this last year that the company has really started to look at other areas of manufacturing, beyond automotive, as Unity senior product manager Ed Martin explained at the recent Unity Unite event in Copenhagen.
“We’ve expanded our focus to include the entire range of manufactured products, including trains, planes, medical equipment, industrial equipment, durable consumer goods – things like bicycles or washing machines – so essentially any manufactured product.”
This isn’t just about delivering high quality, interactive experiences on multiple devices, including desktop, web and VR/AR. Unity is now being used to support a wider range of manufacturing workflows, including PLM integration, simulation and digital twins.
Unity Pro
Hobbyists and small firms can use Unity for free, but most commercial customers — game developers, manufacturing firms, architectural practices, etc. — will need Unity Pro, which costs $125 per month.
Unity Pro is not a product, per se, rather a toolkit on which bespoke applications can be built. In order to get the most out of the engine, manufacturing firms will need C# programmers, and often look to the game industry to recruit talent.
Unlike a few years ago, where manufacturing firms were pretty much left to their own devices, Unity is now supporting its manufacturing customers by offering a layer of industry focused products and services.
There’s a new Unity Enterprise for Product Lifecycle bundle, which features several manufacturing-specific tools.
In addition to Unity Pro, it includes Unity measured materials (containing 300 realistic materials), PiXYZ software for data import, and toolkits that come with ‘ready-to-use building blocks’ to help firms extend the core functionality of the engine.
Currently there are toolkits for designing applications for Human Machine Interfaces (HMI) and Immersive Collaboration (see\ box out below).
Unity plans to add more toolkits in the future, including a Product Configurator to help firms iterate on different design options or to let customers explore the full product line. Training is also becoming a big focus, with new packages targeted at smaller firms.
Data import
One of the biggest challenges for any real-time viz workflow is how to bring in CAD data quickly and easily, while at the same time maintaining its engineering accuracy and ensuring it performs well in 3D.
To help optimise this process, Unity has a close relationship with French software developer PiXYZ. PiXYZ offers a number of tools that can quickly, and sometimes automatically, turn heavy CAD data into optimised meshes, complete with meta data.
“Customers are frequently amazed at how much time it [PiXYZ] saves them,” says Martin.
“I’ve literally had a customer tell me that they took a process that used to take them a few weeks and they took it down to a few minutes, just to be able to produce visualisation data for a fairly complex industrial product.”
PiXYZ’s software offers granular control when bringing in data. It includes the CAD model structure and the ability to reduce polygon counts for different LoDs (Levels of Detail). Python scripts can also be written to create ‘recipes’ that define how to treat certain elements within the dataset.
PiXYZ is investing in machine learning to make its algorithms more intelligent and autonomous, so in the future it could automatically recognise shapes that would be best treated with a specific optimisation.
Unity also has a close relationship with Autodesk, following an extended collaboration announced last year. The two companies are working on an optimised workflow from the automotive-focused viz tool Autodesk VRED, that supports the conversion of geometry, UVs, animations, and materials. 3ds Max is getting a similar treatment.
In the future, we wonder if this might extend to Autodesk Inventor or Fusion 360 to help address the wider manufacturing community. Of course, PiXYZ can already handle many different CAD formats.
Unity has gone one step further for the AEC sector, developing an application called Unity Reflect. It allows firms to quickly and easily bring in CAD or Building Information Modelling (BIM) data and then push it out to multiple devices.
Importantly, Unity Reflect features a live link to Autodesk Revit and other AEC-focused applications, so any changes made to the CAD/BIM model automatically update in Unity Reflect seconds later.
Unlike Unity Pro, which is a development platform and requires programming knowledge, Unity Reflect can be used out of the box. This opens up the game engine to a much wider audience.
While Reflect is heavily focused on AEC, according to Martin it is also relevant to manufacturing, “One common thing about all manufacturing customers is they manufacture things and they manufacture things in factories, and those factories are constantly being modified. I like to refer to a factory as a building that is constantly under construction.”
Looking to the future, we asked McDonough if Unity Reflect could evolve to become a general-purpose tool that would support manufacturing as well as AEC.
“I think it could,” he replied. “I think if you abstract the functionality, it’s an export pipeline, maintaining fidelity to data. It has the ability to synchronise models and let people see them even if they’re on a mobile device,” adding that Unity Reflect is open sourced so apps can also be built on top of it.
“We’ll produce more apps of our own,” he continues. “But I think you’ll see some firms build their own apps for competitive advantage. And you’ll see probably an ISV [Independent Software Vendor] community start building on top of Reflect too.” This is something that is already starting to happen in the AEC sector with SHoP Architects and ISVs including Xinaps (you can read more about Unity Reflect in DEVELOP3D’s sister publication, AEC Magazine).
Lightweight Unity CAD viewer showing model structure and metadata from the original JT data
‘An interactive VR training application, ‘built in 15 minutes using Interact’
Plugging into PLM
Working with CAD files directly is important, but to help Unity become part of an optimised, managed workflow, and to further extend the reach of engineering data, the obvious next step is to plug it into a PLM system.
Unity has worked with Atos and PiXYZ to develop a PLM system integration that can be tailored to individual implementations of Siemens Teamcenter.
Using the Teamcenter connector developed by Atos, firms can ‘quickly and easily’ extract 3D geometry and metadata to create real-time experiences using Unity. This could be for marketing, training, client presentations or for developing VR/AR applications. CAD data or JT files can be pulled in from Unity or pushed out from Teamcenter, automatically processed through PiXYZ Studio.Batch.
Once the data is inside Unity, the system can automatically create a lightweight CAD viewer for multiple platforms, including iOS, Android, Windows, Web and more. Taking this to the next level, Light & Shadows’ Interact tool could be used to create a VR training experience that walks through the steps required to assemble or service a product. Or it could form the basis for immersive, collaborative design / review. Importantly, every asset and revision that is created can be tracked.
Unity – simulating reality
Unity Simulation is a new cloud-based tool designed to test out scenarios, by running thousands or millions of simulations in parallel. It can be used to verify 3D games but there are practical real-world applications where simulations are used for machine learning.
These include training and validating autonomous vehicles. Unity says that in order to be considered safe autonomous vehicles need to ‘drive’ over 10 billion miles, which could take a hundred years on the open road.
Virtual simulation not only rapidly accelerates this process but allows for dangerous scenarios to also be tested – for example, seeing how the car reacts when a child runs out into the street.
Unity Simulation also has applications for robotics, where firms can train and validate robots before building physical products. Most of the projects in this space are shrouded in secrecy but McDonough shares one example of a delivery robot project from a couple of years ago. “It was able to drive on the sidewalks, but it was too timid to cross the street and it kept getting run over,” he smiles. “So we taught it to cross the street with reinforcement learning and it was kind of goofy to watch the progression of it getting killed constantly and then it got good at crossing the street.
“So, we’ve done delivery robot work; household robotics stuff that I can’t talk about is being done. There’s a lot with arms that are picking up soft body things like, I think, a pair of socks, and the physics required to do that is totally different than picking up something rigid.”
Unity Simulation can train robots how to pick up different types of objects
Digital twins
Virtual prototyping and simulation can lay the foundations for digital twins. PREspective is a Unity-based tool from Unit040 that is designed to create and run an identical visual copy of a system in the virtual world, before the real system exists and after the system is operational.
The Netherlands-based firm has a strategic partnership with Unity. In fact, Unity sells the company’s software.
PREspective is used to simulate complex systems, ranging from machines to entire production facilities. Data can be brought in from a variety of applications, including CAD and physics for collision and gravity. The simulation can also be connected to IoT data sources or streams, perhaps for predictive maintenance or for training.
‘Digital Twin’ software PREspective is used to simulate complex systems, ranging from machines to entire production facilities
The thing that stands out about PREspective is that it can run the same PLC control software which will drive the system during operation. It means the digital twin should behave in exactly the same way as it would in the real world, so everything can be properly tested and validated prior to deployment.
“You can run through how it operates; you can put a human in the loop and understand human machine interaction with actual control systems for machines that have not been built yet,” says Martin.
PREspective is being used by Toyota to create virtual versions of its airport security systems, complete with conveyors and sensors for the trays and avatars to simulate how passengers might interact with the system. Engineers can even go into VR to test it out, as Unit040 CTO Guido van Gageldonk explains, “So, you, yourself can throw away a tray, or you can turn off the machine and look what will happen or try to totally mess with the entire system. For example, trying to fool some sensors by putting your hand in front of it.”
It’s impressive stuff and we encourage you to check out the video above.
Test drives to blow your mind
‘Testing new HMI and UX designs at Volvo in Mixed Reality
Volvo is doing some crazy things with Unity, developing applications that allow the Swedish automaker to test new designs by blending the real and virtual worlds in a completely new way.
It uses Varjo’s mixed reality XR-1 headset, which delivers incredible detail so new HMI and UX designs can be overlaid on the dashboard of Volvo’s existing car – which is then physically driven on a real test track.
The car designers can then add virtual automobiles or unexpected road hazards (such as a virtual moose) to see how the car and the drivers react.
The big benefit for Volvo is it can can see how drivers use new functionality or whether they are distracted in any way – and all without having to build costly physical prototypes. We encourage you to check out the presentation below from Volvo’s Timmy Ghiurau at Unity Unite:
Unity – core developments
Unity is working on several new developments that should push the core engine forward in the coming year. The most notable is real time ray tracing, powered by Nvidia RTX technology, which will bring true photorealistic qualities to Unity’s High-Definition Render Pipeline (HDRP).
At Nvidia’s GTC event in April this year, Unity gave its first public demonstration of real-time ray tracing, showing work it had done with BMW, Light and Shadows and Nvidia. It’s currently an ‘experimental technology’ but will soon be available as a preview.
Unity is also working with Google, using its Kubernetes engine to stream high quality rendered models to any device. The idea here is that all the graphics processing is done in the cloud, so you can interact with very high-quality real time models on iPad, PC, laptop or even a mobile phone.
Unity says multiple sessions will be enabled from a single GPU, making it ‘cost effective’ and well suited to product configurators.
Conclusion
Over the last few years Unity (and indeed, Epic Games, the developer of Unreal Engine) has really woken up to the huge potential of 3D game engines in the manufacturing space. While the automotive and aerospace firms took the initiative several years ago, Unity is now actively targeting the wider manufacturing industry through its new engineering business unit.
The challenge for the company will be making its engine more accessible, especially to manufacturing firms that don’t have coding expertise in-house.
Certainly, firms can recruit programming talent from the games sector, and Unity is also doing a lot of the heavy lifting by developing toolkits on which to build workflow-driven applications. However, for the engine to really appeal to the wider manufacturing community, particularly when it comes to smaller firms, there’ll need to be off the shelf applications.
In this respect, it will be interesting to see if Unity develops a manufacturing-focused version of the AEC-centric Unity Reflect, which does the bread and butter work of data prep and conversion, pushing out large models to multiple devices quickly and easily.
Then of course, there are third-party developers like Unit040 and Light & Shadows which are delivering powerful industry-focused solutions based on Unity. We expect to see more of these in the future.
It’s still very early days for the engine but the potential is huge. With broad applications for design, manufacturing, simulation, training, sales and marketing, we’re excited to see where things will go.
Laying the foundations for design review in VR
Unity’s Immersive Collaboration toolkit provides a foundation on which to build custom collaborative design / review applications in VR. It means firms don’t need to reinvent the wheel, getting access to core tools including teleportation, measurement, cross sections, annotation (freehand and text), collision and scenario switching.
There’s also built in support for avatars and multiplayer with voice, so teams in different locations, even different countries, can collaborate on the same model.
According to Kieran Colenutt, art evangelist, Unity, building a base application requires minimal C# coding,
“Everything can be achieved through an interaction menu, or by a simple drag and drop functionality in order to create simple interaction.”
Of course, this is really just a starting point for manufacturing firms. As Unity is fully programmable, pretty much anything can be developed on top of a design / review application that has been built with the help of the Immersive Collaboration toolkit, from custom functionality all the way down to aesthetics.
