Review: Nvidia RTX 6000 Ada Generation

Posted on June 15, 2023 at June 15, 2023 by Greg Corke

1447 0

One can’t deny the breathtaking performance of Nvidia’s new flagship workstation GPU and its potential to completely transform viz workflows, but the eyewatering price will be offputting for some

New GPU architectures are delivered from the top down. And the new Nvidia RTX 6000 Ada Generation is very much at the top of the stack. With a price tag of £7,150 (Ex VAT), this 48 GB professional GPU is reserved for those that take design visualisation, simulation or AI extremely seriously.

Over 40 pages of dedicated workstation reviews, features and coverage. (Click image to read)

The first thing to get out of the way is the name of this new workstation-class GPU. It is built on Nvidia’s Ada Lovelace architecture, named after the English Mathematician credited with being the first computer programmer.

Recently, Nvidia has used a single letter prefix for its pro GPUs — P for Pascal, T for Turing, A for Ampere, and so on.
As ‘A’ was already taken, Nvidia initially referred to the Ada Lovelace GPU as the Nvidia RTX 6000, but soon after tagged ‘Ada Generation’ on to the end, presumably to avoid confusion with 2018’s Turing-based Nvidia Quadro RTX 6000.

We don’t know why Nvidia didn’t use an ‘L’ prefix as it has done for its ‘Ada Lovelace’ datacentre GPUs (the Nvidia L4 and L40), but this is where we are at now. The Nvidia RTX 6000 Ada Generation might be a bit of a mouthful, but at least it has a clear identity.

The workstation card

The Nvidia RTX 6000 Ada Generation is a dual slot, full height, PCIe 4.0 workstation GPU with four DisplayPort 1.4a connectors. It looks virtually identical to its predecessor, the Nvidia RTX A6000, and has a minimal angular black and gold design. The radial type fan blows hot air directly out of the rear of the workstation via the grille on the bracket.

The total board power is 300W, which is less than its triple slot ‘Ada Lovelace’ consumer counterparts — the Nvidia GeForce RTX 4090 and GeForce RTX 4080. Power is delivered via a single PCIe CEM5 16-pin connector.

The card has a phenomenal amount of processors: 142 third-gen RT Cores for ray tracing (delivering 211 TFLOPs), 568 fourth-gen Tensor Cores for AI compute (delivering 1,457 TFLOPs), and 18,176 next-gen CUDA cores for general purpose operations, boasting 91 TFLOPs of single precision performance. This is a significant jump up from the Nvidia RTX A6000 it replaces, which delivers RT Core performance of 76 TFLOPs, Tensor performance of 310 TFLOPs, and single-precision performance of 39 TFLOPS.

The Nvidia RTX 6000 Ada comes with 48 GB of GDDR6 memory, which should be plenty for most viz-centric workflows. However, unlike its predecessor, the RTX 6000 Ada Generation does not support NVLink, so two GPUs cannot be bridged together with an adapter to create a 96 GB memory pool. While this shouldn’t matter to most users, it could be a barrier for those working with exceptionally high poly count models / high resolution textures. It could also limit its use in engineering simulation, including Computational Fluid Dynamics (CFD).

48 GB is still double that on offer in the top-end consumer Nvidia GeForce RTX 40-Series. The RTX 6000 Ada also differentiates itself from Nvidia’s consumer cards in several other ways, including pro drivers, pro software certifications, support for Error Correction code (ECC) memory, and some niche features for pro viz, including stereo and Frame Lock for viz clusters.

It also supports Nvidia virtual GPU (vGPU) software, which allows a workstation to be repurposed into multiple GPU-accelerated virtual workstation instances. With workstation vendors, especially Lenovo and HP, actively making their new ‘Sapphire Rapids’ desktop workstations rack friendly, this feature is likely to be more important than ever before.

Finally, it boasts 3x the video encoding performance of the Nvidia RTX A6000, for streaming multiple simultaneous XR sessions using Nvidia CloudXR.

Optimised for visualisation

The Nvidia RTX 6000 Ada offers all the generational improvements you’d expect from a new GPU architecture, but there are also significant changes in the way the GPU carries out calculations to increase performance in viz-centric workflows.

Deep Learning Super Sampling 3 (DLSS) and Shader Execution Reordering (SER) are the two technologies that stand out.

Nvidia DLSS has been around for several years and with the new Nvidia RTX 6000 Ada, it is now on its third generation. It uses the GPU’s AI Tensor cores to boost performance.

With Nvidia’s previous generation ‘Ampere’ GPUs, DLSS 2 took a low-resolution current frame and the high-resolution previous frame to predict, on a pixel-by-pixel basis, what a high-resolution current frame would look like.

With DLSS 3, the Tensor cores generate entirely new frames rather than just pixels. It processes the new frame, and the prior frame, to discover how the scene is changing, then generates entirely new frames without having to process the graphics pipeline.

So far, we’ve only seen DLSS 3 implemented in Nvidia Omniverse, but we expect others to follow. Enscape and Autodesk VRED, for example, both support DLSS 2.

As a background to Shader Execution Reordering (SER), Nvidia explains that GPUs are most efficient when processing similar work at the same time. However, with ray tracing, rays bounce in different directions and intersect surfaces of various types. This can lead to different threads processing different shaders or accessing memory that is hard to coalesce or cache.

With SER, the Nvidia RTX 6000 Ada Generation can dynamically reorganise its workload, so similar shaders are processed together. According to Nvidia, SER can give a two to three times speed up for ray tracing and a frame rate increase of up to 25%. But these are probably extremes. For offline path tracing in Unreal Engine 5.1, for example, Nvidia quotes speed improvements of 40% or more.

Engineering simulation

In product development, while visualisation is the primary use case for the Nvidia RTX 6000 Ada, the GPU can also be used for engineering simulation.

At launch, Nvidia highlighted the use of Ansys software, including Ansys Discovery and Ansys Fluent for Computational Fluid Dynamics (CFD).

Compared to the RTX A6000, the RTX 6000 Ada not only has more cores and faster cores, but significantly larger L2 cache (96 MB vs 6 MB) and increased memory bandwidth (960 GB/s vs 768 GB/s). According to Ansys, this results in ‘impressive performance gains’ for the broad Ansys application portfolio.

However, the Nvidia RTX 6000 Ada is not suited to all simulation tools. Some simulation solvers require double precision and with relatively poor FP64 performance (which at 1,423 GFLOPSs is 1/64 of its FP32 performance), the RTX 6000 Ada is unlikely to perform that well in those that do. In fact, for double precision solvers, even 2016’s Nvidia Quadro GP100 boasts better FP64 performance of 5.17 TFLOPs.

Testing the Nvidia RTX 6000 Ada Generation

DEVELOP3D put the Nvidia RTX 6000 Ada Generation through a series of real-world application benchmarks, both GPU rendering and real time visualisation.

The GPU is simply overkill for current generation CAD and BIM software, so we didn’t do any testing in that regard. However, it’s important to note that it will still be certified for the likes of Solidworks and Solid Edge, which is useful if you plan to use those types of core applications alongside viz focused tools like KeyShot, V-Ray, and Unreal Engine.

The full spec of our AMD Ryzen Threadripper Pro test machine, the Scan 3XS GWP-ME A1128T can be seen below. Read our full review here.

Scan 3XS GWP-ME A1128T

AMD Ryzen Threadripper Pro 5995WX processor (2.7 GHz, 4.5 GHz boost) (64-cores, 128 threads)
256 GB (8 x 32GB) Samsung ECC Registered DDR4 3200MHz memory
2TB Samsung 990 Pro NVMe PCIe 4.0 SSD
8TB RAID array (4 x 2TB Samsung 990 Pro NVMe PCIe 4.0 SSDs)
Asus Pro WS WRX80E Sage SE WiFi motherboard
Corsair H150i Elite Cappelix RGB with Noctua fans
1,500W Corsair HXi, 80PLUS Platinum
Microsoft Windows 11 Pro

For comparison, we also tested the Nvidia RTX A6000 GPU inside the same machine. Nvidia’s 528.24 pro driver was used for both GPUs.

Real time 3D

Real time 3D visualisation with applications that use OpenGL, DirectX or Vulkan graphics APIs continue to be a very important part of design visualisation. Key applications include Unreal Engine, Autodesk VRED, and the architectural focused TwinMotion and Enscape.

We recorded frame rates (Frame Per Second) within Enscape, Unreal Engine, Nvidia Omniverse, and Autodesk VRED Professional, a pro viz application commonly used in automotive design.

We only tested at 4K (3,840 x 2,160) resolution. At FHD (1,920 x 1,080) , it’s a given that the Nvidia RTX 6000 Ada Generation can deliver more than enough performance.

In Enscape, we tested with five different models. Overall, our experience was incredibly smooth, even with the large RTX-enabled Enscape 3.0 building complex which uses 11 GB of GPU memory.

However, our preferred benchmark model, an urban scene from Enscape 3.1, was a little unresponsive, sometimes taking a few seconds to react to mouse or keyboard movements. We don’t know why this was, but it could be because it includes custom assets and textures and there is a conflict of some sort. Once it got going, however, we recorded a phenomenal 124.95 FPS, 63% faster than the Nvidia RTX A6000.

In Unreal Engine 4.26, the generation-on-generation gains were smaller. The biggest increase came when ray tracing was enabled on our Audi test model, with the RTX 6000 Ada Generation delivering 1.5 times more FPS than the RTX A6000.

In Autodesk VRED Professional 2023, performance increases ranged from 1.63 to 1.89. The biggest came when anti-aliasing was disabled.

In Nvidia Omniverse Create 2022.3.3, we tested with the Brownstone building sample model. In RTX – Real-Time mode with DLSS enabled the RTX 6000 Ada was a whopping 3.62 times faster than the RTX A6000. However, there’s a case of comparing apples with pears here as the RTX 6000 Ada uses DLSS 3 while the RTX A6000 use DLSS 2 (see earlier on). In saying that, we saw no visual difference between the two. In RTX – interactive (path tracing) mode the RTX 6000 Ada was 2.16 times faster.

Ray trace rendering

We tested with a range of GPU-accelerated ray-trace renderers, including V-Ray GPU, KeyShot, Solidworks Visualize, Nvidia Omniverse and blender.

With the V-Ray, KeyShot and blender benchmarks, the RTX 6000 Ada Generation shot ahead, outperforming the RTX A6000 by a factor of 1.93, 2.05 and 2.17 respectively. We saw similar gains in Nvidia Omniverse, with the RTX 6000 Ada taking less than half the time of the RTX A6000 to render the brownstone building scene.

However, in some of our real-world application tests, the gains were nowhere near as large. In Solidworks Visualize 2023, rendering the 1969 Camaro test scene with Nvidia Iray and the new 3DS Stellar Physically Correct global illumination engine, showed the RTX 6000 Ada to be between 32% and 40% faster than the RTX A6000.

In KeyShot 2023 the RTX 6000 Ada Generation rendered our motorbike model between 33% and 41% quicker in a range of stills and turntable animations. With KeyShot ‘s sample drone scene it went down to 23%-25%.

Conclusion

The Nvidia RTX 6000 Ada Generation is a phenomenally powerful GPU. In several visualisation workflows, it delivered more than double the performance of the previous generation Nvidia RTX A6000. And when Nvidia has full control over both software and hardware, and the GPU’s fourth-gen Tensor cores kick in with DLSS 3, the boost in real-time performance (with seemingly no impact on end user experience) is simply breathtaking.

But these are the extremes of what you can expect from this super high-end workstation GPU. In some of our real world tests the performance increases were as low as 23%. But even this can save hours in the working day. When rendering out a 600 frame 4K animation in KeyShot 2023 for example, rendering times dropped from 6 hours 26 mins to 4 hours 50 mins.

There are some downsides to the new GPU. First, there is no NVLink, which may come as a disappointment to those really pushing the boundaries of complexity. Second it is quite power hungry, drawing 300W at peak. And finally, of course, there’s the price.
£7,150 (Ex VAT) is an incredible amount to pay for a single graphics card and considerably more than Nvidia previously charged for its top end workstation GPUs. The Nvidia RTX A6000, for example, only cost £3,730 (Ex VAT) in February 2021.

While some will consider £7,150 to be a price worth paying for the transformative effect it could have on their workflows, others may seek better value elsewhere.

The new AMD Radeon Pro W7900 (48 GB) is one such option, although it does not offer the same breadth of software support. More than ever before, perhaps Nvidia is facing its biggest competition from within. The consumer-focused ‘Ada Generation’ GeForce RTX 4090 is available for around £1,500, but you miss out on some pro features, superior build quality and access to 48 GB of memory, double that of the 4090. And for some viz artists that’s a big deal. 48 GB allows you to work with more complex datasets and render them at higher resolutions. In simulation, engineers can increase the fidelity of the solver for more accurate results.

The additional memory will not just offer potential benefits for single app workflows. These days many viz artists to use multiple apps at the same time – render in V-Ray while working on real-time experiences in Unreal Engine, for example. And the RTX 6000 Ada Generation is much more likely to allow them to do this without having to compromise their workflow or output.

Nvidia RTX 4000 SFF – Ada Generation

Nvidia is having a staggered roll out of its Nvidia RTX Ada Lovelace GPUs for desktop workstations. Following on from the ultra-high-end Nvidia RTX 6000 Ada Generation, which launched at the tail end of 2022, Nvidia has now released a second desktop GPU designed specifically for compact workstations with form factors like those of the HP Z2 SFF, HP Z2 Mini, Lenovo ThinkStation P3 Ultra, Dell Precision 3460 SFF and Dell Precision 3260 Compact.

The GPU will likely be of interest to users of CAD or BIM software who want to extend their workflows into visualisation, VR and simulation with tools like KeyShot, Twinmotion, Lumion, V-Ray, Omniverse and more.

The Nvidia RTX 4000 SFF Ada Generation features 20 GB of graphics memory, nearly double that of its predecessor, the Nvidia RTX A2000 (12 GB), and is said to offer a 2x performance improvement. It also offers greater memory bandwidth, so it can transfer data to and from its memory more quickly. According to Nvidia, this results in improved graphics, compute and rendering performance.

The Nvidia RTX 4000 SFF is a low-profile, double height graphics card, which takes up two slots on the motherboard. It has four mini DisplayPort1.4a connectors.

The GPU is designed to operate with PCIe slot power alone and has a max power consumption of 70W. This is significantly lower than previous ‘4000’ class GPUs, which typically draw up to 140W.

However, Nvidia confirmed that, in the future, it will also launch a standard Nvidia RTX 4000 GPU that can take advantage of 140W in a single slot. We expect both GPUs will be share the same silicon, but the Nvidia RTX 4000 SFF will be clocked lower.

Nvidia RTX 4000 SFF specs include 6,144 CUDA parallel processing cores, 192 Nvidia Tensor Cores and 48 Nvidia RT Cores. Nvidia quotes 19.2 TFLOPs single precision performance, 44.3 TFLOPs RT Core performance, and 306.8 TFLOPS Tensor core performance.

This is significantly more than the Nvidia RTX A2000 (8.0 TFLOPs, 15.6 TFLOPS and 63.9 TFLOPS respectively). And while single precision performance and RT core performance is very close to the 140W Nvidia RTX A4000, Tensor performance for AI operations has doubled.

The Nvidia RTX 4000 SFF GPU costs around $1,250, and will be available from workstation manufacturers later this year.