Rack workstations – Dell Precision R7610

Rack workstations, remote workstations, call them what you will, are nothing new. Just like the iPhone and the Kindle they have been around since the late 2000s.

The idea is simple: your Windows desktop runs on a rack-mounted workstation, which is housed in a data centre. Only the display (pixel) data is transmitted over LAN or WAN to a thin client that sits on an engineer’s desk. All of the CAD data remains in the data centre.

In many ways it’s like streaming video, except the rack workstation generates each frame in real time as the designer pans, zooms and rotates around the 3D CAD model.

The first machines were all about delivering remote workstations over a one to one connection.

Move the workstation to the data centre and benefit in a number of ways: keep intellectual property secure, centralise IT management, log on anywhere (in the office, at home, or at a client meeting) and keep the design office quiet and clutter free.

More recently, the rise of Virtual Desktop Infrastructure (VDI) has added whole new levels of flexibility to how rack workstations are deployed.

By virtualising the CPU and (soon) GPU, multiple virtual machines can be spawned from a single physical rack workstation and given their own desktop OS and applications. And they don’t all have to
be the same.

Virtual machines can be adapted for different workflows and engineers can access high-performance, OpenGL-accelerated 3D CAD/CAM/CAE and PDM applications on demand, on any device including thin clients, tablets, and smartphones.

Moving from the design office to the boardroom, it’s even possible to pause a session, change devices and reconnect to the same desktop.

The user doesn’t even have to be in the same building as the rack workstation. Providing there’s a high-bandwidth, low latency connection in place, design teams can connect remotely, across cities and countries.

This not only means global design teams can share hardware and follow the sun, operating 24 hours a day, but it removes the need to sync huge CAD datasets between sites, which can take time and cause sync errors.

Everyone can work from one version of the truth with data held in a single PDM system.

Dell has been in the rack workstation business since 2008. Its first offering, the Dell Precision R5400, delivered its remote capabilities through a purpose built Teradici PCIe host card, which compressed and encrypted display data before sending it out over the network.

Meanwhile, a zero client that sat on the engineer’s desk decompressed the data and delivered the pixels to the screen.

Dell’s rack workstations have been through a number of revisions since and the company recently released the Precision R7610, a dedicated rack version of the T7610.

The principles remain the same, albeit with much improved Teradici hardware, but Dell has also added virtualisation into its rack workstation offering, providing support for Citrix.

The hardware

The template for the Dell Precision R7610 is the Dell Precision T7610 and features virtually identical components to its desktop counterpart.

According to Dell, this means that certification from leading CAD/CAM/CAE software developers is straightforward as all the hard work has already been done for the desktop machine.

The similarities end there though. The Precision R7610 has been designed from the ground up to sit in a server rack and firms will most likely host tens or hundreds of them in an air-conditioned data centre.

The Precision R7610 features a streamlined 2U design, and while it might not be the slimmest rack workstation on the market, it does pack in an incredible amount of processing power.

Dell makes a lot of noise about its ‘industry leading’ processing capabilities, both for CPU and GPU. While a lot of rack workstations max out at 135W per CPU, Dell can go all the way up to 150W, meaning it can take exactly the same processors as its high-end desktop workstations.

This increased headroom means the Precision R7610 can be kitted out with two top-end Intel Xeon E5-2687W v2 CPUs, each of which boast eight cores running at a very quick 3.4GHz

Depending on workloads, there are plenty of other options for CPUs, including an extensive range of 4, 6, 8 or 10 core chips from the new Intel Xeon E5-2600 v2 family. A 12 core Intel Xeon E5-2697 v2 will be available soon.

Dell is similarly bullish about its GPU capacity and offers capacity for up to three double width 225W cards. This includes the Quadro K6000 (available soon) and the Tesla K20 for GPU compute. With support for GPU virtualisation coming later this year we would also expect Nvidia’s GRID K2 to join this list.

Those with lower-end GPU requirements can squeeze in four single width graphics cards instead, including the AMD FirePro V4900, W5000 & W7000 or the Quadro K2000 & K4000.

There are some limitations to this wave of GPU and CPU power. Once you go above 450W for graphics, you are limited to 135W CPUs. A special 1,400W power supply will also be required instead of the standard 1,100W unit.

With 16 DIMM slots the Precision R7610 currently peaks at 256GB of 1,600MHz ECC RDIMM memory, though capacity will rise to 512GB once 32GB DIMMS become available.

This might sound like a lot but when you consider that a single workstation could possibly be shared between 12 or even 24 concurrent users resources soon get used up.

As with Dell’s desktop workstations, the R7610 features Reliable Memory Technology (RMT), a Dell patented technology that is claimed to “eliminate virtually all memory errors.”

It works in a similar way to a hard disk drive that has suffered physical damage to the platter. When the machine detects a nonrepairable memory error, it reboots and, at a BIOS level, prevents the system from writing to that spot again.

Such errors typically lead to the DIMM being replaced but, with RMT, Dell reckons it can correct up to seven errors over the lifetime of the DIMM, minimising downtime, while keeping the machine stable.

Unlike the desktop T7610 which can take up to eight 2.5-inch drives, storage in the R7610 is limited to six (SSD, HDD or SAS). We use limited in the looser sense of the word, as the reality is most users will rely on some sort of shared storage for data.

This could be a NAS (Network Attached Storage) or a SAN (Storage Area Network) for higher end users.

There are three main ways to utilise the resources in the Precision T7610, from virtualisation to direct connections. This means Dell is able to offer incredible flexibility in how its rack workstation is deployed, theoretically serving from one up to twelve (even 24) 3D CAD users from a single machine.

Let’s look at each of these in turn.

One-to-one connection

A one-to-one connection is the classic way of using a rack workstation and can be a simple way of getting remote access.

Each designer is given a dedicated remote workstation running a client operating system like Windows 7. It’s exactly like having your own desktop workstation except the machine is usually kept in a data centre while the engineer controls it with a small zero client that sits on his or her desk.

3D applications can be served up with a number of protocols including Microsoft RemoteFX and Citrix HDX 3D Pro but with Dell’s Precision R7610 the connection is commonly made using PCoIP (PC-over-IP), developed by Canadian firm Teradici.

With PCoIP the Dell Precision R7610 is kitted out with a dedicated Teradici Tera2 PCoIP host card, which takes the graphics output from the workstation’s GPU and converts it into encrypted IP packets, which are transmitted over the network. Client-side the data is decrypted and converted back into pixel information to be displayed locally.

The workstation can be controlled remotely with any PCoIP client, including VMware View and the new Teradici PCoIP Software Client but, according to Dell, the best experience comes from using a hardware client, such as the Dell Wyse P25 zero client.

With dedicated processors at both ends, specifically designed to handle the compression and decompression, Dell says the experience is just like having a dedicated workstation under your desk.

PCoIP clients can communicate directly with the host card (all you need is the rack workstation’s IP address) so this can be a very low cost, simple way of delivering 3D CAD applications remotely — even from other sites.

However, the more remote workstations you add into the mix the more complex the management becomes and this is when third party broking software, such as VMware View, comes into play, dynamically managing a pool of remote workstations so they can be accessed from a range of PCoIP clients.

With the Dell Precision R7610 a big benefit of a one-to-one connection is that the users can have dedicated use of an exceptionally powerful workstation: up to two Xeon processors, 512GB RAM and three high-performance GPUs (including two dedicated GPUs for GPU compute).

This is great for users of design visualisation or simulation software where multi-core processors offer big benefits.

However, for mainstream CAD, where GHz is more important than the number of cores, a one to one connection might not make the best use of the resources that can be packed into this dual CPU rack workstation.

Here, virtualisation can dramatically increase user density, without adversely impacting performance.

GPU pass-through

GPU pass-through allows a physical GPU to be mapped directly to a virtual machine.

The workstation’s CPU resources are shared but each user has access to a dedicated graphics card.

The Dell Precision R7610 can support up to four physical GPUs so a typical virtual CAD machine might comprise of four virtual CPU cores, 16GB or 32GB RAM and a dedicated Nvidia Quadro K2000 or AMD FirePro W5000 GPU.

GPU pass-through for 3D CAD needs a dedicated virtualisation software stack commonly handled by Citrix XenServer (a hypervisor that creates and runs the virtual machines) and Citrix XenDesktop with HDX 3D Pro (a desktop and app virtualisation solution that is specifically designed to support professional 3D applications (through Microsoft DirectX and OpenGL).

Client side, users need the Citrix Receiver client, an app that runs on pretty much any device, including Windows, Mac, Linux, iOS and Android, so it’s even possible to drive a high-performance workstation from a tablet.

GPU pass-through can be a powerful way of delivering virtual workstations as it gives each user a dedicated physical graphics resource.

This sometimes means a high cost of ownership per connection, but if each GPU is a mid-range graphics card, which is sufficient for mainstream CAD, then the hardware costs don’t actually stack
up that much.

When kitted out with two eight-core Xeon processors a Precision R7610 can deliver four respectable virtual CAD machines. Each one might consist of four CPU cores, an Nvidia Quadro K2000 or AMD FirePro W5000 and 16GB RAM.

With most CAD operations typically using one or two CPU cores and another reserved for the OS this specification would also give some spare CPU capacity for multi tasking or multithreaded operations like rendering or simulation. Alternatively, save money with Quad Core Xeons and create four dual core virtual machines.

GPU pass-through doesn’t offer as much user density as a fully virtualised CAD workstation (CPU and GPU), but it is possible to have more than four virtual machines per workstation. The caveat is only four of these will be GPU accelerated: the rest will use software graphics so will only be suitable for office workers running apps like Outlook and Word.

Virtual GPU (vGPU)

GPU virtualisation is the next step up from CPU virtualisation and allows multiple 3D CAD users to share the resources of powerful individual GPUs. Nvidia is pushing this heavily with its GRID technology, which Dell says will be available in the Precision R7610 later this year.

The virtualisation software to enable this is not quite there yet, but it will be soon.

Citrix is currently previewing a new version of Xen Desktop (7.0) which will open up the Precision R7610 to many more 3D CAD users. With current generation Nvidia GRID technology theoretically this could be as many as 24 but will more likely be 12 for a better 3D CAD user experience.

In the Precision R7610 hardware GPU sharing (vGPU) is likely to be enabled with Nvidia GRID K2. However, this would have to be a special version of the card as the current model is designed to be passively cooled in servers and does not have an on-board fan.

The current Nvidia GRID K2 GPU is a dual height card with 8GB of GDDR5 memory and two GPUs, each the equivalent of a Quadro K5000.

As the Precision R7610 could in theory host three GRID K2s and two 12 core Intel Xeon E5-2697 v2 CPUs, it would be possible to give 12 concurrent users access to a virtual machine with the equivalent of a Quadro K2000 GPU, 2 CPU cores and 16GB RAM.

But the beauty of GPU virtualisation is that specifications don’t have to be rigid. It also offers incredible flexibility to designers and engineers, allowing GPU cores and memory to be physically allocated to each virtual machine.

An engineer that typically models parts and small assemblies could be assigned fewer GPU resources than one who works with large assemblies.

Different virtual machines could be set up specifically to handle different workflows and spawned by users on demand, even part time users who may only need occasional access to a 3D viewer in a
Product Data Management (PDM) system.

Virtualisation is not just about hardware though. A design house that does a lot of sub contracting work for an automotive or aerospace firm may have engineers that work on a project to project basis.

One week that could be Catia, the next NX and the next AutoCAD. Virtual machines can be instantly re-allocated to users according to software and hardware requirements.

Conclusion

Investing in rack workstations is not just a case of buying some machines, plugging them in and away you go.

It’s a complex IT solution that needs careful consideration, not least about how to integrate the technology into your exciting infrastructure. Firms need to think beyond CPUs, GPUs and memory: rack workstations can demand a lot at the back end including the provision of data centres, SANs and well managed gigabit networks.

The good news is many organisations have the foundations in place having already invested in a virtualisation backbone for office PCs. Adding rack workstations into the mix, even with the 3D graphics performance that CAD demands, can actually be a relatively straightforward process.

In addition to the IT infrastructure there’s a big choice in how to best deploy the machines. The Dell Precision R7610 is a hugely flexible piece of kit: equally at home delivering a high-end workstation for design visualisation as it is serving up 12 virtual machines for entry-level 3D CAD.

Then, at 6pm when everyone has gone home, it can even be turned into a render node. While all of this will likely demand careful planning and monitoring to make the best use of resources, with the flexibility it gives you as a firm, being able to support all different types of 3D users, it’s a nice problem to have.

---

Teradici PC-over-IP

Teradici’s PCoIP (PC-over- IP) technology offers the easiest route to getting remote access to your Dell Precision R7610.

And because it uses hardware, rather than software, to compress, encrypt and encode the display data, it promises to deliver a better user experience, particularly on higher latency, lower bandwidth networks.

All you need is a Tera2 host PCoIP card that sits inside your rack workstation, a standard 100Mbps network and a PCoIP client.

Inside the rack workstation the Tera2 card sits close to the GPU but the two cards actually connect externally. Instead of hooking your GPU up to a monitor you connect it to the Tera2 card using a standard DisplayPort cable.

The Tera2 card can remotely support a single monitor up to 2,560 x 1,600 resolution or two displays at 1,920 x 1,200. Dell is currently evaluating a Tera4 card that can support up to four displays at 1,920 x 1,200 or two at 2,560 x 1,600.

Teradici’s Dynamic Network Adaptation technology continually monitors network latency and bandwidth and adjusts the image quality at times of congestion so user experience isn’t unduly affected.

In the context of 3D CAD the model might remain fully interactive but may appear pixelated while moving. The user also has full control over these settings so can prioritise performance or image quality.

In terms of PCoIP clients Teradici recommends a zero client, such as the Dell Wyse P25, which has no operating system nor GPU. USB ports allow users to connect keyboard and mice, even specialist devices like 3Dconnexion’s SpaceMouse Wireless.

However the USB ports can be locked down so it’s not possible to copy or write data to external drives. This can help protect against IP theft or infection by malware.

Teradici also recently released a free software client, acknowledging the need to give its users flexibility at the end point.

The software client is primarily seen as a complementary solution, for occasional use — to connect workers over VPN at home, for example — but Teradici does have some customers using it over LAN instead of zero clients.

With the software client Teradici acknowledges that the user experience is unlikely to be as good as when hardware is used at both ends.

A zero client, for example, can decode a maximum of 200 megapixels per second whereas the software client can do up to 50 megapixels per second. The performance also depends on the speed of the CPU in the client machine.

The software client is currently limited to 1,920 x 1,200 resolution on a single display and is available for Mac and PC. It only supports a direct connection and users will need the IP address of the rack workstation they want to connect to.

Support for software brokers, such as VMware View, will come early 2014.

---

More options – Rack workstations and blade servers

With the rise of Virtual Desktop Infrastructure (VDI) the lines between rack workstations and graphics server blades are blurring.

Broadly speaking rack workstations offer more flexibility as they can run a client or server operating system and feature standard desktop workstation graphics cards. A graphics server blade, on the other hand, can only run a server operating system and uses passively cooled GPUs.

Here’s a quick run down of some of the current options.

The Fujitsu Celsius C620is an entry-level 1U rack workstation. With support for a single Xeon processor (up to 4 cores), a single GPU (from Quadro K600 up to GRID K2) and up to 32GB RAM it’s best suited to a one to one connection using a dedicated Teradici-based CELSIUS RemoteAccess card.

Workstation Specialists WS-r cranks the single CPU rack workstation up a level, offering up to 12 cores, 512GB RAM and two double width GPUs on a one to one connection using Teradici PC-o-IP hardware.

Despite the name the HP ProLiant WS460c Gen8 Graphics Server Blade is arguably a rack workstation as it runs both client and server operating systems.

It features two Intel Xeon E5-2600 v2 CPUs (up to 12 cores) and up to 512GB RAM. A graphics expansion blade, which sits alongside the main blade, supports a vast range of graphics boards: two single width or one double width.

For GPU virtualisation this means the HP blade is limited to one Nvidia GRID K2 card, but it does offer an interesting proposition for GPU pass through by hosting eight Quadro 1000M GPUs or six Quadro 3000M GPUs on a pair of mezzanine PCI Express boards.

The Supermicro Server RZ-1240i-NVK2, available from Boston and direct from Supermicro, is a 1U rack server, very much focused on VDI. It features up to two Xeon E5-2600 v2 CPUs, two Nvidia GRID K2 GPUs and 256GB memory.

The Tyan FT77A-B7059 is a 4U monster, very much focused on GPU intensive workflows. It’s certified for four Nvidia GRID K2 cards (for graphics) or eight Tesla (K10, K20, K20X) for compute, has two Xeon processors and up to 768GB RAM.

Nvidia, best known for its GPUs, is also developing entire systems. The 4U Nvidia GRID VCA Visual Computing Appliance) combines hardware and software for an out-of the box VDI solution.

Nvidia goes all out on graphics with a ‘Max’ version featuring up to eight GRID K2s, two Xeon processor and 384GB RAM.

The system includes all the virtualisation software (including the client and the hypervisor that creates the virtual machines).

The remote experience

A common fear with rack workstations is that the experience won’t be as good as when the workstation is under your desk — that there will be a lag between when you move the mouse and when the 3D model reacts on screen.

This is certainly true with mainstream remote technologies, such as GoToMyPC, where graphics is handled in software, rather than the GPU. But with workstation-class remote technologies it can be extremely hard to tell any difference. Of course, everything depends on the quality of the connection.

While bandwidth is important, latency (reaction time, measured in milliseconds) has the biggest impact.

On a local Gigbabit network where latency is very low (typically between 0-1ms) the user experience should be very good. However, when rack workstations are accessed from a remote location over WAN it may well take a knock, particularly over large distances.

This might not necessarily be in terms of how reactive a 3D model is when a user moves the mouse.

Both Teradici and Citrix technologies are able to temporarily drop model quality to maintain interactivity.

To the end user the 3D model might appear pixelated as it is rotated on screen but it will quickly refine as soon as the model stops moving — much like a progressive render.

Dell says the best user experience comes when using Teradici-based hardware, both in the rack workstation and on the desk. This is because all of the compression and decompression is handled in hardware meaning it can deal with latency issues much better.

With Citrix, as the graphics compression is done in software, user experience will likely take a bigger hit as latency increases.

---

Build yourself a remote workstation

So you want to use your high-performance workstation from home, the board-room or the shop floor, but don’t want to invest in a rack?

Don’t worry; it’s actually very easy to turn any desktop machine into a remote workstation.

One way to do this is using hardware-based PCoIP. Simply buy a Teradici host card, optional zero client and away you go.

Many of the major workstation manufacturers also offer Teradici host cards as a option in their desktop workstations.

AMD’s FirePro R5000 does a similar job, combining Teradici hardware and a midrange professional 3D graphics card on a single PCI Express board.

The advantage is you only use one motherboard slot instead of two and everything is controlled with a single driver.

However, getting remote capabilities for your workstation doesn’t have to cost a bean. HP Z series workstations come free with HP RGS (Remote Graphics Software) a remote desktop protocol specifically designed for 3D graphics.

To get up and running you simply need to install HP RGS sender and receiver software on workstation and client. However, as compression and decompression is done in software it may not be so forgiving on high-latency networks.