Hardware

Benchmarking Workstations – what it all means

Posted on November 13, 2009 at 2:10 pm by DEVELOP3D

2423 0

How the performance of wokstations is assessed at Develop3D
To help assess the performance of all the workstations we test at DEVELOP3D we have recently updated our suite of 3D application benchmarks. We only use application benchmarks with real engineering datasets as we deem these give a much better idea of relative performance of hardware than synthetic benchmarks such as Viewperf.

Heavy machinery manufacturer Mastenbroek provided the model for this test which measures graphics frame rates. Because Inventor is limited by the speed of the CPU, it is the CPU, not the graphics card that dictates performance

For 3D graphics performance we use SolidWorks 2009, 3ds Max Design 2010, and Inventor 2010. While all applications work differently, we believe these applications give a good cross section of the kind of performance you can expect from most 3D software. Of course there’s no substitute for testing all of the major 3D applications, but to do this we would need to dedicate an entire magazine (and our lives) to this cause.

Each application benchmark runs a script that manipulates a 3D model along a set path and rotation and measures the average number of frames it can display each second. The faster the ‘frames per second’, the smoother the 3D experience for the end-user.

When 3D graphics become too jerky it becomes very hard to work fluently. To avoid this, some applications, including SolidWorks and 3ds Max, can automatically reduce the level of detail of the geometry to a point where consistent frame rates can be maintained. However, as the level of detail varies according to how powerful the workstation is we switch this setting off when testing.

Advertisement

Advertisement

This standard SolidWorks assembly of an underwater camera is tested with a rotation script. The model features complex geometry and has RealView switched on to increase the load (and importance) of the graphics card

While the raw power of the graphics card is important for 3D, performance is also governed by the speed of the Central Processing Unit (CPU). In all 3D applications the faster the GHz of the CPU, the faster the graphics performance. In some applications in order to boost graphics performance it is more important to increase the speed of the CPU than to buy a more powerful graphics card. An extreme example of this is Inventor, whose graphics performance is dictated entirely by the speed of the CPU. In this respect, our Inventor graphics test could actually be considered to be a CPU test.

In terms of CPU, most CAD applications only use a single CPU core. This is why in general it is more important to buy a workstation with a high GHz CPU, as opposed to one with lots of cores. There are, of course, exceptions to this, and processes such as simulation, rendering, and other elements of CAD software including model can take advantage of multiple CPU cores.

For graphics an average frame rate is recorded when a model is rotated in ‘smooth and highlights with edged faces’ mode and a part is moved. For CPU the time taken for mental ray to render a single frame at HD resolution is recorded

To test multi-core CPUs in DEVELOP3D we use the mental ray rendering engine in 3ds Max Design 2010 to render a scene in HD. This takes full advantage of all of the workstation’s CPU cores, including the virtual cores created by Intel’s HyperThreading technology. This means a workstation with two CPUs is almost exactly twice as fast at rendering as a workstation with one identical CPU.

While most simulation software can take advantage of multiple CPU cores, they are not as efficient at using all the cores as a rendering tool and it is all highly dependent on the types of simulation and model. This is also true for many CAM applications.

How the performance of wokstations is assessed at Develop3D
To help assess the performance of all the workstations we test at DEVELOP3D we have recently updated our suite of 3D application benchmarks. We only use application benchmarks with real engineering datasets as we deem these give a much better idea of relative performance of hardware than synthetic benchmarks such as Viewperf.

Heavy machinery manufacturer Mastenbroek provided the model for this test which measures graphics frame rates. Because Inventor is limited by the speed of the CPU, it is the CPU, not the graphics card that dictates performance

For 3D graphics performance we use SolidWorks 2009, 3ds Max Design 2010, and Inventor 2010. While all applications work differently, we believe these applications give a good cross section of the kind of performance you can expect from most 3D software. Of course there’s no substitute for testing all of the major 3D applications, but to do this we would need to dedicate an entire magazine (and our lives) to this cause.

Each application benchmark runs a script that manipulates a 3D model along a set path and rotation and measures the average number of frames it can display each second. The faster the ‘frames per second’, the smoother the 3D experience for the end-user.

When 3D graphics become too jerky it becomes very hard to work fluently. To avoid this, some applications, including SolidWorks and 3ds Max, can automatically reduce the level of detail of the geometry to a point where consistent frame rates can be maintained. However, as the level of detail varies according to how powerful the workstation is we switch this setting off when testing.

This standard SolidWorks assembly of an underwater camera is tested with a rotation script. The model features complex geometry and has RealView switched on to increase the load (and importance) of the graphics card

While the raw power of the graphics card is important for 3D, performance is also governed by the speed of the Central Processing Unit (CPU). In all 3D applications the faster the GHz of the CPU, the faster the graphics performance. In some applications in order to boost graphics performance it is more important to increase the speed of the CPU than to buy a more powerful graphics card. An extreme example of this is Inventor, whose graphics performance is dictated entirely by the speed of the CPU. In this respect, our Inventor graphics test could actually be considered to be a CPU test.

In terms of CPU, most CAD applications only use a single CPU core. This is why in general it is more important to buy a workstation with a high GHz CPU, as opposed to one with lots of cores. There are, of course, exceptions to this, and processes such as simulation, rendering, and other elements of CAD software including model can take advantage of multiple CPU cores.

For graphics an average frame rate is recorded when a model is rotated in ‘smooth and highlights with edged faces’ mode and a part is moved. For CPU the time taken for mental ray to render a single frame at HD resolution is recorded

To test multi-core CPUs in DEVELOP3D we use the mental ray rendering engine in 3ds Max Design 2010 to render a scene in HD. This takes full advantage of all of the workstation’s CPU cores, including the virtual cores created by Intel’s HyperThreading technology. This means a workstation with two CPUs is almost exactly twice as fast at rendering as a workstation with one identical CPU.

While most simulation software can take advantage of multiple CPU cores, they are not as efficient at using all the cores as a rendering tool and it is all highly dependent on the types of simulation and model. This is also true for many CAM applications.