We only use application benchmarks with real engineering datasets as we believe these give a better idea of relative performance of hardware than synthetic benchmarks.
To simplify things we have split our benchmarks into four distinct groups: CAD, CAE (Computer Aided Engineering), CAM (Computer Aided Manufacturing) and rendering. Each test focusses predominantly on the CPU and memory and assesses the benefits of multi-core CPUs and dual CPU workstations.
Users of CAE software that multitask may also be interested in the results from our CAM benchmark, though memory can become a bottleneck in simulation.
We understand that all software behaves differently, but have tried to use applications and datasets which we feel display ‘typical’ behaviour for the target group.
Obviously there is no substitute for testing out your own datasets in your application of choice.
The results from all the workstations we test will be published in a database which can be found at DEVELOP3D.com/workstations
CAD (Graphics)
It is very hard to assess 3D graphics performance for CAD as each application behaves differently, as do the datasets loaded within them.
Assemblies with complex feature trees can place a huge load on the CPU and this can become the bottleneck in a system instead of the graphics card. CPU speed is essential for all 3D applications and it is important to buy a CPU with a high GHz in order to expose the true power of a graphics card.
A trend in modern CAD software is to render 3D models in real time with materials, reflections and shadows. In the OpenGL-based SolidWorks 2010 this is called RealView and our benchmark features an assembly of an underwater camera, which is tested with a rotation script. The higher frames per second (FPS), the faster the graphics card.
CAD (CPU)
For pure CAD most CPU operations are still single threaded. This means it is more important to have a processor that runs at a high GHz, than one that has lots of CPU cores.
Our CAD benchmark records the time taken to import a STEP model into SolidWorks 2010. It is a very processor intensive operation, but as the software also writes out part files at the same time, the speed of the hard drive plays its part.
The test assembly is a sea scooter from Sea-Doo, which is made up of many complex parts.
CAM (CPU)
In recent years CAM applications have evolved to take advantage of multiple CPU cores. However, there are diminishing returns as more are used, typically peaking at two or four cores. For our CAM benchmark we use Delcam PowerMill 2010 and the test records the time taken to calculate a couple of toolpaths.
We also use this test to assess a workstation’s ability to multitask, or carry out multiple compute intensive tasks at the same time. Here we run two and then three sessions of the software and calculate multiple copies of the same toolpath in parallel.
We record the average time taken for all tests to finish. Workstations with more CPU cores will perform better as they do not need to share CPU cores between tests.
CAE (CPU)
Computer Aided Engineering (CAE) is a huge area, comprising Finite Element Analysis (FEA), Computational Fluid Dynamics (CFD) and more.
A variety of solvers are used and most are multi-threaded. The benefits of using multiple CPU cores not only depend on the type of simulation, but the nature of the dataset.
In a workstation, which uses a shared memory architecture, there are diminishing returns as more cores are used and this typically peaks at around 2 to 4 cores. In a distributed system (cluster), where each CPU has its own memory and hard drive, the returns are usually much better.
For our CAE benchmark we use SolidWorks Simulation 2010 and apply the FFEplus solver to solve a linear static analysis with contact. While this test is multi-threaded we would expect to see better scaling with a non-linear analysis.
Rendering (CPU)
Rendering is the one process in a product development workflow that can take advantage of every bit of processing power inside a workstation. In addition to using all the available CPU cores, it can benefit from the virtual cores offered by Intel’s HyperThreading technology.
This can boost performance by approximately 15%. The more cores you assign to a rendering task the quicker it delivers the render and this scales linearly to an almost infinite number.
For our rendering benchmark we use Autodesk 3ds Max Design 2011, which features the mental ray rendering engine common in many CAD applications.
Our test records the time taken to render a single frame at HD resolution.
PTC Creo 2.0 graphics (CPU)
SPECapc for PTC Creo 2.0 is produced by the Standard Performance Evaluation Corporation (SPEC), a non-profit corporation. It provides eight workflows that test all aspects of system performance.
For the time being we are only publishing the graphics score, which tests various aspects of graphics performance including:
– shaded with edges display mode
– shaded with reflection display mode
– hidden and no-hidden display mode
– anti-aliasing up to 8X
– edge quality up to very high
The total number of seconds to run each test is normalised based on a reference machine. A Dell Precision 690 with Xeon 5130 2GHz, AMD FirePro V3750, and 16GB ECC RAM gives a score of 1.0.
The benchmark can be downloaded from www.spec.org
