Introduction to Server Benchmarking

Each time we publish a new server platform review, several of our readers inquire about HPC and rendering benchmarks. We're always willing to accommodate reasonable requests, so we're going to start expanding beyond our usual labor intensive virtualization benchmarks. This article is our first attempt. It was a bumpy ride, but this first attempt produced some very interesting insights.

The core counts of modern servers have increased at an incredible pace, making many benchmarks useless if we want to assess the maximum throughput. Just three years ago, we could still run benchmarks like Fritz Chess, Winrar, and zVisuel to satisfy our curiosity. We also performed real-world benchmarks like MySQL OLAP on our octal-core servers. All these benchmarks are pretty useless now on our 48-core Magny-Cours and 80-thread Westmere-EX systems. The number of applications that can really take advantage of the core counts found in quad- and even dual-socket servers continues to get lower and lower.

Most servers are now running hypervisors and virtualization of some form, so we naturally focused on virtualized environments. However, many of our readers are hardware enthusiasts, so while we wait for the new server platforms such as Intel's Romley-EP (Sandy Bridge EP) and AMD's Interlagos (Bulldozer) to appear, we decided to expand our benchmark suite. Our first attempt is not very ambitious: we'll tackle Cinebench (rendering) and Stars Euler 3D CFD (HPC). Both are quick and easy benchmarks to perform... or at least that'ss what we expected going in. On the plus side, our testing results are a lot more interesting than we imagined they would be.

Benchmark Setup
Comments Locked

52 Comments

View All Comments

  • derrickg - Friday, September 30, 2011 - link

    Would love to see them benchmarked using such a powerful machine.
  • JohanAnandtech - Friday, September 30, 2011 - link

    Suggestions how to get this done?
  • derrickg - Friday, September 30, 2011 - link

    simple benchmarking: http://www.linuxhaxor.net/?p=1346

    I am sure there are much more advanced ways of taking benchmarks on chess engines, but I have long since dropped out of those circles. Chess engines usually scale very well from 1P and up.
  • JPQY - Saturday, October 1, 2011 - link

    Hi Johan,

    Here you have my link how people can test with Chess calculatings in a very simple way!

    http://www.xtremesystems.org/forums/showthread.php...

    If you are interested you can always contact me.

    Kind regards,
    Jean-Paul.
  • JohanAnandtech - Monday, October 3, 2011 - link

    Thanks Jean-Paul, Derrick, I will check your suggestions. Great to see the community at work :-).
  • fredisdead - Monday, April 23, 2012 - link

    http://www.theinquirer.net/inquirer/review/2141735...

    dear god, at last the truth. Interlagos is 30% faster

    hey anand, whats up with YOUR testing.
  • fredisdead - Monday, April 23, 2012 - link

    everybody, the opteron is 30% faster

    http://www.theinquirer.net/inquirer/review/2141735...

    follow thew intel ad bucks ... lol
  • anglesmith - Friday, September 30, 2011 - link

    i was in a similar situation on a 48 core opteron machine.

    without numa my app was twice slower than a 4 core i7 920. then did a test with same number of threads but with 2 sockets (24 cores), the app became faster than with 48 cores :~
    then found the issue is all with numa which is not a big issue if you are using a 2 socket machine.

    once i coded the app to be numa aware the app is 6 times faster.

    i know there are few apps that are both numa aware and scale to 50 or so cores but ...
  • tynopik - Friday, September 30, 2011 - link

    benhcmark

    like it Phenom
  • JoeKan - Friday, September 30, 2011 - link

    I'd llove to see single core workstations used as baseline comparisons. In using a server to render, I'd be wondering which would be more cost effective to render animations. Maybe use an animation sequence as a render performance test.

Log in

Don't have an account? Sign up now