New Testing Methodology

Every twelve to eighteen months it makes sense to upgrade our test beds in order to best represent what is available on the market. How the upgrade occurs depends on what is being tested, and in the case of our APU reviews it is clear that due to the wide range of graphics options available, as well as at different price points, that we have to adjust our gaming testing.

For 2015 our CPU performance testing regime remains untouched aside from the late 2014 addition of Linux-Bench for a glimpse into Linux based performance. On the gaming side, our games have been updated to the following:

  • Alien Isolation (First Person Survival-Horror)
  • Total War: Attila (Strategy)
  • Grand Theft Auto V (Open World Sandbox)
  • GRID: Autosport (Driving)
  • Middle-Earth: Shadows of Mordor (Action-Adventure)

Because budgets for gaming graphics cards can vary, or users decide to keep the same card for several generations, we will be testing each of these titles in both low, medium and high end graphics setups. This means we can see where the bottlenecks are for CPU performance at each stage. We have also been able to source both AMD and NVIDIA cards for most of these areas, should one side of the equation scale more than the other.

The GPU sections are split into three based on where they fit in their independent stacks rather than for direct competition:

Low-end:
 - Integrated Graphics
 - ASUS R7 240 2GB DDR3 ($70)
 - Dual Graphics (where applicable)

Mid-range:
 - MSI GTX 770 Lightning 2GB ($245-$255 on eBay/Amazon, $330 new)
 - MSI R9 285 Gaming 2GB ($240)

High-end:
 - ASUS GTX 980 Strix 4GB ($560)
 - MSI R9 290X Gaming LE 4GB ($380)

On the low end, we have selected settings in order to make the current best integrated graphics solutions score between 45 and 60 frames per second. On the mid-range and high-end, we typically pull out 1080p maximum settings or almost-maximum.

The Shadows of Mordor (SoM) benchmark throws up a little interesting teaser as well due to the use of its Dynamic Super Resolution technique. This allows us to render at 3840x2160 (Ultra-HD, or ‘4K’) with our settings despite using a 1080p monitor. As a result, we also test SoM at 4K ultra with our mid-range and high-end graphics setups.

For the high-end setups, as we have managed to source 2 cards of each, means that where applicable we can test both SLI and Crossfire setups. We apply this to Shadows of Mordor at 4K as an extra data point.

For clarity, this means:

  Integrated
R7 240 2GB
Dual Graphics
GTX 770 2GB
R9 285 2GB
GTX 980 4GB
R9 290X 4GB
Alien Isolation 720p Ultra 1080p Ultra 1080p Ultra
Average Frame Rate Average Frame Rate Average Frame Rate
Total War: Attila 720p Performance 1080p Quality 1080p Quality
Average Frame Rate Average Frame Rate Average Frame Rate
Grand Theft Auto V 720p Low 1080p Very High 1080p Very High
Average Frame Rate
%FPS <60 FPS
Average Frame Rate
%FPS <60 FPS
Average Frame Rate
%FPS <60 FPS
GRID: Autosport 1080p Medium 1080p Ultra 1080p Ultra
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Middle-Earth:
Shadows of Mordor
720p Low
 
1080p Ultra
4K Ultra
1080p Ultra
4K Ultra
4K SLI/CFX
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate
Average Frame Rate
Minimum Frame Rate

For drivers, we locked down the 350.12 WHQL versions from NVIDIA soon after the launch of GTA V. Similarly, the 15.4 Beta drivers from AMD are also being used. These will remain consistent over the next 12-18 months until the next update.

All of our old (and new) benchmark data, both for CPU and graphics performance, can be found in our benchmark database, Bench.

We have a variety of benchmarks here, including legacy benchmarks such as CineBench 11.5 and TrueCrypt, which are not published in the main review. All CPUs/APUs that have been tested in our new 2015 style will be labeled in the dropdown menus by having its launch price listed, e.g. ’AMD A10-7850K (95W, $173)’. With any luck over the course of the next six months we will be adding new data and re-testing older processors for the database in order for our readers to compare old with new.

AMD A8-7650K Review AMD A8-7650K Test Setup, Overclocking
Comments Locked

177 Comments

View All Comments

  • TheJian - Friday, May 15, 2015 - link

    I don't think it's sheer luck when you're doing one of two things: 1. you write the compiler they're using. 2. you're the chip/platform etc they are DOING the coding on and thus optimizing for best perf on the platform they're using. Granted, these two things might not help in ALL cases, but it's a pretty sure bet if EVERYONE decided to code their app/game ON Intel/Nvidia, if you're AMD you're not likely to win many things. You may code how you know how to code, but you OPTIMIZE for whatever is in your hands, and get to others if financing allows (or someone pays you, like Dice/B4F netting 8mil for frostbite running on mantle).

    If you don't have access for platform X, and it runs well on it vs. platform Y that you program on, THEN that was luck. But when it runs well on what you're programming/compiling on, that probably has much less to do with luck. It's just common sense to get that. I'm not saying that's the case here, but you're making a general statement that would seem to go against simple logic in what I'd guess was MOST cases. IE, how many ports of console games do you see that are BETTER on a PC. In most cases we get "another crappy port" comments all over the place. Consoles are admittedly (generally) a worst case scenario, but you get the point. Usually the 2nd platform etc is an afterthought to milk the original cow, not coded with the care of the main platform. Large firms with bigger teams (EA, Blizzard etc) may depend on the skill of the teams doing said work (but even then it's quite rare), but for smaller firms where financing is a big issue, other platform optimization may never happen at all.

    Why do you think Nvidia bought a company like PGI? To make sure they were on even footing with Intel compilers for HPC. Being the vid card that ~75% of workstations and 76% of gamers (according to peddie) use doesn't hurt either, but compilers/tools are a big help too.
  • shadowjk - Friday, May 15, 2015 - link

    Linux has adapted to some AMD specialities rather quickly, like the module/core division, and further back in time, discovered you could have iommu on amd cpus before they even were released.

    Unfortunately, I don't think AMD participates as actively in compiler development..
  • LarsBars - Tuesday, May 12, 2015 - link

    Glad to see the IGP benchmarks updated, they are so much more relevant now! No more 1280x1024 ;) Great work!
  • BrokenCrayons - Wednesday, May 13, 2015 - link

    I agree, the new IGP benchmarks are a much-needed realignment to make them more current.
  • darkfalz - Tuesday, May 12, 2015 - link

    I love AMD's naming scheme, mimicking Intel's but using higher numbers. I wonder how many would fall for that? Surely a 7850K is much faster than a 4560K? And an A8 or A10 clearly a better CPU than an i5 or i7? Awesome chutzpah.
  • akamateau - Tuesday, May 12, 2015 - link

    Another piece of JUNK SCIENCE and yellow journalism from the journalistically bankrupt Anand Tech.

    What happened to the API Overhead Tests?

    What HAPPENED to the DX12 benchmarks?

    I am not INTERESTED in OBSOLETE GARBAGE.

    There is nothing that you use for benchmarking that is relevant.

    ALL gaming is now written to DX11 MAXSPEC. DX12 MINSPEC is 12x broader and allows for far more performance.

    When you FAIL to use relevant benchmarks the you are LYING to the consumer.

    ANAND TECH is nothing more than a garbage website.
  • extide - Tuesday, May 12, 2015 - link

    This isnt a GPU benchmark article, it is a CPU benchmark article
  • Crunchy005 - Tuesday, May 12, 2015 - link

    Why are you even here reading the articles or commenting on them if you think they are garbage?
  • Michael Bay - Tuesday, May 12, 2015 - link

    Your post lacks capitalization.
  • NeatOman - Wednesday, May 13, 2015 - link

    this is the only time I've liked what Michael Bay has said or done lol

Log in

Don't have an account? Sign up now