Futuremark Announces PCMark 7 for Windows 7
by Jarred Walton on March 21, 2011 12:21 PM EST- Posted in
- Guides
- Futuremark
- PCMark 7
When 3DMark11 launched earlier this year, it seemed like the time was right for an update to PCMark Vantage as well. While we don't have an exact release date or pricing, Futuremark has now confirmed their intention to update the product, but instead of PCMark11 we're getting PCMark 7. The release does say "coming soon", which usually means Futuremark is less than a month or two away from the official launch (but don't hold us to that).
Over the years, we've complained more than a few times about Futuremark's 3DMark scores; It's a long story, but I've posted a comment below for the interested. PCMark on the other hand has been more useful at giving us an easy to run application suite. (If you haven't tried it, SYSmark requires some serious voodoo to get it to run on most modern systems—many of the patches for Vista will break the benchmark, and let's not even get into Windows 7.) The new version looks to provide an updated performance analysis for modern Windows 7 PCs, which may mean Vista is no longer supported. [Update: It should run on Windows Vista, but "missing API support" means the full suite won't run properly, so the final score will be affected.]
We've been a bit critical of PCMark Vantage with regards to inflated SSD results, so it will be interesting to see if PCMark 7 provides a more realistic picture of what an SSD can do for the overall system performance. Yes, we know they're faster, but the problem with Vantage is that even relatively mediocre SSDs did an excellent job at inflating benchmark scores. The PCMark 7 press release states, "Hardware innovations like solid state drives (SSDs), and new form factors such as netbooks and tablets are greatly increasing the range of PC performance available to businesses and home users," so we should see some new metrics addressing such changes at the very least.
System requirements list a DX9 video card as the minimum, but DX10 and DX11 are necessary for some additional tests (i.e. tests that aren't part of the main PCMark score). There are seven test suites with 20 different workloads designed to give various views of system performance. The seven test suites are PCMark, Lightweight, Entertainment, Creativity, Productivity, Computation, and Storage. The Lightweight suite is the one truly new entrant, and Computation could be interesting if there's any support for GPGPU type workloads; the other suites tend to correlate with one or more of the suites in Vantage (Memories, TV and Movies, Music, Communications, Productivity, and HDD—thankfully, the Gaming Suite that tended to be more of a storage benchmark is now MIA).
The full (and somewhat short) press release is below.
New Benchmark Includes 7 Suites for Complete PC Performance Analysis
HELSINKI, FINLAND – MARCH 21, 2011 – Futuremark®, the developer of the world's most popular benchmarking software, announced PCMark® 7 today, a new benchmark offering complete PC performance analysis for Windows 7. PCMark 7 includes 7 separate test suites combining more than 20 individual workloads covering storage, computation, image and video manipulation, web browsing and gaming. A release date has not been announced though the official website states that PCMark 7 is coming soon.
"Hardware innovations like solid state drives (SSDs), and new form factors such as netbooks and tablets are greatly increasing the range of PC performance available to businesses and home users," said Jani Joki, Director of PC Products and Services at Futuremark. "With so much choice available, PCMark 7 is an essential and easy to use tool to test and compare PC performance accurately and reliably across a wide range of usage scenarios."
PCMark 7 provides a set of 7 suites for measuring different aspects of PC performance with a high degree of accuracy. Overall system performance is measured by the PCMark Suite. The Lightweight Suite measures the capabilities of entry level systems and mobility platforms unable to run the full PCMark suite. Common use performance is measured by the Entertainment, Creativity and Productivity scenario suites. Component performance is measured by the Computation and Storage hardware suites. The Storage suite is ideal for testing SSDs and external hard drives in addition to the system drive.
For more information please visit http://www.pcmark.com/ or follow Futuremark on Facebook.
Facebook
Twitter
RSS
13 Comments
View All Comments
JarredWalton - Wednesday, March 23, 2011 - link
Futuremark actually asked me for some details on my concerns with 3DMark, and since I wrote them a lengthy response I figured some of the readers might want to hear the details as well. So, here's the major portion of the email content:One concern with 3DMark (all iterations) is that they're simply not real games. They are 3D benchmarks that are repeatable, and as such they're somewhat useful as a baseline comparison. However, the only way to really tell how well certain games run is to go and benchmark the games themselves. Batman and HAWX, for example, are far less stressful on hardware than Crysis and Metro 2033; looking at just a 3DMark result, you can't really tell which games will run well on a system and which will have performance issues.
Of course, running gaming benchmarks doesn't really tell you how well other games will run either. A benchmark of Crysis really only tells you how Crysis will run; other games might have similar performance characteristics, but without testing you won't actually know. Thus, the best thing to truly quantify gaming performance is to grab as many games as you can and run benchmarks on all of them. So, even if 3DMark were to switch to the latest Unreal Engine and id Tech engines for the graphics core, while the results may (or may not) correlate better with games using UE/idTech, they still don't do more than give a baseline of performance.
The other concern is probably the bigger issue, and unfortunately it's mostly something Futuremark can't control. When there's a popular benchmark out there, the hardware manufacturers will often spend a lot more effort optimizing to run well on that benchmark rather than working on real performance problems. NVIDIA and ATI/AMD have been doing high performance GPUs long enough that they don't seem to worry about 3DMarks that much these days, but one look at the Intel IGP results tells you that they're focusing on the wrong areas.
Take the comparison between an Arrandale i5-520M with HD Graphics and an Athlon II P340 with HD 4250: http://www.anandtech.com/bench/Product/232?vs=336
If we average the four 3DMark scores, HD Graphics with Core i5 comes out 32% ahead of HD 4250. Do the same thing for the six games we tested, and the Intel "lead" drops to 1.4%.
Or perhaps even better, how about Sandy Bridge i7-2820QM with Intel HD 3000 Graphics against i5-460M with GT 420M: http://www.anandtech.com/bench/Product/327?vs=238
The GT 420M setup wins in 3DMarks on average by 28%; in actual gaming results at low detail, the lead is 37%. Bump up to our medium detail settings, however, and the lead is now 57%; at high (which is generally unplayable on either system), the lead is 69%.
The performance optimizations that Intel has worked on with 3DMark vs. gaming is pretty clear (the GMA 4500MHD was worse back in the Core 2 days), but more importantly, these optimizations take time away from real problems. The fact is, Intel still has driver compatibility issues with quite a few games, but they focus on the stuff people are benchmarking rather than doing the admittedly large amount of work to really make their drivers 100% (or at least 99%) compatible. Having Futuremark certify drivers only does so much here, at least as far as I can tell, and it's almost impossible to catch every "cheat" that might be done to improve scores (as opposed to a driver enhancement that affects a large number of 3D rendering tasks).
So basically, the more weight we put on 3DMarks scores, the less meaningful they become because of the optimization games that companies will play. PCMark is certainly susceptible to this phenomenon as well, but in general my experience is that the PCMark test suite uses more in the way of real-world scenarios. If a manufacturer tries to improve their platform's productivity or multimedia scores, that will show up in actual office and multimedia applications as well.
tipoo - Thursday, March 24, 2011 - link
Wow, awesome. Have they responded? Let us know when they do!Intel certainly seems like the most shoddy player, although AMD and Nvidia are hardly clean players either.
lili94 - Wednesday, March 23, 2011 - link
welcome