System Performance: UL and BAPCo Benchmarks

Our 2022 test suite for Windows 11-based systems carries over some of the standard benchmarks we have been using over the last several years, including UL's PCMark and BAPCo's SYSmark. Starting this year, we are also including BAPCo's CrossMark multi-platform benchmarking tool.

PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked select PCs with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark.

Futuremark PCMark 10 - Essentials

Futuremark PCMark 10 - Productivity

Futuremark PCMark 10 - Gaming

Futuremark PCMark 10 - Digital Content Creation

Futuremark PCMark 10 - Extended

The 'Essentials' workload score is dictated by single-threaded burst performance, and the Tiger Lake-U systems take up the top three spots. However, the 'Productivity' and 'Digital Content Creation' workloads seem to appreciate larger number of cores. The octa-core Renoir systems come out on top, and even the hexa-core equipped ASUS PN51-E1 leapfrogs the Tiger Lake-U systems in the 'Productivity' workload. For the 'Gaming' workload, the TGL-U systems with the new Iris Xe iGPU is able to surpass Renoir. The iGPU in the TGL-U Core i7 in the ASRock Industrial NUC BOX-1165G7 enjoys a healthy lead over the competition. Overall, though, the octa-core Renoir-equipped UCFF PCs get a slight edge over the mid-range TGL-U configuration in the NUC11TNBi5. Another key takeaway here is that the scores for the actively-cooled and passively-cooled versions are pretty much similar, lending credence to the possibility of Akasa having pulled off another excellent offering in the Newton TN.

BAPCo SYSmark 25

BAPCo's SYSmark 25 is an application-based benchmark that uses real-world applications to replay usage patterns of business users in the areas of productivity, creativity, and responsiveness. The 'Productivity Scenario' covers office-centric activities including word processing, spreadsheet usage, financial analysis, software development, application installation, file compression, and e-mail management. The 'Creativity Scenario' represents media-centric activities such as digital photo processing, AI and ML for face recognition in photos and videos for the purpose of content creation, etc. The 'Responsiveness Scenario' evaluates the ability of the system to react in a quick manner to user inputs in areas such as application and file launches, web browsing, and multi-tasking.

Scores are meant to be compared against a reference desktop (the SYSmark 25 calibration system, a Lenovo Thinkcenter M720q with a Core i5-8500T and 8GB of DDR4 memory to go with a 256GB M.2 NVMe SSD). The calibration system scores 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

SYSmark 25 - Productivity

SYSmark 25 - Creativity

SYSmark 25 - Responsiveness

SYSmark 25 - Overall

SYSmark 25 also adds energy measurement to the mix. A high score in the SYSmark benchmarks might be nice to have, but potential customers also need to determine the balance between power consumption and the performance of the system. For example, in the average office scenario, it might not be worth purchasing a noisy and power-hungry PC just because it ends up with a 2000 score in the SYSmark 25 benchmarks. In order to provide a balanced perspective, SYSmark 25 also allows vendors and decision makers to track the energy consumption during each workload. In the graphs below, we find the total energy consumed by the PC under test for a single iteration of each SYSmark 25 workload. For reference, the calibration system consumes 8.88 Wh for productivity, 10.81 Wh for creativity, and 19.69 Wh overall.

SYSmark 25 - Productivity Energy Consumption

SYSmark 25 - Creativity Energy Consumption

SYSmark 25 - Overall Energy Consumption

The scores for the NUC11TNKi5 and the Akasa Newton TN build are approximately the same (within the realm of run to run variations), and there is a sizable different in the energy consumption in favor of the passive build (probably due to the absence of the fan's power consumption). On the raw scores side, the mid-range TGL-U systems make it to the top half dominated by Core i7 offerings with higher package power budgets.

BAPCo CrossMark 1.0.1.86

BAPCo's CrossMark aims to simplify benchmark processing while still delivering scores that roughly tally with SYSmark. The main advantage is the cross-platform nature of the tool - allowing it to be run on smartphones and tablets as well.

CrossMark 1.0.1.86 - Productivity

CrossMark 1.0.1.86 - Creativity

CrossMark 1.0.1.86 - Responsiveness

CrossMark 1.0.1.86 - Overall

The relative performance seen in SYSmark 25 translate to CrossMark also, as expected. The responsiveness ratings point to the first sign of hiccups for the Akasa Newton TN build. While the SSD has a significant impact on this sub-component score, our monitoring software didn't report any dangerous temperatures for the SSD. As such, we trust SYSmark 25 more than CrossMark because it operates with real workloads and without idle time compression, rather than a simulation of the activity.

Setup Notes and Platform Analysis System Performance: Miscellaneous Workloads
Comments Locked

18 Comments

View All Comments

  • meacupla - Sunday, July 24, 2022 - link

    At that point, you may just want to use a white noise maker
  • abufrejoval - Sunday, July 24, 2022 - link

    Thanks, that's what I've been hearing, too!

    And in a way that's what I've been thinking without hinting it explicitely to Ryan1981: Getting yourself tuned to zero noise is both very expensive and counter-productive.

    Humanity has operated on communal and external noises for hundreds of thousands of years: a large part of our brain is designed to do nothing but discriminate between sounds that indicate danger and those that don't. A total absense of sound only has your brain increase the sensitivity of your receptors to the point where minute sounds become a bother.

    Instead of making electronics completely silient, we should have them emit a soft snore or other comforting noises akin to humans being human.

    There is an auditorium in the midst of Gibraltar's rock, that offers a level of silence no recording studio can match. Anyone left alone in there is bound to develop tinnitus as the brain keeps increasing the sensitivity in your in-ear "DSPs" to the point where they get the "social noise" evolution set as a base line.
  • abufrejoval - Saturday, July 23, 2022 - link

    Your previous Akasa tests had me hoping, that I’d be able to silence any NUC, if a passive Mini-ITX based solution, like the one I’d been using for Gemini Lake Atoms, wasn’t going to be available.

    I had sampled a NUC or Brix once before and was quite shocked at the nervous noise it generated: the fan gave you an audio variant of a CPU graph that you couldn’t just click away. And at top load, it was an unacceptable howler.

    I wanted something with a bit more punch than an Atom, but a similar idle power and obviously a notebook SoC should be able to do that. But the only way to get that stationary and at a reasonable price (with a full complement of RAM) was to get a NUC.

    When I hit across a NUC8i7BEK with the “double sized” Iris 655 48EU iGPU for only €300, my resistance to the form factor melted away and I gave it a try, even if the primary use case—a Linux based HCI server—had zero use for a GPU. After all you never know if it might be recycled as a desktop later and I was just curious to see how this “Apple spec” SoC would perform.

    It turned out that they key to making it unnoticeable was to ensure that the fan would never rev beyond 3200rpm and for that I had to ensure that PL2 would never last longer than 10 seconds nor exceed 50 Watts, while a PL1 of 15 Watts ensured low fan revs even for a power virus.
    I had just ordered another, when I saw a hexa-core i7-10700U based NUC (with a very ordinary 24EU iGPU) going for just €50 extra. So I cancelled and got that one instead. It turned out much more difficult to tame, because Intel was desperate to wring performance leadership out of 14nm in a tiny NUC and only Watts can get you there. I managed again, playing with the PL1/PL2/TAU to get a system rather good for those sprints where the Atoms were trying my patience, yet with a low-enough power and noise footprint to operate 24x7 as a server.

    Half a year later in February 2021 I landed a fresh Tiger Lake NUC11PAHi7, that’s played hard to get ever since. But mine is a Panther Canyon variant, evidently consumer optimized, with a completely different layout of ports for which Akasa doesn’t build a chassis. I don’t know if Intel already made these differentiations in earlier generations, but it’s rather annoying when only the number of models increases, not their availability.

    Again, that Tiger could also be tamed to unnoticeability via the excellent control Intel’s NUCs offer in the BIOS. Of course, even better would be a set of CLI tools which allow you to adjust these things from Linux…

    In terms of snappiness, none of them needs to hide, because at least for a couple of seconds they will all clock to 4.5 GHz or more and match any desktop. For brutal workloads I have other machines with 16 or 18 cores and 140-150 Watts of TDP made tolerable via lots of giant Noctua fans and coolers.

    While there is no noticeable difference in scalar performance between the NUC8 and NUC10, the two extra cores on the NUC10 i7-10700U will obviously deliver a bit of extra punch until TAU runs out. But the Tiger Lake annihilates their value with better IPC: with its four cores it matches pretty exactly the six cores of its predecessor on any parallelized workload while the single core performance is on par with a Zen 3 at the same clocks.

    The “double sized” Iris 655 with its 128MB of eDRAM on the NUC8 turned out to be a paper tiger, effectively adding only 50% of extra power vs. a normal 24EU UHD iGPU at the expense of quite a bit of silicon real-estate and production complexity. If Intel were to sell “Apple spec” chips only, I doubt they’d be nearly as profitable. The list price of an i7-8565U is $409 while the list price of an i7-8559U is only $22 higher. They are close to identical on the CPU side, but the GT3e extra die area and the 128MB eDRAM chip must have cost a pretty penny! I still own a notebook with an i5-6267U, a dual-core Skylake variant of GT3e where the CPU cores were probably the smallest piece of the chip’s silicon pie.

    Really astounding was how badly it got beaten by the 96 Tiger Lake Xe iGPU, which doesn’t have eDRAM for extra bandwidth: that one scaled rather nicely to 4x 24EU performance, beating my Ryzen 3 based 5800U APUs in most benchmarks, just as you describe.

    I don’t really know where that performance is coming from, because DRAM bandwidth is very similar across the board and only around 40GB/s. All my NUCs run with 64GB and while the timings may have gone from DDR4-2400 (NUC8) to DDR4-3200 (NUC11), that’s just adding wait states on these low power devise.

    I love running Google Maps in 3D globe view on Chrome derived browsers at 4k, because it really shows what this low power hardware is capable of with perhaps the most efficient 3D pipeline on the planet: it puts Microsoft’s best flight simulator to shame on an RTX 2080ti!

    It proves the main issue is software, not hardware. But existing real-world games are no fun on these boxes, even the Tiger Lake needs another power of 10 to become reasonably attractive at 4k.

    Another aftermarket NUC solution would evidently be one that adds a beefy active cooling, say a Noctua NH-L9i or even a Noctua NH-L9x65. Obviously these chips could sustain 65 Watts with proper cooling and then deliver quite a reasonable desktop performance in only a slightly bigger form factor.

    BTW: for my use as µ-servers I've added TB3 based 10Gbase-T NICs so the NVMe based SSDs contributing bricks to the Gluster file system don't get slowed down to inacceptable levels.

    I'd have preferred to make do with TB3 based networking via direct connect cables, but fell afoul the fact that Thunderbolt ports don't have MACs and will randomly generate them on every boot or plug event. It's the software.... again!
  • xane - Sunday, July 24, 2022 - link

    Interesting to see continued development, but for me nothing beats Cirrus7 cases from Germany. I do understand it's subjective, though.
  • Hixbot - Tuesday, July 26, 2022 - link

    Ganesh, I've been politely asking you add noise testing to your mini-pc tests for the last couple years. Noise is a very important characteristic to home theater PCs.
    Here we are with a fanless offering with some obvious thermal compromises, but your other reviews don't highlight noise at load and therefore cannot be compared.
  • ganeshts - Tuesday, July 26, 2022 - link

    If there is any noise / electrical coil whine, or anything of that sort, I do make a mention of it in the concluding section (like I did in the Zotac ZBOX CI660 nano).

    Other than that, the ambient noise / noise floor is too high in the environment where these systems are tested for a sound meter to pick up anything at all from them.
  • kepstin - Wednesday, August 24, 2022 - link

    You should really consider retiring/updating that Gimp application startup benchmark… The multithreaded scaling being weird is actually a bug where it's doing extra redundant work that it shouldn't have been, and has been fixed (or at least worked around) in newer versions.
  • storapa - Thursday, September 1, 2022 - link

    Had an old NUC3 with the old version of the Akasa Newton. Worked like a charm for years, until the board died (google results suggests it was a common problem with NUC3, not the case).

    But note that the kensington "lock" doesn't add any security, as you can remove the entire backplate with 4 screws..

Log in

Don't have an account? Sign up now