AMD EPYC Milan Review Part 2: Testing 8 to 64 Cores in a Production Platform
by Andrei Frumusanu on June 25, 2021 9:30 AM EST
It’s been a few months since AMD first announced their new third generation EPYC Milan server CPU line-up. We had initially reviewed the first SKUS back in March, covering the core density optimised 64-core EPYC 7763, EPYC 7713 and the core-performance optimised 32-core EPYC 75F3. Since then, we’ve ben able to get our hands on several new mid and lower end SKUs in the form of the new 24-core EPYC 7443, the 16-core 7343, as well as the very curious 8-core EPYC 72F3 which we’ll be reviewing today.
What’s also changed since our initial review back in March, is the release of Intel’s newer 3rd generation Xeon Scalable processors (Ice Lake SP) with our review of the 40-core Xeon 8330 and 28-core Xeon 6330.
Today’s review will be focused around the new performance numbers of AMD’s EPYC CPUs, for a more comprehensive platform and architecture overview I highly recommend reading our respective initial reviews which go into more detail of the current server CPU landscape:
-
(April 6th) Intel 3rd Gen Xeon Scalable (Ice Lake SP) Review: Generationally Big, Competitively Small
-
(March 15th) AMD 3rd Gen EPYC Milan Review: A Peak vs Per Core Performance Balance
-
(December 18th) The Ampere Altra Review: 2x 80 Cores Arm Server Performance Monster
What's New: EPYC 7443, 7343, 72F3 Low Core Count SKUs
In terms of new SKUs that we’re testing today, as mentioned, we’ll be looking at AMD new EPYC 7443, 7343 as well as the 72F3, mid- to low core-count SKUs that come at much more affordable price tags compared to the flagship units we had initially reviewed back in March. As part of the new platform switch, we’ll cover in a bit, we’re also re-reviewing the 64-core EPYC 7763 and the 32-core EPYC 75F3 – resulting in a few surprises and resolving some of the issues we’ve identified with 3rd generation Milan in our first review.
| AMD EPYC 7003 Processors Core Performance Optimized |
||||||
| Cores Threads |
Base Freq |
Turbo Freq |
L3 (MB) |
TDP | Price | |
| F-Series | ||||||
| EPYC 75F3 | 32 / 64 | 2950 | 4000 | 256 MB |
280 W | $4860 |
| EPYC 74F3 | 24 / 48 | 3200 | 4000 | 240 W | $2900 | |
| EPYC 73F3 | 16 / 32 | 3500 | 4000 | 240 W | $3521 | |
| EPYC 72F3 | 8 / 16 | 3700 | 4100 | 180 W | $2468 | |
Starting off with probably the weirdest CPU in AMD’s EPYC 7003 line-up, the new 72F3 is quite the speciality part in the form of it being an 8-core server CPU, yet still featuring the maximum available platform capabilities as well as the full 256MB of L3 cache. AMD achieves this by essentially populating the part with 8 chiplet dies with each a full 32MB of L3 cache, but only one core enabled per die. This enables the part (for a server part) relatively high base frequency of 3.7GHz, boosting up to 4.1GHz and landing with a TDP of 180W, with the part costing $2468.
The unit is a quite extreme case of SKU segmentation and focuses on deployments where per-core performance is paramount, or also use-cases where per-core software licenses vastly outweigh the cost of the actual hardware. We’re also re-reviewing the 32-core 75F3 in this core-performance optimised family, featuring up to 32 cores, but going for much higher 280W TDPs.
| AMD EPYC 7003 Processors Core Density Optimized |
||||||
| Cores Threads |
Base Freq |
Turbo Freq |
L3 (MB) |
TDP | Price | |
| EPYC 7763 | 64 / 128 | 2450 | 3400 | 256 MB |
280 W | $7890 |
| EPYC 7713 | 64 / 128 | 2000 | 3675 | 225 W | $7060 | |
| EPYC 7663 | 56 / 112 | 2000 | 3500 | 240 W | $6366 | |
| EPYC 7643 | 48 / 96 | 2300 | 3600 | 225 W | $4995 | |
| P-Series (Single Socket Only) | ||||||
| EPYC 7713P | 64 / 128 | 2000 | 3675 | 256 | 225 W | $5010 |
In the core-density optimised series, we’re continuing on using the 64-core EPYC 7763 flagship SKU which lands in at 280W TDP and a high cost of $7890 MSRP. Unfortunately, we no longer have access to the EPYC 7713 so we couldn’t re-review this part, and benchmark numbers from this SKU in this review will carry forward our older scores, also being aptly labelled as such in our graphs.
| AMD EPYC 7003 Processors | ||||||
| Cores Threads |
Base Freq |
Turbo Freq |
L3 (MB) |
TDP | Price | |
| EPYC 7543 | 32 / 64 | 2800 | 3700 | 256 MB | 225 W | $3761 |
| EPYC 7513 | 32 / 64 | 2600 | 3650 | 128 MB | 200 W | $2840 |
| EPYC 7453 | 28 / 56 | 2750 | 3450 | 64 MB | 225 W | $1570 |
| EPYC 7443 | 24 / 48 | 2850 | 4000 | 128 MB |
200 W | $2010 |
| EPYC 7413 | 24 / 48 | 2650 | 3600 | 180 W | $1825 | |
| EPYC 7343 | 16 / 32 | 3200 | 3900 | 190 W | $1565 | |
| EPYC 7313 | 16 / 32 | 3000 | 3700 | 155 W | $1083 | |
| P-Series (Single Socket Only) | ||||||
| EPYC 7543P | 32 / 64 | 2800 | 3700 | 256 MB | 225 W | $2730 |
| EPYC 7443P | 24 / 48 | 2850 | 4000 | 128 MB | 200 W | $1337 |
| EPYC 7313P | 16 / 32 | 3000 | 3700 | 155 W | $913 | |
Finally, the most interesting parts of today’s evaluation are AMD’s mid- to low-core count EPYC 7443 and EPYC 7343 CPUs. At 24- and 16-core, the chips feature a fraction of the maximum theoretical core counts of the platform, but also come at much more affordable price points. These parts should especially be interesting for deployments that plan on using the platform’s full memory or I/O capabilities, but don’t require the raw processing power of the higher-end parts.
These two parts are also defined by having only 128MB of L3 cache, meaning the chips are running only 4 active chiplets, with respectively only 6 and 4 cores per chiplet active. The TDPs are also more reasonable at 200W and 190W, with also respectively lower pricing of $2010 and $1565.
Following Intel’s 3rd generation Xeon Ice Lake SP and our testing of the Xeon 28-core 6330 which lands in at an MSRP of $1894, it’s here where we’ll be seeing the most interesting performance and value comparison for today’s review.
Test Platform Change - Production Milan Board from GIGABYTE: MZ72-HB0 (rev. 3.0)
In our initial Milan review, we unfortunately had to work with AMD to remotely test newest Milan parts within the company’s local datacentre, as our own Daytona reference server platform encountered an unrecoverable hardware failure.
In general, if possible, we also prefer to test things on production systems as they represent a more mature and representative firmware stack.
A few weeks ago, at Computex, GIGABYTE had revealed their newest revision of the company’s dual-socket EPYC board, the E-ATX MZ72-HB0 rev.3.0, which now comes with out-of-box support for the newest 3rd generation Milan parts (The prior rev.1.0 boards don’t support the new CPUs).
The E-ATX form-factor allows for more test-bench setups and noiseless operation (Thanks to Noctua’s massive NH-U14S TR4-SP3 coolers) in more conventional workstation setups.
The platform change away from AMD’s Daytona reference server to the GIGABYTE system also have some significant impacts in regards to the 3rd generation Milan SKUs’ performance, behaving notably different in terms of power characteristics than what we saw on AMD’s system, allowing the chips to achieve even higher performance than what we had tested and published in our initial review.
Facebook
Twitter
RSS
58 Comments
View All Comments
Andrei Frumusanu - Friday, June 25, 2021 - link
Those results don't contradict anything I'm saying. Given a normalised throughput performance of the socket, for example here where the 16- and 24- core Milan equals or beats the 28-core ICL-SP in many workloads, the Xeon still handily beats those Milan parts in transactional workloads. The 40-core Xeon has 77% of the jbb performance of the 64-core EPYC even though in the int suite it's only at 60%. Those particular STH results work out because the 7543P is $1000 cheaper than the 7543, but for the SKUs we had in today, Intel still is on equal footing in terms of DB performance value.Cllaymenn - Friday, June 25, 2021 - link
Whatever one says about some insignificant single anomaly in some DB test... The fact is that ANY company, from small to large, any corporation needing power, any data centre, hosting, cloud computing, research institutes, universities, will choose EPYC on ZEN3 over even the 8320, because it will allow them to compute faster, make more money per month, and less stress for administrators when there are higher network loads, clouds because AMD will "grind" / process faster the requests/needs of thousands of of thousands of clients simultaneously using servers, because in addition to more compute power has more bandwidth AMD platform especially with 256 threads and 8 channel memory and fast Infinity Fabric and many of the ZEN3 optimizations... and is more flexible (harder to clog or jam Zen2/Zen3 from what I've noticed. ) These processors grind through anything you throw at them without any breathlessness.schujj07 - Friday, June 25, 2021 - link
While Spec is an "industry standard" benchmark, vendors spend hours optimizing for their servers to look better. Therefore as an administrator and designer of a high performance data-center I personally look at Spec results with a grain of salt. For example, Super Micro submitted data for 2 of their A+ AS-1124US-TNRP with dual 75F3 on April 26, 2021. One system has max-jOPS of 276,317 and critial-jOPS of 116,628. The other has a score of 211,179 max-jOPS & 191,813 critical-jOPS. They also have 2 X12DPG-QT6 with dual 8380's and one has scores of 272,500 for max-jOPS & 147,409 for critical-jOPS. The other has scores of 258,368 for max-jOPS & 201,334 for critical-jOPS. In these cases the 75F3 with few cores and threads ends up in a virtual tie with the 8380 in the transactional workload for one of the results, but the second result in the database is a 22-30% lower based on comparison systems. https://www.spec.org/jbb2015/results/res2021q2/Depending on the results you want, the 75F3 is a much better value or of equal value to the 8380. I think now you can see why I take Spec with a grain of salt on their results. Globally saying that Milan has issues in transactional DBs based solely on Spec results isn't a good idea. While I know it is the benchmarks that you choose as they are "industry standard," I think it would be worth while to invest in creating an actual real world scenario DB benchmark that doesn't use Spec.
Andrei Frumusanu - Friday, June 25, 2021 - link
> One system has max-jOPS of 276,317 and critial-jOPS of 116,628. The other has a score of 211,179 max-jOPS & 191,813 critical-jOPS.Which generally makes submitted scores not very useful, we're using apples-to-apples runs here, and while you can argue they're not as optimised, they're comparable to each other.
And I also never said that Milan has *issues*, I'm simply saying that compared to other workloads where there's a massive performance lead for AMD, Intel is still competitive, a view that falls in line with many industry customers.
Cllaymenn - Friday, June 25, 2021 - link
We know that Intel watches the Anandtech website, and that you are aware of this, they also send you expensive hardware for testing, and hope that the results will be more favourable to their new development (e.g. 8320) which they have been working on for a long time. I think it would be unpleasant and uncomfortable to criticise their new products harshly if I were writing a review, but I would rather gently point out which is good at what, which is leading and which still needs to catch up. Because of the awareness of the efforts of hundreds or even thousands of Intel engineers I would not have the heart to criticize their new product, or sharply, clearly say who wins everything and the rest can hide. I know that even the engineers, designers and CPU architects like to read about their new baby after work, and they go to sites like Anandtech with enthusiasm and quiet hope that they have made a better impression on the reviewer and readers, than their previous older products, that we have noticed a significant difference, jump in performance and that it has been appreciated and maybe there will be some nice, positive comments, feedback. It probably gives them a lot of happiness to see people out there enjoying the results of their hard work and another success for the company. Because the 8320 was a huge challenge for these people, it's a brand new fresh 10nm SuperFin technology and a mega monolithic 40 core big piece of silicon. And it works! It may not catch up with the 64 core competition but it's still a huge step forward for them, reaching a significant milestone. Once they mastered this SuperFin 10nm technology to create monolithic 40 core chips they now have a lot of experience and know how to do it even better, especially in a modular architecture where the silicon pieces will be smaller. Many of the threads stem from the creation of the Xeon 8320, so I understand the reviewer's attitude of appreciating the level of technology, sophistication, and performance of their new design. (sorry for some grammatical errors, I'm still improving)bwhitty - Friday, June 25, 2021 - link
Can't tell if you're very subtly implying Andrei is coloring the results in favor of Intel? Perhaps you're not, but anyways it doesn't seem he is. Other than that, I agree thatSmall correction: Ice Lake is on 10nm+, not Super Fin. Tiger Lake is 10SF (10++), and Sapphire Rapids will be on 10 Enhance Super Fin, so 10nm+++.
Tangent: I think that Ice Lake being on the non-SF process actually bodes extremely well for Sapphire Rapids because Ice Lake even in laptops is just not that good from a mfg perspective. It's basically Intel 10nm's first shippable and salvaged process. Super Fin appears far, far better in Tiger Lake versus Ice Lake, and so an improvement on top of that thusly should perhaps finally bring Intel's mfg in line with TSMC 7nm. That gives Sapphire Rapids a good place to be in the first half of 2022 until Genoa rolls out on TSMC 5nm is late 2022 / early 2023.
Cllaymenn - Friday, June 25, 2021 - link
bwhitty. I did not mean favoring Intel products, but a more subdued way of speaking about their performance in relation to ZEN3, a way other than the popular Linus on YT, which is sharply pressing Intel with each premiere of new AMD products.As for Super Fin, I read about it recently in one of the popular IT websites. I typed in google and found a quote
"Intel Xeon Scalable Ice Lake-SP processors were announced some time ago, but we had to wait a while for their premiere. We finally got it - we got to know the technical details of the units, as well as their performance results. Intel Xeon Scalable units (Ice Lake-SP) use the new Sunny Cove microarchitecture, which is expected to translate into up to a 20% increase in IPC over the previous generation Skylake. The chipsets are manufactured using a new 10nm SuperFin process.
As I checked with a few other sources, I now know that this site was wrong about the 83xx series.
Ian Cutress - Friday, June 25, 2021 - link
On 10nm naming, Intel has changed it twice. There are no + or ++ any more.https://www.anandtech.com/show/16107/what-products...
bwhitty - Monday, June 28, 2021 - link
Oh yes, Dr Cutress, I know all these Intel mfg node specifics purely from Anandtech’s breakdownsoutsideloop - Friday, June 25, 2021 - link
Far, far better? Tiger Lake H still sucks power like an anebriated Cleopatra.