Intel Addresses Desktop Raptor Lake Instability Issues: Faults Excessive Voltage from Microcode, Fix Coming in August
by Ryan Smith on July 22, 2024 7:00 PM EST- Posted in
- CPUs
- Intel
- 13th Gen Core
- Raptor Lake
- 14th Gen Core
In what started last year as a handful of reports about instability with Intel's Raptor Lake desktop chips has, over the last several months, grown into a much larger saga. Facing their biggest client chip instability impediment in decades, Intel has been under increasing pressure to figure out the root cause of the issue and fix it, as claims of damaged chips have stacked up and rumors have swirled amidst the silence from Intel. But, at long last, it looks like Intel's latest saga is about to reach its end, as today the company has announced that they've found the cause of the issue, and will be rolling out a microcode fix next month to resolve it.
Officially, Intel has been working to identify the cause of desktop Raptor Lake’s instability issues since at least February of this year, if not sooner. In the interim they have discovered a couple of correlating factors – telling motherboard vendors to stop using ridiculous power settings for their out-of-the-box configurations, and finding a voltage-related bug in Enhanced Thermal Velocity Boost (eTVB) – but neither factor was the smoking gun that set all of this into motion. All of which had left Intel to continue searching for the root cause in private, and lots of awkward silence to fill the gaps in the public.
But it looks like Intel’s search has finally come to an end – even if Intel isn’t putting the smoking gun on public display quite yet. According to a fresh update posted to the company’s community website, Intel has determined the root cause at last, and has a fix in the works.
Per the company’s announcement, Intel has tracked down the cause of the instability issue to “elevated operating voltages”, that at its heart, stems from a flawed algorithm in Intel’s microcode that requested the wrong voltage. Consequently, Intel will be able to resolve the issue through a new microcode update, which pending validation, is expected to be released in the middle of August.
Intel is delivering a microcode patch which addresses the root cause of exposure to elevated voltages. We are continuing validation to ensure that scenarios of instability reported to Intel regarding its Core 13th/14th Gen desktop processors are addressed. Intel is currently targeting mid-August for patch release to partners following full validation.
Intel is committed to making this right with our customers, and we continue asking any customers currently experiencing instability issues on their Intel Core 13th/14th Gen desktop processors reach out to Intel Customer Support for further assistance.
-Intel Community Post
And while there’s nothing good for Intel about Raptor Lake’s instability issues or the need to fix them, that the problem can be ascribed to (or at least fixed by) microcode is about the best possible outcome the company could hope for. Across the full spectrum of potential causes, microcode is the easiest to fix at scale – microcode updates are already distributed through OS updates, and all chips of a given stepping (millions in all) run the same microcode. Even a motherboard BIOS-related issue would be much harder to fix given the vast number of different boards out there, never mind a true hardware flaw that would require Intel to replace even more chips than they already have.
Still, we’d also be remiss if we didn’t note that microcode is regularly used to paper over issues further down in the processor, as we’ve most famously seen with the Meltdown/Spectre fixes several years ago. So while Intel is publicly attributing the issue to microcode bugs, there are several more layers to the onion that is modern CPUs that could be playing a part. In that respect, a microcode fix grants the least amount of insight into the bug and the performance implications about its fix, since microcode can be used to mitigate so many different issues.
But for now, Intel’s focus is on communicating that they have fix and establishing a timeline for distributing it. The matter has certainly caused them a lot of consternation over the last year, and it will continue to do so for at least another month.
In the meantime, we’ve reached out to our Intel contacts to see if the company will be publishing additional details about the voltage bug and its fix. “Elevated operating voltages” is not a very satisfying answer on its own, and given the unprecedented nature of the issue, we’re hoping that Intel will be able to share additional details as to what’s going on, and how Intel will be preventing it in the future.
Intel Also Confirms a Via Oxidation Manufacturing Issue Affected Early Raptor Lake Chips
Tangential to this news, Intel has also made a couple of other statements regarding chip instability to the press and public over the last 48 hours that also warrant some attention.
First and foremost, leading up to Intel’s official root cause analysis of the desktop Raptor Lake instability issues, one possibility that couldn’t be written off at the time was that the root cause of the issue was a hardware flaw of some kind. And while the answer to that turned out to be “no,” there is a rather important “but” in there, as well.
As it turns out, Intel did have an early manufacturing flaw in the enhanced version of the Intel 7 process node used to build Raptor Lake. According to a post made by Intel to Reddit this afternoon, a “via Oxidation manufacturing issue” was addressed in 2023. However, despite the suspicious timing, according to Intel this is separate from the microcode issue driving instability issues with Raptor Lake desktop processors up to today.
Long answer: We can confirm that the via Oxidation manufacturing issue affected some early Intel Core 13th Gen desktop processors. However, the issue was root caused and addressed with manufacturing improvements and screens in 2023. We have also looked at it from the instability reports on Intel Core 13th Gen desktop processors and the analysis to-date has determined that only a small number of instability reports can be connected to the manufacturing issue.
For the Instability issue, we are delivering a microcode patch which addresses exposure to elevated voltages which is a key element of the Instability issue. We are currently validating the microcode patch to ensure the instability issues for 13th/14th Gen are addressed.
-Intel Reddit Post
Ultimately, Intel says that they caught the issue early-on, and that only a small number of Raptor Lake were affected by the via oxidation manufacturing flaw. Which is hardly going to come as a comfort to Raptor Lake owners who are already worried about the instability issue, but if nothing else, it’s helpful that the issue is being publicly documented. Typically, these sorts of early teething issues go unmentioned, as even in the best of scenarios, some chips inevitably fail prematurely.
Unfortunately, Intel’s revelation here doesn’t offer any further details on what the issue is, or how it manifests itself beyond further instability. Though at the end of the day, as with the microcode voltage issue, the fix for any affected chips will be to RMA them with Intel to get a replacement.
Laptops Not Affected by Raptor Lake Microcode Issue
Finally, ahead of the previous two statements, Intel also released a statement to Digital Trends and a few other tech websites over the weekend, in response to accusations that Intel’s 13th generation Core mobile CPUs were also impacted by what we now know to be the microcode flaw. In the statement, Intel refuted those claims, stating that laptop chips were not suffering from the same instability issue.
-Intel Rep to Digital Trends
Instead, Intel attributed any laptop instability issues to typical hardware and software issues – essentially claiming that they weren’t experiencing elevated instability issues. Whether this statement accounts for the via oxidation manufacturing issue is unclear (in large part because not all 13th Gen Core Mobile parts are Raptor Lake), but this is consistent with Intel’s statements from earlier this year, which have always explicitly cited the instability issues as desktop issues.
49 Comments
View All Comments
kkilobyte - Wednesday, July 24, 2024 - link
Good. Then the question is: why wasn't it done yet, and how is that taking so much time?As a reminder, one of Anandtech's authors claimed he'd do just that "over the weekend" in the article about the i9-14900KS at the beginning of May. So I guess that if it was not going to take longer than a weekend *then* for a given CPU shouldn't take much longer *now* (I'd expect a little more time would be needed, to rebuild the test configuration, etc), right?
Hopefully at least in the meantime, I guess you'll add clear warnings on the charts in the recent July's buyer's guide article, so that readers don't get the wrong information about those CPUs and can make an educated choice based on the content of that article ?
Oxford Guy - Friday, July 26, 2024 - link
It's not May anymore? I guess I've taken a long weekend, indeed.fallaha56 - Tuesday, July 23, 2024 - link
Errr come on Intel…the voltage problem has likely caused degradation issues ie chips already malfunctioning will not be fixed by thisAnd of course you’ll notice Intels new RL stepping planned for release
Time overdue for Intel to come clean about this (Intel shareholder here)
evanh - Tuesday, July 23, 2024 - link
Intel have already said "contact us" when the patch fails to solve it. That's already them saying they expect many permanently broken CPUs. Which they will replace for free obviously.fallaha56 - Tuesday, July 23, 2024 - link
That's fine but as one of many shareholders sitting on losses due to material dishonesty we'll be looking for compensationcatavalon21 - Sunday, July 28, 2024 - link
So go the risks of investment, dude.Bruzzone - Tuesday, July 23, 2024 - link
On INTC financial on channel average weighed price I calculate 125,966,924 to 150 million Raptor desktop components (maybe a smidge more on bundle deal) that are 62,598,714 13th and 38,173,935 14th generation components.On replacement Intel's looking at around $36 at variable cost and $60 marginal cost per component plus handling and administration.
14900/13900K_ represent 1/3rd of Raptors production and KS alone = 6%. 12900K_ volume is 20.1% with KS alone 2.4%. 9900K_ mature 14+ shows a similar volume to Raptors on the maturity of SF10 at 34% however 9th KS represent a meager 0.0017%.
10th i9's represent 10% and adding i7's = 22.7%. Obviously noteworthy reference concerning the stability and evolutionary improvement of design process to hit frequency targets.
All i9/7/5 K_ by generation.
14th = 73.9% and Intel needed them to drive up full line gross margin
13th = 62.5% similar for margin and AMD did same w/R7K emphasizing 16/12C volume.
12th = 46.8%
11th = 41.9% and 18.7% are i9
10th = 27.1% and 10% are i9
9th = 49.35% and 34% are i9
Here's Raptor percent by SKU on channel supply weight.
14900KS 1.38%
14900K 30.29%
14900KF 5.09%
14900 0.02%
14900F 1.21%
14900T 0.77%
14790F 0.35%
14700K 14.07%
14700KF 5.22%
14700 0.04%
14700F 3.09%
14700T 0.66%
14600K 14.71%
14600KF 3.24%
14600 0.01%
14600T 0.05%
14500 2.79%
14500T 0.37%
14490F 0.02%
14400 6.11%
14400T 0.08%
14400F 2.35%
14100 7.19%
14100F 0.76%
14100T 0.12%
100.00%
13900KS 7.06%
13900K 28.57%
13900 3.84%
13900KF 2.39%
13900F 1.77%
13900T 0.40%
13790F 0.18%
13700K 14.04%
13700 4.45%
13700KF 2.40%
13700F 2.83%
13700T 1.47%
13600K 5.68%
13600 0.13%
13600KF 2.40%
13600T 0.59%
13500 5.51%
13500T 2.75%
13490F 0.21%
13400 2.98%
13400F 2.15%
13400T 1.51%
13100 4.79%
13100F 1.13%
13100T 0.76%
100.00%
mb
Bruzzone - Tuesday, July 23, 2024 - link
Clarification, 14th generation run to date is 63,368,210 and 13th 62,598,714 components for range low 125,966,924 units produced. mbballsystemlord - Tuesday, July 23, 2024 - link
Maybe I misunderstood something, but when you wrote: "On replacement Intel's looking at around $36 at variable cost and $60 marginal cost per component plus handling and administration."You meant it will only cost Intel $60 plus handling to replace each CPU? That's a bit low compared to the prices of even the cheaper SKUs sold today.
Bruzzone - Thursday, July 25, 2024 - link
@bellsysemlord, the average marginal cost of Raptors is approximately $60 and subject depreciated process no more than $67 currently.At $60 Intel price in volume to OEMs and large retail is $247.50 in a full line procurement mirroring sort out of the fab; 46.9% i9, 23.9% i7, 8.9%, 22.7% i5, 6.2% i3.
Intel wholesale trade price at x4 over marginal cost of production equals a traditional Intel competitive profit point.
In a full line procurement since i5 1x500 at $1K is underwater at $247 they present a negotiating point between the buyer and Intel. Savvy procurement negotiates for n/c 'bundle deal' that is sales close incentive and Intel might agree as a 'reward' for contract performance. Buyers are willing to pay $247 for a 1x500 or less to get the high margin i7/i9 allocation. High volume can be interpreted as full line production volume divided among seven to eleven top customers.
Otherwise, the basic rule for determining what some mid volume customer, say an SI pays through distribution. is $1K / 2. For some local operator for a tray of 10 CPUs < $1K x 10 < 10%.
mb