Silent Data Errors, or Silent Data Corruption (Silent Data Errors/Silent Data Corruption), reveal a growing gap between a chip passing conventional manufacturing tests and its ability to produce correct results throughout its operating life. A processor may pass ATPG tests, transition-fault and stuck-at-fault tests, and at-speed structural tests, then perform a calculation incorrectly without recording an obvious failure, before the corrupted result reaches an application or an AI training task.
This analysis is based on an article published by Semiconductor Engineering on September 10, 2026, and brings together views and findings from Advantest, Siemens EDA, NXP, Synopsys, Intel, Meta, Google, and proteanTecs. It does not concern the launch of a specific product, but rather a shift in how processor quality and test coverage are defined and managed after the processor reaches the data center.
A Rare Problem at the Chip Level, Widespread Across the Fleet
Silent data errors appear rare when measured on an individual device, but they become frequent and costly when millions of processors operate at high utilization rates. Early analyses from Google and Meta indicate that these errors may affect one server in every thousand, equivalent to a level ranging from 100 to 1,000 defective parts per million. Even a rate of 10 FIT, or one failure per billion operating hours, could mean one error approximately every four days when 10 million devices are deployed.
The danger is that the error may appear as an incorrect numerical result or an undefined value, then cause database corruption, unexpected behavior in an artificial intelligence model, or inconsistent analytical results. Because the processor itself may not send an error signal, the fault may not be discovered until an illogical result appears at the end of a lengthy task.
Why Do Conventional Tests Fail?
Silent errors are associated with marginal defects such as high-resistance metal interconnects, weak bridge defects, timing and voltage variations, as well as the effects of aging, radiation, temperature, and load conditions. Their likelihood increases with advanced process nodes, where margins narrow and interconnects become smaller and more resistive. Chiplet-based packages also increase the complexity of verification paths.
The industry estimates that approximately 80% of corrupted-execution errors are associated with defects that escape zero-time testing, while the remaining 20% appear intermittently or as a result of aging. However, testing all possible combinations of voltage, frequency, temperature, age, and workload type is impractical. Linking a failure that appears at the system level to a specific defect pattern in chip testing may also take weeks and require cooperation among design, test, failure-analysis, and systems-integration teams.
Siemens EDA notes that relying on a single-input transition when testing for delay defects may not emulate actual functional operation, since switching multiple inputs can produce larger delays. Tests therefore need to target multiple voltage, temperature, and frequency corners rather than relying solely on structural fault models that do not cover every usage condition.
Deeper Testing from the Factory to the Data Center
This problem is redefining the concept of test coverage. Instead of counting the percentage of known manufacturing defects that a test can detect, coverage is also becoming linked to the probability of detecting an incorrect computational answer during realistic operation under an actual workload. Companies are therefore moving toward system-level functional tests, workload-aware tests, and mission-mode tests, alongside improvements to built-in test design and monitoring of on-chip margins.
Intel’s experience illustrates the scale of the challenge. After testing 1.2 million processors across five generations of Intel Xeon processors, the company needed more than 1,000 functional tests in the DCDiag suite and 5,000 synthetic stress tests to detect silent-error defects. The tests were not evenly distributed across defects: approximately 50% of defective parts could be detected using just 5% of the tests, while detecting 90% of errors required more than half of the 1,000 tests. The results also showed that more than 70% of defects were detected by only one test, and that an effective test recipe for one product generation could not be transferred directly to the next generation.
What Is Changing in Fleets in Practice?
The response does not stop at the factory gate. Data center operators use layers of software screening and field testing to isolate servers or cores that exhibit abnormal behavior. At Meta, the Fleetscanner program takes a server out of service and runs it through computational tests with known results, while the Ripple program executes short patterns during normal operation. Hardware Sentinel analyzes application exceptions and system behavior without assigning test workloads, and Meta reported that it improved detection by 40% compared with test-based approaches across different architectures, applications, and data centers.
Google uses a set of protective measures, including end-to-end validation of test suites, redundant computations and comparisons, invariant checks and assertions, data validation during transfer, and periodic validation of stored data. Application-level measurements, such as the Spanner approach, can also detect corruption and remove suspected devices from the fleet, while screening methods are adjusted when devices return in order to identify cores vulnerable to the problem before it worsens.
From Shipping Tests to Silicon Life-Cycle Management
These trends are driving continuous monitoring of timing, voltage, temperature, and degradation margins, along with the use of time-series analysis and machine learning to detect deviations before they become silent errors. Parametric measurements and machine-learning models may help isolate devices that deviate from their expected profile, while diverse instruction tests, redundant execution, and comparisons between cores can improve detection opportunities.
However, this approach does not eliminate the limitations. No single method captures every error mechanism, test results do not automatically transfer from one design to another, and root-cause diagnosis remains difficult when the effect appears in an application far from the physical defect. The analysis also indicates that the sharing of failure data among manufacturers, test-tool providers, data center operators, and universities remains limited because of commercial and legal considerations. The actual change, therefore, is not the addition of a single test, but the construction of a chain of visibility extending from manufacturing to operation, while recognizing that processor quality is not determined entirely at the moment it is shipped.