Non-Determinism and Overcount on Modern Hardware Performance Counter Implementations
Curated from Lobsters
If you rely on hardware performance counters for regression testing or strict SLO enforcement, this paper is a necessary reality check. We often assume that metrics like retired instructions are absolute ground truth, but modern x86_64 architectures introduce subtle, non-deterministic variations that can skew your data. This matters because many automated profiling tools and capacity planning models assume deterministic outputs to detect micro-architectural regressions. When the underlying hardware is slightly "noisy," your baseline shifts, leading to false positives in performance degradation alerts. The authors dissect specific PMU behaviors that cause overcounting, even in controlled environments, highlighting a gap between theoretical expectations and silicon reality. For practitioners, this means you must treat PMU data as statistical rather than exact. Before trusting a counter-based metric for critical decisions, validate its variance across multiple runs to ensure your threshold accounts for inherent hardware noise.
Abstract: Ideal hardware performance counters provide exact deterministic results. Real-world performance monitoring unit (PMU) implementations do not always live up to this ideal. Events that should be exact and deterministic (such as retired instructions) show run-to-run variation and overcount on x86_64 machines, even when run in strictly controlled environments.
— Lobsters