1. More data, worse answer
Realized variance — the sum of squared returns, RV = Σ r² — has a comforting theory behind it: sample the price on a finer and finer grid and RV converges to the true integrated variance. In continuous-time theory, infinitely frequent sampling gives you the exact answer. In practice, it gives you nonsense. A realized variance computed from every tick is typically several times too large, and the finer you sample the worse it gets. The clean asymptotic promise collides with the messy reality of how prices are actually recorded.
The culprit is microstructure noise: the observed price is not the efficient price, but the efficient price plus a layer of frictional wobble.
2. Where the noise comes from
Model the log price you record as the true price plus an error term:
pobserved = pefficient + u (u = noise)
That noise u is the accumulated residue of market mechanics: bid-ask bounce (consecutive trades printing at the bid, then the ask, then the bid, with no change in fair value), price discreteness (the tick grid rounds the true price), and the small timing and latency effects of how quotes and trades are stamped. None of it is information; all of it is friction. Crucially, taking a return differences the noise — r = r * + (ut − ut−1) — and that difference is what poisons realized variance. Each squared return now carries a chunk of noise variance, and because there are n of them, the total contamination grows with the sampling frequency. Sample twice as often and you roughly double the noise you're accumulating while adding almost nothing to your measurement of the real price path.
The signature of this is a tell-tale negative first-order autocorrelation in high-frequency returns: an up-tick from a bid-to-ask bounce is systematically followed by a down-tick back. That alternating pattern is pure noise, and it is exactly what inflates the sum of squares.
3. The volatility signature plot
The single most useful diagnostic in this whole area is embarrassingly simple: compute realized variance at a range of sampling intervals — every tick, every 2nd, every 5th, every 30th — and plot the result against the interval. This is the volatility signature plot (Andersen, Bollerslev, Diebold & Labys, 2000).
Its shape is diagnostic. At the finest intervals the curve is sky-high — noise dominates. As you coarsen the sampling, the noise averages away and the curve drops steeply, then flattens into a plateau: the region where you're measuring genuine volatility rather than friction. That plateau is your estimate of integrated variance, and where the curve first levels off tells you the fastest frequency you can trust for this instrument. A liquid future plateaus at a few seconds; an illiquid name might not settle until minutes. You never have to guess the "right" sampling frequency — the signature plot shows you.
4. Measuring the noise itself
The same fact that ruins fine-grid RV gives you a free estimate of how much noise there is. Because the finest-grid realized variance is dominated by noise, and each of the n returns contributes about twice the noise variance, the noise variance is recovered by simply dividing:
noise variance ≈ RVfinest / (2n)
This estimate (Zhang, Mykland & Aït-Sahalia, 2005) turns a nuisance into a number. It's a compact read on execution quality and data cleanliness: a name with a large noise variance relative to its true volatility is one where every basis point of the spread matters and where naive tick-level analytics will mislead you.
5. The fix, part one: sample sparsely — but use all the data
The obvious response to the signature plot is to just sample at the plateau: take every k-th observation and compute RV there. That works, but it throws away most of your data — of every k ticks, you use one. The refinement is subsampling: form the coarse grid k different ways, once starting from each of the first k observations, compute RV on each, and average. You get the noise-reduction benefit of coarse sampling while still touching every observation.
This is a genuine bias-variance trade-off, made explicit. Sample too finely and RV is badly biased by noise; sample too coarsely and you have too few returns, so RV is noisy in the statistical sense. Subsampled sparse RV lets you sit at the sweet spot the signature plot identified without wasting information.
6. The fix, part two: two-scale estimators
Subsampling reduces the bias but doesn't fully remove it. The estimator-grade solution combines two scales: take the subsampled sparse RV, and subtract a bias correction estimated from the fast, all-ticks RV (which, as we saw, is essentially a pure measurement of the noise). The result — the two-scale realized variance (TSRV) — is a consistent, noise-robust estimator of integrated variance that uses every tick optimally, not a hand-picked subset. Its cousins, the realized kernel and pre-averaging estimators, achieve the same end by different weighting. For most monitoring the signature plot plus a sensible sparse RV is enough; when you need an unbiased number with a known error rate, these are the tools, and the noise variance you already measured is the ingredient they run on.
7. The same trap in two dimensions
Microstructure noise has a close relative in the cross-asset world. When you measure the realized correlation between two assets and push the sampling finer, the correlation appears to melt toward zero — the Epps effect — because the two assets don't trade at exactly the same instants, and fine sampling pairs a real move in one with a stale print in the other. It is the two-dimensional version of the same lesson: at the finest frequency, the data is describing the plumbing, not the price. The cure is the same in spirit — sample coarsely enough that the friction averages out, or sample by activity rather than the clock with information-driven bars.
8. Computing it
Our open-source orderflow-metrics library ships the diagnostics, dependency-free, in TypeScript and Python:
import {
noiseVariance,
sparseRealizedVariance,
volatilitySignature,
} from "orderflow-metrics";
// a bouncing tick series — heavy bid-ask-bounce noise
const r = [0.002, -0.0015, 0.0018, -0.0016, 0.0021,
-0.0014, 0.0019, -0.0017, 0.0022, -0.0015, 0.002, -0.0016];
noiseVariance(r); // 1.61e-6 — the wobble, quantified
sparseRealizedVariance(r, 1); // 3.86e-5 — raw RV, noise-inflated
sparseRealizedVariance(r, 2); // 1.28e-6 — ~30x lower once the bounce averages out
// the signature plot as data: RV at each sampling step
volatilitySignature(r, [1, 2, 3, 4]);
// [{step:1, realizedVariance:3.86e-5}, {step:2, ...}, ...]
The Python distribution exposes the same functions (noise_variance, sparse_realized_variance, volatility_signature). They sit directly beside the volatility estimators they protect — plain realized and streaming volatility and the jump-robust MinRV / MedRV — in the wider market microstructure toolkit we build in the open. Run the signature plot first; it tells you whether the volatility number underneath is signal or friction.
9. Conclusion
The finest data is not the truest data. Below some frequency, every extra tick you fold into realized variance adds more noise than price, and a naive high-frequency volatility number can be several times too large. The volatility signature plot makes the problem visible; the noise-variance estimate makes it a number; subsampled and two-scale estimators make it go away. Measure volatility at the frequency the market can actually support — not the fastest one your data feed can print — and the estimate stops lying to you. Explore the rest of the toolkit in our quantitative research library, or read the implementation in our open-source metrics.