Four families of generators, eight statistical tests, one attack. This report asks whether "passing the statistical tests" means "being secure". It does not — and the proof sits in two rows of the table below.
The claim. A statistical test establishes the absence of visible structure. It says nothing about predictability. These are two different properties, and only the second one protects a secret.
Two results from this bench settle it: a generator that passes all eight tests and whose future output is predicted 100 % of the time, and an entropy source containing exactly zero entropy that also passes all eight.
1 000 000 bits sampled per source, rejection threshold at 1 %. Hover a cell to read its p-value.
| Monobit | Runs | Longest run | Chi-square | Shannon | Autocorr. 1-32 | Spectral | Incompress. | score | |
|---|---|---|---|---|---|---|---|---|---|
| LCG (low-order byte)PRNG | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | 4/8 |
| Mersenne Twister (MT19937)PRNG | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 8/8 |
| secrets (system CSPRNG)CSPRNG | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 8/8 |
| CPU jitter — naive extractionSoftware TRNG | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 0/8 |
| CPU jitter — rawSoftware TRNG | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | 0/8 |
| CPU jitter — debiased (von Neumann)Software TRNG | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✓ | 2/8 |
| CPU jitter — SHA-256 conditionedSoftware TRNG | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 8/8 |
| NULL source + SHA-256Counter-example | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 8/8 |
How to read it: "8/8" does not mean "secure". Three rows score 8/8 for three completely different reasons — one of them being a source that holds no entropy at all.
The linear congruential generator is the designated bad student. We publish its low-order byte, which is the classic beginner's mistake: the low bits of an LCG have tiny periods.
Yet it is far from failing everywhere. Its Shannon entropy is 8.00000 bits per byte — the exact theoretical maximum — and its chi-square over the 256 byte values returns p = 1, a flawless uniformity. Its distribution is beyond reproach.
And still, zlib compresses it down to 0.0065 of its
size, a factor of 153. Truly random data is incompressible;
this stream is almost entirely structured.
Histogram of byte values. The LCG (red) is a perfectly flat line — flatter than the CSPRNG, whose fluctuations are precisely the signature of randomness. A distribution that is too perfect is a symptom, not a quality.
The lesson: Shannon entropy and chi-square measure the marginal distribution, i.e. how often each value occurs in isolation. They are blind to the order of those values. Catching the sequence takes structural tests — compression, spectral, autocorrelation.
Autocorrelation at lags 1 to 32, in standard deviations. Past the dashed line, the dependency is statistically established.
This is where the homemade hardware-generator idea meets reality. Our physical source is clock jitter: we run a fixed computational load and measure how long it takes, which varies with cache, scheduler, frequency scaling and interrupts. Genuine physical noise, with nothing to solder.
The obvious extraction takes the low-order bit of the gap between two clock readings.
On Windows, perf_counter_ns has a 100 ns granularity: every gap is a multiple
of 100, and its low bit is invariably 0.
Measured: P(bit = 1) = 0.0000, min-entropy 0.000 bits per bit. The source emits a stream of zeros at 8 Mbit/s. It "works", it raises no error, and it contains nothing.
With a correct extraction the source comes alive: P(bit = 1) = 0.3970. Biased, then, but real. The classic remedy is von Neumann debiasing, which guarantees a perfectly balanced output — provided the bits are independent.
After von Neumann, we measure P(bit = 1) = 0.5024. Over a million bits, that departure from 0.5 is massively significant. This is not measurement noise: it is proof that the raw bits are not independent. Von Neumann's assumption is violated, so its guarantee does not hold. The debiased stream still fails 6 of the 8 tests.
Min-entropy per bit — -log2(max(p0, p1)). This is the
measure that matters for an entropy source (NIST SP 800-90B) because it keys on the most
likely outcome, hence on an attacker's best guess. Shannon entropy, which averages, is far
more flattering.
Von Neumann yield: 21.9 % of bits survive, for a final throughput of 76.2 kbit/s. The system CSPRNG delivers 15.4 Gbit/s — a ratio of 604 522× in favour of the software route.
Production throughput, logarithmic scale.
Final step of the classic design: condition the output through a hash function. Applied to our debiased jitter, SHA-256 lifts the source from 2/8 to 8/8. Victory?
Then let us apply the very same conditioning to the null source — the stream of zeros from Act II, entropy measured at 0.000 bits. Result: 8/8 as well, with equally comfortable p-values.
A generator holding no entropy whatsoever is statistically indistinguishable from a good one. The tests do not measure the source: they measure the conditioner. SHA-256 turns any input into flawless-looking output, a counter included.
This is exactly why a homemade hardware generator is risky. If the diode ages, saturates or comes unplugged, the output stays impeccable under test — and becomes fully predictable to anyone who guesses the counter. Certified TRNGs (AIS-31, NIST SP 800-90B) therefore mandate continuous health tests on the raw source, ahead of any conditioning.
MT19937 is the generator behind Python's random, and behind PHP, Ruby and
Excel. It passes all eight tests without a weakness. Its period is 219937−1.
It holds 624 words of 32-bit internal state. Each output is produced by applying a "tempering" function — shifts and XORs — to one state word. That function is bijective: it scrambles, it does not hide. So we invert it.
y ^= y >> 18 → invertible in one pass y ^= (y << 15) & 0xEFC60000 → invertible by iteration y ^= (y << 7) & 0x9D2C5680 → invertible by iteration y ^= y >> 11 → invertible by iteration
Observing 624 consecutive outputs therefore means reading the entire internal state. We rebuild a clone, and predict what comes next.
None of the eight tests sees this weakness, and none of them could: MT19937's output is statistically excellent. The flaw is not in the distribution, it is in the structure of the algorithm.
| Finding | Consequence |
|---|---|
| The LCG has perfect Shannon entropy and compresses by a factor of 153 | Measuring the entropy of an output says nothing about its quality |
| Naive jitter yields 0 bits of entropy without raising an error | A physical source must be measured, never assumed |
| Von Neumann leaves an imbalance of 0.5024 | Fixing bias does not fix correlation |
| A null source conditioned by SHA-256 passes all 8 tests | The tests judge the conditioner, not the source |
| MT19937 passes all 8 tests and is predicted 100 % of the time | Passing the tests is not evidence of security |
| The CSPRNG is 604 522× faster than the software TRNG | Rolling your own randomness costs more and delivers less |
Hence the practical conclusion, which is also the design principle behind KeyMint: for a secret, use the operating system's CSPRNG. It is already seeded by hardware entropy, already conditioned, already audited — and it does not die in silence.
The eight tests are implemented inside this project, without a statistics library. The p-values rest on a regularised incomplete gamma function written by hand and cross-validated against SciPy in the test suite. Every test is also exercised on deliberately defective sequences whose flaw is known in advance — a test that never rejects anything is worthless.
py run_lab.py # the bench → report/results.json py make_report.py # → report/index.html + index.en.html py tests/test_rnglab.py