Hutter Prize compression: enwik9

Hutter Prize enwik9 record

Definition

The task is to compress the first 10^9 bytes of a fixed XML dump of English Wikipedia, scored on the compressed size including the size of the decompression program. The corpus was frozen in 2006, so the task has no benchmark drift by construction, and counting the decompressor closes the loophole of hiding the model in the program.

The prize caps resources:

"must run in ≲50 hours using a single CPU core and <10GB RAM and <100GB HDD on our test machine" — Hutter Prize rules, prize.hutter1.net, read 2026 [@hutter2026prize]

The cap excludes GPUs, so two series are plotted rather than one: the prize is the constrained series, and Matt Mahoney's Large Text Compression Benchmark is the same corpus with no cap, admitting GPU and TPU compressors [@mahoney2026ltcb].

A "discovery" is an awarded record, dated by the award. The prize pays only for improvements of at least 1% over the standing record, so the ledger records steps that cleared a hurdle, not every improvement made.

Facts

The retired 100 MB enwik8 prize is a different corpus and is not joined to the curve. Its complete five-row chronology:

Date Program Total bytes Status
2006-03-24 paq8f -7 18,324,887 pre-prize baseline
2006-09-25 paq8hp5 -7 17,073,018 first award
2007-05-14 paq8hp12 -7 16,481,655 second award
2009-05-23 decomp8 15,949,688 third award
2017-11-04 phda9 15,284,944 fourth award

Alexander Rhatushnyak set all four enwik8 awards; the gap before the last runs eight and a half years. The prize moved to enwik9 in February 2020.

The collection-wide cumulative index redraws this series as the standing record's value over time:

Standing record for total size in MB over time.

Method

The rows are transcribed by hand from the two upstream pages, so there is no fetcher that rebuilds them. fetch.py is a staleness probe instead: it reads the prize page and reports if the standing awarded record has been displaced, leaving the CSV and this document to be updated by hand. check.py recomputes the fact lines above from the CSV.

figure.py calls the shared compression_chart() in ../../lib/families.py, which reads enwik9-records.csv, keeps the rows whose series column matches, and plots total_bytes divided by 10^6 against the year fraction of date as a step function. Rows whose award column is anything other than pending are drawn filled and joined by the solid line, which therefore includes the unawarded 2019 baseline; the pending row is drawn as an open marker labelled from its program column, "cmix-lex, pending". When the series is hutter_enwik9, the function additionally selects the ltcb_enwik9 rows and draws them in grey dashes, with a corner note reporting the uncapped frontier as flat since October 2023 at 107.3 MB. The axis is linear and in megabytes; January 2026 onward is shaded, as in every figure here.

Limitations

AI attribution

No record on either the capped or the uncapped series credits a language model or an agent, as of the prize-site read of 2026-07-28 and the benchmark page's update of 2026-07-08. The nearby AI-credited results in this domain are not compression records: the 23% GPU kernel speedup reported in the AlphaEvolve paper [@novikov2025alphaevolve], and GPU kernels found by test-time training that beat the best human submissions by 15 to 51% [@yuksekgonul2026learning].

Sources