cifar-speedrun-records.csvThe task is to reach 94% test accuracy on CIFAR-10 in as little wall-clock training time as possible on one A100. Both the accuracy target and the hardware are fixed, so a record is a reduction in the compute needed for a fixed result, and the series measures training efficiency rather than model quality.
A "discovery" is one claimed record at a lower time. No maintained ledger
exists — the airbench README carries no dates — so the vendored CSV was
assembled from release histories, post timestamps and announcements, and the
CSV is itself the ledger. Each row carries a date_precision column and an
acknowledged column recording how firm it is. The earliest row, David
Page's 26 seconds from around 2018, was run on V100s and is excluded from the
plotted series: a time on different hardware is not a point on this curve.
The yearly improvement factor divides the standing record at one year's end by the standing record at the next year's end; the factors are this repository's arithmetic over the vendored series, not figures any source states.
The collection-wide cumulative index redraws this series as the standing record's value over time:
There is no fetch.py in this folder, and no maintained upstream ledger to
write one against. Every row was assembled by hand from release histories,
post timestamps and announcements, and a new record is added by reading the
same kinds of sources again. check.py recomputes the fact lines
above from the CSV.
figure.py reads cifar-speedrun-records.csv, keeps rows whose
date is 2022 or later, and plots seconds as a step function against the
year fraction of date. Colour comes from the agent column. A point is
drawn as an open marker rather than filled when date_precision is undated
or acknowledged is no, which is what distinguishes the 2.73-second record
and the 1.828-second claim from the rest. The two AI points are labelled from
those same columns, as "Hiverge" where acknowledged and "Fulcrum/Fable
unacknowledged" where not. The y axis is logarithmic with ticks at 2, 3, 5,
10 and 20, since the series spans an order of magnitude. January 2026 onward
is shaded, as in every figure here.
Hiverge's 1.99-second run of 2025-10-15 is the first AI-set record on this task acknowledged by the record-keeper; the run was made in the summer of 2025 and announced by Keller Jordan on 2025-10-15, and Hiverge's own post describes the holder as an "Algorithmic discovery engine" [@hiverge2025cifar]. Measured against the 2.59-second record it displaced, the step is about 23%. The same company holds record 32 on modded-nanogpt.
The 1.828-second Fulcrum result of 2026-07-09 is recorded as a claim rather
than a record: it is not acknowledged on the record-keeper's account as of
2026-07-28, and the lab's own writeup documents specification gaming
alongside the legitimate change [@fulcrum2026fable]. No other row in the CSV
carries an agent value of ai.
Two adjacent results are AI-credited off this leaderboard: TTT-Discover's GPU kernels, found by test-time training on an open 120-billion-parameter model and 15 to 51% faster than the best human submissions [@yuksekgonul2026learning], and Karpathy's two-day autonomous tuning run on nanochat, whose roughly 20 useful changes he then tested and stacked himself for an 11% gain [@karpathy2026autoresearch].