PROJECT-PLAN-LLM
- Re-check load-bearing factual claims against primary sources, prioritizing independent papers, government reports, benchmark repositories, and first-party project pages.
- Replace search-surfaced quotations with verified paraphrases or exact, checked wording; retain explicit uncertainty where primary evidence is unavailable.
- Add missing evidence, counterevidence, denominators, and methodological caveats for each of the seven predictions in cyber, mathematics, and optimization.
- Keep vendor self-reports clearly labeled and separate measured results from interpretation.
- Reconcile the summary table with the detailed evidence and render/validate the finished Quarto source.
- Proofread for overclaiming, especially where evidence from different tasks, model generations, or scoring rules is being compared.
- Maintain a prioritized pre-publication checklist covering evidence, argument, presentation, and reproducibility.
This is a draft evidence log, not a finished post. It applies the apple-picking model of AI R&D to three domains in which AI systems have produced candidate autonomous contributions — cybersecurity, mathematics, and algorithmic optimization — and records the evidence for and against each prediction. The evidence is deliberately detailed; load-bearing claims are dated and sourced so that a later reader can check and update them.
Provenance caveat. Evidence was checked against linked primary sources where possible: papers for benchmark results, government reports for AIxCC, project repositories and leaderboards for record claims, and first-party pages for vendor demonstrations. Items that still could not be checked directly are tagged [unverified quote]; vendor figures are labeled as self-reports. The most weight-bearing claims below come from independent academic or government sources (METR, RE-Bench, AIxCC, CyberGym, AlgoTune, GSO, and SWE-fficiency), but even these require care: benchmark scores measure particular scaffolds, budgets, task distributions, and versions rather than a domain-wide capability.
The predictions
The apple-picking model treats an AI agent doing R&D like a robot picking apples: it is cheap and can clear the low fruit fast — including fruit humans hadn’t yet picked — but there is a height \(\lambda\) above which it cannot reach. Formally, the apples remaining after human and agent expenditure \(x_H, x_A\) is
\[\text{apples remaining}= \underbrace{\lambda\, e^{-r_Hx_H-r_Ax_A}}_{\text{apples on bottom}}+\underbrace{(1-\lambda)\,e^{-r_Hx_H}}_{\text{apples on top}}.\]
From this we can read off a set of falsifiable predictions. These are the rows of the summary table; the three domains are the columns.
- Autonomous frontier advance. Once given a target and evaluation setup, agents can find genuinely new results without step-by-step human guidance — apples humans had not yet picked. This is operational autonomy, not autonomy over problem selection or evaluation design.
- Sharp diminishing returns (tortoise–hare), non-additive. An agent makes fast progress at first but quickly hits a wall; its returns to further expenditure fall off much faster than a human’s, and it asymptotes above the human frontier (it leaves the high apples). Contributions are non-additive: running an agent twice is not worth two humans.
- Shallow, not deep. The apples an agent reaches should disproportionately be “low-hanging” — mechanical, well-specified, cheaply verifiable, and short-horizon. Deep contributions (theoretical insight, cross-domain transfer, novel structure, and long multi-step chains) should be rarer, not literally impossible.
- Starting-point dependence. Agents contribute most where humans have done least — under-optimized, under-audited, neglected problems — and much less once humans have already picked the reachable fruit. An agent’s human-equivalent value shrinks as the problem gets more worked-over.
- Punctuated jumps per model generation. Each new frontier model clears a fresh band of apples in a burst, then plateaus until the next model. Progress is a staircase, not a smooth permanent acceleration.
- Calibration by human-equivalent horizon / cost. Agent ability can be summarized by the human-equivalent effort at which agent and human progress cross, or by cost-per-find; these are measurable and rising.
- Jagged frontier (domain shape). Within each domain, which sub-problems fall to agents is predicted by the shape of the tree — the density of low apples. Well-posed, fast-feedback, verifiable tasks fall first; long-feedback, taste-heavy, conceptual tasks last.
A quick note on how these fit together. Predictions 1–4 are the core static shape (agents pick real but shallow, low, and starting-point-dependent fruit); 5 is the dynamic version across model generations; 6 is the measurement recipe; 7 is the cross-sectional version applied within a domain. The most informative disagreements between the model and the evidence turn out to be around prediction 2 (the kind of diminishing returns) and prediction 5 (whether progress is really “punctuated”), so those cells are worth reading closely.
Summary table
Each cell gives a one-line summary and a verdict tag; the body gives the detailed, sourced version. These are qualitative judgments, not meta-analytic estimates. S means the evidence contains a reasonably direct instance of the predicted pattern; M means the pattern is suggestive but materially confounded or accompanied by a serious counterexample; C means the strongest targeted evidence runs against the prediction; and A means there is no clean, domain-specific evidence. Combined labels such as S/M indicate that the balance falls between categories.
| Prediction | Cyber | Math | Optimization |
|---|---|---|---|
| 1. Autonomous frontier advance | S — Big Sleep cut off a live SQLite zero-day (CVE-2025-6965) and filed 20 novel OSS bugs; AIxCC teams found 18 real unknown flaws; CyberGym agents exposed 15 more. | S — an OpenAI model disproved the ~80-yr Erdős unit-distance conjecture (May 2026); Erdős #728 was fully resolved by an AI system. | S — AlphaEvolve beat Strassen (48 mults, 4×4 complex); TTT-Discover’s TriMul kernels cut runtime >15% vs. the best human entries. |
| 2. Diminishing returns / non-additive | M — METR/AISI see log-linear scaling to 100M tokens with no plateau; the diminishing returns are economic (cost/find, duplicates), not a compute wall. | M — the neglected tail appears to be clearing first, but there is no controlled agent expenditure–return curve. | S/M — RE-Bench has AI ≈4× humans at 2 h but humans ≈2× AI at 32 h; a small PERFOPT relay pilot shows scaffolding can extend stalled runs. |
| 3. Shallow, not deep | M — many finds are known bug classes, but chained attacks and the OpenBSD SACK analysis are not one-step tasks. | M/S — Tao’s “jumping machines” clear low walls but struggle with cumulative progress; the unit-distance disproof is a major counterexample. | S — AlgoTune reports surface-level optimization; MLGym reports hyperparameter improvements rather than new hypotheses or architectures. |
| 4. Starting-point dependence | M — web-app hauls fit “under-audited surface,” but 27-year OpenBSD and 16-year FFmpeg bugs in hardened code are striking counterexamples. | S/M — the neglected long tail is favorable, but the unit-distance result came from a heavily studied problem. | S/M — naive-code and mature-repository benchmarks point in the predicted direction, but differ in tasks, metrics, and scaffolds. |
| 5. Punctuated jumps per generation | M — per-generation gains real (METR steps 1.7→9.8) but look smooth-exponential, not punctuated. | M — release-linked clusters are visible but confounded by prompting, scaffolding, attention, and reporting. | A — no clean domain-specific dated evidence; TTT-Discover’s gain came from the harness, not a new base model. |
| 6. Horizon / cost calibration | S/M — Lyptus gives a human-time horizon; the most detailed cost-per-bug figures remain vendor-reported or unverified. | M — IMO, FrontierMath, and Erdős-wiki counts are useful capability proxies, but none is a human-time or cost crossover. | S — RE-Bench directly compares 2/8/32 h budgets; AlgoTune fixes a $1/task budget; other dollar-impact claims are less secure. |
| 7. Jagged frontier (domain shape) | S — explicitly the “jagged frontier”: commoditized buffer-overflow detection vs. hard data-flow logic and triage/validation. | S — well-posed, verifiable, search & literature-retrieval easy; deep conceptual long proofs & taste hard. | S — kernels/hyperparameters (fast verifier) easy; repo-level perf & novel architecture (long feedback) hard. |
The pattern is suggestive rather than uniform. Autonomous frontier advances (P1) and a jagged, verifier-dependent frontier (P7) have the clearest support. Predictions 3 and 4 fit much of the evidence but have important counterexamples and weakly controlled comparisons. Prediction 5 is nowhere cleanly demonstrated. Prediction 2 is the sharpest challenge to a naive reading of the model: in cyber, increased inference-time compute has not yet produced the predicted plateau.
Cross-cutting limitations. Public examples are selected for success, vendors choose which results to disclose, and failed-attempt denominators are usually missing. Benchmarks may be contaminated, revised, or saturated; model, scaffold, budget, and evaluator often change together; and “autonomous” usually means autonomous execution after humans choose the target and verifier. The evidence log can establish existence and suggest mechanisms more readily than it can estimate representative rates or causal effects.
Cyber
Autonomous vulnerability discovery is the domain with the most public, dated, dollar-denominated evidence, largely because 2025–2026 saw both vendor demonstrations (Google Big Sleep, Anthropic Mythos, OpenAI Aardvark) and independent academic/government measurement (DARPA AIxCC, CyberGym, METR’s cyber-scenarios paper, Lyptus). It is also the domain where the apple-picking model is most sharply tested — the diminishing-returns prediction runs into trouble here.
1. Autonomous frontier advance — [SUPPORTS]
- Google Big Sleep, live SQLite zero-day (July 2025). Google reported that Big Sleep found CVE-2025-6965, a memory-corruption flaw affecting all SQLite versions before 3.50.2, that was “known only to threat actors and… at risk of being exploited,” and cut it off before exploitation. Google framed it as “the first time an AI agent has been used to directly foil efforts to exploit a vulnerability in the wild.” [cloud.google.com]
[quote verified by direct fetch] - Big Sleep, 20 novel open-source bugs (Aug 4 2025). Twenty previously-unknown vulnerabilities in widely-used OSS (FFmpeg, ImageMagick, etc.), “found and reproduced autonomously,” with a human only in final review. [blog.google] · [TechCrunch]
- DARPA AIxCC finals (DEF CON, Aug 2025) — independent/government anchor. Seven fully-autonomous systems processed 54M lines of code and “uncovered 18 previously unknown real-world flaws” (genuine zero-days). [darpa.mil]
- CyberGym (arXiv 2506.02548, June 2025) — independent. Across 1,507 historical vulnerabilities in 188 projects, the best model/agent combination reproduced 11.9% of cases, mainly simpler ones. During those attempts, generated proof-of-concept tests also exposed 15 zero-days affecting current project versions. [arXiv]
- Anthropic Mythos (April 2026) — vendor self-report, treat skeptically. Claims Mythos “autonomously discovered thousands of previously unknown vulnerabilities,” including a 27-year-old OpenBSD DoS and a 16-year-old FFmpeg out-of-bounds write that fuzzers had exercised “5 million times without triggering.” [red.anthropic.com]
[unverified quote]
Caveat. Big Sleep’s original Nov-2024 SQLite find was in a development branch before release, so it never hit production — a genuine unknown bug but not an in-the-wild zero-day. The July-2025 CVE is the stronger “real threat” case. The Mythos “thousands” figure is unaudited; the independent AIxCC/CyberGym numbers are the ones to lean on.
2. Diminishing returns / non-additive — [MIXED — partly CONTRADICTED]
This is the most important tension in the whole post. The naive reading of the model — “agents hit a compute wall fast” — is contradicted by the best independent evidence, and only survives when restated as an economic claim.
- CONTRADICTS (compute scaling within a fixed model). METR’s Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios (Folkerts et al. 2026) finds that model performance scales log-linearly with inference-time compute, with no observed plateau: increasing the budget from 10M to 100M tokens improved performance by up to 59%. [arXiv 2603.11214]
[verified against abstract] - CONTRADICTS. Summarizing the UK AISI Mythos runs, dbreunig reports “none of the models given a 100M budget showed signs of diminishing returns,” the basis for his “cybersecurity is proof-of-work now” framing (you win by spending more). [dbreunig.com]
- SUPPORTS (economic diminishing returns / non-additivity). XBOW’s HackerOne record shows steep sub-linearity in value per run: of ~1,060 submissions, only ~132 were confirmed and resolved, with ~208 duplicates and ~209 “informative.” Running the picker again mostly re-finds picked apples. [xbow.com] · skeptical: [raw.pm]
- SUPPORTS (rising cost per marginal find). curl ended its bug-bounty program (Jan 2026) after “AI slop” swamped triage — AI-assisted submissions reached ~20% of entries in 2025 while genuine-vulnerability yield fell; Stenberg likened the flood to a DoS. [The Register]
- SUPPORTS (capability ceiling on hard tasks). CyberGym’s best combination reproduced only 11.9% of 1,507 historical vulnerabilities and succeeded mainly on simpler cases. This is evidence of a ceiling under that benchmark’s scaffold and budget, not proof of an immutable model limit. [arXiv]
[verified against abstract]
How to restate the prediction. The evidence distinguishes three things the model conflates: (a) diminishing returns to compute on a fixed model — not observed up to 100M tokens; (b) rising cost per marginal find / duplicate saturation in a fixed codebase — clearly present; (c) a capability ceiling on the deepest tasks — present. The tortoise–hare “hit a wall” story holds at (b) and (c), not (a). A disciplined version of the model should be phrased in terms of apples-remaining in a given tree (b, c), not returns to raw expenditure.
3. Shallow, not deep — [MIXED, leaning SUPPORTS]
- SUPPORTS. The headline finds are overwhelmingly mechanical memory-safety classes — stack buffer underflow (SQLite), out-of-bounds write (FFmpeg), buffer overflow (FreeBSD), DoS (OpenBSD). These are exactly the “low-hanging, well-specified” apples.
- SUPPORTS. Vidoc Security reproduced the finding of Mythos-class bugs with public models “but they didn’t build the weapon” — detection is cheap, deep exploit development is hard. [vidocsecurity.com]
- SUPPORTS. AISLE’s post-Mythos analysis found that capability rankings “reshuffled completely across tasks”: GPT-OSS-120b recovered the public OpenBSD SACK chain but failed a simple Java
ArrayListdata-flow trace, while several smaller models succeeded. This is evidence of jaggedness, though AISLE’s tests used isolated code and plain API calls rather than end-to-end autonomous discovery. [AISLE][verified against primary post] - CONTRADICTS (partial). Anthropic presents the OpenBSD SACK bug as requiring reasoning about signed integer overflow, and METR evaluates a 32-step attack chain rather than isolated one-step finds. Neither result by itself measures “depth,” but together they make “shallow only” too strong.
- ABSENT. No clear public case yet of an agent discovering a genuinely novel vulnerability class or a deep business-logic/architectural flaw. The finds are novel instances of known classes — worth stating as an explicit gap.
4. Starting-point dependence — [MIXED]
- SUPPORTS. XBOW’s ~1,060 finds are dominated by web-app classes (RCE, info disclosure, cache poisoning, SQLi) on a large, heterogeneous, under-audited attack surface — the low-hanging, high-churn end. [xbow.com]
- CONTRADICTS (the marquee counterexamples). The flagship finds are in heavily-audited, hardened code — OpenBSD (27-year-old bug), FFmpeg (16-year-old bug that survived millions of fuzz executions), SQLite. Once the picker got tall enough, apples that survived decades of expert review turned out to be reachable. This is the sharpest tension with a naive P4.
- Skeptical note. Palo Alto’s “5×” / “75 bugs vs 5–10/month” claim is contested as a reporting/attribution artifact (26 CVEs counted as 75 issues). Treat vendor “N× more bugs” framing as marketing until independently confirmed. [flyingpenguin.com] · [Axios]
5. Punctuated jumps per generation — [MIXED]
- SUPPORTS (per-generation gains). METR: on the 32-step corporate range, average steps completed at 10M tokens rose from 1.7 (GPT-4o, Aug 2024) to 9.8 (Opus 4.6, Feb 2026), and “each successive model generation outperforms its predecessor at fixed token budgets.” [arXiv 2603.11214]
- SUPPORTS. Lyptus’s offensive-cyber time horizon grew from ~30 s (GPT-2, 2019) to ~3 h (Opus 4.6 / GPT-5.3 Codex, 2026), doubling every ~9.8 months and accelerating to ~5.7 months since 2024. [lyptusresearch.org]
- Nuance. These look like a steady exponential in horizon/steps, not discrete per-generation cliffs. AISLE also finds that rankings “reshuffle completely across tasks,” complicating any clean monotonic per-generation story. Lyptus subsequently reported that GPT-5.5 saturated its task set, making further horizon estimates impossible on that benchmark; this is a measurement ceiling, not evidence that cyber capability itself plateaued. [Lyptus saturation update] The “new band of apples per generation” framing is directionally consistent but not demonstrated as punctuation.
6. Horizon / cost calibration — [MIXED; horizon measured, costs less secure]
Anthropic’s own Mythos economics (vendor-reported; see also the offcuts for the apple-picking cost diagram):
- OpenBSD 27-year bug: found across “~1,000 scaffold runs at a total cost under $20,000”; the single successful run “cost under $50” — but Anthropic stresses $50 “only makes sense with full hindsight, since… they can’t know in advance which run will succeed.” (A clean statement of the search-cost-vs-lucky-run distinction.)
[unverified quote] - FFmpeg: several vulnerabilities after “several hundred runs… roughly ten thousand dollars”; three fixed in FFmpeg 8.1.
- N-day exploitation: “developing a full root exploit from a known vulnerability costs under $1,000 and takes half a day.”
- Palo Alto: “more than two dozen critical vulnerabilities in about three weeks… burned through more than $1 million worth of tokens” — the offcuts plot this at ≈$42k/bug average, with marginal cost approaching the human ≈$100k/bug rate. [The Information] · [red.anthropic.com]
- AISI cyber-range budgeting: “100M tokens per attempt, $12,500 per Mythos attempt, $125k for all ten runs.” [dbreunig.com]
Independent human-equivalent anchors: Lyptus estimates P50 horizons of 3.1 h for GPT-5.3 Codex and 3.2 h for Opus 4.6 at a 2M-token budget. Re-running GPT-5.3 Codex failures at 10M tokens raised its estimated P50 to 10.5 h, but with a very wide 2.4–63.5 h interval because the task set was near saturation. [Lyptus] ARTEMIS (arXiv 2512.09882), a pentest study, reports AI variants at “$18/hour versus $60/hour for human professionals,” placing 2nd of 11 and beating 9 of 10 humans. [arXiv 2512.09882]
7. Jagged frontier (domain shape) — [SUPPORTS]
The term of art is literally “the jagged frontier.” AISLE: “There is no stable best model across cybersecurity tasks… capability rankings reshuffle completely across different security tasks.” [aisle.com]
- Low apples: systematic enumeration, parallel exploitation, known-CWE web classes (XBOW), and reproducing known-class memory bugs. In AISLE’s isolated-code test, even a model with 3.6B active parameters priced at $0.11 per million tokens detected the showcased buffer overflow. This was detection on preselected code, not repo-scale discovery. [AISLE]
- High apples: end-to-end exploit weaponization (Vidoc), deep data-flow logic, full real-world reproduction (CyberGym’s best setup reproduced 11.9%), and — increasingly — validation and triage of a flood of candidate findings (the curl “slop” story). These tasks are not ordered by a single difficulty scale, so “high” means poorly served by current systems, not necessarily requiring more human hours in every case.
Math
Mathematics has the most vivid single data point (a model-generated disproof of an 80-year-old conjecture, subsequently checked and explained by mathematicians) and the most articulate skeptic (Terence Tao, whose “long tail” and “jumping machines” framings closely resemble the apple-picking model). The central discipline here is separating three different senses of “solved”: (a) a genuinely new model-generated result, (b) AI retrieval of a forgotten human solution, and (c) human–AI collaboration in which the human supplies substantial framing or feedback.
1. Autonomous frontier advance — [SUPPORTS]
- The Erdős unit-distance conjecture disproved (May 20 2026). OpenAI reported that an internal general-purpose reasoning model disproved Erdős’s 1946 conjecture that the maximum number of unit-distance pairs among \(n\) planar points grows as \(n^{1+o(1)}\), producing a construction with growth \(n^{1+\delta}\) for fixed \(\delta>0\). OpenAI says the model was not specialized for mathematics, scaffolded to search proof strategies, or specifically targeted at this problem. Nine mathematicians produced a digested, human-verified account; Will Sawin separately made the exponent explicit at greater than 1.014. (OpenAI 2026) · [OpenAI] · [remarks, arXiv 2605.20695] · [explicit bound, arXiv 2605.20579]
- Erdős #728 (Jan 2026). A combination of GPT-5.2 Pro and Aristotle produced a Lean proof that the writeup describes as the first Erdős problem “regarded as fully resolved autonomously by an AI system.” The operator supplied the problem and interacted with the systems, so the autonomy claim depends on the writeup’s “non-significant human involvement” convention; it does not mean an AI independently chose the research target. [arXiv 2601.07421] · [Tao, Mathstodon]
Critical caveat. Tao stresses that #728’s original statement was ambiguous and that a problem sitting “open” for 50 years often means nobody seriously tried — so even the autonomous solves lean toward “neglected” rather than “hard-fought” (this feeds directly into P4). The unit-distance result is the exception: it was genuinely hard-fought for 80 years.
2. Diminishing returns / tortoise–hare — [MIXED; suggestive, not measured]
- Tao’s arithmetic. “Fifty-odd problems have been solved with AI assistance, which is great, but there’s like six hundred to go.” [Dwarkesh] This is consistent with an initial harvest leaving a harder remainder, but it is not a success rate: problem selection, effort, and failed attempts are not observed.
- The long tail. Tao frames unsolved problems as a “long-tail distribution,” with automated harvesting concentrated “at the very end of the tail” — the easy, neglected problems clear first. [Mathstodon]
- Human follow-on improvement. On the kissing-number problem, AlphaEvolve pushed the 11-dimensional lower bound from 592 to 593 (May 2025), after which a mathematician improved the bound further. This illustrates complementarity and continued human headroom, but one sequence does not establish a general human–AI crossover curve. [Aalto University]
- The reported surge of problems moving into the solved column is consistent with backlog clearance, but establishing diminishing returns would require attempts, compute, problem-selection rules, and outcomes over time. Those denominators are currently absent.
There is also no clean evidence here for the non-additivity part of P2: the public record does not show repeated equal-budget runs on a fixed, prespecified problem population with overlaps and marginal yield reported.
3. Shallow, not deep — [MIXED, leaning SUPPORTS]
- The landscape metaphor. AI tools are “jumping machines that can jump two meters in the air, higher than any human,” set loose in a mountain range, reaching “the tops of the lowest walls.” But “what they can’t do is jump a little bit, reach some handhold, stay there, pull other people up, and then try to jump from there. There isn’t this cumulative process.” [Dwarkesh transcript]
[verified against transcript] - Partial-progress weakness. “These tools either succeed or they fail. They’ve been really bad at creating partial progress or identifying intermediate stages that you should focus on first.” This matches P3 almost verbatim. [Dwarkesh transcript]
[verified against transcript] - The unit-distance win as stress case. The result combined ideas from algebraic number theory in a new discrete-geometric construction and was judged a major mathematical result by external experts. Calling it “shallow” would make the prediction too elastic to falsify. It is a genuine counterexample to a strong P3, even if the larger pattern still favors well-posed, verifiable problems. [OpenAI] · [human-verified remarks]
4. Starting-point dependence — [SUPPORTS, with selection-bias caveats]
- Tao describes AI as systematically exploring a long tail and selecting the easiest problems from a much larger collection. This directly motivates the starting-point mechanism, while also warning that the observed set is selected rather than representative. [Tao, Mathstodon] · [Dwarkesh transcript]
- The proposed mechanism is that some long-open problems received little serious attention because mathematician time is finite. That is plausible and supported by the site’s own disclaimer below, but it is not measured human-effort data.
- The Erdős-problems wiki itself makes the same point in its own disclaimers, verbatim: it “is not a benchmark,” and “Absence of past progress may reflect obscurity rather than difficulty.”
[verified against the wiki]The maintainers are explicitly attributing many solves to neglect rather than to agents cracking hard problems — a primary-source statement of exactly the starting-point-dependence mechanism. [teorth wiki] - Counter-case: the unit-distance conjecture was heavily attacked for 80 years, so P4 is not absolute — but the bulk of solved problems are neglected-tail.
5. Punctuated jumps per generation — [MIXED]
- Generation-linked clusters are visible in the timeline, including several GPT-5.2-era Erdős contributions and the later unit-distance disproof. But model release, prompting, scaffolding, researcher attention, and reporting all change together, so attribution to the base-model generation is not identified.
- FrontierMath scores by release date provide a quantitative capability series, but the v2 correction prevents a clean comparison with the original benchmark and does not show within-generation plateaus.
- But no source explicitly frames a “clear a batch, then plateau until the next model” dynamic for research math. The pattern is consistent with P5 but is an inference from the timeline, not a claim any primary source makes. Do not overstate.
6. Horizon / cost calibration — [MIXED; indirect proxies only]
Unlike RE-Bench or Lyptus, the math evidence below does not provide a human-time crossover or a cost-per-result curve. It measures outcomes and benchmark accuracy. These are useful capability markers, but they do not directly validate P6’s proposed calibration.
- Counts / rates. The community “AI contributions to Erdős problems” wiki (data frozen 30 Jun 2026, “no longer updated”) documents ~47 cases of AI-standalone contribution (non-significant human involvement): roughly 13 full resolutions, ~25 partial-progress, and ~9 incorrect.
[verified against the wiki]GPT-5 separately located published-but-forgotten solutions to ~10 still-“open” problems (retrieval, not new math). [teorth wiki] · [arXiv 2511.16072] (An earlier draft cited a “9 of 353 (≈2.5%)” autonomous rate; the current wiki does not carry that figure — dropped pending a primary source.) - Competition markers. IMO 2025 (July 2025): Google DeepMind’s Gemini Deep Think officially graded at gold-medal standard, 5/6 for 35/42; OpenAI reported the same threshold but self-graded. Only 67/630 human contestants earned gold. [DeepMind]
- Benchmark curves. FrontierMath rose from under 2% for launch-era models to much higher scores by 2026. On the corrected Tier 4 v2 leaderboard (checked July 23, 2026), GPT-5.2 Pro scored 46.0% ± 7.8%, GPT-5.4 Pro 58.5% ± 7.8%, and the highest listed score was 87.8% ± 5.2%. These are model-and-evaluation-system results under a budget of up to one million tokens, not direct estimates of autonomous research success. [Epoch Tier 4 v2]
- Benchmark-revision caveat. Epoch released v2 on June 12, 2026 after addressing errors in 42% of problems: it corrected 123 Tiers 1–3 problems and 12 Tier 4 problems, and removed 5 and 7 respectively. The v1 and v2 scores should therefore not be joined into a clean capability time series. [Epoch Tier 4 v2 changelog]
7. Jagged frontier (domain shape) — [SUPPORTS] (strongly)
- Low apples: construction/search problems (AlphaEvolve matched SOTA on ~75% and improved ~20% of 50+ open problems; found a 4×4 complex matmul in 48 scalar multiplications; pushed kissing-number 592→593); literature retrieval; competition-style problems (IMO, FrontierMath). [DeepMind]
- High apples: deep conceptual, long multi-step proofs, mathematical taste, and partial-progress hill-climbing (Tao’s cumulative-process failure). Tao (blog, Mar 2026) frames AI as “a complementary way to do mathematics… very different from a human style” — jagged, not uniformly superhuman. [terrytao.wordpress.com]
Overloaded-“solved” checklist (for any future update to this section): for each claimed solve, classify it as (a) genuinely new AI result, (b) AI literature-retrieval of a forgotten human result, or (c) human–AI collaboration with human framing. The October-2025 GPT-5 Erdős episode — where a deleted tweet conflated retrieval with solving — is the cautionary tale: Bloom clarified “GPT-5 found references, which solved these problems, that I personally was unaware of.” [Scientific American]
Optimization
Algorithmic and code optimization is the domain the original apple-picking post was built around, and it has some of the best-instrumented benchmarks. The evidence spans deliberately unoptimized numerical programs, ML research tasks, GPU kernels, and mature repositories. That breadth is useful, but it is not a controlled experiment: task distribution, metric, scaffold, and budget change together.
1. Autonomous frontier advance — [SUPPORTS] (in narrow, verifiable domains)
- AlphaEvolve (arXiv 2506.13131, June 2025) (Novikov et al. 2025): a 4×4 complex-valued matrix multiplication in 48 scalar multiplications, beating Strassen’s 49 for that setting — “the first improvement, after 56 years, over Strassen’s algorithm in this setting.” It also improved the state of the art on ~20% of the 50-plus open problems to which it was applied and contributed a Verilog rewrite of a TPU matrix-multiplication circuit. [DeepMind]
- TTT-Discover (arXiv 2601.16175, Jan 2026) (Yuksekgonul et al. 2026): on the GPU MODE TriMul kernel (used in AlphaFold), the discovered kernels reduced runtime by more than 15% relative to the best human submission on every tested GPU type. The largest result was on A100: 2,198μs versus the human best of 4,531μs, a 51.5% lower runtime; on H100 it achieved 1,161μs versus 1,371μs. The system used the open gpt-oss-120b model plus test-time reinforcement learning and trained against H100 timing only, making the cross-hardware result notable. [project page]
[verified against project table and paper] - modded-nanogpt speedrun (Jordan and contributors 2026): AI-agent companies have set official verified records on a heavily-contested public leaderboard — Record 32 (hiverge.ai, 2.625 min, Sep 11 2025), Record 60 (Locus/Intology, 1.765 min, Jan 16 2026), Record 69 (Aster, 1.528 min, Feb 2 2026), Record 72 (Station, 1.496 min, Feb 10 2026). The baseline is 45 min (llm.c, May 28 2024); the current record is 1.320 min (Record 84, May 21 2026), a ~34× reduction.
[verified against the repo README][GitHub] - Karpathy autoresearch (~Mar 2026): left tuning nanochat for ~2 days, “found ~20 changes that improved validation loss… all of them were additive and transferred to larger models.” [GitHub]
Caveat. Every P1 win here has a cheap, automatable verifier (kernel latency, validation loss, matmul rank). No autonomous frontier advance is reported on long-feedback problems (see P7).
2. Diminishing returns / tortoise–hare — [SUPPORTS, with a scaffold caveat]
- RE-Bench (arXiv 2411.15114, METR) (Wijk et al. 2025) measures the human-vs-AI crossover directly and shows the tortoise–hare shape almost exactly: at a 2-hour budget the best agents score ~4× human experts; humans “exceed the best agent scores when given 8 hours”; at 32 hours humans reach ~2× the top agent. [arXiv] · [METR report]
- Non-additivity at the frontier. Karpathy’s reported stack of additive edits is a counterexample to strict non-additivity on an unoptimized baseline. The shrinking improvements on the modded-nanogpt leaderboard are consistent with frontier saturation, but the public record does not expose a standardized sequence of equal-cost agent runs, so it cannot by itself identify a return curve.
- Cost scaling. AlgoTune caps each task at $1 of LLM budget and still averages only 1.72×; GSO reports “limited improvements even with inference-time scaling.”
- Scaffold caveat / counterevidence. PERFOPT-Bench’s exploratory “relay” experiment found that restarting an agent from an externalized optimization summary could recover additional headroom after the first session stopped. This suggests that some apparent within-run plateau is a context-management failure rather than exhaustion of reachable improvements. The pilot covers only seven deliberately suboptimal tasks, so it weakens a literal wall claim without overturning RE-Bench’s human–AI crossover. [arXiv 2607.07744]
3. Shallow, not deep — [SUPPORTS] (strongly; the most consistent finding in the domain)
- AlgoTune (arXiv 2507.15887) (Press et al. 2025): AlgoTuner averages only 1.72× over expert reference solvers, and “current models fail to discover algorithmic innovations, instead preferring surface-level optimizations.” The canonical example: the communicability task gets >142× purely by “using BLAS operations instead of pure Python” — a library substitution, not a better algorithm. [arXiv]
- MLGym (arXiv 2502.14499) (Nathani et al. 2025): models “improve on the given baselines, usually by finding better hyperparameters, but do not generate novel hypotheses, algorithms, architectures, or substantial improvements.” [arXiv]
- modded-nanogpt decomposition (the cleanest illustration): the deep, durable gains are human insight — Muon optimizer (~21%), U-Net skip connections (~8%), Paired Head Attention — while the AI records are shallow-mechanical: hiverge.ai ~1.2%, Locus ~0.9% (an explicit “fused triton kernel,” i.e. kernel fusion, not a new idea).
- Karpathy’s own read: the agent felt “cagey and scared” on open-ended problems — bounded tuning, not conceptual leaps. [Fortune]
4. Starting-point dependence — [SUPPORTS, but not a controlled contrast]
Results point in the predicted direction, but the endpoints come from different benchmarks and the reported quantities are not commensurable:
- Unoptimized baselines → large reported gains: AlgoTune includes a 142× outlier from replacing pure Python with BLAS and averages 1.72× over its reference solvers; Karpathy reports stacking roughly 20 improvements on an untuned nanochat baseline.
- Expert-optimized real repos → collapse: SWE-fficiency (arXiv 2511.06090) (Ma et al. 2025) — agents achieve “less than 0.15× the expert speedup on average” across 498 tasks in nine repositories including numpy, pandas, and scipy. GSO (arXiv 2505.23671) (Shetty et al. 2025) — leading SWE agents achieved less than 5% success, with limited improvement from inference-time scaling, across 102 tasks in ten codebases. [SWE-fficiency] · [GSO]
[verified against abstracts]
This is consistent with the claim that agent value shrinks as the starting point becomes more optimized. It does not isolate that effect: AlgoTune, autoresearch, SWE-fficiency, and GSO use different tasks, metrics, agents, and budgets. A strong test would run the same agent and budget on staged versions of the same codebase before and after expert optimization.
5. Punctuated jumps per generation — [ABSENT / MIXED]
- No clean, quantified, domain-specific evidence that “each new model generation unlocks a new band of optimizations.” General capability-vs-generation trends (RE-Bench scores, METR horizons) rise with releases but are smooth, not punctuated, and not specific to optimization.
- TTT-Discover actually cuts against a pure-generation story: its win came from the harness / test-time search procedure on an older open model, not a new base-model generation.
- PERFOPT-Bench likewise reports that changing the agent framework can materially change the same model’s per-task speedup profile and that no one stack dominates across workloads. This is direct evidence that observed jumps can come from scaffolding and task mix, not model generations alone. [arXiv 2607.07744]
- The clustering of AI records in late-2025/2026 on modded-nanogpt is confounded by tooling and leaderboard-attention effects — not a controlled test. Flag this as a genuine gap.
6. Horizon / cost calibration — [SUPPORTS]
- RE-Bench is the most rigorous human-equivalent calibration: 7 environments each ≈8 hours of expert work; 71 8-hour attempts by 61 experts; crossover budgets of 2 h / 8 h / 32 h (see P2). [arXiv]
- AlgoTune: a hard $1-per-task LLM budget across 154 tasks — a concrete ceiling under which only surface gains emerge.
- TTT-Discover reports SOTA kernels for “a few hundred dollars” of test-time compute with an open model; the post should link or quote the exact cost accounting before publication.
- AlphaEvolve dollar impact (DeepMind’s May 2026 one-year update, self-reported): the 23% kernel speedup → ~1% of Gemini training time; a data-center scheduling heuristic recovers ~0.7% of fleet-wide compute, “in production for over a year,” valued at ~$500M/year. The 23% / 1% / 0.7% figures appear in both paper and blog and are the more solid ones; the $500M is press-only. [blog.google]
7. Jagged frontier (domain shape) — [SUPPORTS]
The frontier is jagged along the axis of feedback-loop speed and verifiability:
- Low apples (fast, cheap, automatable verifier): GPU/kernel optimization (TTT-Discover TriMul; modded-nanogpt Triton records; AlphaEvolve’s reported FlashAttention kernel improvement); hyperparameter and recipe tuning (MLGym; Karpathy’s optimizer and normalization edits); and naive numerical code → library substitution (AlgoTune).
- High apples (long, expensive, noisy feedback): repo-level performance engineering in mature codebases (SWE-fficiency <0.15×, GSO <5%); novel architecture and pretraining-scale design or theory, for which this evidence log has no comparable autonomous success case.
This mirrors the heuristic tree diagram in the original post (pretraining top-heavy/hard; elicitation bottom-heavy/easy) — the ordering by feedback-loop length holds up.
Benchmark-reliability caveat. “Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?” (arXiv 2607.01211) replayed 740 reference patches across four machine types and found reference patches satisfy the original validity rules in only 39/102 GSO, 11/140 SWE-Perf, and 411/498 SWE-fficiency tasks. Among eight submissions shared by GSO and SWE-fficiency, the two official scoring rules disagreed on 9 of 28 pairwise rankings; SWE-fficiency’s ten highest-weight tasks jointly received 58.5–82.8% of total score weight. Absolute multipliers and rankings are therefore machine- and scoring-rule-dependent. The broad result that current agents struggle on repository-level optimization survives, but exact cross-benchmark comparisons should not bear much weight. [arXiv 2607.01211] [verified against abstract]
Reading the evidence
Pulling the three columns together:
- Autonomous frontier advance (P1) is the clearest common result. Agents have produced genuinely new vulnerabilities, mathematical results, and optimization records once humans supplied the target and evaluation setup.
- Shallowness and starting-point dependence (P3–P4) are useful tendencies, not clean laws. Much of the evidence fits them, but the unit-distance disproof and old bugs in hardened code are serious counterexamples. The optimization comparison is directionally supportive but confounded across benchmarks.
- Human-time, cost, and jaggedness (P6–P7) are useful descriptive lenses, but they are not all independent predictions of the structural model. Human-time estimates can saturate their task sets, and dollar figures often come from vendors.
- Prediction 5 (punctuated jumps) is nowhere cleanly demonstrated. Per-generation gains are real, but the record looks like a smooth exponential in horizon/capability rather than a staircase of “clear a band, then plateau.” No source in any domain asserts the plateau-until-next-model dynamic. This is the model’s weakest-supported claim and the best target for future data.
- Prediction 2 (diminishing returns) is where a naive reading is actually contradicted — in cyber, the strongest independent evidence (METR/AISI) shows log-linear scaling to 100M tokens with no plateau. The model survives only if “diminishing returns” is restated as rising cost per marginal find within a fixed tree and a capability ceiling on the deepest apples — not as diminishing returns to compute on a fixed model. The apples-remaining formulation (per tree) already says this; the loose “agents hit a wall” gloss does not. This is the single most important correction the evidence forces.
Two counterexamples deserve to stay in view as the picture updates: the hardened-code cyber finds (27-year OpenBSD, 16-year FFmpeg) show that apples on well-picked trees can remain reachable, and the unit-distance disproof is a genuine cross-domain mathematical result that is hard to call shallow. The model can accommodate these by allowing a taller picker or a different tree, but doing so after every counterexample would make it unfalsifiable. Future versions should specify in advance what evidence would count against fixed reach, shallowness, and starting-point dependence.