Research library
AI research
View the four-date completion graphs and reconstruction notes.
Promise and evidence
AI that improves AI research is the most direct test of the singularity's proposed feedback mechanism. It could discover more effective learning methods, improve experiments, and reduce the resources required for subsequent advances. RE-Bench compares agents and human experts on open-ended machine-learning research engineering tasks; it does not by itself demonstrate autonomous recursive improvement.
Apply the shared measurement protocol. Proposed targets are not claims of achieved self-improvement.
Dated baseline coverage
All linked measurable-outcome files below contain an evidence baseline reviewed November 7, 2026, with source dates, findings, limitations and follow-up measurements. These distinguish measured results from proxies and unknown values; they do not assert that proposed milestones are achieved. See the baseline coverage notes.
Progress estimates
Judgment-based estimates as of November 7, 2026, using the same 0–100% scale and five-part rubric as mathematics. 0% anchors the period before ChatGPT's public release in 2022; 100% requires AI to manage this field's continuing advancement without human participation. Confidence is low. These are evidence-based judgments, not empirical completion measurements, forecasts or percentages of jobs replaced.
| Measurable outcome | Estimated progress | Execution / breadth / agenda / validation / improvement (each out of 20) |
|---|---|---|
| Acceleration | 30% | 10 / 5 / 5 / 5 / 5 |
| Algorithmic efficiency | 40% | 15 / 5 / 5 / 10 / 5 |
| Experiment efficiency | 25% | 5 / 5 / 5 / 5 / 5 |
| Research autonomy | 30% | 10 / 5 / 5 / 5 / 5 |
| Research task performance | 40% | 10 / 10 / 5 / 10 / 5 |
| Successive improvement | 25% | 5 / 5 / 5 / 5 / 5 |
| Valid discoveries | 30% | 10 / 5 / 5 / 5 / 5 |
Each linked outcome explains its score, cites its dated evidence baseline and identifies the next measurement. Zero indicates no qualifying post-anchor progress established by the reviewed evidence for that outcome, not proof of no relevant capability. Do not average these overlapping indicators into a field-wide percentage or interpret their differences as a precise ranking of fields. All outcomes reaching 100% would still require a separate field-wide demonstration of sustained autonomous advancement.
Measurable features
| Feature | Measurement and acceptance rule |
|---|---|
| Research task performance | Held-out objective improvement on fresh research tasks at fixed time, compute, and data budgets, relative to expert researchers and automated-search baselines. |
| Algorithmic efficiency | Compute, energy, and total dollars needed to reach a fixed independently evaluated capability target. Report training and inference separately. |
| Valid discoveries | Fraction of AI-proposed improvements reproduced by an independent team across at least three seeds and two model sizes or settings. |
| Experiment efficiency | Validated improvements per 100 experiments and per total dollar; include failed runs and researcher support. |
| Research autonomy | Human hours and interventions from research question through implementation, evaluation, and reproducibility package. |
| Successive improvement | Change in the next system's ability to perform fresh AI research tasks caused by the previous system's contributions. Measure each generation, not just the final model. |
| Acceleration | Elapsed time and resources per comparable validated capability increment across generations; report whether research gains persist, flatten, or regress. |
Evidence requirements
Freeze private evaluations outside the research agent's control. Publish ablations comparing the same development pipeline with and without the AI-produced change. Control for additional compute, new data, expert interventions, and evaluator changes. Distinguish improving a narrow kernel from improving general research ability. Do not allow an agent to redefine its own success metric.
Proposed milestones
Independently reproduce one AI-originated improvement yielding at least 10% less compute to reach unchanged held-out quality, across three seeds.
Reproduce improvements in three distinct research task families, with at least 25% less total human effort than the matched baseline.
Run three consecutive development generations in which each successor demonstrates better fresh-task AI research performance attributable to its predecessor's contributions. Report uncertainty and full resource changes.
Replicate the multi-generation effect in a second organization. Sustained research feedback is stronger evidence than isolated benchmark gains, but three generations alone cannot prove an unlimited intelligence explosion.