Fresh issue resolution

Area: definition. Apply the shared measurement protocol.

Estimated progress toward AI-managed software

20% complete — judgment-based estimate as of November 7, 2026. Confidence: low. The scale starts at 0% before ChatGPT's public release in 2022 and ends at 100% when AI manages the field's continuing advancement without human participation. This rates the outcome's contribution to that transition, not the percentage of tasks solved, a measured fraction of the field, or time remaining.

The shared scoring rubric uses five equally weighted dimensions: execution, breadth, agenda-setting, validation, and sustained autonomous improvement. Points for this outcome are 5 + 5 + 5 + 5 + 0 = 20/100, in that order; each dimension has a maximum of 20. These component judgments make the estimate inspectable, but are not independently measured scores.

Reason for the estimate: An updated issue-evaluation framework is useful, but no verified score meeting the stricter freshness and maintainer-acceptance rule is available in the baseline. The dated baseline below supplies the findings and source links used for this assessment. Unmeasured components receive no demonstrated-progress credit; missing evidence does not prove a missing capability.

What would move the score: Archive a dated leaderboard export and issue creation dates; evaluate unseen issues with hidden regression tests and independent maintainer acceptance. Higher autonomy scores additionally require AI to select worthwhile work, validate results and improve the process over repeated cycles without human research management, repair or adjudication. Lower the score if replication fails, quality regresses or previously hidden human assistance is found. Preserve this dated estimate when adding future assessments.

Measurement and acceptance rule

Accepted fixes / all assigned issues created after the evaluation cutoff, using hidden functional tests, regression tests, and maintainer review. Report by language and repository.

Evidence requirements

Archive specifications, initial repository states, patches, independent tests, run logs, reviewer decisions, and follow-up defects. Reject test deletion, weakened acceptance criteria, hard-coded hidden answers, and fixes that pass narrow tests while violating the specification. Evaluate the tests themselves. Separate public benchmark results from private prospective evaluations.

Related milestones

See the area's proposed milestones. Proposed targets are distinct from the dated evidence below.

Baseline as of November 7 2026

Status: Current evaluation located; qualifying score not verified. Sources accessed November 7, 2026. This is a documentary baseline, not an independent replication or an assertion of the worldwide best result. An unknown measurement is not zero.

Dated findings

September 22, 2026: SWE-Bench Pro Public V2 refreshed its public split in September 2026 with a locked evaluation setup. Scale: SWE-Bench Pro Public V2.

Interpretation and tracking value

The accessible source did not provide a verifiable current score for this project's stricter post-cutoff, hidden-test and maintainer-accepted metric. A refreshed public split alone does not prove training exclusion.

Next comparable measurement

Archive a dated leaderboard export and issue creation dates; evaluate unseen issues with hidden regression tests and independent maintainer acceptance. Preserve this dated observation; append future results with their model/system version, denominator, budget and evidence level rather than overwriting it.