Forecast skill

Area: definition. Apply the shared measurement protocol.

Estimated progress toward AI-managed weather

45% complete — judgment-based estimate as of November 7, 2026. Confidence: low. The scale starts at 0% before ChatGPT's public release in 2022 and ends at 100% when AI manages the field's continuing advancement without human participation. This rates the outcome's contribution to that transition, not the percentage of tasks solved, a measured fraction of the field, or time remaining.

The shared scoring rubric uses five equally weighted dimensions: execution, breadth, agenda-setting, validation, and sustained autonomous improvement. Points for this outcome are 15 + 10 + 5 + 10 + 5 = 45/100, in that order; each dimension has a maximum of 20. These component judgments make the estimate inspectable, but are not independently measured scores.

Reason for the estimate: Broad historical skill evaluation and an operational model lineage support substantial bounded progress. Human-designed verification, initialization and research agendas remain necessary. The dated baseline below supplies the findings and source links used for this assessment. Unmeasured components receive no demonstrated-progress credit; missing evidence does not prove a missing capability.

What would move the score: Archive CRPS/Brier/error tables by variable, region and lead time for contemporaneous systems; preserve initialization, resolution, ensemble size and uncertainty. Higher autonomy scores additionally require AI to select worthwhile work, validate results and improve the process over repeated cycles without human research management, repair or adjudication. Lower the score if replication fails, quality regresses or previously hidden human assistance is found. Preserve this dated estimate when adding future assessments.

Measurement and acceptance rule

Continuous ranked probability score for forecast distributions, Brier score for specified events, and appropriate deterministic error metrics, by variable, region, and lead time.

Evidence requirements

Archive forecasts before outcomes occur. Use the same information availability, issue times, spatial resolution, and observation reference for comparators. Prespecify which events and variables will be scored to prevent selective reporting. Historical hindcasts support evaluation but cannot directly verify future century-scale projections. Distinguish better warnings from realized reductions in harm, which also depend on response systems.

Related milestones

See the area's proposed milestones. Proposed targets are distinct from the dated evidence below.

Baseline as of November 7 2026

Status: Measured historical benchmark; current operational version identified. Sources accessed November 7, 2026. This is a documentary baseline, not an independent replication or an assertion of the worldwide best result. An unknown measurement is not zero.

Dated findings

December 4, 2024: GenCast's published evaluation outperformed ECMWF ENS on 97.2% of 1,320 evaluated targets. GenCast, Nature.

May 12, 2026 deployment: ECMWF records operational AIFS Single v2 and Ensemble v2 from May 12, 2026. The historical GenCast score is not their score. ECMWF AIFS version history.

Interpretation and tracking value

This is the fraction of target metrics improved, not forecast accuracy. Historical ENS and evaluation periods must not be confused with November 2026 operational comparisons.

Next comparable measurement

Archive CRPS/Brier/error tables by variable, region and lead time for contemporaneous systems; preserve initialization, resolution, ensemble size and uncertainty. Preserve this dated observation; append future results with their model/system version, denominator, budget and evidence level rather than overwriting it.