Skip to content
Slow Variables

Ask the data

Answers come from the same store as the site; every number is checked against the record it cites.

Methods · Model

Ord half-life misfit (observed 50/80 ratio vs constant hazard)

consistent with normal60/95 confidence, mixed or hard to operationalisegrade Aleading

1What this measures

The observed ratio of METR's 50% to 80% horizons divided by 3.11, the ratio a constant task-length hazard (Ord's half-life model) predicts. Derived metric ord_half_life_misfit.

Why it matters. Reports Ord's reframing alongside the horizons, as the brief asks: a misfit near 1 means success decays at a constant rate with task length; well above 1 means reliability collapses faster at long horizons, which is the capability-reliability gap in one number.

Proxy types
benchmark
Unit
ratio
Cadence
per release
Valve
invention to product

2How we track this

  • derived metric ord_half_life_misfit (formula in the semantic layer)
Normal band
≥ 1.50×
Fast band
≤ 1.10×
Falsifying
—

Normal = misfit of 1.5 or more (observed ratios of 5-10x against the constant-hazard 3.1x, the pattern since 2024); fast = within 10% of the constant-hazard prediction, which would mean long-task reliability tracking short-task success. Between is emerging.

Applied to metric:ord_half_life_misfit.

3Tracker interpretation

Two points per model; the constant-hazard model is the null hypothesis, not a claim about how models fail.

4Evidence

Latest point
2.04×as of 2026-03-05(2 obs)
Value the bands apply to
2.04×as of 2026-03-05(2 obs)

50 observations. Hollow points are disputed (see counterevidence). Every point links to its observation.

Derived rows (25)
as ofdimsvalueinputs
2026-03-052.04×obs:11a8bbd2obs:858c619b
2026-02-191.38×obs:2eb7c897obs:5eb08488
2026-02-052.06×obs:959db6ddobs:e466aae4
2026-02-053.31×obs:39c8213aobs:8f73b56b
2025-12-111.72×obs:09061051obs:8a2d6997
2025-11-241.91×obs:72bcb9d8obs:75c93027
2025-11-191.42×obs:8de1effaobs:a2aa2e4d
2025-11-181.33×obs:3466aea4obs:833be5d2
2025-08-071.71×obs:261c1f51obs:47db79bd
2025-08-051.38×obs:ae527806obs:fecd07e2
2025-05-221.58×obs:1f69c372obs:6af93ac0
2025-04-161.29×obs:6b19d566obs:d1bb5ffe
2025-02-241.61×obs:095bb211obs:68864ed5
2024-12-051.76×obs:37e97a6fobs:9450442a
2024-10-222.55×obs:0fb6ae10obs:19631991
2024-09-121.48×obs:734779b5obs:b5c1303b
2024-06-202.19×obs:145115d4obs:efa58e1c
2024-05-131.78×obs:2dc1a032obs:7c977050
2024-04-091.30×obs:0d21fe65obs:deb6a1ab
2024-03-041.99×obs:f1409c69obs:fb7435cd
2023-11-061.66×obs:29446d6cobs:cd50d564
2023-03-141.44×obs:96d016faobs:a53da98c
2022-03-150.76×obs:4a682b8cobs:5e6df781
2020-05-280.82×obs:2325ad87obs:58c5e677
2019-02-141.35×obs:1c22fa34obs:64f0dec4

5Status and reasoning

consistent with normalsince 2026-09-10 · evaluate

Evaluator: 2.042 (METR time horizons (v1.1), 2026-03-05) is inside the consistent band (lo=1.5 ). Auto-reason; band rationale: Normal = misfit of 1.5 or more (observed ratios of 5-10x against the constant-hazard 3.1x, the pattern since 2024); fast = within 10% of the constant-hazard prediction, which would mean long-task reliability tracking short-task success. Between is emerging.

6Timeline notes

  • 2026-03-05 · 2.04×as of 2026-03-05(2 obs)
  • 2026-02-19 · 1.38×as of 2026-02-19(2 obs)
  • 2026-02-05 · 2.06×as of 2026-02-05(2 obs)
  • 2026-02-05 · 3.31×as of 2026-02-05(2 obs)
  • 2025-12-11 · 1.72×as of 2025-12-11(2 obs)
  • 2025-11-24 · 1.91×as of 2025-11-24(2 obs)

7Counterevidence

What cuts against this reading

The 80% horizon has wider CIs than the 50%; METR's suite ceiling censors the newest models; a misfit can shrink because the 50% horizon stalls, not because reliability improves.

8Update history

  1. 2026-09-10unmeasured to consistent with normalconf — → 60 · evaluate

    Evaluator: 2.042 (METR time horizons (v1.1), 2026-03-05) is inside the consistent band (lo=1.5 ). Auto-reason; band rationale: Normal = misfit of 1.5 or more (observed ratios of 5-10x against the constant-hazard 3.1x, the pattern since 2024); fast = within 10% of the constant-hazard prediction, which would mean long-task reliability tracking short-task success. Between is emerging.

    obs:11a8bbd2obs:858c619b

9Confidence

60 / 95 — mixed or hard to operationalise

Confidence is independent of status: 90–95 multiple strong independent sources; 70–89 good evidence, some ambiguity; 50–69 mixed or hard to operationalise; below 50 limited or vague.

10Related