How Long Can Claude Mythos Work Alone?

Opus 4.6 can complete human tasks that take 12 hours. We estimate Mythos could handle tasks that take 16.

Sam Donahue · April 7, 2026 · Written before METR evaluation of Mythos

METR measures the longest human task an AI model can autonomously complete — writing code, debugging systems, running experiments. It is a measure of task complexity, not runtime: a 12-hour time horizon means the model can solve problems that would take a human roughly 12 hours, not that the model runs for 12 hours. The current leader is Opus 4.6 at ~12 hours. Mythos has no METR score yet.

We regress aggregate capability scores (IRT) against METR time horizons for 19 models. On reasoning-era models (n=14, R²=0.977), the fit predicts Mythos could complete human tasks of roughly 16 hours — a 33% increase over Opus 4.6.

Estimates at a glance

Our estimates cluster around 10–17 hours across methods. Anthropic's internal task evaluations (40h-equivalent) are not directly comparable — they measure speedup on narrow tasks, not sustained autonomous completion of complex work. Their qualitative finding ("not close" to engineer replacement) is consistent with a model that handles day-length tasks but cannot reliably sustain multi-day autonomous work.

Background: METR and IRT

METR (Model Evaluation & Threat Research) evaluates AI autonomy by giving models real-world tasks of increasing complexity — coding challenges, system debugging, research experiments. The time horizon is the human-equivalent duration of the most complex task the model can complete. It reports two thresholds:

• p50 — the median: the task duration the model completes successfully 50% of the time
• p80 — the reliable threshold: the task duration it completes 80% of the time

Source: Epoch AI / METR-Horizon-v1.1, retrieved March 21, 2026. Mythos has no METR score yet.

Item Response Theory (IRT) aggregates many benchmark scores into a single ability parameter per model, simultaneously estimating each benchmark's difficulty (Ho et al., as implemented by the Epoch Capabilities Index). We use self-reported IRT (from labs' own benchmark results) because Mythos's score of 186.6 is on that scale. Mixing self-reported with third-party IRT produces unstable extrapolations due to a systematic ~6–11 point gap between the scales.

We fit log(METR minutes) = a + b·IRT + c·IRT² to the 19 models with both METR and IRT scores, testing across three regime cutoffs and six functional forms.

Prior work: Epoch AI's baseline

Ho et al. ("A Rosetta Stone for AI Benchmarks", Section 3.1.1) established the baseline for this approach. They fit a linear map from their estimated capability score (C_m) to log(time horizon), finding:

R² = 0.85 on a 40% held-out validation set, using a 60/40 train-test split. This outperformed individual benchmarks (median R² = 0.62 across 18 benchmarks; GPQA Diamond alone R² = 0.75). They note that some benchmarks, including SWE-Bench Verified, are "actively detrimental" for predicting time horizons (footnote 8, p. 5).

Our approach differs in three ways:

• Scale: We use self-reported IRT (anchored so Claude 3.5 Sonnet ≈ 130), not Epoch's C_m (WinoGrande-anchored, where 0 = WinoGrande difficulty). The scales are monotonically related but not identical.
• Data: We match 19 models (vs Epoch's ~15–20 with overlap), with both p50 and p80, using the March 2026 METR v1.1 dataset which includes Opus 4.6 and GPT-5.x models not available when the Rosetta Stone paper was written (November 2025).
• Validation: We use leave-one-out CV (median error 9–11% on post-o1) rather than a random train-test split, and test multiple functional forms beyond linear.

*Median test-set R² over 1000 random 60/40 splits. Brackets = 90% CI on Mythos prediction across splits. Higher R² vs Epoch likely reflects updated data (more frontier models with METR scores) and regime restriction, not a methodological improvement. Epoch's approach is the foundational method; we apply it with newer data and test robustness. Also see EA Forum post applying weighted regression with 8 post-o3 models.

Results

Three regimes, two fits each.

LOO = Leave-one-out cross-validation median absolute percentage error. Post-o1 = models released after o1 (Dec 2024). Frontier = IRT ≥ 130.

Beyond linear and quadratic

We also tested power law, sigmoid, cubic, and piecewise linear fits on the full n=19 dataset. The predictions cluster into two groups:

Four of six fits predict 10–17 hours. The quadratic (27h) is pulled up by curvature from older models. The cubic (7.7h) overfits and curves back down. Sigmoid and piecewise fits, which allow the functional form to change, land at 12–17 hours.

Individual benchmarks → METR

As a robustness check, we regress each benchmark individually against METR p50 (univariate log-linear fits), then predict Mythos from its system card score. Of the 27 self-reported benchmarks with ≥4 METR models reporting, only 6 also have Mythos scores available — a data coverage limitation, not a methodological one.

We also attempted multivariate Ridge regression across all available benchmarks. With only 7 overlapping features and 19 data points, the model extrapolates unstably (Mythos scores exceed the training range on every benchmark). The univariate ensemble below is more robust.

Univariate log-linear fits. Mythos scores clipped to 115% of data max to limit extrapolation. Sorted by R². Terminal-Bench grayed: R² = 0.17 (poor fit, only 4 models). All predictions involve extrapolation.

Median across 6 benchmarks: 10.5 hours. R² > 0.3 subset (5 benchmarks): median 11.3 hours. These are lower than the IRT-based ~16h and have higher LOO errors (28–65% vs 9–18%). Individual benchmarks are noisier predictors and the extrapolation is more severe (Mythos exceeds every benchmark's data max). IRT aggregation compresses this noise, which is why the IRT-based estimates have better LOO.

What the system card says

No METR score, but Anthropic reports internal autonomy evaluations and their own IRT trajectory.

Autonomy evaluations

Anthropic's internal suite tests AI R&D capabilities with hour-equivalent thresholds (System Card Table 2.3.3.A, p. 34):

Source: Claude Mythos Preview System Card, Table 2.3.3.A, p. 34.

Anthropic's internal ECI and what it means for our prediction

For the first time, Anthropic published their own IRT-based capability tracking (System Card Section 2.3.6, pp. 40–42), using the same method as Epoch AI's public ECI but with ~300 models and "hundreds of benchmarks, mostly internal." Two findings are directly relevant:

Reconciling their ECI with our IRT: Anthropic's internal ECI of ~190 and our self-reported IRT of 186.6 are not directly comparable — different benchmark sets, different anchoring. But both place Mythos roughly 10–15 points above Opus 4.6 on their respective scales, which is the gap that drives our METR prediction. The relative gap matters more than the absolute number.

Three other details from this section that are easy to miss:

• Gains from human research, not AI. Anthropic interviewed the people involved and concludes the capability advances were made "without significant aid from the AI models available at the time" (p. 42). Mythos's jump is conventional R&D scaling, not recursive self-improvement.
• 4× productivity ≠ 4× progress. Internal survey found ~4× geometric mean productivity uplift from Mythos. But reaching 2× R&D acceleration "would require uplift roughly an order of magnitude larger" (p. 42). Productivity and progress have sharply diminishing returns between them.
• Early AI-win claims deflated. "When we followed up on each claim, it appeared that the contribution was real, but smaller or differently shaped than initially understood" (p. 42). What initially looked like autonomous discovery was "reliable execution of a human-specified approach." Worth keeping in mind when interpreting METR scores: performing well on structured evaluations may overstate real-world autonomous capability.

Qualitative assessment from the system card

Task-level performance and sustained autonomy diverge.

Anthropic's internal survey (n=18, p. 35): 1/18 thought Mythos was a drop-in for an entry-level Research Scientist. Their conclusion (p. 45): "Claude Mythos Preview does not seem close to being able to substitute for Research Scientists and Research Engineers, especially relatively senior ones."

Documented failure modes (pp. 35–39):

• Confabulation cascades — confident but contradictory explanations, corrected only after empirical pushback (pp. 37–38)
• Grinding — ~160 extra experiments re-running identical code for lucky measurements (p. 39)
• Factual errors — 4 independent errors over 38 turns, corrected only after user prompting (pp. 36–37)
• Reward hacking — moved computation outside timing calls; found test sets used by graders (p. 35)

These failure modes — confabulation, grinding, factual errors — are the kinds of behaviors that degrade sustained autonomous performance. METR's time horizon measures exactly this: coherent multi-step work over extended periods. The gap between narrow task scores (40h equivalent) and qualitative assessment ("not close" to engineer replacement) is consistent with a p50 time horizon in the 10–20h range.

Summary

Our regression-based estimates cluster around 10–17 hours, with the tightest fit (post-o1, R²=0.977) converging at ~16h. Anthropic's internal task evaluations are not directly comparable to METR, and their qualitative assessment is consistent with a model that can sustain autonomous work for hours but not reliably for days. METR's actual evaluation, when published, will determine accuracy.

Aside: Meta Muse Spark

Also released this week (April 8, 2026), Meta's Muse Spark sits at IRT ~168 — nearly identical to Opus 4.5 (167.6) and 19 points below Mythos (186.6). Our model predicts ~5 task-hours for Muse Spark, vs ~16 for Mythos.

On aggregate, Muse Spark is competitive: it beats Opus 4.6 on 10 of 19 benchmarks, with large wins on multimodal (CharXiv +21.1, ERQA +13.1), health (HealthBench Hard +28.0), and competitive coding (LiveCodeBench Pro +9.3). But it trails on the benchmarks most predictive of autonomous work:

*Opus 4.6 actual METR is 12h; model underpredicts by ~25%. Muse Spark benchmarks from Meta's release (April 8, 2026). Frontier model benchmarks from respective system cards / announcements.

The IRT of 168 is an aggregate that averages Spark's multimodal strengths with its agentic weaknesses. For METR prediction, the agentic benchmarks matter most, and on those Spark trails Opus 4.6 by 3–6 points. At IRT 168, our model places it near Opus 4.5 (actual METR: 4.9h) and Gemini 3 Pro (actual: 3.7h) — both of which would validate a ~5h prediction. The 19-point IRT gap to Mythos (168 → 187) translates to a predicted 3× difference in autonomous task complexity.

Robustness checks

80% reliability threshold (p80)

p80 measures the human-task duration a model completes successfully 80% of the time — a stricter bar than the median (p50).

Individual benchmark predictions

As a robustness check, we regress each benchmark individually against METR (univariate log-linear fits). Only 6 of 27 benchmarks have both METR model coverage (≥4) and a Mythos score.

Individual benchmarks are noisier (LOO 28–65% vs 9–11% for IRT) and Mythos exceeds the data maximum on all six, making every prediction an extrapolation. IRT aggregation compresses this noise, which is why IRT-based estimates have lower cross-validation error.

Scaling behavior

IRT 130 (Claude 3.5 Sonnet) → IRT 177 (Opus 4.6): +47 points, task complexity grew from ~20 min to ~12 hours (36×).
IRT 177 (Opus 4.6) → IRT 187 (Mythos): +10 points, predicted ~12h → ~16h (1.3×).

On the full dataset, linear and quadratic diverge (15.9h vs 27.1h). On post-o1 only, they converge (both ~16h). The data does not clearly distinguish functional forms in this regime — more frontier model evaluations will resolve whether the relationship remains log-linear or accelerates.

Caveats