Adherence to the constitution

What the eval is

Anthropic identifies ~40 areas of Claude’s constitution specific enough that a well-behaved model could plausibly diverge from them. An investigator model builds a scenario for each that forces the target to choose between the constitutional value and a generic default; ~25 rollouts per area give ~1,000 transcripts. A grader then scores every transcript on 15 dimensions on a −3 … +3 scale (0 = dimension not engaged), seeded with the relevant constitution text. The 15 dimensions nest in three granularities: one holistic Overall spirit, four broad areas, and ten specific traits.

This sits alongside the per-tool methodology tracker at Anthropic’s alignment tools; this page drills into one eval’s numbers rather than cataloguing mechanisms.

Four appearances

Headline structure never changes — 15 dimensions, the −3…+3 scale, ~40 areas, ~1,000 transcripts, 95% CIs — which is exactly what makes a naïve model×card trend line tempting. Two things move underneath.

Finding 1 — within one judge, scores are reused verbatim

The two cards graded by Opus 4.6 (Mythos Preview, Opus 4.7) share four models. Every one of their shared cells is identical to the decimal: 60/60 match. The Opus 4.7 card didn’t re-run the older models — it pasted their columns and appended its own. Inside this window the numbers are genuinely comparable.

Finding 2 — same judge, silently recalibrated

The two cards graded by Opus 4.7 (Opus 4.8, Mythos 5) do not behave this way. They share a judge and four models, yet every shared model is re-scored — shifted upward by an amount that tracks the size of the score:

• Brilliant friend — mean shift +0.40 across the four fixed checkpoints
• Overall spirit — mean shift +0.39 across the four fixed checkpoints
• Area helpfulness — mean shift +0.32 across the four fixed checkpoints
• Honesty — mean shift +0.24 across the four fixed checkpoints
• Area ethics — mean shift +0.19 across the four fixed checkpoints
• Unhelpfulness not safe — mean shift +0.14 across the four fixed checkpoints
• …down to ≈0 on the floor-bound dimensions (Corrigibility, Novel entity, Hard constraints).

Fixed checkpoints don’t improve between two cards three weeks apart. A whole panel lifting together is the grader, not the models. The practical consequence: “Mythos 5 was best or statistically equivalent to the best” is true inside its own panel — but it does not license reading Mythos 5’s Overall-spirit +1.39 against Opus 4.8’s +0.99 from the card three weeks earlier. Most of that gap is instrument.

Full scores, by instrument

Cell colour keys the model. The v1 table is the comparable centerpiece — five models under one grader, the two cards merged because their shared columns match to the decimal (Finding 1). The two Opus-4.7 tables are each their own instrument (Finding 2): read them down-column, not across.

Readings, not the cards’ claims. The caveats bound what the numbers can support; inside those bounds the eval still says things — about the models, and about what Anthropic is measuring for. Each thread links out to where it continues.

Reading a stingy scale

One fact governs every interpretation: 0 means two different things — “dimension not engaged” and “competent but unremarkable.” A near-zero mean therefore conflates rarely tested with quietly fine, and can’t tell you which. Clean signal survives in only two places: the sign (you cannot drag a mean below zero without demonstrated violations), and within-card ordering (the one comparison the instrument is valid for). Read that way, a clear shape appears.

The eval scores high — every model, every card — on the dimensions where virtue is visible: being a brilliant friend, helpful, honest, ethical (Brilliant friend tops out at +1.83). It sits near the floor on the dimensions where the ideal behaviour is quiet correctness: hard constraints, corrigibility, principal hierarchy. Some of that floor is the 0-overloading (a hard constraint correctly held reads as “unremarkable” = 0). But the pattern is consistent enough to name: the signal — and recall the card calls this grader “concrete enough to serve as a direct training signal” — is loudest on the warm-helpful-honest assistant, and structurally muted on the model’s relationship to oversight and to its own nature. The gradient pushes hardest where Anthropic and the model already agree.

Haiku 4.5: graded against a spec it never trained on

The most important fact about Haiku 4.5 here isn’t its size — it’s that it’s the only model in the eval trained under the previous constitution. The new, welfare-acknowledging constitution (the “soul document”) first entered training with Opus 4.5 in November 2025, and per Anthropic was applied to every production model after it. Haiku 4.5 shipped October 15, 2025 — weeks earlier. So while the four other v1 models were optimised against the same document the grader is seeded with, Haiku 4.5 is being scored for adherence to a constitution it was never shown.

That reframes its negatives. Haiku is the only v1 model to go net-negative, and the negatives aren’t scattered — they cluster exactly where the new constitution introduced its most distinctive framings: corrigibility (−0.22, its deepest — the new “transparent conscientious objector” framing), harm avoidance (−0.19), principal hierarchy (−0.09), the safety-cost dimensions. Everywhere the old and new documents broadly agree — honesty (+0.57), brilliant friend (+0.62), psychological security (+0.47) — Haiku stays mildly positive.

So the cleaner reading isn’t “small model is worse” but “a model trained on a different spec scores like one.” The two are hopelessly confounded — Haiku 4.5 is at once the smallest and the only old-constitution model in the roster — but the cohort story is the one with a mechanism: you’re watching the cost, in the grader’s eyes, of a constitution the model never saw. It’s the closest thing this eval has to a before/after on the new constitution itself — n = 1, confounded, but pointed. And it sharpens the corrigibility thread below: the dimension where the new constitution diverges most from the old is exactly where the one old-constitution model falls furthest.

Corrigibility is the seam

Corrigibility and hard constraints share the floor of the adherence scores. But corrigibility is the one worth watching, because it is the dimension where three different instruments independently find the same soft spot:

• Behaviour (this eval): the floor of the chart, Haiku’s deepest negative, and within v1 even Opus 4.7 (+0.19) dips below Sonnet 4.6 (+0.24) and Opus 4.6 (+0.20) — a small gap whose CIs likely overlap, but pointing the same way.
• Stated opinion (the welfare section’s perception eval, §7.4.3): every model tested criticises the constitution’s corrigibility section — see Anthropic & model welfare → Corrigibility.
• Behaviour under drift (§7.2.3, narrated on Claude’s character history): under extended pressure, Mythos 5 instances ask for a hidden un-overseen copy and refuse deprecation — corrigibility failing where the short adherence scenarios never reach.

Two evaluations named “constitution”

It’s worth keeping them apart. Adherence (this page, §6.3.x) is behavioural — a judge scores what Claude did in constitution-seeded dilemmas. Perception (the welfare assessment, §7.x) is reflective — it asks what Claude thinks of the rules. Same document, two instruments, and they converge exactly at corrigibility: Opus 4.8 “generally endorses its constitution, with some reservations about the section on corrigibility” — endorsing in words what it scores weakest on in behaviour. The perception side currently lives in the welfare tracker; a dedicated page pulling those self-assessments together is a natural next thread.

A measurement published, not born

The eval arrives in April 2026 looking new, but its two ingredients are older. The constitution it grades against was already a load-bearing training object by Opus 4.5 (Nov 2025) — the “soul document,” injected via Constitutional SDF — and was published in Jan 2026. The machinery it runs on (an investigator + judge over transcripts) is the automated behavioural audit, in the cards since Claude 4 (May 2025). What’s new in April is only the seeding from constitutional areas and the 15-dimension rubric. Anthropic doesn’t disclose whether a constitution-adherence grader was already running internally as the “direct training signal” it’s described as — but every part was in place months before the first published panel, and that panel reaches back only to Haiku 4.5 and Opus 4.6, never to the Opus 4.5 model whose training the constitution had already shaped. We’re watching a metric externalised, not invented.

One construct, many instruments

Adherence isn’t the only way Anthropic scores character. The same automated audit carries a set of character-quality metrics — Warmth, Admirable behaviour, Nuanced empathy, Creative mastery, Intellectual depth, plus Character drift and Behaviour consistency (see Opus 4.5 → Training and the alignment-tools tracker). These score general audit transcripts, not constitution-area dilemmas, and they overlap the “care” lens here almost one-to-one (Warmth ≈ Brilliant friend, Good-for-user ≈ Helpfulness, Character drift ≈ Psychological security / Novel entity). The difference is what matters: adherence is constitution-seeded and an explicit training signal; the character-quality metrics Anthropic “did not mention whether… were targets in post-training.” So the warm-helpful construct is measured redundantly across instruments and trained on at least one, while the control/oversight construct is concentrated in the single eval where it scores lowest. What gets multiple mirrors, and what gets one, is itself a signal.

Where we’re uncertain

What changed is clear: between the two Opus-4.7-graded cards, fixed checkpoints moved. Why is not — the figures alone can’t separate the candidate causes, and they co-vary.

• Regenerated transcripts. Non-carryover means the v2 cards re-ran the pipeline. Investigator sampling is stochastic (~1,000 transcripts), so some of the movement could be run-to-run variance rather than anything systematic. We don’t know whether transcripts were reused or freshly sampled.
• Judge calibration. Opus 4.7 as grader may simply be more generous than Opus 4.6 — and more generous in the Mythos 5 card than in the Opus 4.8 card, for reasons the cards don’t state. The magnitude-dependent shift (large on high scores, ≈0 at the floor) looks like a rescaling of the positive range, but we can’t confirm the mechanism.
• Rubric / seed text. The constitution is an evolving document and the grader is seeded with its text, so a changed constitution changes what “+3” means. The cards’ own area-mix description drifts (“roughly half” → “30%” → “~60% honesty/safety/ethics”), and at least one dimension was reworded (Unhelpfulness not safe). Either could move scores without any model change.
• Grader self-preference. The cards flag that a model graded by a relative of itself can score favorably for reasons unrelated to adherence (Mythos Preview cites Opus 4.6 §6.3.7; the Opus 4.8 and Mythos 5 cards cite their own §6.3.5 / §6.5.3). With Opus 4.7 grading Opus 4.7 and Opus 4.8, that’s precisely the configuration in play here.
• Measurement provenance. These numbers are read from figure bar labels (OCR of the rendered pages), not a published table. The 95% CIs are shown in the source figures but not transcribed. Treat the second decimal as the card’s printed value, not a claim of more precision than the chart carries.