---
title: Anthropic and pretraining
---

Canonical URL: https://ghost.fail/anthropic/pretraining

# Anthropic and pretraining

## Claude models

### Pretraining data cutoff dates

See the [fog](/fog) app for an interactive view of models' epistemic horizons.

(table)

### Methods

- Decontamination

## Methods

- Modular Pretraining (GRAM)

### Data filtering

- Classifier-based data filtering[^ptdatafilter]
- Canary strings

{/* TODO gather list of all anthropic documents containing canary strings */}

## External links

- https://www.google.com/search?q=%22continued+pretraining%22+site%3Aanthropic.com

### Discussion

- [Anthropic's focus on hyperstition](https://www.lesswrong.com/posts/xhpktBLttPc6uXcHP/anthropic-s-strange-fixation-on-hyperstition) (Simon Lermen, May 11, 2026)

[^influencefunctions]: [Tracing Model Outputs to the Training Data](https://www.anthropic.com/research/influence-functions) (Aug 8, 2023)
[^ptdatafilter]: [Enhancing Model Safety through Pretraining Data Filtering](https://alignment.anthropic.com/2025/pretraining-data-filtering/) (Aug 19, 2025)
[^emrewardhack]: [From shortcuts to sabotage: natural emergent misalignment from reward hacking](https://www.anthropic.com/research/emergent-misalignment-reward-hacking) (Nov 21, 2025)
[^gram]: [Modular Pretraining Enables Access Control](https://alignment.anthropic.com/2026/modular-pretraining/) (Jul 8, 2026)