16 million Claude exchanges: anatomy of industrial distillation attacks
Anthropic reported 24,000 accounts and 16 million exchanges. Learn model extraction, detection signals, limits, watermarking, risks and legitimate distillation.
- AUTHOR
- Karol Rapacz / CEO of Breachroad · OSCP · PNPT
- PUBLISHED
- 23 February 2026
- READING TIME
- 17 min read
- TOPIC
- AI Security
On 23 February 2026 Anthropic said it detected industrial-scale campaigns extracting Claude capabilities. The company attributed more than 16 million exchanges across roughly 24,000 fraudulent accounts to DeepSeek, Moonshot and MiniMax, describing violations of terms and regional restrictions and framing the activity as illicit competitive distillation.
Separate evidence from a party’s interpretation. Counts and attribution are Anthropic claims based on private telemetry, and named organisations may interpret events differently. The technical description still matters because it shows how model extraction differs from an ordinary high-volume customer.
Legitimate distillation versus attack
Distillation is a standard technique: a strong teacher generates outputs or targets used to train a smaller student. A lab can distil its own models or use licensed synthetic data. The boundary is crossed when an operator circumvents limits, creates fraudulent identities, breaches terms and systematically reconstructs another service’s capabilities.
No weights need to be stolen. Carefully designed prompt-response pairs cover reasoning, coding, tool use, language and refusals. SFT, preference tuning or reinforcement learning then teaches a student to imitate teacher behaviour.
Telemetry signals
Anthropic highlights high volume concentrated in valuable capabilities, repetitive structures and training-like content. Individual requests look legitimate; the signal emerges across accounts, payments, devices, networks and semantics.
Features include similar account life cycles, template-shaped parallel requests, network rotation with stable client fingerprints, systematic task matrices, machine-like cadence, repeated variants or reasoning-trace requests, and immediate migration after limits. Per-key rate limiting fails against thousands of keys. Graph analysis and behavioural clustering need false-positive controls for legitimate evaluation firms and integrators.
Defend without punishing good customers
Use identity and payment verification proportional to scale, organisation-level and linked-identity limits, semantic extraction scoring and targeted friction for high-value query classes.
Additional controls include canary prompts, statistical watermarking with realistic expectations, step-up verification, restricted mass access to logprobs or reasoning traces, audited high-volume agreements and research into teacher-specific behavioural fingerprints.
No signature is perfect. Overblocking harms evaluation, accessibility, batch processing and research, so high-impact decisions need review and appeal.
Risk outside frontier labs
A company exposing fine-tuned know-how through an API can also be extracted. Attackers may approximate pricing, fraud, recommendation or specialist classifiers without stealing weight files.
Use authentication, quotas, tenant analytics and enumeration detection. Avoid returning internal prompts, hidden reasoning or raw logits unless required. A system prompt is not a secret vault and must never contain credentials.
Measure resistance
Red-team extraction under a defined budget using systematic queries, semantic variants and controlled multi-account simulation. Measure queries needed to reproduce task quality, detection time, cluster quality and false-positive impact.
Telemetry that separates batch use from extraction
Volume alone is a weak signal. Profiles should combine rate, topic coverage, template similarity, length distribution, systematic variants of the same problem and links among accounts, payments and devices. A legitimate batch customer normally has a stable workload and can explain its purpose; extraction campaigns seek broad coverage of behavioural boundaries. Analyse at organisation and cluster level while minimising data and restricting telemetry access.
Use graduated response: observe, throttle, require stronger verification, restrict selected query classes and only then suspend. Validate every threshold against legitimate traffic. A false-positive detector that interrupts a customer’s production batch is also an incident.
A tabletop exercise for model owners
Assume a competing model suddenly reproduces characteristic answers from your endpoint. The team should preserve logs, reconstruct query scope, assess contractual impact, contain active accounts and retain evidence for counsel. Predefine decision owners, escalation thresholds and canary-pattern rotation. Distillation defence joins security engineering, fraud, privacy and terms-of-service enforcement.
Combine these controls with model theft testing, an LLM gateway and AI red teaming. If you expose a model API, BreachRoad can assess extraction and abuse resistance.
Primary source and one party’s account: Anthropic — Detecting and preventing distillation attacks.
