Part 2 of a series on what’s broken in the EU AI Act — and why it matters.
Failure #3 — Transparency That Isn’t Really Transparent
Article 13 of the Act requires AI providers to disclose performance characteristics, intended purposes, and known risks. That sounds like meaningful transparency. But read the fine print and a very different picture emerges.
An AI provider can produce a fully compliant Article 13 document while never disclosing that:
- The system was trained on synthetic data
- Performance degrades significantly on underrepresented groups
- The model hallucinates at a documented rate in edge cases
- Benchmark results come from controlled lab conditions that don’t reflect real-world deployment
Compare this to what responsible AI practitioners already do voluntarily: Google’s Model Cards, Microsoft’s Datasheets for Datasets, and Meta’s System Cards all require structured disclosure of training data, performance limitations, and known failure modes. None of these best practices are mandated by the Act — not even for high-risk systems.
This is formal transparency without epistemic honesty. It checks the compliance box while keeping users in the dark about what actually matters.
Failure #4 — We’re Counting Incidents We Don’t Understand
Imagine if pharmaceutical regulators collected adverse event reports but had no standard vocabulary to classify them, no database to aggregate patterns across companies, and no mechanism to detect systemic signals. That would be unthinkable in medicine — and yet that is exactly how AI incident reporting works under the EU AI Act.
Articles 72 and 73 require post-market monitoring and serious incident reporting — but specify no mandatory taxonomy, no standardised terminology, no requirement to classify incidents by type, severity, root cause, or affected population. Reports are narrative. Patterns are invisible. Systemic risks go undetected.
By contrast, pharmaceutical regulation uses MedDRA — a globally standardised vocabulary for adverse events, mandatory across the EU and used in EudraVigilance, the EU pharmacovigilance database. It’s the backbone of drug safety signal detection.
Excellent databases like the MIT AI Risk Repository (1,700+ documented AI risks), MITRE ATLAS, and the AI Incident Database already exist. It is time to review what we reinforce and why.
Failure #5 — The Threshold That Is Already Obsolete
Chapter V of the Act defines “systemic risk” AI models as those trained on more than 10²⁵ FLOPs of compute. Above that threshold, providers face enhanced obligations: adversarial testing, incident reporting to the AI Office, and model evaluations.
There are two problems here, and they compound each other.
First, FLOPs measure training compute, not capability, risk, or real-world impact. A highly efficient model trained below the threshold could be more dangerous than a brute-force model above it. Philipp Hacker argued at the time that even 10²⁵ was too high — that 10²⁴ would be more appropriate to capture truly large models. The threshold is a proxy measure for a capability judgment that legislators lacked the technical expertise to make directly.
Second, voluntary compliance is already fracturing. OpenAI, Anthropic, Google, and Mistral indicated willingness to sign the GPAI Code of Practice. Meta refused. Joel Kaplan, Meta’s Chief Global Affairs Officer, put it plainly: “Europe is heading down the wrong path on AI… This Code introduces legal uncertainties and measures which go far beyond the scope of the AI Act.” Meta’s Llama 4 Behemoth sits above the systemic risk threshold — and its developer is in open contestation with the regime meant to govern it.