Every AI executive I talk to has started using the phrase "we red-team our models" the way people used to say "we have a firewall." It signals seriousness without anyone checking what's behind it. And increasingly, someone is going to check. Not just a regulator — a plaintiff's attorney, an insurance underwriter, a board committee reviewing an incident after the fact. When that happens, the question isn't whether you red-teamed. It's whether what you did would survive being read out loud in a deposition.
I want to walk through what's actually happening in the law and the standards world right now, because "red teaming" is quietly becoming one of the load-bearing concepts in AI governance — the thing regulators point to when they want evidence that a company exercised reasonable care. But there is no blanket statute anywhere that says "if you red-team, you're immune." The safe harbor, where it exists at all, is conditional, and the conditions are specific. Get the form wrong and you've built an expensive theater production instead of a legal defense.
What "Safe Harbor" Actually Means Here
In my experience advising companies through AI governance builds, people hear "safe harbor" and imagine a shield — do the thing, walk away protected. That's not quite what's on offer in AI regulation today. What's emerging instead is something closer to a rebuttable presumption: if you can show you followed a recognized risk management process, including adversarial testing, the burden shifts. You're not automatically cleared, but you're not starting from zero either.
Colorado's SB 24-205, the Colorado AI Act enacted May 17, 2024 with compliance obligations now delayed to June 30, 2026, states this almost explicitly. It grants developers and deployers a rebuttable presumption of "reasonable care" if they've complied with a nationally or internationally recognized AI risk management framework. That's a legislature telling you, in writing, that following a framework changes your liability posture. It's not a hall pass. It's a head start, and only if the framework was actually followed rather than named in a slide deck.
The EU AI Act, Regulation (EU) 2024/1689, goes further for a narrower slice of the market. Article 55 requires providers of general-purpose AI models classified as carrying systemic risk to conduct and document adversarial testing specifically aimed at identifying and mitigating that risk, with these obligations applying from August 2, 2025. This isn't a suggestion buried in a recital. It's a binding obligation with the word "document" written directly into the operative text.
So the pattern across jurisdictions is consistent even where the mechanisms differ: adversarial testing is moving from best practice to evidentiary requirement. The safe harbor isn't the testing itself. It's what the testing lets you prove.
The Federal Picture Just Got More Complicated, Not Less
Here's where the timing matters. Executive Order 14110, signed October 30, 2023, required developers of the largest frontier models to report red-team safety test results to the federal government under the Defense Production Act. That created something close to a mandatory floor. President Trump's Executive Order 14179, signed January 20, 2025, rescinded EO 14110 entirely. As of today, there is no active federal red-team reporting mandate in the United States.
That doesn't mean the pressure disappeared. It means the pressure moved to state law, to international regulation that still reaches U.S. companies selling into the EU, and to the voluntary frameworks that insurers, enterprise customers, and boards now expect as a baseline. The White House's voluntary AI commitments, signed by major AI labs on July 21, 2023, already had internal and external red-teaming baked in as a pledge before public release. That pledge didn't evaporate when the executive order did. It's now simply unenforced by statute and enforced instead by market expectation, procurement contracts, and the plain fact that your competitors are doing it and citing it.
In my view, this is actually the harder environment to operate in, not the easier one. A single federal mandate is at least a fixed target. A patchwork of state rebuttable-presumption laws, an EU regulation with extraterritorial reach, and a set of voluntary commitments that function as de facto industry standard means you have to build a program robust enough to satisfy all of them at once, because you don't get to choose which one a regulator, a customer's legal team, or a plaintiff's expert witness reaches for first.
Why "We Tested It" Isn't Enough
I've reviewed AI governance documentation from companies that will tell you, with complete sincerity, that they red-team their models. What they usually mean is that an engineer spent an afternoon trying to get the chatbot to say something embarrassing, took a few screenshots, and moved on. That's not nothing. It's also not the thing any of these frameworks are describing, and it will not hold up as evidence of reasonable care.
The gap between real adversarial testing and the appearance of it comes down to a small number of structural features, and every one of them is checkable after the fact by someone who wasn't in the room when the testing happened.
Independence matters more than expertise. A tester who reports to the product manager whose launch depends on the model passing has an incentive problem, even if they're skilled and honest. The most defensible programs separate the red team's reporting line from the team that owns the ship date.
Scope has to be adversarial, not confirmatory. Testing that only checks whether the model does what it's supposed to do is quality assurance. Red teaming means actively trying to make it fail: prompt injection, jailbreaks, data extraction attempts, bias probes across protected classes, testing for the specific harms named in your own risk assessment. If your red team's test plan wasn't written by someone thinking like an adversary, it isn't red teaming.
The findings have to go somewhere and something has to happen to them. A report that identifies a vulnerability and then sits in a shared drive is worse than no report in a legal sense, because now there's a document proving you knew and didn't act. The remediation has to be tracked, timestamped, and closed out, or reopened with a documented risk acceptance decision made by someone with the authority to accept that risk.
It has to repeat. A single pre-launch red-team exercise tells you about the model as it existed on one day. Models drift, get fine-tuned, get connected to new tools and data sources after launch. A testing program that only fires once, before ship, is describing a snapshot as if it were an ongoing state of assurance.
The technical term regulators and standards bodies use for this whole apparatus, when it's done right, is a control. Not an event.
What Internal Adversarial Testing Has to Look Like to Count
Pulling that together, here's the comparison I use when I'm assessing whether a client's existing program would hold up under scrutiny versus whether it's mostly performance.
| Dimension | Testing That Won't Hold Up | Testing That Counts |
|---|---|---|
| Reporting line | Testers report to the launch owner | Testers report outside the product P&L, to risk, compliance, or an AI governance function |
| Scope definition | Ad hoc, tester's judgment on the day | Written test plan mapped to the risks identified in a prior AI risk assessment |
| Attack diversity | One tester, one technique (usually prompt-based jailbreaks) | Multiple vectors: prompt injection, data exfiltration, bias/fairness probes, tool-use abuse, adversarial inputs at the data layer |
| Documentation | Slack thread or a slide with screenshots | Dated report with methodology, findings severity, and reproduction steps |
| Remediation | Findings noted, no tracked closure | Each finding tracked to closure or to a documented, signed-off risk acceptance |
| Cadence | Once, before launch | Recurring, and triggered by material model or integration changes |
| Ownership of the record | No single owner; scattered across tickets | Held inside the AI management system's records, subject to management review |
That last row is the one people skip and regret skipping. A red-team report that lives inside a formal management system, where it's subject to periodic management review, behaves completely differently in a dispute than the same report sitting in an engineer's personal notes folder. Same testing. Different evidentiary weight.
Building the Paper Trail
ISO/IEC 42001:2023, the AI management system standard, gives you the scaffolding to make the second version happen without inventing your own bureaucracy from scratch. Clause 6.1.2 requires a documented AI risk assessment process — this is where your red-team scope should trace back to, so a tester isn't guessing at what matters. Clause 8.1, operational planning and control, is where the red-teaming activity itself gets defined as a repeatable operational control rather than a one-off favor someone did before a launch. And clause 9.1, monitoring, measurement, analysis and evaluation, is where the findings get reviewed on a cadence, rather than filed and forgotten.
None of those clauses mention "red team" by name. That's fine. The standard gives you the container; you put the adversarial testing inside it. What changes the evidentiary value isn't the label on the activity, it's whether the activity is traceable through a documented system with defined ownership, defined inputs, and a defined review cycle. A regulator or a plaintiff's expert reading an ISO 42001-conformant record set is reading a company that can show its work. A regulator reading a screenshot in a Slack channel is reading a company that got lucky the testing happened at all.
There's a useful historical parallel outside AI entirely. The Department of Justice's policy update on May 19, 2022 revised how federal prosecutors charge cases under the Computer Fraud and Abuse Act, explicitly carving out "good-faith security research" from prosecution. That change didn't happen because security researchers asked nicely. It happened because a body of documented, good-faith adversarial testing had built enough of a track record and enough public understanding that the law adjusted around it. AI red teaming is walking the same path, several years behind. The documentation discipline you build now is the raw material that future safe harbors, statutory or judicial, will eventually be built on.
Where This Fits Inside a Broader AI Program
None of this works as a bolt-on. I've seen companies try to stand up a red-teaming function in isolation, disconnected from any underlying risk assessment, and the result is testing that looks busy but can't answer the most basic question an auditor will ask: how did you decide what to test? Red teaming only counts as evidence of reasonable care when it's clearly downstream of a risk assessment and clearly upstream of a remediation process that a management review actually looks at.
If you're building or auditing that chain for the first time, the place to start is almost never the red team itself — it's an honest inventory of what your AI systems can actually do and what could go wrong, which is exactly what an AI readiness assessment is built to surface before you spend money on testing methodology. For regulated industries specifically, I've written before about how ISO 42001 gives pharmaceutical manufacturers a structured way to bring AI oversight into an existing quality system rather than treating it as a parallel compliance track, and the same logic applies to red-teaming records: they belong inside your existing management system architecture, not next to it.
Frequently Asked Questions
Is there an actual legal "safe harbor" for companies that red-team their AI systems?
Not a blanket one. What exists is closer to a rebuttable presumption in specific statutes — Colorado's AI Act ties reasonable care to compliance with a recognized risk management framework — and a documented-testing requirement under the EU AI Act's Article 55 for the highest-risk general-purpose models. Neither grants automatic immunity; both change who has to prove what.
What's the difference between red teaming and routine QA testing?
QA testing checks whether the system does what it's supposed to do. Red teaming actively tries to make the system fail in ways that matter — jailbreaks, data extraction, bias, misuse of connected tools — using an adversary's mindset rather than a checklist. A program that only confirms intended behavior isn't adversarial testing regardless of what it's called internally.
Does red teaming have to be done by an outside firm to count?
No, but the team doing it has to be structurally independent from whoever owns the launch decision. Internal testing can absolutely satisfy the frameworks discussed here, including NIST's AI Risk Management Framework, as long as the reporting line, scope, and remediation tracking are separated from the team incentivized to ship.
How often should internal adversarial testing happen?
At minimum, before any material release and after any material change to the model, its fine-tuning, or the tools and data it's connected to. A single pre-launch exercise describes the system as it existed on one day; regulators and courts increasingly expect an ongoing control, not a one-time event.
What should a red-team report actually contain to be useful as evidence?
A dated methodology tied to a prior risk assessment, the specific attack vectors attempted, findings with severity ratings and reproduction steps, and a tracked remediation status for each finding — either closed or formally risk-accepted by someone with the authority to accept it. Anything less is hard to distinguish, after the fact, from testing that never happened.
If you're trying to figure out whether your current testing program would hold up under that kind of scrutiny, that's a conversation worth having before an incident forces it. You can reach out through the contact page to talk through where your program stands.
Last updated: 2026-08-14
Jared Clark
AI Strategy Consultant, AI Strategies Consulting
Jared Clark is the founder of AI Strategies Consulting, helping organizations design and implement practical AI systems that integrate with existing operations.