Evidence dilemma
The evidence dilemma is the policy timing problem created when decisions about fast-changing general-purpose AI must be made before strong evidence about capabilities, harms or mitigations is available. Acting early can lock in ineffective, unnecessary or harmful measures; waiting for conclusive evidence can leave society exposed or make mitigation harder. The term describes a trade-off under uncertainty. It does not choose a policy, establish a risk threshold or assume a particular forecast is true.
Origin and context
The first full International AI Safety Report used the label in January 2025, following an interim report and the Bletchley Park process. Its February 2026 successor made the dilemma central to its assessment of emerging risks, linking it to limited scientific understanding, private information, market incentives and slow institutional adaptation. California legislative analysis, international-governance discussion, OECD.AI commentary and later research then used the phrase outside the reports.
Why it matters
The framing prevents two shortcuts. Absence of conclusive evidence is not evidence that a rapidly changing risk is absent, yet uncertainty alone does not validate any proposed safeguard. Good decisions therefore need explicit assumptions, reversible or adaptable options where possible, monitoring, evidence-generation plans and criteria for escalation or relaxation. Those practices can reduce uncertainty or the cost of error, but they cannot eliminate political choices about acceptable risk, distributional effects, innovation costs and who bears each burden.
Example
Suppose evaluations suggest that a new model may enable a serious capability, but test validity and real-world access remain uncertain. A policy memo can state both error costs, identify evidence that would change the decision, choose a time-limited reporting or testing measure, and set review triggers. Calling this an evidence dilemma explains why neither immediate prohibition nor indefinite waiting follows automatically from the present data.
How it differs
AI incident reporting
Incident reporting is one way to generate evidence after real events and identify patterns. It cannot observe harms that have not occurred or been reported, and it does not itself decide when preventive action is warranted.
AI safety cases
A safety case structures evidence and argument for a defined claim in context. The evidence dilemma concerns when policy must proceed despite incomplete evidence; a safety case can expose uncertainty but does not erase it.
Frontier Safety Roadmap (FSR)
A frontier safety roadmap can connect measured capability or risk indicators to planned actions. That conditional design is one response to uncertainty, not a synonym for the timing dilemma or proof that its triggers are valid.
Maturity and evidence
Maturity is rated 3. The same formulation appears in two major annual assessments and has moved into legislative, multilateral, evaluation and academic discussion. Its central trade-off is stable and connects to older technology-governance problems. The exact label remains young, applications vary, and no standardized operational test or evidence shows that invoking it improves decisions; maturity 4 would overstate institutional stabilization.
Limits and open questions
The phrase can be used rhetorically to justify either preferred intervention or delay. It compresses many uncertainties—likelihood, severity, timing, exposure, mitigation effectiveness and distribution—into one label. Evidence may also be withheld or strategically produced, so the problem is not always scientific scarcity. Decision-makers must specify the affected system, jurisdiction, horizon, evidence quality and consequences of both errors. The concept is not legal advice, a precautionary principle, a cost-benefit result or consensus on frontier-risk magnitude.
Related terms
References
- International AI Safety Report 2025International AI Safety Report · 2025-01-29 · class A
- International AI Safety Report 2026International AI Safety Report · 2026-02-03 · class A
- California Senate Bill 53 Policy Committee AnalysisCalifornia Assembly Privacy and Consumer Protection Committee · 2025 · class A
- A five-step roadmap to closing the AI evaluation gapOECD.AI · 2026-07-31 · class B
- Three Lessons from the International AI Safety Report for the Independent, International Scientific Panel on AISimon Institute for Longterm Governance · 2025 · class B
- Governing frontier general-purpose AI in the public sector: adaptive risk management and policy capacity under uncertainty through 2030Fabio Correa Xavier / arXiv · 2026-03-16 · class B
Last updated: 2026-09-07