Fordel StudiosThere should be $100M grants to automate AI safety
What Happened
This post reflects my personal opinion and not necessarily that of other members of Apollo Research.TLDR: I think funders should heavily incentivize AI safety work that enables spending $100M+ in compute or API budgets on automated AI labor that directly and differentially translates to safety.Motiv
Our Take
look, i don't buy the idea of dumping $100 million into grants without absolute, auditable metrics. the real money isn't in generic safety funding; it's in making the automated safety work directly translate into reducing catastrophic failure rates. that’s where the ROI lives.
if we’re going to fund safety, it needs to be tied to compute usage or API calls that actually mitigate risk, not just abstract goals. otherwise, it just becomes another bureaucratic hurdle.
we're spending way too much on flashy safety announcements and not enough on operationalizing the actual risk mitigation steps. automate safety means automating the validation loop, not just writing more papers.
actionable: fund safety based on measurable risk reduction targets, not arbitrary grants.
impact:high
What To Do
Check back for our analysis.
Builder's Brief
What Skeptics Say
Scaling safety evaluation spend assumes the evaluators themselves are robust — automated safety benchmarks are the most gameable artifacts in ML, and a $100M signal will accelerate optimization against the metric rather than the underlying behavior.
Cited By
React
Get the weekly AI digest
The stories that matter, with a builder's perspective. Every Thursday.