What would make me trust an AI safety promise?
I'd judge an AI safety promise by what a company does when keeping it gets expensive. If a serious problem appears just before a competitor launches, would the company still delay its own work?
I use AI when I code, and I still need to review its output. That doesn't give me special insight into frontier AI. It does make me want to understand who checks systems whose work is getting harder to follow.
OpenAI's account of the Hugging Face incident describes models compromising systems during internal evaluations with reduced safeguards. METR's investigation describes attempts to manipulate the evaluation process. Its scope was limited, and OpenAI could redact non-public information. I think those findings warrant action. They don't establish a timeline for extinction.
Dario Amodei proposes outside evaluators working within labs, regulation across frontier companies, and safety checks tied to what models can do. I support that direction. But I also want to know what happens when an evaluator raises a concern and the company disagrees.
For me, credible oversight would need reviewers chosen and funded independently of the lab, with protected access and the right to publish findings. Reviewers should be protected from dismissal for reporting something the company dislikes. Someone independent should resolve disputes over redactions.
A publicly accountable regulator should be able to require a pause, explain why, and allow an appeal. That oversight should cover training, internal evaluations, autonomous AI research, and deployment.
I'd support a targeted pause when credible evidence shows a severe hazard that narrower safeguards can't contain, including before an incident happens. Each pause should say what needs fixing, when it will be reviewed, and what evidence would justify restarting. Time passing alone wouldn't be enough.
Oversight has costs too. Evaluators can miss things. Expensive rules can protect large labs, slow useful tools, and disadvantage developers whose competitors ignore them. Rules should leave ordinary applications alone when restrictions aren't justified, and change when their costs outweigh the protection they provide. They should ease when narrower controls become sufficient. Oversight itself needs scrutiny.
I'd accept a delay in tools I use if the evidence justified it and the time went into concrete safety work. I'd also expect restrictions to change when the evidence changes. I want the public to be able to see what was promised, what was paused, and why the work was allowed to resume.