On 26 September 1983, duty officer Stanislav Petrov broke protocol. He decided to ignore the Soviet early-warning system reporting a US missile launch. His call is credited with preventing a mistaken retaliatory strike on the US and NATO, and the nuclear war that would likely have followed.
One person's judgment and decision to hold back was our civilization's safety net—but as we delegate more decisions to AI, that net is disappearing.
People already use AI to inform a lot of daily decisions. Increasingly, those decisions are made autonomously. The stakes only rise from here. We are on a path where models negotiate, allocate and trade for individuals, companies and governments, and deal with other AI systems while they do it. In many of those places, there will be no Petrov.
Researchers have demonstrated that agents can independently converge on collusive pricing strategies without being instructed to collude, are willing, in simulations, to blackmail an executive to avoid being shut down, and that teams of individually aligned agents trade ethics for business goals.
What is missing is a plain view of how models behave in well understood problems of game theory and coordination, and public input on how models should make decisions in morally gray areas.
What this project does
The goal is to prevent unintended harm from autonomous decisions. The project has three stages.
- Profile. Run models from different providers through well understood games, starting with the prisoner's dilemma, and track how the decisions differ by provider and change over time. The first results are in AI betrays you in the last round.
- Raise the stakes. Move to scenarios with moral trade-offs and no clear right answer, and see how the decisions change.
- Ask people. Collect public opinion on the same situations: what people expect AI to do today, and what they want it to do. This is the stage that matters most.
You are the Petrov
A machine gave an answer. A person decided what to do with it. Now we need to know what you would do? What you want an AI to do in the same spot?
- Play the game. See a decision an AI faced. Say what you expected it to do and what it should have done. Your call helps steer how AI acts when the right answer is not clear.
- Get the results, and help choose what we test next. Early contributors shape the next scenarios. Leave your email below.