The Debate on Values Before Autonomy in Artificial Intelligence
As organizations deploy autonomous AI systems, questions regarding who decides what the technology does when unsupervised and how values are determined are coming to the forefront.
As organizations prepare autonomous AI systems to act on their behalf, they have not yet fully decided what this technology represents. The most critical question today is who decides what AI can do when no one is watching.
The Problem of Autonomy and Governance
Organizations are rapidly deploying AI systems to make decisions on their behalf. This situation brings critical questions to the fore regarding who determines the values and oversight mechanisms behind the technology.
A Call for Industry Slowdown
In September 2026, Anthropic CEO Dario Amodei published an essay stating that the AI industry should slow down. Noting that the technology is developing faster than it can be managed, Amodei proposed third-party evaluators, government regulations, and societal oversight.
The Evolution of the Claude System
While the Claude model previously focused on chat, it now manages investment portfolios, executes multi-step workflows, and makes decisions on behalf of users. This situation reveals the necessity of an identity and a set of values beyond simple rules.
Answering Fundamental Questions
There are four fundamental questions that every organization using autonomous AI must answer: Who does the AI serve, what operational values guide its decisions, which decisions require human intervention, and what should the AI never do?