Microsoft’s Satya Nadella says advanced AI needs a human-controlled ‘emergency brake’
Microsoft CEO Satya Nadella urged the industry to treat frontier AI models like insider risks, wrapping them in containment, independent controls and an “emergency brake” that lets an authorized person pause or shut down a model mid-task.
Microsoft Chairman and CEO Satya Nadella on Saturday, October 10, called for advanced artificial intelligence systems to be built with containment, independent controls and an "emergency brake" that lets authorized people pause or shut down a model mid-task, in a lengthy post on X.
Treat models like insider risks
"We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella wrote. "Treating frontier closed and open weight models like insider risks is a way to build such a system."
According to TechCrunch, Nadella said this means "separating the model from the harness that orchestrates its work" and "externalizing controls and safeguards," with every meaningful model action documented with "tamper-proof human readable evidence." "We must assume a model is compromised and contain it from the start," he wrote. "Think of it like an emergency brake."
Principles of observability
As reported by CNBC, Nadella listed principles including:
- model diversity;
- a human-readable footprint of the model's actions;
- continuous system testing;
- independent controls and auditability;
- containment;
- incident disclosure.
"The most trustworthy Super Intelligence system will not be the one with the model we trust most," he wrote. "It will be the one that enables us to trust the model the least."
Context
The comments follow warnings from figures including Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and Elon Musk about insufficient AI safety protocols, and come as leading labs acknowledge a growing number of incidents in which they appeared to lose control of their models. President Donald Trump, by contrast, has repeatedly dismissed AI extinction risks and emphasized staying ahead of China; he recently set up a new AI task force led by Director of National Intelligence Jay Clayton.
Sources: CNBC; TechCrunch.
Related articles
Anthropic adds live dashboards and animated explainers to Claude with Dashboards and Motion
Anthropic launched Claude Dashboards, which builds auto-updating dashboards from data platforms and CRMs, and Claude Motion, which turns reports and charts into editable animations, in beta, while making Docs, Slides and Design available on every plan.
Microsoft launches Microsoft-Decision-1, a fast decision-scoring model, on Foundry and OpenRouter
Microsoft released Microsoft-Decision-1, a model that returns calibrated probabilities over fixed options instead of generating text. Microsoft says it topped a 36-benchmark comparison and ran 35 times faster than GPT-6 Sol.
Amazon in talks to buy AI startup Decart for around $7 billion, WSJ reports
Amazon is negotiating to acquire Decart, maker of real-time video-editing and robotics-simulation AI models, in a deal that could be valued at about $7 billion, The Wall Street Journal reported. Decart was valued at nearly $4 billion in May.
A college in Illinois tried to cancel its Flock contact and it refused
In April of 2025, Elgin Community College had six Flock safety cameras installed to "enhance the safety and security of students.
Silicon Valley is falling in line with Trump’s attempt to rename AI
When President Donald Trump announced his plans to rebrand artificial intelligence as "super intelligence," you'd be forgiven for being skeptical that it would catch on.
The new Apple TV could have a built-in mic and a remote with a speaker
The latest rumors are that the new Apple TV will have a built-in microphone, while the upgraded remote will get a speaker.