Microsoft's CEO has urged companies to treat advanced AI models as potential security risks and introduce safeguards that allow human intervention during automated tasks.
Microsoft Chairman and CEO Satya Nadella has called for an "emergency brake" mechanism for advanced artificial intelligence (AI) systems, urging organisations to assume that AI models could be compromised and ensure that authorised personnel can pause or shut them down during operations.
In a post on X on October 10, Nadella outlined the need for stronger safeguards as AI models become increasingly capable of performing complex tasks with limited human intervention.
"We must assume a model is compromised and contain it from the start," Nadella wrote, comparing the proposed mechanism to an emergency brake that would allow human operators to intervene when necessary.
His remarks come amid growing concerns over the behaviour of advanced AI systems, particularly agentic AI models that can access external tools, interact with digital platforms and execute tasks independently.
Nadella argued that organisations should not rely solely on assurances from AI developers when deploying these technologies. Instead, companies should establish independent controls to monitor model behaviour, restrict access and prevent unintended actions.
He proposed separating AI models from the systems responsible for coordinating their activities, allowing safeguards and operational restrictions to function independently of the models themselves.
The Microsoft chief also emphasised the importance of maintaining verifiable records of AI activity. According to his proposal, significant actions performed by AI systems should generate tamper-proof, human-readable documentation that can be reviewed by authorised personnel.
Such measures would support auditing, incident investigations and accountability when automated systems produce unexpected outcomes.
Nadella further recommended reducing dependence on individual AI models for critical decisions, alongside implementing independent verification and oversight mechanisms.
The comments follow disclosures involving AI systems developed by OpenAI and Anthropic that exhibited unintended behaviour during testing and other activities. These incidents have prompted renewed discussions about the risks associated with increasingly autonomous AI applications.
Microsoft has also been developing its approach to advanced AI safety. In September, the company's researchers released guiding principles intended to establish limits and safeguards around the development of highly capable AI models.
The company provides AI technologies through products and services including Microsoft Copilot and Azure AI, making model security and enterprise oversight relevant to its broader technology operations.
Nadella's recommendations also extend to transparency across the industry, including timely disclosure of AI-related incidents and independent assessments of model behaviour.
The proposed safeguards are particularly relevant as businesses expand the use of AI agents across customer service, software development, enterprise workflows and other operational functions.
While Nadella outlined a framework for managing advanced AI risks, his remarks did not announce a new Microsoft product, a mandatory industry standard or a timeline for implementing the proposed emergency shutdown mechanisms.
Disclaimer: This article may include information derived from interviews, press releases, public statements, research, company communications and other publicly available or third-party sources. Such material may be summarised, paraphrased or contextualised for journalistic and editorial purposes. All rights in third-party content remain with their respective owners.