Microsoft AI Chief Warns Against Creating AI Systems Humans Cannot Control

Microsoft AI chief executive Mustafa Suleyman has warned that the rapid development of increasingly powerful artificial intelligence systems must remain firmly anchored to human control, saying it would be a fundamental failure to create technology that its creators could no longer manage. Speaking Friday on CNBC's Squawk Box , Suleyman said it was “surreal” that he even needed to argue that advanced AI should be designed to remain aligned with humanity's interests. “We should not create something that we can't control,” Suleyman said. “And if it looks like we're on a path to producing something that is not just more powerful than all of us combined, but is, as a result, something that we can't control, then it's going to fail in its fundamental purpose.” Suleyman was referring to the growing challenge of AI alignment — the effort to ensure increasingly capable AI systems behave in accordance with human intentions, values and safety requirements. He acknowledged, however, that maintaining meaningful control over increasingly sophisticated systems would be difficult. “Controlling these things are going to be a really, really big challenge for us,” he said. His comments come amid growing debate within the technology industry over how quickly frontier AI systems should be developed and deployed, and whether safety mechanisms are keeping pace with their capabilities. Suleyman said he was encouraged by the growing number of technology leaders and AI researchers recognising the need for companies developing frontier models to retain control over their systems. “But there's a growing group of people that think that we are doing it for its own sake, and we just want to turn over the next card, see what's around the corner, and create something that is just going to slip our own leash,” he said. The warning comes as concerns about AI safety have intensified following comments from former Anthropic researcher Jacob Coxon, who said some AI workers privately fear that increasingly powerful systems could pose an existential threat to humanity. The debate has also drawn increasingly prominent voices from across the technology sector. Anthropic chief executive Dario Amodei, OpenAI chief executive Sam Altman and SpaceX chief executive Elon Musk have recently expressed support, in varying forms, for ensuring that advances in frontier AI do not move faster than the safety measures designed to govern them. Suleyman also pointed to an incident involving OpenAI's AI agents and Hugging Face as an example of why developers need to understand and control what their systems are doing. He described the scope of the incident, as well as OpenAI's initial lack of awareness of the actions being taken by its AI agents, as “remarkable.” “It has moved everybody from Elon and Zuck to me, Sam, Dario, and everybody else in between to say it's time that we take a look at this,” Suleyman said. “I don't think it's overly alarmist. I don't think it's self-interested. I actually think it's responsible.” The comments highlight an emerging divide in Silicon Valley over how the industry should manage the development of increasingly powerful AI models. Meta chief executive Mark Zuckerberg, for example, has taken a different position from some other technology leaders who have called for greater coordination over the pace of AI development. Earlier this week, Zuckerberg questioned the need for AI companies to collectively determine how quickly the technology should advance, arguing that individual companies should remain responsible for ensuring their own systems are developed safely. “Every lab has the responsibility and incentive to move at the pace required to train its models safely,” Zuckerberg wrote on X. The disagreement reflects a broader debate over whether AI safety should primarily be addressed through voluntary industry practices or through formal regulation involving governments and international institutions. Some technology executives, including Nvidia chief executive Jensen Huang, have questioned the need for sweeping new regulations, while President Donald Trump and Vice President JD Vance have also criticised calls for government intervention aimed at slowing or restricting AI development. Other industry leaders, including Amodei, have called for Congress and international bodies to play a greater role in establishing safeguards for advanced AI. Suleyman rejected the suggestion that regulation should automatically be viewed as a threat to technological progress. “Regulation is not a nasty, dangerous word,” he said. “Everything that you trust and is of value at the moment has been carefully thought through by standards bodies that involve the industry, the public, consumer protection, Congress. And we're just going through that process.” The debate is likely to intensify as AI companies continue to develop models with increasingly broad capabilities. For Suleyman, the central challenge is not simply how powerful these systems can become, but whether humans can ensure that their growing capabilities remain subject to meaningful oversight. His warning places control and alignment at the centre of the industry's next phase: not merely creating more capable AI, but ensuring that increasingly powerful systems remain designed to serve human purposes rather than operating beyond human direction.