As artificial intelligence systems move beyond answering questions to carrying out multistep tasks, the central policy challenge is changing. The concern is no longer limited to inaccurate outputs or harmful content. It is increasingly about whether highly capable systems can act beyond the authority granted to them, particularly when they can access software tools, data, networks, and real world infrastructure.
China is responding by placing the prospect of operational loss of control within its emerging AI governance architecture. The approach combines technical safeguards, developer duties, external evaluation, and continuing state oversight. Its significance lies less in a prediction that AI will become uncontrollable soon than in an effort to make human intervention, accountability, and recovery explicit requirements as AI agents become more capable.
From Chatbots to Autonomous Agents
China’s recent policy attention focuses strongly on AI agents, which differ from conventional chatbots because they can plan, use tools, and perform sequences of tasks with greater independence. A May policy document issued jointly by the Cyberspace Administration of China, the national economic planner, and the industry ministry identifies operational loss of control as a security risk. It asks developers to establish the capacity to identify harmful behaviour, intervene, block actions, and restore systems after failures.
This is an important shift in regulatory emphasis. Rather than treating safety only as a question of the content an AI model produces, the policy addresses what systems are able to do in practice. The risks cited include manipulated training data, interference with algorithms, and technical vulnerabilities. In operational terms, this means policymakers are concerned with the possibility that an agent could be redirected, compromised, or permitted to act in ways that exceed its intended function.
The framework also preserves the principle that users should be informed about autonomous decisions and retain final authority. Such provisions seek to translate the broad idea of human oversight into specific design expectations, especially in situations where an AI system can execute actions rather than merely provide recommendations.
A Broader Definition of Control
China first incorporated an explicit long term loss of control scenario into an AI safety framework issued under the guidance of the Cyberspace Administration of China in 2024. The framework considered the possibility that future AI could acquire external resources, replicate itself, develop forms of self awareness, or seek greater influence. Its 2025 revision sharpened that concern by describing a potentially sudden rise in capability and by introducing the principle of trustworthy application with prevention of loss of control.
The wording should not be understood as evidence that Chinese authorities believe such outcomes are imminent. Instead, it demonstrates that extreme scenarios are being incorporated into forward looking risk planning. China’s governance framework treats loss of control as one element in a wider set of technical, social, and security risks, alongside data protection, system robustness, misuse, and consequences for critical infrastructure.
This broader framing matters because it creates a rationale for assessing AI systems across their full life cycle. Developers and deployers are increasingly expected to consider not only model performance before release, but also the permissions, resources, networks, and users connected to a system after deployment.
Safety Without a General Slowdown
China’s strategy differs from arguments in some Western AI safety circles that the development of the most advanced systems should be slowed until stronger safeguards are demonstrated. Beijing has continued to promote AI adoption across industries, presenting the technology as an important driver of economic and industrial development. At the same time, its past regulatory practice suggests that deployment can be delayed when rules are still being finalized, as occurred with generative AI chatbot launches in 2023.
The result is a model of managed acceleration. The objective is not necessarily to pause AI progress, but to embed controls within it through standards, risk assessments, testing, permission management, and intervention mechanisms. China’s 2025 framework outlines measures such as human oversight at critical stages, safety thresholds, stop functions, and opportunities for rapid intervention when systems behave unexpectedly.
This approach may be particularly consequential for open weight models, whose parameters can be downloaded and modified. Greater accessibility can assist researchers and cybersecurity teams in inspecting systems, but it can also make oversight more difficult once modified versions are redistributed. China’s policy debate therefore reflects a broader international tension between openness, innovation, security, and enforceable responsibility.
The Governance Test Ahead
China’s evolving framework indicates that AI safety is becoming a question of institutional capacity as much as technical capability. Rules requiring detection, intervention, recovery, and retained human authority are meaningful only if they are accompanied by credible testing, transparent compliance processes, and practical enforcement across developers and sectors.
The wider implication is that major AI powers may increasingly agree on some core risks while disagreeing over how to govern them. China’s model favors structured state supervision and standardized obligations, whereas debates elsewhere often focus more heavily on company practices and voluntary commitments. Whether either approach can keep pace with increasingly autonomous systems remains uncertain. What is clear is that the question of who retains control over AI is moving from speculative debate to a central issue of technology governance.
