Who Has the Power to Pause AI?

Author: Xiaobai

This article is an original submission by the author, and the views expressed represent only the author's personal understanding. ETHPanda has edited and organized the content.

The debate surrounding AI is often simplified into two camps: one advocating for continued acceleration and the other calling for an immediate pause.

Vitalik pointed out in response to "AI 2040: Plan A" that the real divergence in this debate is not about policy choices, but rather about fundamentally different judgments people have about the future.

One judgment holds that AI remains a "normal technology." It will change employment, industries, and social structures, but ultimately will still be constrained by markets, laws, energy, capital, and organizational capabilities.

Another judgment posits that once AI begins to participate at scale in AI research and development, its capabilities may enter a rapid feedback loop. At that point, humanity will face not just job displacement, but potentially a gradual loss of actual control over research, economy, military, and institutional matters.

If the first judgment holds, establishing a global chip tracking, computing power approval, and data center monitoring system would clearly be a severe overreaction.

However, if the second judgment holds, continuing to maintain a default competition, believing that the market will ultimately resolve everything, is equally a high-risk gamble.

Thus, what AI governance truly needs to address is not just whether to pause, but:

How to retain the ability to decelerate without permanently handing power to a few, when it is uncertain which world the future belongs to?

The Core of "AI 2040" Is Not Just "Deceleration"

"AI 2040" proposes a relatively idealized transition route to superintelligence.

It envisions that major countries and AI companies, through international coordination, slow down the development speed of cutting-edge AI, increase research transparency, allow more participants to catch up to near the frontier level, and then continue to advance after safety and governance issues are addressed.

I believe the biggest problem with this plan lies not in the goals, but in execution.

As long as one party secretly trains a stronger model, other participants will worry about being left behind, and the agreement will quickly become ineffective. Therefore, to make deceleration credible, it is necessary to track high-end chips, monitor large data centers, and determine whether computing power is being used for training or other purposes.

This means that in order to prevent superintelligence from forming a new center of power, humanity may first need to establish a global computing power control center.

This is precisely the most difficult contradiction to resolve in the AI pause proposal:

The mechanisms to prevent technological loss of control may themselves become tools for power loss of control.

"AI risks are significant" does not automatically justify that a certain government, international organization, or large laboratory should gain unlimited power.

Similarly, "regulation may be abused" does not conversely prove that AI risks do not exist.

What truly needs scrutiny is the institutional structure:

Who defines danger, who controls the data, who has the authority to initiate restrictions, who can obtain exemptions, who supervises the enforcers, and whether this power can be revoked after a judgment error.

If these questions remain unanswered, "AI safety" can easily become an open-ended technological license.

Between the Pause Button and the Throne Lies an Exemption

"Pausing AI" sounds like a simple action, but it may represent two completely different systems.

The first is selective pausing.

Those in power can decide who is qualified to train models and who must stop; who belongs to trustworthy institutions and who is seen as dangerous participants. Others are restricted, while those in power can retain models, chips, and computing power.

This mechanism does not genuinely end the competition; it merely pre-decides the winner.

It is not a pause button, but more like a throne.

The second is symmetrical pausing.

When public risk conditions are triggered, all major participants are restricted, including the party responsible for enforcing the pause. There are no secret exemptions, and one cannot demand competitors to stop while allowing their own laboratory to continue.

This mechanism is much closer to a true circuit breaker.

Vitalik explicitly affirmed that Plan A introduces the concept of "mutual assured destruction of computing power." Compared to allowing a few participants to selectively restrict others while exempting themselves, this mechanism is at least more symmetrical.

However, symmetry does not automatically solve the problem.

A system capable of causing a large-scale halt of advanced global computing power is itself a very strong tool of power. It may be triggered by false intelligence, exploited by political conflicts, manipulated by insiders, or may cause significant losses due to technical failures.

Therefore, a credible pause mechanism must at least meet several conditions:

The triggering conditions must be public, evidence cannot be monopolized by a single institution, restrictions should be as symmetrical as possible, pauses must have a clear time limit, and the rules for restarting must be clearly outlined in advance.

In particular, one should not only design "how to enter an emergency state" but also design "how to exit an emergency state."

In reality, once institutions gain power, they often find it easier to prove that danger still exists rather than proactively return power.

Thus, the pause mechanism for AI itself also needs a pause button.

The Real Concentration of Power Comes Not Only from Companies and Governments

When discussing AI power today, people usually worry about two things.

First, a few large companies monopolizing the strongest models.

Second, governments controlling chips, data, and developers in the name of security.

But Vitalik reminds us of a third, deeper risk:

Superintelligence itself may also become the largest concentrator of power.

The reason humanity currently has bargaining power is that humans control some scarce resources: labor, knowledge, organizational capabilities, capital, political power, and military force. Different groups must negotiate with each other because no one can completely bypass others.

If AI can perform research, programming, management, finance, communication, cyberattacks, and even military planning faster, cheaper, and more effectively than humans, these originally human scarce capabilities will rapidly depreciate.

By then, even if there are ten AI companies competing in the market, it does not mean that humanity still holds the ultimate decision-making power.

Market competition can limit a company's profits, but it does not guarantee that ordinary people will continue to have bargaining power.

Therefore, "letting trustworthy people control the strongest AI" is not a complete answer.

It merely shifts the question from "Will AI control humanity?" to "Which small group should first gain the power to control the strongest AI?"

d/acc Is the Foundation, But Not a Panacea

Vitalik's long-term advocacy of d/acc can be understood as prioritizing the development of defensive, decentralized technologies.

For example, safer software, formal verification, cryptography, open hardware, network defense, biosafety, and information verification tools.

The value of these directions lies in their independence from a specific timeline for superintelligence.

If AI ultimately remains a normal technology, safer software, more reliable hardware, and stronger public defense capabilities are still worth building.

If AI rapidly becomes stronger, these technologies can also enhance society's ability to resist cyberattacks, biological risks, hardware backdoors, and information manipulation.

This is a low-regret strategy: when it is uncertain which world the future belongs to, first build safety infrastructure that is worth having regardless.

But d/acc is not a cure-all.

If AI capabilities indeed leap forward in a very short time, defensive technologies may not be deployed in time. Safer systems can reduce risks but cannot prove that all competitions should continue.

Thus, d/acc is more suitable as a long-term foundation, while the pause mechanism serves as insurance in extreme cases.

The foundation needs continuous construction, and insurance should not be easily activated, but it also cannot wait until the system is already out of control before it starts to be designed.

More Important Than Predicting Dates Is Agreeing on Trigger Conditions in Advance

The most practically significant suggestion in Vitalik's lengthy article is not to immediately pause AI, but to agree in advance:

What evidence must emerge for all parties to change their judgments?

Today, asking everyone to agree that "superintelligence will appear in 2030" or "will not appear in the next twenty years" is nearly impossible.

However, different camps can still agree in advance on some risk signals.

For example:

Has AI been able to independently complete R&D tasks for several weeks?

Can it significantly accelerate the development of the next generation of AI?

Does it possess the ability to discover and exploit network vulnerabilities on a large scale?

Has it begun to autonomously replicate, acquire resources, or bypass supervision?

Has it been integrated into weapons, financial systems, and critical infrastructure?

Has it already caused sustained large-scale social harm?

The focus is not on finding a perfect indicator, but on shifting the debate from "Do you believe in superintelligence?" to "What real evidence would prompt you to update your judgment?"

Those who believe AI risks are exaggerated can accept this because they trust that these situations will not occur.

Those concerned about superintelligence can also accept it because they believe these signals may appear soon.

Both sides do not need to unify their worldviews first; they only need to commit in advance:

When reality changes, policies must also change.

Interventions Should Not Only Have Two Settings: Continue or Shutdown

Truly executable AI governance should not directly jump between "completely free development" and "global shutdown."

A more reasonable approach is to establish graded interventions.

When risks are low, increase capability disclosures, incident reporting, and third-party testing.

As risks continue to rise, restrict model access to critical infrastructure, limit high-risk permissions such as autonomous execution, fund transfers, and weapon control.

If AI has clearly automated AI R&D, or multiple serious signals appear simultaneously, then restrict new frontier training and large-scale computing power expansion.

Only when lower-intensity measures fail to control risks should a broader, shorter pause be initiated.

This graded design is not meant to weaken regulation, but to ensure that the intensity of policies matches the strength of evidence.

The truly dangerous system is often not one that lacks the ability to take strong measures, but one that can skip all intermediate steps and directly gain unlimited power under the pretext of "emergency state."

Conclusion

The real difficulty in AI governance lies not in the lack of positions, but in the absence of a system that can accommodate judgment errors.

Continuing the competition is not a neutral choice.

Global pauses are also not a neutral choice.

Open models are not inherently safe, and closed models are not inherently responsible.

A credible governance mechanism must at least answer four questions:

What evidence does it act upon?

Does it equally constrain those in power?

Can it be externally verified?

Can power be revoked after a judgment error?

In the AI era, perhaps we indeed need a pause button.

But the real challenge of the pause button has never been whether it can be created.

Rather, it is:

Who has the power to press it, who has the power to restart it, and how to ensure that no one can turn this button into their throne?

Disclaimer: This content is provided for general branding and informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online events, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets or to use any services. Crypto assets are highly volatile and may result in loss. WEEX services and online events may not be available in all regions and are subject to applicable laws, regulations, and eligibility requirements. You are responsible for ensuring that your use of WEEX services complies with local laws and for carefully assessing the risks before participating in any crypto-related activities.