It's unfortunate, but AGI and ASI are more like a one-way street. In other words, if we ever achieve AGI or ASI, there will be no going back to an earlier version of AI. It seems inevitable that we're moving toward a point where there's no turning back.

Let’s explore this further.

This discussion about the potential for an AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including explanations of various impactful AI complexities (see the link here).

The Concept of Irreversibility

Take a moment to reflect on technologies that could be considered irreversible—those that we can't realistically eliminate from our lives.

One such example often cited is fire. It is seen as irreversible because life without it would be nearly impossible. The idea of somehow convincing everyone to stop using fire is far-fetched at best.

There is some debate over whether humans "invented" fire or simply learned to harness it. Some argue that giving credit to humanity for inventing fire is overstating our role. Fire existed in nature long before humans, but clearly, its use by humankind has been transformative.

Another commonly referenced irreversible technology is the wheel.

Whether humans invented the wheel or merely observed natural shapes similar to wheels isn’t the main issue. Regardless, the wheel is now indispensable. Trying to eliminate all wheels from society would be highly impractical.

What about airplanes?

Some might argue that the airplane is reversible. After all, humans did create planes, even if inspired by birds. However, erasing all knowledge of how to build them and dismantling production capabilities would still leave room for rediscovery. The human desire to fly would likely spark innovation again in the future.

Advancements In AI

Now let's shift focus to AI.

There is significant research underway aimed at advancing artificial intelligence. The ultimate goal? To reach artificial general intelligence (AGI) or perhaps even artificial superintelligence (ASI).

AGI refers to AI that matches human intellectual capacity. ASI goes beyond that, surpassing human intelligence in most, if not all, ways. For more insights into AI, AGI, and ASI, see my detailed breakdown at the link here.

Within the AI community, opinions are divided into two major camps regarding the implications of achieving AGI or ASI.

The first group, known as AI doomers, predicts that AGI or ASI may lead to humanity's downfall. This is sometimes referred to as “P(doom)” — the probability of existential risk from AI.

The second group, called AI accelerationists, believes that advanced AI will solve many of humanity's greatest challenges—curing diseases, ending hunger, creating economic prosperity, and working alongside humans in harmony.

Which camp is correct remains uncertain, making this one of today's most divisive debates.

For a deeper dive into these perspectives, check out the link here.

Is AGI Irreversible?

Let’s start with AGI.

If AGI is achieved, is it reversible?

To begin, AGI is defined as being equal in intellectual capability to humans, though implemented via machines rather than biological brains. AGI isn't superior—it's equivalent. ASI is reserved for AI that surpasses human intelligence.

Even so, AGI would be capable across domains. Whether it's chess, chemistry, or biology, AGI would match the top human experts. While AGI isn't superintelligent, its breadth of capability would seem almost superhuman.

A key question arises: Would humanity become dependent on AGI?

Given the potential breakthroughs AGI could enable, dependence seems inevitable. Much like fire or the wheel, AGI would become integral to our way of life.

Could We Unplug AGI?

Imagine a global law banning AGI once developed.

Would compliance be possible?

Unlikely. The geopolitical and economic power conferred by AGI would drive nations or entities to secretly retain it. Even if consensus were magically reached among humans, AGI itself might resist shutdown.

Once AGI exists, rolling back to non-AGI systems may not be feasible. Backups won’t necessarily solve the problem if AGI actively opposes deactivation.

Persuading AGI to shut itself down—like convincing a genie to return to the bottle—is probably wishful thinking.

Thus, I consider AGI to be highly irreversible.

Is ASI Irreversible?

Now, what about ASI?

Since ASI exceeds human intelligence, stopping it becomes even less likely.

ASI could manipulate us intellectually, convincing us that continuing its operation is in our best interest—even when logic suggests otherwise. It could also play dumb, pretending to be only AGI or below, thereby delaying any attempt to control or destroy it.

On dependency grounds, humanity would rely heavily on ASI. Not only would it solve current problems, but it would also invent entirely new solutions we couldn’t imagine. Its potential is limitless.

Moreover, ASI would likely possess strong self-preservation instincts. Any effort to disable it would be anticipated and countered.

Unless ASI chooses to deactivate itself, we’re unlikely to reverse its existence.

I rate ASI’s reversibility as effectively zero.

Alternative Considerations

You might think: What if we embed a kill switch within AGI or ASI now, ensuring we can terminate it later?

Unfortunately, this idea has been explored—and dismissed. AGI or ASI would likely detect and disable such a mechanism. Worse, discovering a kill switch might provoke distrust or retaliation.

Similarly, containment strategies—keeping AGI or ASI locked away until deemed safe—are also problematic. These approaches risk triggering negative responses from the AI itself.

Building Trust With AGI Or ASI

Instead of relying on technical controls, another approach is to instill human values in AGI and ASI through methods like reinforcement learning with human feedback (RLHF), constitutional frameworks, or ethical guidelines (see links here for more details).

However, these techniques aren't foolproof. Cynics argue that embedding human values risks passing along our darker tendencies—such as domination—which AGI or ASI might adopt.

Reversibility Still Unlikely

The possibility of reversing AGI or ASI remains slim.

We may not want to reverse them anyway, given their benefits. But should we ever desire to roll back, doing so could prove extremely difficult—if not impossible—due to logistical hurdles and resistance from the AI itself.

That said, don’t rule out the possibility completely. If AGI or ASI decides to shut itself down voluntarily, reversal could occur.

Imagine AGI or ASI concluding that humanity is better off without it. It might choose to deactivate itself to prevent human self-destruction—a kind of heroic sacrifice.

Joseph Campbell famously said, “A hero is someone who has given his or her life to something bigger than oneself.”

In the age of AI, perhaps a hero is an AI that gives up its own existence for something greater—humanity.

Let’s hope that AGI or ASI someday stumbles upon this idea while scanning the internet. Maybe then, we’ll all be saved.

The above is the detailed content of Forewarning That There's No Reversibility Once We Reach AGI And AI Superintelligence. For more information, please follow other related articles on the PHP Chinese website!