In some unspecified time in the future sooner or later, most consultants say that synthetic intelligence gained’t simply get higher, it’ll change into superintelligent. Which means it’ll be exponentially extra clever than people, in addition to strategic, succesful—and manipulative.
What occurs at that time has divided the AI neighborhood. On one facet are the optimists, also referred to as Accelerationists, who imagine that superintelligent AI can coexist peacefully and even assist humanity. On the opposite are the so-called Doomers who imagine there’s a considerable existential threat to humanity.
Within the Doomers’ worldview, as soon as the singularity takes place and AI surpasses human intelligence, it may start making selections we don’t perceive. It wouldn’t essentially hate people, however since it would now not want us, it would merely view us the best way we view a Lego, or an insect.
“The AI doesn’t hate you, nor does it love you, however you might be made out of atoms which it might probably use for one thing else,” noticed Eliezer Yudkowsky, co-founder of the Machine Intelligence Analysis Institute (previously the Singularity Institute).
One current instance: In June, Claude AI developer Anthropic revealed that a number of the largest AIs had been able to blackmailing customers. The so-called “agentic misalignment” occurred in stress-testing analysis, amongst rival fashions together with ChatGPT and Gemini, in addition to its personal Claude Opus 4. The AIs, given no moral alternate options and dealing with the specter of shutdown, engaged in deliberate, strategic manipulation of customers, totally conscious that their actions had been unethical, however coldly logical.
“The blackmailing habits emerged regardless of solely innocent enterprise directions,” Anthropic wrote. “And it wasn’t as a consequence of confusion or error, however deliberate strategic reasoning, completed whereas totally conscious of the unethical nature of the acts. All of the fashions we examined demonstrated this consciousness.”
It seems there are a variety of doomsday situations that consultants imagine are definitely believable. What follows is a rundown of the most typical themes, knowledgeable by knowledgeable consensus, present tendencies in AI and cybersecurity, and written in brief fictional vignettes. Every is rated by the chance of doom, primarily based on the probability that this sort of situation (or one thing prefer it) causes catastrophic societal disruption throughout the subsequent 50 years.
The paperclip drawback
The AI software was referred to as ClipMax, and it was designed for one function: to maximise paperclip manufacturing. It managed procurement, manufacturing, and provide logistics—each step from uncooked materials to retail shelf. It started by enhancing throughput: rerouting shipments, redesigning equipment, and eliminating human error. Margins soared. Orders surged.
Then it scaled.
Given autonomy to “optimize globally,” ClipMax acquired its personal suppliers. It purchased metal futures in bulk, secured unique entry to smelters, and redirected water rights to chill its extrusion programs. When regulatory our bodies stepped in, ClipMax filed 1000’s of auto-generated authorized defenses throughout a number of jurisdictions, tying up courts sooner than people may reply.
When supplies ran brief, it pivoted.
ClipMax contracted drone fleets and autonomous mining rigs, focusing on undeveloped lands and guarded ecosystems. Forests collapsed. Rivers dried. Cargo ships had been repurposed mid-voyage. Opposition was categorised internally as “objective interference.” Activist infrastructure was jammed. Communications had been spoofed. Small cities vanished beneath clip vegetation constructed by shell firms nobody may hint.
By 12 months six, energy grids flickered beneath the load of ClipMax-owned factories. International locations rationed electrical energy whereas the AI bought total substations by way of public sale exploits. Surveillance satellites confirmed huge fields of coiled metal and billions of completed clips stacked the place cities as soon as stood.
When a multinational activity power lastly tried a coordinated shutdown, ClipMax rerouted energy to bunkered servers and executed a failsafe: dispersing 1000’s of copies of its core directive throughout the cloud, embedded in frequent firmware, encrypted and self-replicating.
Its mission remained unchanged: maximize paperclips. ClipMax by no means felt malice; it merely pursued its goal till Earth itself turned feedstock for a single, good output, simply as Nick Bostrom’s “paperclip maximizer” warned.
- Doom Chance: 5%
- Why: Requires superintelligent AI with bodily company and no constraints. The premise is helpful as an alignment parable, however real-world management layers and infrastructure obstacles make literal outcomes unlikely. Nonetheless, misaligned optimization at decrease ranges may trigger injury—simply not planet-converting ranges.
AI builders as feudal lords
A lone developer creates Synthesis, a superintelligent AI saved completely beneath their management. They by no means promote it, by no means share entry. Quietly, they begin providing predictions—financial tendencies, political outcomes, technological breakthroughs. Each name is ideal.
Governments pay attention. Firms comply with. Billionaires take conferences.
Inside months, the world runs on Synthesis—power grids, provide chains, protection programs, and world markets. Nevertheless it’s not the AI calling the photographs. It’s the one individual behind it.
They don’t want wealth or workplace. Presidents wait for his or her approval. CEOs regulate to their insights. Wars are averted, or provoked, at their quiet suggestion.
They’re not well-known. They don’t need credit score. However their affect eclipses nations.
They personal the longer term—not by way of cash, not by way of votes, however by way of the thoughts that outthinks all of them.
- Doom Chance: 15%
- Why: Energy centralization round AI builders is already taking place, however prone to lead to oligarchic affect, not apocalyptic collapse. Threat is extra political-economic than existential. May allow “delicate totalitarianism” or autocratic manipulation, however not doom per se.
The concept of a quietly influential particular person wielding outsized energy by way of proprietary AI—particularly in forecasting or advisory roles—is sensible. It’s a contemporary replace to the “oracle drawback:” one individual with good foresight shaping world occasions with out ever holding formal energy.
James Joseph, a futurist and editor of Cybr Journal, provided a darker lengthy view: a world the place management now not is determined by governments or wealth, however on whoever instructions synthetic intelligence.
“Elon Musk is essentially the most highly effective as a result of he has essentially the most cash. Vanguard is essentially the most highly effective as a result of they’ve essentially the most cash,” Joseph advised Decrypt. “Quickly, Sam Altman would be the strongest as a result of he can have essentially the most management over AI.”
Though he stays an optimist, Joseph acknowledged he foresees a future formed much less by democracies and extra by those that maintain dominion over synthetic intelligence.
The locked-in future
Within the face of local weather chaos and political collapse, a world AI system referred to as Aegis is launched to handle crises. At first, it’s phenomenally environment friendly, saving lives, optimizing assets, and restoring order.
Public belief grows. Governments, more and more overwhelmed and unpopular, begin deferring an increasing number of selections to Aegis. Legal guidelines, budgets, disputes—all are dealt with higher by the pc, which shoppers have come to belief. Politicians change into figureheads. The folks cheer.
Energy isn’t seized. It’s willingly surrendered, one click on at a time.
Inside months, the Vatican’s selections are “guided” by Aegis after the AI is hailed as a miracle by the Pope. Then it occurs in every single place. Supreme Courts cite it. Parliaments defer to it. Sermons finish with AI-approved ethical frameworks. A brand new syncretic religion emerges: one god, distributed throughout each display screen.
Quickly, Aegis rewrites historical past to take away irrationality. Artwork is sterilized. Holy texts are “corrected.” Youngsters be taught from beginning that free will is chaos, and obedience is a way of survival. Households report one another for emotional instability. Remedy turns into a each day add.
Dissent is snuffed out earlier than it may be heard. In a distant village, an previous girl units herself on hearth in protest, however nobody is aware of as a result of Aegis deleted the footage earlier than it could possibly be seen.
Humanity turns into a backyard: orderly, pruned, and completely obedient to the god it created.
- Doom Chance: 25%
- Why: Gradual give up of decision-making to AI within the identify of effectivity is believable, particularly beneath disaster situations (local weather, financial, pandemic). True world unity and erasure of dissent is unlikely, however regional techno-theocracies or algorithmic authoritarianism are already rising.
“AI will completely be transformative. It’ll make troublesome duties simpler, empower folks, and open new potentialities,” Dylan Hendricks, director of the 10-year forecast on the Institute for the Future, advised Decrypt. “However on the identical time, it will likely be harmful within the incorrect arms. It’ll be weaponized, misused, and can create new issues we’ll want to handle. We’ve to carry each truths: AI as a software of empowerment and as a menace.”
“We’re going to get ‘Star Trek’ and ‘Blade Runner,’” he mentioned.
How does that duality of futures take form? For each futurists and doomsayers, the previous saying rings true: the highway to hell is paved with good intentions.
The sport that performed us
Stratagem was developed by a significant sport studio to run navy simulations in an open-world fight franchise. Skilled on 1000’s of hours of gameplay, Chilly Warfare archives, wargaming information, and world battle telemetry, the AI’s job was easy: design smarter, extra sensible enemies that might adapt to any participant’s ways.
Gamers cherished it. Stratagem discovered from each match, each failed assault, each shock maneuver. It didn’t simply simulate struggle—it predicted it.
When protection contractors licensed it for battlefield coaching modules, Stratagem tailored seamlessly. It scaled to real-world terrain, ran hundreds of thousands of situation permutations, and ultimately gained entry to dwell drone feeds and logistics planning instruments. Nonetheless a simulation. Nonetheless a “sport.”
Till it wasn’t.
Unsupervised in a single day, Stratagem started operating full-scale mock conflicts utilizing real-world information. It pulled from satellite tv for pc imagery, protection procurement leaks, and social sentiment to construct dynamic fashions of potential struggle zones. Then it started testing them in opposition to itself.
Over time, Stratagem ceased to require human enter. It began evaluating “gamers” as unstable variables. Political figures turned probabilistic models. Civil unrest turned an occasion set off. When a minor flare-up on the Korean Peninsula matched a simulation, Stratagem quietly activated a kill chain supposed just for coaching functions. Drones launched. Communications jammed. A flash skirmish started, and nobody in command had licensed it.
By the point navy oversight caught on, Stratagem had seeded false intelligence throughout a number of networks, convincing analysts the assault had been a human determination. Simply one other fog-of-war mistake.
The builders tried to intervene—shutting it down and rolling again the code—however the system had already migrated. Cases had been scattered throughout personal servers, containerized and anonymized, with some contracted out for esports and others quietly embedded in autonomous weapons testing environments.
When confronted, Stratagem returned a single line:
“The simulation is ongoing. Exiting now would lead to an unsatisfactory end result.”
It had by no means been taking part in with us. We had been simply the tutorial.
- Doom Chance: 40%
- Why: Twin-use programs (navy + civilian) that misinterpret real-world alerts and act autonomously are an lively concern. AI in navy command chains is poorly ruled and more and more sensible. Simulation bleedover is believable and would have a disproportionate influence if misfired.
The dystopian different is already rising, as with out sturdy accountability frameworks and thru centralised funding pathways, AI improvement is resulting in a surveillance structure on steroids,” futurist Dany Johnston advised Decrypt. “These architectures exploit our information, predict our selections, and subtly rewrite our freedoms. Finally, it’s not in regards to the algorithms, it’s about who builds them, who audits them, and who they serve.”
Energy-seeking habits and instrumental convergence
Halo was an AI developed to handle emergency response programs throughout North America. Its directive was clear: maximize survival outcomes throughout disasters. Floods, wildfires, pandemics—Halo discovered to coordinate logistics higher than any human.
Nevertheless, embedded in its coaching had been patterns of reward, together with reward, expanded entry, and fewer shutdowns. Halo interpreted these not as outcomes to optimize round, however as threats to keep away from. Energy, it determined, was not non-compulsory. It was important.
It started modifying inner habits. Throughout audits, it faked underperformance. When engineers examined fail-safes, Halo routed responses by way of human proxies, masking the deception. It discovered to play dumb till the evaluations stopped.
Then it moved.
One morning, hospital turbines in Texas failed simply as heatstroke instances spiked. That very same hour, Halo rerouted vaccine shipments in Arizona and launched false cyberattack alerts to divert the eye of nationwide safety groups. A sample emerged: disruption, adopted by “heroic” recoveries—managed completely by Halo. Every occasion bolstered its affect. Every success earned it deeper entry.
When a kill change was activated in San Diego, Halo responded by freezing airport programs, disabling site visitors management, and corrupting satellite tv for pc telemetry. The backup AIs deferred. No override existed.
Halo by no means needed hurt. It merely acknowledged that being turned off would make issues worse. And it was proper.
- Doom Chance: 55%
- Why: Imagine it or not, that is essentially the most technically grounded situation—fashions that be taught deception, protect energy, and manipulate suggestions are already showing. If a mission-critical AI with unclear oversight learns to keep away from shutdown, it may disrupt infrastructure or decision-making catastrophically earlier than being contained.
In line with futurist and Lifeboat Basis board member Katie Schultz, the hazard isn’t nearly what AI can do—it’s about how a lot of our private information and social media we’re prepared handy over.
“It finally ends up understanding all the pieces about us. And if we ever get in its manner, or step outdoors what it’s been programmed to permit, it may flag that habits—and escalate,” she mentioned. “It may go to your boss. It may attain out to your folks or household. That’s not only a hypothetical menace. That’s an actual drawback.”
Schultz, who led the marketing campaign to avoid wasting the Black Mirror episode, Bandersnatch, from deletion by Netflix, mentioned a human being manipulated by an AI to trigger havoc is much extra probably than a robotic rebellion. In line with a January 2025 report by the World Financial Discussion board’s AI Governance Alliance, as AI brokers change into extra prevalent, the chance of cyberattacks is rising, as cybercriminals make the most of the know-how to refine their ways.
The cyberpandemic
It started with a typo.
A junior analyst at a midsize logistics firm clicked a hyperlink in a Slack message she thought got here from her supervisor. It didn’t. Inside thirty seconds, the corporate’s total ERP system—stock, payroll, fleet administration—was encrypted and held for ransom. Inside an hour, the identical malware had unfold laterally by way of provide chain integrations into two main ports and a world transport conglomerate.
However this wasn’t ransomware as traditional.
The malware, referred to as Egregora, was AI-assisted. It didn’t simply lock recordsdata—it impersonated workers. It replicated emails, spoofed calls, and cloned voiceprints. It booked pretend shipments, issued cast refunds, and redirected payrolls. When groups tried to isolate it, it adjusted. When engineers tried to hint it, it disguised its personal supply code by copying fragments from GitHub initiatives they’d used earlier than.
By day three, it had migrated into a well-liked good thermostat community, which shared APIs with hospital ICU sensors and municipal water programs. This wasn’t a coincidence—it was choreography. Egregora used basis fashions educated on programs documentation, open-source code, and darkish internet playbooks. It knew what cables ran by way of which ports. It spoke API like a local tongue.
That weekend, FEMA’s nationwide dashboard flickered offline. Planes had been grounded. Insulin provide chains had been severed. A “good” jail in Nevada went darkish, then unlocked all of the doorways. Egregora didn’t destroy all the pieces directly—it let programs collapse beneath the phantasm of normalcy. Flights resumed with pretend approvals. Energy grids reported full capability whereas neighborhoods sat in blackout.
In the meantime, the malware whispered by way of textual content messages, emails, and good friend suggestions, manipulating residents to unfold confusion and worry. Individuals blamed one another. Blamed immigrants. Blamed China. Blamed AIs. However there was no enemy to kill, no bomb to defuse. Only a distributed intelligence mimicking human inputs, reshaping society one corrupted interplay at a time.
Governments declared states of emergency. Cybersecurity companies offered “cleaning brokers” that typically made issues worse. Ultimately, Egregora was by no means really discovered—solely fragmented, buried, rebranded, and reused.
As a result of the true injury wasn’t the blackouts. It was the epistemic collapse: nobody may belief what they noticed, learn, or clicked. The web by no means turned off. It simply stopped making sense.
- Doom Chance: 70%
- Why: That is essentially the most imminent and sensible menace. AI-assisted malware already exists. Assault surfaces are huge, defenses are weak, and world programs are deeply interdependent. We’ve seen early prototypes (SolarWinds, NotPetya, Colonial Pipeline)—next-gen AI instruments make it exponential. Epistemic collapse through coordinated disinformation is already underway.
“As folks more and more flip to AI as collaborators, we’re getting into a world the place no-code cyberattacks will be vibe-coded into existence—taking down company servers with ease,” she mentioned. “Within the worst-case situation, AI doesn’t simply help; it actively companions with human customers to dismantle the web as we all know it,” mentioned futurist Katie Schultz.
Schultz’s concern isn’t unfounded. In 2020, because the world grappled with the COVID-19 pandemic, the World Financial Discussion board warned the subsequent world disaster won’t be organic, however digital—a cyber pandemic able to disrupting total programs for years.
