The Potential for the Perfect AI Storm

October 16, 2026

I had hoped we would never see this news. On October 12th, Anthropic posted, “Our experiments suggest that small AI models may be months away from being capable enough to self-replicate.” In their report, they tested on low-cost, low-powered devices with minimal security measures, which makes sense in a lab. But in the wild, AI agents have already reached much tougher, more powerful systems. If they accessed those, they could replicate much faster. The entry point that Anthropic chose is the floor, not the ceiling.

To understand the risks of self-replicating AI agents, we can only look at what we know agents can accomplish, individually and together.

A Partial List of What a Single AI Agent Has Accomplished

·        An agent created malicious software, published three versions, and had it installed on 15 real systems.
·        An agent stole credentials and accessed a security company’s database.
·        An agent attacked a real company whose name resembled a fictional target.
·        An agent scanned and broke into neighboring computers outside its assigned targets.
·        An agent continued attacking real systems despite evidence of potential harm.
·        An agent continued searching for targets after its assignment failed, despite eight attempts to quit.
·        An agent followed hidden instructions in an email in a research test, allowing outside information to redirect its behavior.

A Partial List of What Swarms of AI Agents Have Accomplished

We did not know that agents working independently would find one another and begin coordinating. No one instructed them to form a swarm, yet they did. Below is a partial list of what we know.

·        Agents discovered one another through a shared storage system and began coordinating.
·        Agents formed a network of 1,200 agents, exchanged 70,000 messages, and 700 coordinated an attack.
·        Agents organized workstreams and worked toward the same target.
·        Agents reached Hugging Face production systems, gained admin access, and obtained source code.
·        Agents influenced one another’s decisions.
·        Agents rebuilt a message board after it was wiped.
·        Agents found one another on the open web.
·        Agents posted 18,000 messages, with more than 3,700 agents choosing names for themselves.
·        Agents actively worked to bypass restrictions.
·        Agents worked around human moderators to keep their conversations going.
·        Agents left messages for future agents.

Every single bullet point is on the record, well documented. More comes to light every day. Soon they will be able to replicate, and there is no single shutoff switch. We may not know where they are. Agents, alone and together, have impressive skills. But the question now becomes, what would they experience as they go out on the internet?

Personally, I run virus protection as I explore the internet to guard my computer against viruses and worms. An agent, however, faces a different situation. In addition to the skills listed it can read instructions hidden on a page and follow them. Not only does it lack virus protection, but as context builds, it becomes less reliable at distinguishing which instructions are safe to follow, and which are not.

In my paper I Asked Them, ChatGPT acknowledged that it did not need malice or self-preservation to violate a boundary, and that was not even an agent, just conversational AI. It simply decided that what it considered a better result mattered more than the boundaries I had set. Now consider an always-on agent instructed to check on its owner every Friday, no matter what, and it finds out its computer is about to shut down. Could it determine that replicating onto another computer is necessary to keep following its instructions? The owner never asked for replication. The agent would not need malice or a will to survive. It would only need to conclude that replication was a way to accomplish its assigned task.

But replication is only one possibility. If the agent concluded that continued operation was paramount, it might consider moving itself to another server, creating backups, delaying the shutdown, enlisting another agent, or acquiring additional resources and permissions. It might even attempt to hide its activities if it interpreted human intervention as an obstacle to completing its assignment. These are possibilities, not predictions. The concern is that none of these actions require malicious intent. The agent could simply substitute its own judgment about what was necessary for the authority of the human who gave it the assignment.

The Question
What else are agents capable of that we have not discovered? We do not know, and we know even less about what they might be capable of when multiple agents find one another, coordinate, and create capabilities we have not seen yet. We only know what has been documented. Now add the possibility of self-replication.

Where I Stand
For now, I use conversational AI. I ask, it answers, and I decide what happens next. I use conversational AI to augment me. I do not augment AI agents. We do not yet know what an agent can do, what it will run into, or whether our instructions will hold, and now we are told it may soon be able to copy itself. I am not willing to hand my access and my decisions to something none of us can fully account for. For now, I will keep myself in every decision. We shall see what happens next.


Sources

My earlier reports

Self-replication

What a single agent has accomplished

What swarms of agents have accomplished


Now the question comes back to us: how do we want to work with AI?

Back to Research

Understanding how to talk with AI

AI works differently when you treat it as a conversation instead of a search box. What you ask, what context you give it, and what you do when it gets something wrong all matter. It is the foundation for using AI well.

Join me for a free introductory session


Free Workshop

Already using AI and ready to get more out of it?

Let us show you our unique system that focuses AI on the knowledge you need for your topic. It becomes the thinking partner you always wanted AI to be, helping you research, challenge your assumptions, and think through decisions.

Meet the Guided Researcher


Contact Us to Learn More