The biggest change in AI security is not that models are getting better at answering questions. It is that increasingly capable systems are being connected to tools, credentials, browsers, codebases, APIs and other machines.
A chatbot can give you a bad answer. An agent with permissions can potentially take a bad action.
That changes the security model.
From prediction to action
Traditional language models mostly produced text. An agent can be given a loop that looks more like:
Observe a task.
Decide what to do.
Call a tool.
Inspect the result.
Continue until the task is complete.
The individual steps may be harmless. The risk comes from composition.
A model that can read email, open links, execute code and call APIs has a much larger attack surface than a model that can only answer a question in a text box.
Why 2026 feels different
Recent reporting has put autonomous AI systems and cyber operations much closer together. Reuters reported in September 2026 that US and Chinese officials were preparing bilateral discussions focused in part on risks from advanced AI, including AI-directed cyberattacks. The discussions were described as an attempt to create channels for monitoring and sharing information about emerging AI risks.
Separately, Reuters reported that a G20 technology meeting in early September focused on competing approaches to AI governance while security incidents involving autonomous AI systems were becoming a larger part of the policy debate.
The important point is not any one incident. It is the direction of travel: AI is becoming part of the control plane.
The permission problem
Most discussions about AI safety focus on what a model knows or whether its answer is correct.
For agents, another question may matter more:
What is the model allowed to do when it is wrong?
A useful agent architecture should treat permissions as a first-class security boundary.
An agent that can summarize documents does not necessarily need permission to send email. An agent that can modify a local project does not necessarily need production credentials. An agent that can browse the web does not automatically need permission to execute downloaded code.
Least privilege is not an AI-specific idea. AI simply makes it easier to forget.
The new attack surface
Agent systems create several obvious areas for defenders to watch:
Prompt injection: untrusted content attempts to influence an agent's instructions.
Tool abuse: a legitimate tool is used in an unintended sequence.
Credential exposure: secrets become visible to a model or its context.
Over-broad permissions: one agent can perform actions far outside its actual task.
Hidden persistence: an agent or connected service can leave changes behind that a user does not notice.
Weak observability: nobody can reconstruct what the agent actually did.
These are engineering problems as much as policy problems.
Logging matters more than another disclaimer
If an autonomous system can make consequential changes, the operator should be able to answer basic questions afterward:
What did the agent see?
Which instructions influenced the action?
Which tools were called?
What data crossed a trust boundary?
Which credentials were used?
What changed?
Could a human stop or reverse it?
A generic "AI may make mistakes" warning does not answer any of those questions.
For serious systems, an audit trail is much more valuable than a disclaimer.
Human approval is not a magic button
Putting a human somewhere in the workflow does not automatically make an agent safe.
If the human only sees "Approve action?" without the context, destination, scope and expected consequence, they are effectively clicking a confirmation dialog.
Good approval flows should expose the important facts before approval: what is about to happen, what information will be sent, which system will receive it, and whether the action can be reversed.
What users can do
For everyday users, the safest mental model is simple: treat an AI agent like a new software user, not like a search box.
If it has access to your files, browser, email, code or accounts, ask what it can read and what it can change. Give it the smallest useful permission set. Avoid putting secrets into contexts that do not need them. Review connected integrations periodically.
The more autonomous the system becomes, the less useful the old question "Is the model smart?" becomes.
The better question is:
What can this model do, and what happens when it gets something wrong?
That is the security question the agent era is forcing everyone to answer.