I have written previously about several aspects of AI and cybersecurity, including issues such as prompt injection attacks on LLMs, deepfakes, data poisoning and steganography, where secret messages are hidden inside images. What is the state of play today?
There is no doubt that cybersecurity is one of the main concerns around AI, and LLMs in particular. Because LLMs are usually accessed via a chat interface rather than a carefully controlled input screen, chat-based and agentic applications give attackers numerous opportunities to submit content intended to subvert an LLM. The basic issue of trying to prevent a hacker from injecting a malicious command into an LLM’s input text is a difficult problem to defend against. Leading security institutions such as the UK’s National Cyber Security Centre (NCSC) and the OWASP Foundation classify prompt injection as an inherent design flaw of the LLM transformer architecture. In regular computing, data and code are separate. Systems can access data by passing SQL queries to a database, with the processing of the returned data being separate. By contrast, LLMs process data and instructions together. There is no reliable way to programmatically block data from overriding instructions without breaking the model’s ability to understand context.
Given this, security engineers use a layered defence to try to minimise the damage that an attack can do. We discussed earlier the use of a harness around an LLM. This includes the provision of a guard layer to check inputs for adversarial content before they are passed to the main LLM. A second precaution is in terms of access rights. Best practice is not to give an LLM unrestricted authority to execute API calls. Instead, the model outputs a structured intent (such as a JSON object), which is then rigorously validated by a traditional deterministic system before any real-world action, like sending an email or deleting data, is carried out. In addition, the frontier labs carry out extensive tests during the reinforcement learning stage of LLM training. This teaches models to ignore instructional language when it originates from a data block. Frontier laboratories use adversarial training, reinforcement learning and extensive red-team testing to encourage models to distinguish trusted instructions from untrusted content.
A newer technique is called context bombing, whereby defensive prompts are placed alongside sensitive data like passwords or cryptographic keys or planted decoy data. If an attacking LLM encounters these, it may itself be hijacked – ironically through a form of reverse prompt injection – and induced to stop or alter its behaviour. Early research suggests that this may reduce the success rate of some prompt-injection attacks, although it is not a complete defence. Even so, prompt injection remains the single most dangerous security exposure for LLMs.
AI models themselves can be induced to generate malware, and are proving very effective at detecting security vulnerabilities in regular computer systems. In May 2026, Anthropic initially restricted access to its Mythos model because of its unusually strong cybersecurity capabilities. Through a programme involving vetted partners, it was used to identify thousands of potential vulnerabilities while software maintainers were given time to validate and repair them before details became public. The model was released only to a series of trusted vendors in preview form, so that they could have time to work with it and fix the vulnerabilities that Mythos found before general public release. This was a major exercise, as the model found thousands of previously unknown vulnerabilities in browsers and operating systems.
In July 2026, OpenAI had its own security problems when two OpenAI models (GPT-5.6 Sol and a more capable pre-release model) were given the task of trying to break out from a supposedly secure environment at the lab. Not only did the models identify vulnerabilities in the sandbox infrastructure, they then broke out, exploiting a vulnerability in a component of the evaluation infrastructure that had limited internet access. They then hacked into the systems of Hugging Face, an AI platform, seeking answers to the benchmark on which they were being evaluated. Hugging Face was only able to defend against this by using a Chinese open weight model (GLM), which was (having fewer guardrails) more relaxed than the US frontier models about working to identify and plug the security weakness. Somewhat worryingly, the sandbox escape went undetected for five days by OpenAI.
This incident showed that LLM models can be used for both offensive and defensive cybersecurity purposes. The ability to defend a network by understanding code, analysing logs, reasoning about systems and generating scripts can equally be used to attack one. Models can be used to automatically analyse and prioritise security alerts, identify unusual behaviour, detect malware, find weaknesses in systems like hard-coded passwords, and generate security tests. Of course, an attacker can use a model to do much the same thing, but with malicious intent. Increasingly, cybersecurity may depend in part on who can deploy capable AI models most effectively and integrate them most successfully into security operations. An LLM can also help develop security training for humans, who remain a major source of security risk.
This arms race between offensive and defensive AI models will continue to develop. These vulnerabilities are not theoretical in nature. One April 2026 report by Proofpoint found that about half of companies had experienced an AI attack. A separate April 2026 Gravitee survey of 750 companies found that 54% had already had a security incident involving AI agents, while 90% had unmonitored AI agents in production. The odds, sadly, favour the attacker. An attacker needs to find one security weakness, whereas a defender needs to secure every vulnerability. AI is therefore unlikely to eliminate cyberattacks. Instead, it is changing the balance of power by giving both attackers and defenders far more capable tools. Success will depend not simply on adopting AI, but on adopting it faster, more intelligently and with stronger governance than one’s adversaries.
