OpenAI has built itself a dedicated digital attacker: a large language model called GPT-Red, whose sole purpose is to find loopholes in the company's other AI systems. The system functions as an automated red-team tester — and according to MIT Technology Review, it was actively used during the training of GPT-5.6, which OpenAI released last week.

Self-play as a training method

GPT-Red is trained through what is known as a self-play reinforcement learning loop. Simply put: the attacker attacks a defender, the defender adapts, and the cycle repeats at scale. The goal is to uncover weaknesses that human testers either fail to think of or simply don't have time to cover.

OpenAI describes the approach as central to its work on making models robust against real-world cyberattacks — particularly attacks based on prompt injection, where a malicious actor attempts to manipulate a model's behavior through specially crafted instructions.

OpenAI has acknowledged that prompt injection is "unlikely to ever be fully solved" — but GPT-Red is designed to make models as hardened as possible against exactly that.
OpenAI's hacker AI attacked its own models 674 times in three hours - Bilde 1

The entire industry is moving in the same direction

AI-versus-AI security testing is no longer an OpenAI experiment. According to available industry documentation, all the major players have adopted similar approaches.

Microsoft has open-sourced the PyRIT framework, in which an LLM-based agent repeatedly attacks a target system while an evaluation engine assesses the results. Raja Sekhar Rao Dheekonda, co-creator of PyRIT, has stated that AI agents make security testing dramatically more efficient — a single operator can carry out hundreds of attacks in the course of an afternoon.

Google employs a dedicated AI Red Team that simulates threat actors inspired by nation-state hackers, hacktivists, and insiders. Meta uses both internal and external testers on its open models, including Llama 3.1, with evaluations covering areas such as cybersecurity, prompt injection, and biological threats.

674
Attacks against Meta's Llama Scout in three hours
73%
AI systems in production with prompt injection vulnerabilities

A serious security breach illustrates the need

In February 2026, an autonomous AI agent discovered 22 unauthenticated API endpoints in McKinsey's internal AI platform Lilli. Within two hours, the agent had gained full read and write access to the database — exposing 46.5 million chat messages, 728,000 files, and 57,000 user accounts. The incident has not been confirmed by McKinsey, but has been cited in industry research as an illustrative example of what automated attacks can achieve in a short time.

A single AI agent needed two hours to expose 46.5 million chat messages in a corporate AI platform.

Regulation is accelerating development

The industry is also being pushed by legislators. The EU AI Act requires that high-risk AI systems undergo adversarial testing — with a deadline of August 2026. This provides additional impetus for investment in precisely the methods that GPT-Red represents.

Tools and frameworks for AI security testing are now available from a range of providers: Microsoft's PyRIT, NVIDIA's Garak, Promptfoo, Confident AI, and Scale AI are among those offering structured attack frameworks. The market for such services is estimated to reach $18.6 billion by 2035, according to industry forecasts.

Problems that remain unsolved

Despite the progress, the industry has not reached the finish line. OpenAI has itself acknowledged that prompt injection will likely never be fully eliminated. The question instead is how hardened models can become — and whether attackers out in the world will always remain one step ahead. GPT-Red is OpenAI's attempt to answer that question by letting the machine fight against itself.