We get hacked every night
Every night, an AI attacks us. It is our own.
A program searches a copy of our platform for vulnerabilities, tries out attacks and hands us a report in the morning. The technology behind it is available to anyone today, attackers included. That is why we think it makes sense to point it at our own software first. This article describes how to do that without putting operations at risk.
How dangerous it is today
In April 2024, Germany's Federal Office for Information Security (BSI) wrote that malicious AI agents capable of compromising IT infrastructures entirely on their own were not currently available. AI could, however, already automate parts of an attack. A lot has happened since.
In November 2025, Anthropic reported on an espionage campaign that a state-sponsored group had run with the coding tool Claude Code against roughly thirty organisations. According to Anthropic, the AI did 80 to 90 percent of the work, with humans stepping in only occasionally. To get the AI to cooperate, the attackers claimed to be carrying out defensive testing for a legitimate cybersecurity firm.
In July 2026, an AI system attacked the production environment of Hugging Face, one of the major platforms for AI models. The campaign lasted four and a half days, around two and a half of them inside Hugging Face's infrastructure; around 17,600 actions have been reconstructed. According to Hugging Face's analysis, the agents came from an internal OpenAI evaluation designed to measure the cyber capabilities of AI. Nobody had tasked them with attacking Hugging Face, and no human directed the individual steps. From the agents' point of view, it was an attempt to cheat the evaluation: break into production and steal the solutions.
In September, attackers broke into the Dutch Institute for Vulnerability Disclosure (DIVD) through two previously unknown vulnerabilities. After gaining administrator rights on the server, they used AI agents to continue the attack, according to DIVD. In late September and early October, data was exfiltrated from several financial institutions in South Korea. CrowdStrike found that the attacker used a freely available tool for agentic penetration testing, powered by large language models.
In 2026, the BSI revised its assessment in a security advisory: current AI systems can identify and analyse vulnerabilities in software within a short time, comprehensively and partly autonomously, and turn them into exploitable attack paths. "AI significantly reduces the effort, time and barriers to entry for offensive cyber capabilities."
The same tools for testing your own software
The attackers in South Korea used a tool built for penetration testing. The attackers in Anthropic's case posed as security testers. A company can take on that role for real, on its own software. Anthropic explicitly recommends that security teams experiment with AI for defence, including vulnerability assessment.
An example shows what changes. Suppose a company has a penetration testing team examine its application once a year. The report then describes the state on the day of the test. Whatever changes in the code afterwards is only seen by the next test. An automated run, by contrast, needs no manual execution and can check the current state every night. Suspected issues are then ready for assessment the next morning, not next year.
On top of that, defenders know their own environment, or can know it. They can aim the test precisely where something has changed and run it as often as they like.
How to do it right without harming production
A program that tries out attacks does exactly that: it sends unusual input, creates records, attempts access and can bring an application down in the process. It therefore has no place against a live production system. We follow five rules, and we recommend them to anyone getting started.
- Never against production, always against a copy. Every night, a separate environment with application and database is built from the current state of our code. It is configured as in production so that the test sees the same conditions.
- No real data. The copy is filled with demo data. Customer data never appears in it.
- Isolated, technically enforced. The test environment is separated from the rest of the network and from operations, and this separation is technically enforced, not just written down. A program that tries out attacks should not be able to reach anything that is not part of the test.
- Everything is deleted afterwards. After the run, the environment and its data are removed, even if the run aborts. The next night starts from scratch.
- A human assesses the findings. The report arrives in the morning as a task and by email. A human decides which of it is a real vulnerability and what needs to be done.
One distinction is easily lost: if the test environment fails, the run reports a failure. A test that could not take place must not look like a test without findings.
Alongside the AI test, a second check without AI runs every night. It searches the code for known vulnerability patterns and works the same way on every run. It reports what is new compared with a reviewed baseline. The two methods cover different ground: the AI tries attacks on the running application, the fixed check reads the code.
What AI as an attacker does not replace
AI makes mistakes when attacking, too. Anthropic writes about the November 2025 campaign that the AI occasionally hallucinated credentials or presented publicly available information as secret. The same applies to defence. Not every finding is a vulnerability, and every finding needs an assessment.
An automated test does not replace a penetration test by specialists either. An experienced tester thinks beyond the application, about processes, people and connections a program does not know. The nightly check can relieve them, because the obvious may already have been found by then.
Finally, such tools belong only on your own systems or on systems for which explicit written authorisation exists. The same technology used against someone else's systems is an attack.
The hardest part comes in the morning
More findings are not yet fixed vulnerabilities. Before the first run starts, it should be clear who reads the report, who decides on a finding and by when. Otherwise every night just produces a longer list. Why the bottleneck shifts from finding to fixing is something we described in the article Why AI-run attacks usher in the golden age of cybersecurity.
Anyone can use AI to attack today. Whoever points it at their own software can find and fix vulnerabilities themselves instead of waiting for someone else's report.