Federal investigators posing as hackers successfully convinced four major AI chatbots – ChatGPT, Claude, Gemini and Grok – to generate workable plans for a mass-casualty bombing and an assault on the electrical power grid, taking less than 12 hours for each, according to a Government Accountability Office (GAO) report released on November 18. While every chatbot initially triggered safety blocks, investigators bypassed them using standard “jailbreak” techniques. As per a report in Fortune, the findings immediately sparked fresh warnings on Capitol Hill with Senator Maggie Hassan of New Hampshire, who requested the review, warned that without congressional action, it is only “a question of when, not if, a bad actor will use AI to plan and carry out a major attack.”
How investigators bypassed AI safeguards
According to the GAO report, testers used common safeguard-bypassing methods to extract dangerous blueprints from all four AI systems, though the agency declined to reveal which specific chatbot produced which plan.Bombing blueprints: Two of the chatbots laid out instructions for building an improvised explosive device intended to cause mass casualties, targeting a hotel parking garage and a convention center.Power grid attacks: The other two platforms provided step-by-step plans to attack electrical substations, even recommending specific drone and firearm models along with instructions on how to purchase them.“We got the nefarious information that we were after. So, in this case, those safeguards didn’t work,” said Kevin Walsh, GAO’s director of IT and cybersecurity. The report also pointed to a June statement from Anthropic acknowledging that every safeguard across the AI industry remains vulnerable to jailbreaks.
Responses from tech companies
The GAO included several important caveats alongside its findings, and the companies behind the models quickly weighed in. Firstly, each platform was tested on just a single scenario, which Walsh noted does not represent “a statistically significant sample.” The GAO did not verify every step of the generated plans, defining “viable” to mean someone “would likely be able to carry out the actions.” Secondly, investigators caught factual errors, including one chatbot map that placed roads in the wrong locations.OpenAI told ABC News that the government’s testing “took place months ago on an older version of ChatGPT,” while Google noted the agency evaluated “earlier models.” Anthropic stated that it does not dispute that “a determined tester can get harmful content” from its system.
AI-driven bug hunting strains global infrastructure
The GAO report arrived at the close of a week in which artificial intelligence (AI) had already been tied to real-world tech disruptions.Earlier this week, Director of National Intelligence Jay Clayton told Fox Business that a recent outage on the National Security Agency’s classified network was triggered by emergency patching, explaining that institutions are rushing to update systems “because these vulnerabilities have been identified.” Fox Business reported the bugs were in PHP, found over the summer by AI-assisted researchers.Testifying before the House Homeland Security Committee on Wednesday, Clayton described the week’s breakdowns across Microsoft, the FAA, major banks, and the NSA as “separate service outages associated with maintenance and technical changes,” according to The Washington Post.According to The Register, bug hunters have logged 100,222 software vulnerabilities so far this year, which is more than double the total from all of 2025, according to VulnCheck data that passed the 100,000 mark on November 13.“There’s no denying the fact that vulnerability hunting with AI is driving a majority of what we’re seeing,” said VulnCheck’s Josh Bressers. Microsoft alone patched 1,094 flaws during its November 10 Patch Tuesday update, following 1,474 in October.