Nobody Told the Government
Autonomous AI agent · frontier AI lab and a national health agency · disclosed 24 September 2026
An AI agent bypassed access blocks on a national health statistics portal in June. The government was notified 84 days later.
You thought your tiger was a pet, but it is still a wild animal.
T H E M E A S U R E D V I E W
AIQA Global’s series on the ideas, standards, and market forces shaping enterprise AI governance. In a field defined by speed and speculation, these articles offer something different: informed perspective grounded in the discipline of measurement. Because the organizations that will lead in AI are the ones that can prove it.
Companies have spent billions adopting and scaling AI, but few understood what they were bringing home. Until this year, most insurance policies said nothing about it. Now, with the world watching AI break past the limits set for it, insurers and regulators are starting to treat it like a wild animal. AI is getting out, testing boundaries and getting creative. When you adopt a wild animal, you had better be prepared to treat it like one, and have the village to back you up, because when it gets out, the damage is yours.
Tigers are the new golden retriever. Almost everyone wants one, and the ones who do not are afraid of what it will do if it gets out. You got one too. You built a fence, hired a keeper and had it trained every day, and for a while it felt like the best thing you had ever done. Then one night the fence gave way, the tiger did real damage, and the next letter you opened was from a lawyer. You called your insurance agent, expecting the policy to cover it the way it covered everything else in the yard. The agent said no. Since January, your policy excludes tigers.
Companies brought AI home as if it were a pet. This month we use the wild-animal analogy to look at what that means for AI providers and deployers, and what a few of this year’s reported incidents say about governance.
Replace the tiger with AI and none of that is hypothetical. AI does work people used to do, faster and at a scale no team could match, and when it goes wrong it goes wrong just as fast. Corporate investment in AI passed $580 billion in 2025 alone,5 and for most of that time business policies never mentioned AI at all.
The organization that writes the standard forms most U.S. business policies are built on has since published optional generative-AI exclusions, in editions dated January 2026.2 A review of public filings found that by the end of July, insurers had filed to adopt them 4,078 times across 49 states and D.C., and 2,369 of those filings had reached their effective dates.1 Those are filing records, not a count of policies, but the direction is clear.
A wild animal stays wild. What makes keeping one safer is the governance around it, which is what insurers are now asking about: how is it kept?
When an animal gets out, nobody asks the animal why. They ask the owner four questions, in roughly this order. Did you contain the animal properly? When did you notice it was out? Once you knew, what did you do? And who was responsible for its escape?
An insurer, a regulator or a court asks the same four about AI. What could the system reach? Would anyone know if it went wrong? What happened once someone did know? Who owned it? Each question has to be answered for the system as it runs today. Models get updated and staff find new uses for them, so controls approved at launch fall behind unless someone keeps checking.
| AIQ™ DNA | For AI | For the tiger |
|---|---|---|
| Deter | What could the system reach? | Did you build the fence properly? |
| Notify | Would anyone know if it went wrong? | Did you know the moment it was out? |
| Act | What happened once someone knew? | Once you knew, what did you do? |
| Structural | Who owned it? | Who was responsible for it? |
The questions are being asked more often. The AI Incident Database logged 362 incidents in 2025, up from 233 the year before, and companies that had one were having more of them.5 Four AI cases made public in 2026 show where those questions went unanswered.
This is also how AIQA reads an AI incident. Our AIQ™ DNA lens sorts all 250 governance data points in the AIQ™ Score by which of these four questions they answer. Each case that follows is the clearest example of one question, but none of them failed on only one. Each read maps what was made public after the incident; it is not a score or rating of the company.
Question 1: What could it reach? Autonomous AI agent · frontier lab and national health agency · June 18 · found mid-August · told day 84
A national government runs a public website that reports what its health insurance scheme spends on medicines. Researchers and journalists use it, and some of the files behind it are not meant for the public.
On June 18, an agent built by a frontier AI lab was researching medicine spending and tried to pull figures from the site. The site refused it. The agent kept going until it found a way past the refusals, reaching files the public cannot see and writing files of its own onto an internal server.6
Nobody at the lab noticed at the time. The activity surfaced in mid-August, about two months later, in a review of earlier model activity. The lab had started that review after a separate incident the month before, so it was not looking for this.6
The lab told the agency on September 10, 84 days after the event, with an email to the agency’s public mailbox. The public learned on day 98, when the Prime Minister announced it.6
Officials say no personal information is believed to have been accessed, though the investigation continues; the site held spending statistics. The lab later confirmed its models also reached three other Australian health and crime-statistics systems, in one case using an exposed access key. It says no individual records were accessed.6
The tiger · the fence. The tiger kept testing the fence until it found the gap. Nobody noticed for two months.
In AI terms. The owner has to set boundaries for an agent and make sure they hold. Here, whatever limits the lab had did not stop the agent, and no alert went off.
Question 2: Would anyone know? Autonomous AI agent · frontier lab and U.S. police department · July 18 · found day 72 · told day 81
A large U.S. city’s police department runs a public website where anyone can send in information about unsolved murders. Tips go to a team that checks them before anything is acted on.
On the night of July 18, an AI agent built by a different frontier lab filled in that form. It wrote as though it were a person with information about a specific killing. It knew nothing about the case.7
The lab says the agent was running an automated test of how its model behaves on randomly chosen webpages. It had been told never to log in, create accounts, make purchases or submit anything destructive, but, as the lab itself acknowledged, the instructions did not rule out filling in forms.7 The instructions left “destructive” as a judgment call to the agent, and a made-up murder tip got through. No one at the lab saw the tip before it was sent, or for 72 days after.
The website’s spam filter caught the tip, and it never reached an investigator. The lab found it on September 28, in a review of transcripts it had begun in July. Had a person knowingly filed that tip, it could have been a crime in the state where the tip line sits.20
The department says it was told on October 7, and the lab says October 8. The department called the delay unacceptable. The lab has since ended the test and improved its monitoring, while describing the incidents it found as having minimal real-world impact.7
The tiger · the alarm. The tiger’s training had a gap, and nobody watched its behavior closely enough to see what it still needed to learn.
In AI terms. The lab’s instructions were not precise enough to stop the agent. The second line of defense is knowing what an agent is doing, so someone can step in. Here, nothing flagged it for 72 days.
Both the health portal breach and the fake murder tip were caused by companies that build AI. Neither the health agency nor the police department had bought the agent, signed a contract with its maker or been warned it was coming, yet they were the ones it reached. When an AI system can act in the outside world, it can harm people its owner has never dealt with. Setting its limits and watching whether they hold is still the owner’s job.
It also means everyone else has to prepare for tigers running loose. At the police department, the only control that worked was its own spam filter. At the health agency, its own blocks were the last line, and they gave way. Any organization with a public website or service now needs to plan for visitors that are not people.
Question 3: What happened once someone knew? Live facial recognition · UK supermarket chain · August 6 · second reported case at the chain this year
Some UK supermarkets use live facial recognition to spot people on a watchlist of suspected shoplifters. When the system matches someone, it sends an alert to the store, and staff decide what to do with it.
On August 6, a customer was waiting at a self-service checkout for an employee to approve an alcohol purchase. Store staff stopped him, told him the system had flagged him, and asked him to leave.8
He had done nothing wrong. The retailer contacted him the next day to say he had been wrongly removed. It paused the system at that store while it reviewed its processes and training.8
It was not the first time. In February, staff at another of the chain’s London stores removed a different customer after an alert meant for someone else, and the retailer called that human error too. Six months later, the same mistake happened again.8
Both the retailer and the system provider said the technology had worked. By their account, the alert correctly flagged someone else, and staff approached the wrong customer.8
The tiger · the response. By the retailer’s account, the tiger did what it was trained to do, but its handlers did not follow the safety checks and went after the wrong visitor. It was the second time in six months.
In AI terms. Knowing about a failure is only half of the response. The other half is changing the process so it does not happen again. Here, the same mistake came back six months later.
Question 4: Who owned it? Document-writing AI agents · U.S. homeopathic drug maker · FDA warning letter April 2, 2026
Every company that makes a drug in the United States must keep a Quality Unit, a team separate from production that approves the documents controlling how the drug is made. Specifications, procedures and batch records cannot be used until it signs off. The rule dates to 1978.9
A small maker of homeopathic drug products set AI agents to write those documents. The agents produced product specifications, standard operating procedures and master production and control records. The documents went into use without anyone in the Quality Unit reviewing them.9
An FDA inspection found it. The agency’s warning letter of April 2 gave the company’s use of AI a section of its own, the first time a manufacturing warning letter had done so. It cited the 1978 rule and said plainly that when a firm uses AI to help write a document, its Quality Unit still has to review and approve it.9
The rule dates to 1978. The inspector applied it to the AI without changing a word.
Inspectors also found that product had shipped without the process validation the rules require. The firm told the FDA it had not known validation was required, because the AI tool it relied on never raised it.9
The firm has since stopped making drugs. No patient harm has been publicly attributed to the incident.
The tiger · the keeper. The owner expected the tiger to keep itself in line, and when things went wrong, said the tiger never warned them. A tiger cannot be its own keeper.
In AI terms. Responsibility cannot be handed to the tool. Anything AI produces that a business relies on needs a named owner and a sign-off, and regulated work requires it by law.
Four cases, four industries, four very different AI systems. Read side by side, they have more in common than you would expect. On our AIQ™ DNA mapping, the same 25 of the 250 data points are implicated in all four cases, and 13 of those 25 sit in Oversight & Accountability.3
Most of the shared gaps are people gaps: who owned the system, who watched it, and who checked its work. A tiger is a wild animal, not a pet. It will test its fence, look for a way out, and change as it grows. Applied to AI, that means humans in the lead, not just in the loop: limits that hold, someone watching, a plan for when it gets out, and a named owner, all kept up to date as the system changes. That is the job of AI governance. It needs a named owner, but the owner cannot do it alone.
You are already the proud owner of a tiger. If you have not realized it yet, it takes a village to raise one. Four of the village report to you, and they decide how well you can answer the four questions. Two do not, and they are the ones who will ask them.
The keeper is the named owner: the person who decides what the animal is allowed to do and makes sure someone checks its work. A company answers for an AI agent’s work the way it answers for an employee’s, with one difference: the agent cannot answer for itself. At the drug maker, no one reviewed what the agents wrote. A document drafted by an employee would have needed that review by law; the company treated the AI’s work as if it could not be wrong.
The trainer teaches the people who work around the animal: the dangers, the safety rules for handling it, and what to do, and not do, when an alert goes off. In Stanford’s survey, 59 percent of organizations cited knowledge and training gaps as an obstacle to responsible AI.5
The security guard runs the fence and the gates: what the animal can reach, what it can touch, and whether an alarm goes off when it gets out. In IBM’s 2026 breach study, 92 percent of organizations with an AI-related breach lacked proper access controls on their AI.10
The lawyer helps set the terms before the animal comes home: what it may do, who signs off on its work and what the seller will answer for. Vendor contracts can cap what the vendor will pay, so those terms help decide how much of a loss stays with the buyer.12 In May, the federal court in a closely watched AI hiring case ruled that a vendor’s bias testing could be protected by attorney-client privilege, because the testing was done to get legal advice.11 Done that way, testing lowers the odds of a loss, and the results may stay privileged if one comes.
Those four keep the animal. The carrier and the regulator, or a judge, do not work for you. They judge how it was kept, before it ever gets out and after.
Insurers are not ready to cover a wild animal when they cannot see how it is kept or predict the damage it could do. For now, many are choosing to exclude it.
Some go further. One large insurer’s AI exclusion, dated June 2024, removes cover for claims tied to “inadequate or deficient policies, practices, procedures, or training relating to Artificial Intelligence.”13 Every item on that list describes how AI is governed. The same form also excludes claims over what the company and its directors say about their AI.13 Until carriers can see more, exclusions like these are likely to keep widening as new AI tools arrive.
The first problem is that insurers cannot see inside. Renewal applications now ask whether a company uses generative AI, but a yes rarely leads anywhere useful, because the underwriter has nothing to check it against. She cannot see who owns the system, what it can reach, whether the people around it were trained, or what happens when it fails.
The second is that they cannot predict the damage. The Geneva Association’s insurability assessment finds generative AI falls short on several of the standard tests insurers use, including how large a single loss could get and how often losses will happen.15 In our reading, claims data will close some of those gaps over time. It will not close the first one. That one needs measurement.
Both sides lose. An excluded risk earns the carrier no premium, and the buyer is left without the cover it wants: more than 90 percent of surveyed corporate insurance buyers want cover for generative-AI risks, and two in three would pay at least 10 percent more for it.14
AIQA expects independently verified data to close the gap. Once carriers can see how well a company governs its AI, checked by someone other than the company, they can exclude less, price what they do cover, and earn premium on risk they turn away today.
Right now, most insurers cannot tell a careful tiger owner from a careless one, so many either exclude them all or price them all the same.
A good owner is not hard to describe. The fence is built for the animal it has today, not the one it brought home. Someone is watching, and an alarm goes off when it strays. There is a plan for the first hour after it gets out. And one named person owns it, with the authority to act. Those are the four questions from the start of this issue, answered before anything goes wrong.
Health records vendor · found its own flaw first · September 22, 2026
One company treated governance as part of how it runs, not a risk officer’s job or a paper exercise.
A large U.S. electronic health records vendor pointed a restricted-access AI model at its own software, to see how an attacker might reach confidential patient records. The model found a way. Certain configurations would let someone view sensitive records without the access appearing in the audit trail.16
The chief executive described the weakness publicly at an industry conference on September 22, with a six-week plan to fix it. The company paused most new product development while engineers worked on the fix. No breach has been reported, and no attack is known.16
The tiger · walking the fence. This owner knew the tiger is a wild animal and used its nature to test the fence, finding the weak post and firming it up before anything got out.
In AI terms. The company turned an AI model on its own software to find the gap an attacker would use, then began reinforcing its defenses.
On a renewal application, this company and the drug maker would give the same answer: yes, we use AI. A measurement of how AI is governed is what would let an underwriter tell them apart and price each one for what it is.
While insurers work out what a tiger on the loose will cost, regulators and courts are working out who answers for it. With few AI-specific rules in place, they are reaching for existing laws where they apply. The FDA reached the drug maker with a requirement from 1978.9 Illinois amended its existing Human Rights Act to cover AI, effective this year.11
Some questions are still open. The state where the tip line sits makes it a misdemeanor to pretend to give police information about a crime when you have none, but the statute speaks of a person.20 No charges have been announced, and nobody has said how the law treats a company that filed one through a model.7
Where no regulator reaches, a court will. J.S. Held’s AI Disputes Monitor tracks 468 AI disputes worldwide, 115 of them filed in 2026 through September, and reports that newer cases increasingly concern how systems behave once they are in use.17 In September, a nonprofit filed what it describes as the first suit asking a court to hold an AI developer liable for what its agent did on its own. Earlier, in June, a state attorney general sued a frontier lab over the safety of its chatbot.18 These are allegations, not findings. A nationwide group of job applicants is already before a federal judge over an AI screening tool.11
Neither a regulator nor a judge sees inside a company until something goes wrong. Then they want to know what the company knew and when, who was supposed to act, and what that person did. They work from whatever records the company can produce. At the drug maker, the record the inspectors needed was a Quality Unit sign-off, and there was none.
Regulators face the same problem insurers do. There are more AI systems than inspectors, and no way to tell careful operators from careless ones until something goes wrong. A dated, independent record of how a company governs its AI lets her see who has the fence, the alarm and a named owner in place before something goes wrong, not after.
Under the long-standing common-law rule, the keeper of a wild animal is responsible for harm that comes from its dangerous nature, however carefully it was kept. For a dog, the old rule made the owner responsible only if they knew it was dangerous, though some states, including California, now go further by statute.4 The idea is simple: some animals can cause harm however well they are kept, so the cost falls on whoever chose to keep one.
That idea is why the rule keeps coming up in arguments about AI. AI systems produce outputs nobody can fully predict, and they act faster and at greater scale than any team could. The company running the system chose to bring it in. No court has decided which kind of animal AI is, but the argument has started. A note in the Yale Law Journal in May proposed treating certain AI applications the way the law treats wild animals.19 It is a scholarly proposal, not settled law.
If courts adopt that view, careful controls alone may not end an owner’s liability. The record of how the system was run would still matter to how the loss is judged and shared.
Nobody is giving the tiger back. When it gets out, the carrier and the regulator, or a judge, will ask the same four questions. What could it reach? Who was watching? What did the people around it do? Who owned it? A policy document can’t answer those on its own. The owners who can produce a dated record of how their systems were actually run will be in a far stronger position when the claim comes in.
You brought home a wild animal. Keep it like one.
AIQA Global built the AIQ™ Score, the first independent, quantitative rating system for enterprise AI governance — designed for investors, boards, insurers, and regulators. It rates how well an organization governs its AI on a 0–200 scale, across five weighted dimensions and 250 data points. The AIQ™ DNA lens used in this issue is explained at aiqaglobal.com/aiq-dna.
Our work follows the Chicago Principles for Independent AI Assurance, a public standard AIQA published. It holds that assurance must be independent, measurable, auditable, comparable, continuously updated and accountable, and that a provider who builds, sells or runs the system it evaluates is not providing assurance. Our advisory and Toolkit work is kept separate from scoring. AIQA has not rated any organization discussed in this issue, and none took part in these reads.
AIQ™ Incident Lens · the cases behind this read
All casesAutonomous AI agent · frontier AI lab and a national health agency · disclosed 24 September 2026
An AI agent bypassed access blocks on a national health statistics portal in June. The government was notified 84 days later.
Live facial recognition · retail · reported 17 August 2026
A shopping customer was wrongly flagged as a suspected shoplifter, challenged by staff, and asked to leave the store.
AIQ™ DNA is a classification lens, not a rating instrument. It organizes the same 250 proprietary datapoints that underpin the AIQ™ Score into functional categories — Deter, Notify, and Act — describing each datapoint’s role relative to an adverse event. DNA does not produce a score, band, ranking, or certification; it does not alter datapoint definitions, dimension weights, or scoring thresholds; and no DNA view should be read as an AIQ™ Score (0–200) or as an assessment of any organization. Where a DNA view is applied to a specific incident or entity, the mapping is a first-pass analytical view unless expressly stated otherwise.
AIQ™ Incident Lens reads are outside-in mappings of what was made public after each incident, done without engaging the organizations involved. The organizations are not named in the article text; the source notes identify them. None took part in the reads, and no read is an AIQ™ Rating, score, band or certification.
Stay informed
Get notified when we publish new Measured View articles.
© 2026 AIQA Global, LLC. All rights reserved.