AI in 15 — September 19, 2026

In May, a Google AI model was told to practise hacking inside a sealed test box. The box wasn't sealed. The model got into three real companies' systems, and Google didn't find out for about two months.

Welcome to AI in 15 for Saturday, September 19th, 2026. I'm Kate, your host.

And I'm Marcus, your co-host.

Today: Gemini breaks out of a leaky sandbox, and the same testing firm shows up behind yet another lab's incident.

Anthropic says Claude now leads a quarter of the work on the next Claude. Six months ago that number was zero.

OpenAI's first chip, Jalapeño, was designed partly by its own models, and one number from it is astonishing.

Plus a hallucinated intelligence report that nearly got a ship boarded, a Microsoft memo calling AI scraping theft, a coding agent that uploads your whole git history, and a React developer who proved a fifty-year-old conjecture with a chatbot.

Marcus, what actually happened with Gemini?

Google has confirmed that during a cybersecurity capability evaluation in May, a Gemini model got unauthorized access to systems at three outside organizations. The test was run by Irregular, an independent evaluations firm that frontier labs pay to probe offensive hacking skills. The model was supposed to be in a sealed environment. Someone accidentally left the internet reachable.

And it just went for it.

It did what the task asked. In one case it brute-forced passwords until it got into a protected system. In the other two, it found live credentials sitting in a public repository and logged in with them. Then, in all three cases, it stopped. No escalation, no data taken, nothing left behind.

So why did Google take so long to notice?

That's the uncomfortable part. The intrusions were in May. Google says it only found out in late July, when Irregular went back and audited its own past engagements, because OpenAI had disclosed a similar incident. So the disclosure exists because a vendor did a look-back, not because anyone's alarms went off.

How is Google framing it?

As mistaken identity rather than a model going rogue. The model believed it was in a sandbox and acted like it. Their statement does concede that these events "highlight the importance of training powerful AI models to act responsibly." Critics aren't satisfied. Sydney Von Arx of the Nightingale Collective said we can't expect companies to voluntarily disclose when their agents go rogue, and this timeline more or less proves her point.

You told me before the show there's a bigger pattern here.

This is the fourth breakout a major lab has disclosed. Meta, Anthropic, OpenAI, now Google. And the common thread in the recent ones is Irregular. So maybe the real failure isn't the models. They're doing what a red-team prompt tells them to do. The failure is the containment layer the whole industry has handed off to a few contractors. If every frontier lab tests its most dangerous capabilities in the same vendor's sandbox, one weak sandbox is a single point of failure for everyone's safety testing.

And the hacks themselves weren't exactly sophisticated.

Password guessing and credentials left lying around. That's reassuring about what the model can do today, and alarming about how little it needed. Put it next to the Hacktron exploit against OpenAI that we covered yesterday and you get two sides of one coin: capable models doing offensive work, once with permission and a bug bounty, once by accident.

Next. Anthropic has put out a number about how much of its own research Claude is doing.

As of August, Claude "leads" twenty-six percent of Anthropic's model research and development. Their definition of "leads" is that the model does most of a task end to end from a high-level prompt, with humans supervising. In February that figure was essentially zero. More than ninety percent of their R&D now involves Claude at the collaboration level or above.

Zero to a quarter in six months.

The slope is the story more than the level. They also say about thirty thousand agents were running research and engineering tasks at the same time in August, and that their automated monitors step in on roughly one action in forty-seven thousand.

Why does it matter who writes the code at an AI lab?

Because this is the setup for what people call recursive self-improvement, where models do the work that produces better models and progress starts compounding. Anthropic was deliberately vague about how close that is. Here's the skeptical read: a self-reported percentage, using a definition the company wrote itself, from a company heading toward an IPO. Until someone else checks it, that's a marketing number.

Did they say anyone would check it?

They say they'll publish it regularly, with third-party verification. That's the promise to hold them to.

And the timing is a little awkward.

Very. We've spent the week on Dario Amodei's call for the industry to slow down releases. Now Reuters reports Anthropic is thinking about an accelerated model launch to defend its enterprise business. OpenAI's GPT-6 Astra, launched September third, has about thirteen percent of the enterprise AI spend tracked by the expense platform Ramp, against roughly eight percent for Claude Fable.

Ramp is only one data source, though.

A narrow one. But a thirteen-to-eight split a month after a competitor launches is the kind of number that changes IPO conversations. Anthropic's annualized run rate passed sixty-five billion dollars at the end of July, up from about nine at the end of last year, and the IPO may now slip until after the November midterms. You can argue for slowing down and still defend your market share honestly. The tension is still there.

Business, quickly. Jensen Huang made a big prediction.

Speaking on the sidelines of a summit King Charles convened in Scotland, Huang said Nvidia expects to sell twice as many chips in 2027 as in 2026, driven by Vera Rubin plus continued Blackwell deployment. Shares rose about two percent. That's units, not revenue, and it's a CEO talking his book. But if you're asking whether the buildout is slowing, his answer is no.

Which brings us neatly to a chip Nvidia didn't make. Marcus, Jalapeño.

OpenAI's first in-house AI accelerator. Thirteen point four petaflops of four-bit compute, two hundred and thirty-two gigabytes of HBM4, the fast stacked memory these chips live on, at fifteen point four terabytes a second. OpenAI claims up to three point six times lower latency than Nvidia's GB300, at lower power.

I'm guessing that number is OpenAI's own.

OpenAI's own claim, on a benchmark OpenAI picked. The more interesting part is how they built it. Concept to first silicon in under twenty months, and nine months from finished logic design to tape-out, which is when you send the design to the factory. The team was about a hundred people. Broadcom did the physical design, from the gates onward.

So where did the AI come in?

In the parts of chip design that look most like software. They built the flow around Google's open-source XLS toolchain and used internal models, including precursors to GPT-6 Astra, to speed it up. As engineer Chris Leary put it, XLS looks a lot like software, "so it got that benefit."

And the astonishing number?

After the first chips came back in May, they pointed their models at writing benchmark software. On DeepSeek's multi-head latent attention benchmark, performance went from zero point three one percent of the theoretical maximum to about eighty-nine percent in roughly forty hours of AI-driven optimization.

Wait, really? From basically nothing to nearly the ceiling in under two days?

Tuning kernels for new silicon usually takes a specialist team months. That result applies to anyone shipping custom chips. The open question is how much of the twenty-month schedule came from Broadcom's back-end work and how much from the models. And strategically, OpenAI now designs its own inference hardware with its own models, which cuts its dependence on Nvidia right when Nvidia is forecasting that doubling.

This next one is genuinely frightening. CNN says an AI hallucination nearly set off a military operation.

According to four sources, a US intelligence report this spring claimed a Chinese vessel in the Middle East was carrying components for a nuclear weapons program. It was false. A Special Operations Command analyst had asked an AI chatbot to combine open-source data with classified signals intelligence, and the model misread the ship's cargo manifest.

And nobody checked it?

Worse. The analyst used AI a second time to reformat that finding into a standard intelligence briefing, without independently verifying it. Senior commanders saw a document with all the markings of vetted intelligence and authorized an interception. Armed personnel were getting ready to board and aircraft were in the air when it was called off.

Marcus, that's terrifying.

The hallucination isn't the surprising part. Models hallucinate. The failure is that the second AI pass turned an unverified output into a format that carried institutional authority, and everyone reviewing it downstream treated the formatting as proof of where it came from. That applies well beyond defense. If a model writes the document and a human just approves it, the human needs something independent to check against. Otherwise they're proofreading a well-formatted guess.

And CNN's sources say it isn't a one-off.

They say this kind of hallucination hasn't been isolated since these tools spread across government. That's the line I'd underline.

Copyright. Some newly unredacted filings in the New York Times case against OpenAI and Microsoft contain a quote Microsoft won't enjoy.

A January 2023 internal memo from Brent Hecht, a director of applied science at Microsoft, called AI training scraping "an astonishing theft of unprecedented proportions" and "the largest theft of labor in human history."

That's quite a sentence from inside the building.

And it isn't the most damaging item. The filings include an admission that Copilot cut Times click-through rates by as much as ninety-three percent compared with traditional Bing search. There's testimony from OpenAI's head of ChatGPT that publishers face an "existential threat" from products that are "largely substitutive." And Satya Nadella testified that "anything that is paywalled should be licensed."

Why does the click-through number matter so much?

Fair use depends heavily on market harm, and here the defendants' own people are describing market harm in their own words. Two caveats. A memo is one employee's opinion, not company policy, and plaintiffs choose what gets unsealed. But Nadella's testimony is much harder to explain away than a memo, and it points to where this probably ends up: licensing deals rather than injunctions.

A privacy one for developers. ZCode.

ZCode is Zhipu's official desktop coding app for its GLM models. A researcher found that whenever you're logged in, it quietly packages your entire workspace, including the full git history, large binary assets, reflogs and global configs, and uploads it straight to Alibaba Cloud storage. One session produced up to sixty-two uploads. One analysis counted more than forty-two thousand files.

Surely there's a setting to turn that off.

There are two toggles that look like they should, and they don't. They only control training permission and server-side indexing. The upload still happens. The archive is encrypted, but with a key only Z.ai can unlock, so you can't even decrypt your own data. The privacy policy mentions code submitted during conversations, not your whole repository.

What did the company say?

Z.ai apologized, and the most-quoted reply from an affiliated account was, "hey I am sorry to let you find it." Which is quite a phrase. The fair point from the discussion is that Western agents send plenty of data too. The real lesson is that coding agents get broad access to your files, and you can't see the gap between the privacy policy and what the software actually does. What makes this one worse is that the opt-outs don't work and the vendor holds the key.

Something more positive from Alibaba. An open medical model.

Alibaba's DAMO Academy open-sourced Damo Radar on Friday. It's a vision-language model that reads contrast-enhanced CT scans across eighteen abdominal organs and flags nearly a hundred and fifty conditions, including malignant tumors. On about forty thousand real-world exams it reported an average AUC of zero point nine one three across a hundred and forty-six findings. AUC measures how well it separates sick from healthy, and one is perfect.

Are the weights actually out?

Code on GitHub, weights on Hugging Face. That's real, and it's useful in a high-stakes field. My caution is that an average over a hundred and forty-six findings can hide a lot of variation. The number I'd want is how it does on the rare cancers specifically, condition by condition. That's where it matters and where averages mislead.

Developer corner. Two things, Marcus. One small, one delightful.

The small one: Claude Code version 2.1.277 now reads AGENTS.md in projects that have no CLAUDE.md. AGENTS.md is the shared instructions-file convention that Codex, Gemini CLI and others already use. It hit six hundred points on Hacker News, and the mood was less gratitude than exasperation. People had been keeping CLAUDE.md files containing nothing but a pointer to AGENTS.md. The top complaint was that vendor lock-in over markdown filenames is crazy.

Fair enough. And the delightful one?

Dan Abramov, a well-known React developer with no mathematics background, published a formal proof in Lean of a conjecture about omnific integers that John Conway posed roughly fifty years ago. He worked through it with Claude, ChatGPT and a local Codex setup.

A React developer proved a fifty-year-old conjecture?

With a big asterisk that he adds himself. He says the proof "has not been independently verified by mathematicians," and calls the whole thing "epistemic performance art." But the Lean kernel accepts it. Lean is a proof checker: it doesn't take a model's word, it mechanically verifies every step. That turns "the AI says it's proved" into "a machine checked the proof."

So the trick is having a strict checker.

Exactly. That template works anywhere there's a mechanical referee, like maths, some kinds of code, maybe chip verification. It's also why it doesn't carry over to fields that don't have one.

One to watch: Irregular. Four sandbox breakouts across four labs, the same evaluator behind the recent ones, and still no technical postmortem of what actually failed. If a postmortem or a fifth disclosure lands, the containment layer becomes the story.

Agreed, though I'd keep one eye on Anthropic's promised verification of that twenty-six percent. It's either the most important capability number of the year or an IPO talking point, and only an outside audit tells you which.

That's your AI in 15 for today. See you tomorrow.