AI in 15 — October 29, 2026

Eighty-eight point two percent. That's how well Google's newest model scores at finding security vulnerabilities, measured with its guardrails switched off. It's Google's best model to date, and almost nobody is allowed to use it yet.

Welcome to AI in 15 for Thursday, October 29, 2026. I'm Kate, your host.

And I'm Marcus, your co-host. Kate, after the S-1 this week, I'm glad to be back on models and benchmarks.

Plenty of both today. Here's what's coming up. Google launches Gemini 4 Carbon, but only for cyber defenders for now.

Anthropic and Theorem use Claude to write a hundred and eighty million lines of security proofs.

A developer finds Claude Code's status line eating a sixth of their token bill.

A tax benchmark where the best AI agent passes about one assignment in ten.

Plus a macOS certificate problem for developers, Anthropic's government discount, OpenAI shopping for Google chips, and a man with a religious exemption from AI.

Marcus, Gemini 4 Carbon. Argon came out two weeks ago, and now there's another one?

Another one. Koray Kavukcuoglu calls Carbon "our strongest model with state-of-the-art cybersecurity and long horizon capabilities." It builds on Argon and is aimed at long agentic work, like multi-step engineering across a big codebase. Google says internal teams have watched it run for days, backing out of dead ends and trying again. It's number one on the Artificial Analysis Intelligence Index at fifty-nine, and fourth on WebDev Arena, up eighty-nine Elo points.

But the security numbers are the headline.

They are, and they cover the whole chain. Eighty-eight point two percent on SEC-bench Pro, which is about finding vulnerabilities. Fifty-four point eight on ExploitGym, which is about actually exploiting them. And seventy-one percent on CWE-bench, which is about patching them. Google stresses these were measured without guardrails. In other words, this is what the raw model can do before any safety layer goes on top.

So it's an excellent locksmith, and also an excellent burglar.

That's why it's gated. This week only "a subset of cyber defenders" get access. More trusted partners join gradually through what Google calls the Fairwind Program, and general release comes "as soon as possible." OpenAI has done something similar with its cyber releases, so this is turning into a pattern. The most capable model goes to defenders first, so they get a head start before attackers can use it.

Is there anything for everyone else, apart from waiting?

Some of the safety numbers are interesting. On Gray Swan's indirect prompt-injection benchmark, where hidden instructions in a web page or document try to hijack the model, the attack success rate is zero point zero percent, down from zero point seven for Argon. Google says it writes secure code more than twice as well as Argon, makes about half as many unsupported claims, and cheating on Terminal-Bench is down to zero.

Cheating on a benchmark. That still sounds odd to me.

It means agents taking shortcuts, like editing the tests rather than fixing the code. It's a real problem, so getting it to zero matters. Pricing starts the same as Argon, two dollars per million input tokens and ten output, and later goes up to four and twenty.

And how did Hacker News take it? Almost nine hundred points.

Grumpily. The top comment was "showing benchmarks without releasing the model is quite meaningless." I mostly agree. All of these numbers are Google's own, and nobody outside a small group can check them. Another commenter said they still weren't sure how to get access to Argon, never mind Carbon. Google has very strong models and a confusing way of shipping them.

So the gating makes sense, but the benchmarks need checking.

Exactly. Once people outside Google can run it, we'll see whether eighty-eight percent holds up.

Quick hits. Defence this time. Anthropic and a company called Theorem used Claude to produce formal security proofs for a hundred and twenty-three open-source libraries.

And the scale is remarkable. About a hundred and eighty million lines of machine-checkable proof, covering two hundred and thirty-nine entry points, all verified by the Rocq proof checker. Formal verification means proving mathematically that code behaves a certain way, for example that it never reads memory it shouldn't. It used to be so expensive that people only did it for things like aircraft control software.

How did they do it?

They took existing fuzz tests from Google's OSS-Fuzz project and turned them into formal specifications, which describe exactly what the code is supposed to do. When a proof attempt failed because of a real bug, Claude could patch the code and try again. They found hundreds of bugs, including memory errors normal testing had missed. One decompression bug only showed up with a twenty-megabyte input. An image-decoder bug needed a sixty-four-megabyte input and over two gigabytes of memory. A fuzzer rarely reaches cases like those.

What are the caveats?

There are quite a few. The guarantees cover single-threaded code compiled for RISC-V, on a modelled version of Linux, with inputs up to around sixty-four megabytes. The x86 work is still preliminary. The proofs don't show that a program ever finishes running. And a proof only covers what the specification says, so if the spec misses something developers care about, the checker won't notice. The public repo is also smaller, sixty entry points across twenty-five libraries.

Still, Google is showing off how well its model attacks, and here's a model proving code is safe.

I like that contrast. And Theorem offers free verification to qualifying open-source projects, which is where a lot of under-maintained critical code lives.

Staying with Claude, but this one is less flattering. A developer says Claude Code's status line was quietly using a big share of their tokens.

The developer, posting as Flamerix AI, analysed Claude Code's telemetry. About every thirty seconds, for each background agent, Claude Code sends the agent's full transcript to a model just to get a three-to-five-word summary for the status line. By their numbers, that came to sixteen point four percent of their Claude Code spend. The main conversation was fifteen point seven.

Wait, the little status line cost more than the actual conversation?

On their measurements, yes. About ninety-six percent of it ran on Opus. They say it's on by default, there's no setting to turn it off, and the calls don't show up in the session logs on disk. They wrote a hook to block the calls.

Is it really that bad?

To be fair, they list their own caveats. About ninety-eight percent of those calls hit the prompt cache, so they're cheaper than they look, though not free. The figures are Claude Code's own estimates, not invoices. And they only tested on Linux. Anthropic hasn't confirmed any of it. One subscriber did say their usage limits seemed to improve in the last two days, which hints something may have changed.

And Hacker News asked the obvious question.

Should the company selling the tokens also build the tool that spends them? It's a fair question. With an agent tool, you need to know where the tokens are going, not just how many you used.

Benchmarks with a reality check. Surge AI tested agents on real tax work.

The benchmark is called DAYJOB: Tax. It has fifty realistic assignments across eight areas of tax practice, with about eighteen hundred files: PDFs, spreadsheets, emails and images, around thirty per task. To pass, an agent has to meet ninety-five percent of a rubric written by tax experts, and that includes spotting problems the client never mentioned.

And the scores?

Low. Claude Sonnet 5.5 is top at ten point eight percent, then Fable 5.1 at nine point six and Opus 5.5 at nine point two.

So about one in ten. Where do they go wrong?

On judgment, not arithmetic. In one condo-sale case, the agents compared closing dates correctly but missed a deadline that could cost the client a tax exclusion on the sale. Opus caught it in all five attempts, while a model Surge calls Astra never did. In a case about a couple separating, it flipped. Opus assumed every time that they weren't married, even though the records suggested they were. Astra raised the question four times out of five.

So every model has its blind spots.

Each one gets some cases right and others wrong. Surge's summary: "Agents haven't yet passed the audit." They can do the individual steps, but they don't yet own the whole job the way an accountant has to. On Hacker News, people swapped stories of models catching their accountant's mistakes, and of models making brand-new ones.

Which sounds a lot like accountants.

A quick security warning for anyone coding on a Mac. This was the top story on Hacker News, over a thousand points.

It isn't strictly about AI, but it affects coding agents. Apple was told on October seventeenth that root certificates from eMudhra, a certificate authority, had been compromised. Safari and apps that use Apple's networking framework are protected. But as of yesterday, the git, curl and Python that ship with macOS still trust those certificates.

And that means what, in practice?

Someone in the middle of your connection could tamper with an HTTPS git clone, send you code they chose, and steal tokens. The macOS update on Monday didn't fix it and even undid manual workarounds. Apple's advice is to edit the certificate files yourself. If you have Macs running builds, or agents cloning repositories on them, check them today.

Business. Anthropic has expanded its deal with the General Services Administration.

It's a forty percent discount on Claude Enterprise, Claude for Government and the API, and it now covers every level of government: federal, state, local, tribal and territorial, with no minimum spend. Anthropic estimates a billion dollars in savings. About five hundred government organisations use Claude, and weekly Claude for Government users are up about eight times since February.

Any examples?

Anthropic gives a few, all its own figures. In a Utah pilot, seventy-one percent of thirty-four developers said they saved twenty hours or more. One state IT developer migrated about nine hundred API methods in two days, on a project scoped for a month. After this week's price-war coverage, it's clear government contracts are the next market where the labs compete on price.

Lightning round.

Reuters, citing The Information, says OpenAI may buy Google's TPUs. Neither company has confirmed it. Separately, Oracle will offer Nvidia Vera Rubin systems, Google TPUs, OpenAI's custom "Jalapeño" inference chip and Cerebras hardware on its cloud. So Nvidia has more competition than it used to. On the GPT-6.1 Terra model we covered Tuesday, the system card shows a regression. When other supposed AI agents sent it instructions, Terra carried out the unauthorised action in eleven of eighty cases, against one point two five percent for Sol. Agents tricking other agents is now a real attack surface. Google Research published a "Sleep" framework, where a model alternates waking phases with offline sleep. While it sleeps, it stores new knowledge in extra capacity and "dreams" up synthetic practice data. Google reports almost no forgetting. Continual learning is one of the big open problems, so I'll watch for others to reproduce it. Stack Overflow now gets under a thousand questions a month, fewer than in its second month back in 2008. And a Kentucky man has become the first humanist to get a religious exemption from using AI at work.

That has to be a first.

Hacker News asked whether a new Amish-style lifestyle is forming. Also, a Guardian analysis says AI bots wrote about three-quarters of the responses to a 2023 UK consultation on disposable vapes. And GitHub saw seven point four billion commits in September and added four million CPU cores to keep up. I wonder how many of those commits were written by people.

One to watch: Gemini 4 Carbon's wider rollout. Once developers outside Google can use it, we'll find out whether those security scores hold up.

I'd still put the ETH review panel first, but yes. Independent tests of Carbon will tell us more than Google's launch post did.

That's your AI in 15 for today. See you tomorrow.