Anthropic Warns AI May Soon Sharply Accelerate Its Own Development

Read the episode (Narration script)

Good morning and welcome to Cyber World News. Anthropic has told investors that AI may soon be speeding up its own development, one of its models built a working attack on a serious Linux flaw that Ubuntu still hasn't patched, and researchers let an AI operate directly on another model's insides. Let's get started.

Anthropic has amended its paperwork to go public, and among the revenue figures sits a sentence the AI industry has argued about for years.

The company says its internal estimates for future models indicate it "may imminently meet" its own safety policy's threshold for "dramatically accelerated progress attributable to AI." In plain terms: AI doing enough of the work of building AI that progress speeds up sharply.

No current model has been designated as crossing that line, and Anthropic is still assessing the data. It also admits its measurements lag. In the past, it says, it failed to notice a significant acceleration until after it had happened.

Anthropic says the value of its own internal Claude usage, priced at list rates, grew more than two hundred and fifty times between January and September. By April, that monthly figure exceeded what it spends on staff, not counting stock compensation.

Its stated limit is human control: it does not expect to safely let Claude build successive frontier models without meaningful human supervision, and it is considering policy changes to keep people in charge.

A securities filing is a legal warning, and also a document written by a company that wants investors excited. Both readings apply. But companies get sued over risk factors, so lawyers choose these words carefully. They signed off on "imminently."

What that capability looks like in practice is visible in Linux right now.

Last week Anthropic's red team said an unreleased research model built a working remote attack on a flaw in how the Linux kernel handles new internet connections. Against a server with a multipath connection listener, an attacker with no password could end up running commands as root, which means full control of the machine. Anthropic's proof of concept took about fifty three minutes, and a failed attempt only drops the connection, so an attacker can keep trying. The code hasn't been published.

The bug dates to 2016 and carries a severity score of 9.8 out of 10. Debian and Oracle shipped fixes in mid October, and Red Hat followed on Sunday. Ubuntu, the default choice at many hosting providers, still lists its supported releases as vulnerable, with fixes in progress. A developer's post titled "WTF Ubuntu?" topped Hacker News yesterday. He took his servers offline rather than wait.

Multipath connections are off by default on many systems, but programs written in Go have used them by default since version 1.24. Go's security team also warns that servers without those listeners may be open to similar attacks, because the flaw sits in ordinary connection handling. If you run Ubuntu servers, plan a kernel update and reboot as soon as the fix lands.

Researchers at MIT have taken the idea of AI working on AI somewhere stranger. They had OpenAI's GPT-6 Astra read and write a neural network's raw numbers directly, with no training process at all.

Asked to write a Breakout player from scratch, it produced one that scored 517, against about 25 for a conventionally trained network after five hundred million steps.

The most practical result came from Qwen, a small open model that often gets stuck repeating itself. Astra traced the loop to a single direction inside two layers, then removed it from about one sixtieth of one percent of the weights. On one math benchmark, scores rose from about eight percent to forty eight. The authors are clear about the limit: "Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked." They call the work very early. It's still a little dizzying.

Y Combinator's Garry Tan told Walter Isaacson's podcast he spends "maybe" one hundred to two hundred thousand dollars a year on AI tokens. His agent has read his email, his Dropbox and about twenty years of his therapy notes. He calls it a Jiminy Cricket on his shoulder. A very well informed cricket. Tan also says it's too expensive right now.

Apple's listening features for the newest Apple Watches are now in developer beta. Double press the Digital Crown and Live Rewind shows the last fifteen seconds of speech as text. Siri Recaps condenses conversations into dates, deadlines and prices. Apple says it doesn't identify speakers or keep raw audio, and unsaved recaps are deleted after seven days.

A couple more before we go. Elon Musk says the cost of serving Grok Bot, SpaceX's AI agent, was "unsustainably high relative to revenue," after users found their weekly limits suddenly maxed out over Halloween weekend. And Marvell has agreed to buy Lumentum, which makes lasers for moving data around AI data centers, in an all stock deal valued at one hundred and fifteen billion dollars including debt.

Finally, a startup called Subvocal wants you to talk to your computer without making a sound. Its glasses send low power radio waves through your mouth and throat, then decode silently mouthed words into text. The founders gave up on face electrodes, partly because people don't want electrodes on their faces. Shipping is targeted for early next year, with no published accuracy figure yet. Hacker News commenters mostly asked where your unspoken words get processed. For now, the cloud.

That about wraps it up for today, friends. Thanks for listening; see you tomorrow!