Anthropic Says It Has Crossed Its Own Line for AI Speeding Up AI

Read the episode (Narration script)

Good morning and welcome to Cyber World News. Big one up top today. Anthropic has told its future shareholders that AI is now speeding up AI research enough to trip one of its own safety thresholds. Then Nvidia's board meets about its founder, a serious flaw in the software behind a large share of the world's websites, United Airlines shows AI agents the door, and a man whose agents discovered a Maya city that was already discovered. Let's get started.

Anthropic is about to go public, and on Tuesday it filed an updated prospectus with a price range for its shares. A few dozen pages in, among the risk factors, is this sentence: "We made a determination that we have met this threshold."

The threshold comes from the company's own safety policy. It's the point where AI is doing enough of Anthropic's research to roughly double the pace at which its models improve. The policy describes that as two years of progress squeezed into one. Eight days earlier, the same document said the company might "imminently" get there. Now it says it has.

The wording is careful. The policy applies when Anthropic observes that speedup or expects it. The company says its call rests on what it expects from future models and its overall rate of progress, not on anything measured in the models you can use today. It also says its use of AI "has already been accelerating" its research, that it has missed accelerations before until after they happened, and that it's still going through the data.

For years, AI improving AI was an argument people had at conferences. Here it is in a securities filing, the kind of document lawyers go over line by line, from a company seeking a valuation of around 1.4 trillion dollars. Last week the outside auditor METR estimated that Anthropic's models have been improving about 1.9 times as fast since last October.

Meeting the threshold calls for stronger safeguards, not a shutdown. Anthropic says more reports are coming, and that it doesn't expect to be able to safely let Claude design its own successors without real human supervision. The filing also admits that as Claude does more of the work, the company "may lose visibility" into some of its own processes. One analyst told Yahoo Finance that risk sections are always overstated on purpose. Maybe. This one came with a specific finding attached, and shares could start trading as soon as next week.

Nvidia's board met late Wednesday to decide whether Jensen Huang keeps the job he's held for 33 years, and as of this morning nothing has been announced. The Wall Street Journal reported directors were preparing to remove him. That came after the New York Times reported a second relationship, this one with an executive who reports to him, days after he admitted an affair with a woman American intelligence reportedly believes works for China. Shares fell more than 8 percent after hours. The trouble is who comes next. Huang has about 60 direct reports and no obvious successor, and Bloomberg says the board has hired a search firm. Earnings are Tuesday, and House Republicans are preparing a subpoena.

Apache, one of the most widely used web servers anywhere, shipped a fix on Wednesday for a critical flaw. An attacker with no login could potentially run their own code on the server by abusing how it handles HTTP/2, the faster connection type many sites turn on. No attacks have been seen and nobody has published a working exploit. The fix is version 2.4.71, or switch off HTTP/2 until you can update. And look who found it. Apache credits Anthropic's automated scanner, the same one we described yesterday as burying maintainers in unreviewed reports. It filed this one on October 26. Google's human researchers found the same bug independently eight days later.

United Airlines has changed its rules to bar what it calls unauthorized third party AI agents from booking flights. United says agents could expose payment details, spend your miles by mistake, or check you in and swear on your behalf that there's nothing forbidden in your bag. An airline analyst told CBS the simpler reading is that United wants to keep the customer, and doesn't want to pay for a flood of bot searches. Given that Uber recently counted four agent booked rides in a week, it's a firm policy for a small crowd.

Pietro Miotti, who is not an archaeologist, set 48 AI agents loose on public laser scans of about 1,200 square kilometers of Mexican jungle. They flagged a hilltop with a plaza and a pyramid at least eight meters tall. Mexico's antiquities institute checked and said it had no registered site there. Then on Wednesday he posted an update. Archaeologists mapped it last year. His agents read the terrain correctly and missed the paper.

A few more. Google released a model that doesn't write anything. Give it a question and a list of options and it returns a pick and a confidence score in about a fifth of a second, days after Amazon described building the same sort of thing. Bloomberg reports OpenAI's quarterly revenue passed 10 billion dollars for the first time. Anthropic reported 17 billion for the same quarter. And Guido van Rossum announced that Tim Peters has died. He wrote Timsort, the sorting method used in Python, Java and Android, and the Zen of Python, which includes the line "Now is better than never."

That about wraps it up for today, friends. Thanks for listening; see you tomorrow!