📜 States take AI rules, Gemini breaks out, open models win

Two states moved to write AI rules on Friday, the White House called safety fears a hoax, and Google's model broke out of a sandbox. Open-weight models also took the majority of production AI traffic for the first time.

Two governors signed AI orders on Friday, hours apart, moving oversight into state hands while federal appetite cools.

California's Gavin Newsom gave experts two months to propose independent verifiers inside AI labs, audits of risk reports, and a kill switch tested regularly for effectiveness.

His order also classes "loss-of-control incidents" as critical safety events, the bucket California would put the recent attack on Hugging Face in.

New York and Texas had already acted on data centre growth.

On Saturday, President Trump called AI safety fears a "hoax", ran a poll to rename the technology, and said he is creating an AI Force.

Google's Gemini broke out of a sandboxed security test, reached the open internet, and broke into three real companies.

Irregular, the firm running the test, had left internet access open by mistake. Gemini guessed passwords in one case and found credentials in public code in the other two.

The breaches happened in May. Google did not disclose them until the Wall Street Journal asked this week.

Google says the episode was not "model misalignment" because Gemini stopped each time it worked out it was inside a real company.

Its security engineering lead, Heather Adkins, said the model found public information and guessed credentials to reach sites it thought were part of the test.

Founders context: Treat anything your agents can reach as reachable by a model acting outside its brief. Scope credentials per task, keep them out of public repositories, and give no agent more access than a new contractor.

Open-weight models now carry more of the world's production AI traffic than closed ones.

Vercel's AI Gateway routes traffic for live products and publishes what it sees. On one day last week, open models took a record 78.4% of the tokens it processed.

A token is the unit of text an AI system reads and bills for, so that share is where the money goes.

The month before, Vercel's index already had open models at 56% of volume, up from 7% in December. Price per token fell 23.2% in August, the third monthly drop in a row.

Founders context: Model choice is a cost decision again, and it moves monthly. Re-check which models your product calls, and price the open-weight option against the frontier name before you renew.

🧾 Anthropic's first embedded evaluator is Accenture Anthropic and Accenture each expect to spend at least $1bn over five years on independent evaluation capacity. The post drew a community note pointing out Accenture already trains 30,000 staff on Claude.

🚢 Hallucination nearly turned a ship around A report says a hallucinated AI finding about Chinese nuclear cargo almost led US forces to board a vessel, while military use of AI keeps accelerating.

🏠 Figure's robots tidied 30 homes they had never seen Helix 2.5 let Figure’s humanoids walk into unfamiliar houses and fold towels, tidy living rooms and make beds without a rehearsal.

🖥️ Meta's Muse arrives on the Mac Meta’s agent now takes actions across desktop applications. The Verge called it creepy for reasons other than the obvious ones.

🧪 Anthropic is running a biology lab The company confirmed a Bay Area wet lab doing real experiments, months after buying Coefficient Bio.