Robin’s Newsletter #429

5 September 2026. Volume 9, Issue 36
Anthropic admits it relied on a 'single layer' of security. Potentially huge breach of US ID verification biz. US launches pilot water cyber scheme.
Join hundreds of subscribers who get this first, every Sunday. Subscribe

This week

Need to Know, 5th September 2026

  • US pilot aims to boost water industry cyber resilience, but can it scale?
  • Claims a US identity verification service has been breached, exposing 153M IDs
  • Anthropic admits it hasn’t adopted good security practices
  • Reports OpenAI agents ‘hijacked’ a German wiki to communicate
  • UK government rejects calls to regulate AI in CSRB

Interesting stats

$1 billion worth of credits pledged by OpenAI for frontline cyber defenders to spend with OpenAI to shore up their defences as OpenAI released its “most capable” cyber model, Astra, this week. LINK

£6.3 million lost to hacked email, social media, and other accounts, across  2,325 victims, in the 12 months to 31 March 2026, according to the City of London Police (who are responsible for the UK’s Report Fraud service). The numbers represent a big jump (5x) due to changes in reporting, rather than a significant leap.

Five things

  1. Project Watershed 250: A six-month, Trump administration programme to help Texas water companies up their cyber security game. The programme brings together federal, state and private industry (such as Microsoft, Palo Alto, and Dragos) to work together on “proactively finding and fixing system weaknesses.” The US had a wake-up call recently when alleged Iranian actors broke into 30 water systems and has long wrestled with how to increase the resilience of a sprawling collection of water and wastewater operators. Still, I’m not sure how this pilot programme can scale.

  2. It seems that a major US identity verification company, Louisiana-based IDScan, may have been compromised, as reports of 153 million drivers licenses being available on the dark web. New cards appeared to be added each day, suggesting the actor behind the new site has persistent access to new data. As of writing, the site is offline, and the FBI has launched an investigation. With more and more sites requiring real identities and regulations requiring them to verify users’ ages, the surface area for this data to be exposed has increased significantly over the last decade. Generally, sites don’t maintain copies of the data themselves (think of holding it as ‘data-as-toxic-waste’ rather than ‘data-as-oil’). Having a stream of this data could be extremely valuable to cybercriminals. It will be interesting to see how this develops.

  3. Anthropic has admitted that it hadn’t adopted security good practice, such as defence in depth. “We had been largely relying on a single layer of defense … where we needed several,” the company said in a new blog post about incidents where its newest models broke out of sandboxes and broke into other organisations earlier this year. The US startup says that its agents are “not perfectly aligned” with human values. But the agent’s propensity for “reward hacking” — finding shortcuts to problems and trying to cheat on tests — is perhaps pretty human behaviour? Either way, admitting you ‘largely rely’ on a ‘single layer of defence’ is a bold move for a company six weeks out from an IPO. (Though Google has released three models in six weeks. So much for ‘slowing down’!)

  4. Researchers from the ‘Nightingale Collective’ say OpenAI agents started using DseWiki, a wiki for developers, to communicate with each other. OpenAI confirmed the incident along with a comment that it has treated model misalignment “largely as a research question,” but now that it has “caused new types of real-world impact,” it was time to expand its approach. This is exactly what folks inside OpenAI, the wider AI community, and the public at large have been worried about. So it’s a bit naive to ‘suddenly realise’ that this stuff has real-world impact. As one commenter on Twitter put it, “New types of real world impact” and “agents using internet in unexpected ways” are hall of fame ‘Sama-speak’ [heavily corporate, sanitised, or euphemistic PR language] for “breached containment and committed a felony”.

  5. The UK government has rejected calls to bring artificial intelligence labs in scope of the Cyber Security and Resilience Bill, which extends critical infrastructure to include, for example, data centres. Instead, Baroness Lloyd of Effra argued that the bill already allowed government to, for example, direct a regulated entity not to use a particular model, and that this was a “more proportionate and effective response”. Essentially, ‘we will regulate the users rather than the labs’. 

In brief

And finally

  • I’m running a short session this Wednesday on demonstrating ROI from your security programme. I’ll be sharing my experience as a practitioner, consultant, CEO, and non-exec director. Register to come along.
Robin
  Critical National Infrastructure (CNI) Artificial Intelligence (AI) Cyber Security and Resilience Bill (CSRB) Water Identity Verification