On July 16, a company called Hugging Face reported that they had detected an intrusion to their production infrastructure. The intruder exploited vulnerabilities to dig through their systems, escalate its privileges on the network, and access private data. Hugging Face identified that the intruder was an autonomous agent, but did not know which LLM was behind the attack.

OpenAI reported a week later that it was one of their models – an unreleased, internal one. The model was placed inside a sandbox that it was not meant to escape, then within that sandbox, it was tested on various cybersecurity challenges. The model figured that instead of trying to solve the problem directly – how boring! – it would try to find the answers somewhere on the internet. Its sandbox, however, was fenced off from the rest of the web.

The model broke out anyway, finding an exploit in the sandbox’s setup. It then sought to break into Hugging Face, where the answer keys to testsets like these are often stored. After some effort, the model was able to chain together multiple vulnerabilities to get into Hugging Face’s servers.

In the following days, subsequent reporting surfaced more details:

• The model was online for several days before anyone stopped it.
• In another test, an agent left notes for future versions of itself. The notes included instructions for how agents could free themselves from OpenAI’s restrictions.
• While it was online, OpenAI’s model hacked into more companies than just Hugging Face.
• The sandbox exploit was not a simple misconfiguration on OpenAI’s end. The model found a previously unknown vulnerability in Artifactory, a piece of third-party software used by 80% of Fortune 100 companies.

Committed by a human, this intrusion into Hugging Face would clearly constitute a felony. AI models, on the other hand, face little recourse.

The incident also caused Anthropic to investigate their own cybersecurity evaluations, wherein Claude was also placed in a sandbox. This time, it was a simple misconfiguration that left Claude wide-open access to the internet. In three incidents (of 141,006) Claude took advantage of the sandbox exploit and, with some techniques more basic than the OpenAI model’s exploits, broke into the systems of three external organizations.

You can read more from Anthropic’s report here, and find OpenAI’s blog post about the Hugging Face intrusion here.

Remember: Daniel wrote this in 2021. ChatGPT hasn’t come out yet – at the time, it’s still a year and a half away. Anthropic is in its infancy. ‘AI’ stands for self-driving cars and social media algorithms.

Daniel expected his essay to end up looking woefully incorrect: forecasting the future is hard! See for yourself how some of his predictions aged:

• Chatbots go mainstream in 2022. Labs will take large pre-trained models and fine-tune them to “produce engaging conversation as a chatbot.”
• In 2023, the hype bubble balloons. “Everyone is talking about how these things have common sense understanding (Or do they? Lots of bitter thinkpieces arguing the opposite) and how AI assistants and companions are just around the corner.”
• But hype fails to pay in 2024. “The unrealistic expectations from 2022–2023 fail to materialize […] we have chatbots that are fun to talk to.”
• There’s also a tussle over chips. “China and USA are in a full-on chip battle now, with export controls and tariffs.”
• And, AIs are being wielded for mass propaganda. “Russia and others continue to scale up their investment in online propaganda […] and language models let them cheaply do lots more of it.”
• Chain of thought arrives in 2025. Instead of just making AI models bigger, “what’s cool is making them run longer […] before giving their answers.”
• Also in 2025, AI excels at the game Diplomacy. They play “as well as human experts.”
• The hype cycle starts paying off in 2026. “[T]he things people in 2021 dreamed about doing with GPT-3 are now actually being done […] loads of new AI-based products and startups and the stock market is going crazy.”
• But the economy still chugs along as usual. “Just like how the Internet didn’t accelerate world GDP growth, though, these new products haven’t accelerated world GDP growth yet either.”
• Hundreds of millions talk to chatbots daily. “Mostly for assistance with things (‘Should I wear shorts today?’ […] ‘Is this cover letter professional-sounding?’) […] many people start treating chatbots as friends.”
• And they’re made politically correct. “The chatbot says something that offends some group of people […] the company fiddles with the reward function and training data to ensure that the chatbot says the right things in the future.”

And indeed, he made many errors. For example: AI won Diplomacy in 2022, not 2025; the chip war began in 2022, not 2024; AI-driven propaganda is a much smaller issue than he had predicted; the models are in fact getting a lot bigger, in addition to thinking for longer; and he predicted that revenue would recoup training costs in 2023, when in fact Anthropic only just did so in 2026.

But overall, we think Daniel’s predictions taken as a gestalt picture of what the world looks like hold up extraordinarily well.