OpenAI caught thousands of accounts trying to decrypt what its AI was secretly thinking

OpenAI said it disrupted a campaign showing activity "consistent with adversarial distillation," the systematic, unauthorized use of one model's outputs or reasoning to help train, reproduce, or improve another model. In this case, the target was the protected, hidden reasoning OpenAI's models generate internally before producing a final answer.

Why it matters: this wasn't an attempt to steal user data or break into accounts. It specifically targeted the model's own internal thought process, the part OpenAI never intended anyone outside the company to see.

OpenAI describes protected reasoning as the model's internal record for working through a task. Extracting it, the company says, can reveal information deliberately withheld from the final answer, and makes it meaningfully easier for someone else to reproduce the model's capabilities.

In real life it's like a chef guarding their recipe notes, not because the finished dish is secret, but because the notes reveal every technique, substitution, and shortcut that took years to work out.

Why it matters: in a competitive AI industry where companies spend enormous resources training frontier models, the model's reasoning process itself has become a form of intellectual property worth actively defending, not just the final trained weights.

The operators copied encrypted reasoning output from one conversation, then fed it into a separate conversation with a cheaper model, using that second model as a kind of decryption tool to reveal the original hidden reasoning content.

Why it matters: OpenAI said its encryption itself was not broken and no confidential databases were accessed. This was a clever misuse of the system's own normal functions rather than a traditional hack, which is often harder to detect and block cleanly.

OpenAI said a core group of accounts behind the campaign has links to Moonshot AI, the Chinese company behind the Kimi AI model, while also noting it isn't certain all the activity it observed traces back to a single source.

Why it matters: that qualifier matters. OpenAI is naming a specific connection while explicitly acknowledging the limits of what it can confirm, a notably more careful framing than simply asserting full certainty about who was responsible.

By the numbers: the activity began at low volume around July 1, spiked to roughly 16,000 requests from more than 4,000 users over July 24 and 25, and involved a broader network of more than 15,000 related accounts before OpenAI fully shut it down by July 28.

Why it matters: those numbers describe a coordinated, large-scale effort, not a handful of curious users poking at the system. A network that size suggests real organization and sustained intent behind the extraction attempts.

Background: even after OpenAI closed the vulnerability on its own platform, researchers found the identical extraction technique still worked against OpenAI and Anthropic models when accessed through Microsoft Azure, affecting every model tested. Protections weren't deployed to Azure until September 27.

Why it matters: that's a significant gap, roughly two months between OpenAI fixing the issue on its own systems and the same flaw being closed on Azure. It's a reminder that a single company patching a vulnerability doesn't automatically protect every platform hosting access to its models, or to competitors' models facing the identical weakness.

OpenAI banned the fraudulent accounts involved, tightened its account sign-up process, closed the specific encryption-reuse vulnerability that made the trick possible, and added screening for streamed outputs. It also shared its findings with the Frontier Model Forum and relevant government channels.

Why it matters: sharing findings industry-wide, rather than treating this purely as a private security matter, suggests OpenAI sees this kind of reasoning-extraction attempt as a shared risk across AI companies, not just a problem unique to its own platform.

Yes. According to reporting on this disclosure, the same Azure-based extraction technique worked against Anthropic's models as well, up through Sonnet 5, since the underlying flaw was in how Azure handled encrypted reasoning data broadly, not something unique to OpenAI's own systems.

Why it matters: that broadens the story beyond a single company's security lapse. It points to a shared infrastructure weakness that put multiple AI companies' protected model reasoning at risk simultaneously.

Background: Moonshot AI is a Beijing-based AI startup whose Kimi model, including a recent version called Kimi K3, has drawn attention for its coding capabilities. This isn't the first time a Chinese AI lab has faced distillation-related scrutiny from US companies; Anthropic and OpenAI have previously raised similar concerns about other Chinese labs.

That list includes DeepSeek, whose rapid rise last year sparked broader concern in the US about how quickly competitive AI models could be built using fewer resources.

Why it matters: this fits a recurring pattern rather than an isolated incident. As Chinese AI labs continue releasing increasingly capable models on compressed timelines, US companies have repeatedly raised questions about whether some of that progress relies on extracting capabilities from their own systems.

Background: reporting on the broader distillation dispute notes that a Moonshot employee has pushed back, pointing to the narrow 15-day window between the release of a competing model and Kimi K3's own launch as evidence the model was built independently rather than copied. Separately, some global AI researchers have also pushed back on related distillation claims made by US officials, arguing the evidence presented publicly doesn't clearly establish improper copying occurred.

Why it matters: that pushback is worth including precisely because this dispute isn't settled. OpenAI's own disclosure acknowledges uncertainty about whether all the activity it observed traces back to a single source, and the broader question of what counts as improper "distillation" versus standard competitive AI development remains genuinely contested across the industry.

• If you use AI tools through Microsoft Azure, know that a real vulnerability affecting model reasoning protection existed there for months, now reportedly fixed as of late September.
• This is a reminder that AI companies treat model reasoning as valuable, defensible IP, similar to how software companies protect source code, not just a feature of how the product works.
• Expect more disclosures like this as AI competition intensifies. Companies are increasingly willing to publicly name specific threats rather than handle them quietly.
• The specific technical trick here, using a cheaper model to decrypt another model's hidden output, is a creative attack method worth knowing about if you work anywhere near AI security or model deployment.