For weeks, Microsoft 365 Copilot quietly learn, summarized, and surfaced emails that organizations had explicitly marked as confidential.
Authorized memos. Enterprise agreements. Authorities correspondence. Protected well being data. All processed by an AI that was by no means supposed to the touch it.
The bug — tracked as CW1226324 — was first reported by prospects on Jan. 21, 2026. Microsoft didn’t start rolling out a repair till early February. As of mid-February, remediation nonetheless isn’t full. The UK’s Nationwide Well being Service flagged the difficulty internally. And Microsoft nonetheless hasn’t stated what number of organizations had been affected.
Right here’s the half that ought to preserve each safety chief up at night time: The sensitivity labels had been in place. The knowledge loss prevention (DLP) insurance policies had been configured accurately. Each field was checked. And none of it mattered.
What truly broke
The small print are frustratingly easy.
A code error in Copilot Chat’s “Work” tab allowed the AI to drag emails from customers’ Despatched Objects and Drafts folders — even when these emails carried confidentiality labels and had DLP guidelines explicitly configured to dam AI processing. The labels stated “arms off.” Copilot ignored them.
Microsoft’s public response was rigorously worded: Customers solely accessed data they had been already licensed to see. Technically true. Additionally fully irrelevant.
The query was by no means whether or not the person had clearance. The query is whether or not the AI was licensed to ingest, course of, and summarize confidential content material. It wasn’t. The insurance policies stated so. The code disagreed.
The deeper downside no one desires to speak about
This isn’t a narrative about one bug. Bugs occur. Code ships with errors. Patches roll out.
The actual story is architectural. Each safety management that was supposed to forestall this — sensitivity labels, DLP insurance policies, entry restrictions — lived inside the identical platform because the AI itself. When the platform broke, all the things broke. No second layer. No impartial verify. No backstop.
Consider it this fashion: Think about a financial institution the place the vault door, the alarm, and the safety cameras all run on a single circuit breaker. One tripped wire, and also you’ve obtained an open vault, no alarm, and no footage. That’s what occurred right here.
And this isn’t a theoretical danger anymore.
The World Financial Discussion board’s 2026 International Cybersecurity Outlook discovered that knowledge leaks by generative AI are actually the primary cybersecurity concern amongst CEOs globally, cited by 30% of respondents. The WEF report additionally warned that roughly one-third of organizations nonetheless don’t have any course of to validate AI safety earlier than deployment. The Copilot incident is what that hole appears to be like like when it hits manufacturing.
Why ‘belief however confirm’ fails when the verifier can also be the seller
Microsoft is each the AI supplier and the entity liable for the controls governing that AI. When these controls failed, organizations had no impartial strategy to know. They discovered when Microsoft instructed them — weeks later.
No impartial audit path. No anomaly detection flagging uncommon entry patterns. No real-time alerting when Copilot abruptly began processing confidential content material it had by no means touched earlier than. Simply silence, then a service advisory.
That is the governance hole that platforms like Kiteworks are constructed to deal with. Their argument — and the Copilot bug makes it arduous to disagree — is that AI governance controls have to function on a separate layer from the AI platform itself. Not as a coverage throughout the identical ecosystem. As an impartial management aircraft.
The logic is easy: If AI should authenticate by an exterior governance layer earlier than touching delicate knowledge, a bug contained in the AI platform doesn’t robotically grant entry to all the things. Goal binding restricts which knowledge classifications AI can course of. Least-privilege entry means AI brokers get solely what they want, not broad entry to whole repositories. And impartial audit trails imply you realize what occurred no matter what the seller’s logs present.
Compliance time bomb
Right here’s the place it will get costly.
If Copilot processed emails containing protected well being data — and the NHS flagged this internally, in order that’s not hypothetical — organizations could face breach notification obligations beneath HIPAA. If GDPR-covered private knowledge was processed, Article 32’s requirement for “acceptable technical measures” comes into play. Telling a regulator your sole safeguard was a vendor management that failed for weeks is a tricky promote.
The EU AI Act’s Article 12 record-keeping necessities add one other wrinkle. In case your solely proof of what the AI accessed comes from the seller that had the failure, that’s a documentation hole no authorized workforce desires to inherit.
What comes subsequent
The repair is to not cease utilizing AI. That ship has sailed. The repair is to cease treating AI platform controls as enough on their very own.
Protection in-depth isn’t new. We don’t defend networks with a single firewall. We don’t safe buildings with only a entrance door lock. However someway, we’ve been trusting AI governance to a single layer of sensitivity labels managed by the identical platform working the AI.
The Copilot bug didn’t reveal a brand new danger class. It confirmed one which safety leaders have been warning about since enterprise AI adoption took off. The organizations that climate these incidents — and there will probably be extra — are those constructing impartial governance now, earlier than the following service advisory lands.
The labels had been in place. The insurance policies had been configured. And the AI learn your confidential emails anyway.
If that doesn’t change how you concentrate on AI governance, what’s going to?
Additionally learn: The Reprompt assault reveals how a single hyperlink can hijack Copilot periods and exfiltrate knowledge.
