Nobody Could Say When It Last Worked

The team had been early. Microsoft was letting customers use its new agent security tools before they were finished, and this team signed up. They put agents into a few real workflows and switched on a real-time protection rule that stops an agent leaking credentials or sensitive data through the tools it calls, or sending email or data to an untrusted domain (Microsoft Security Blog, 27 July 2026). Most organizations would not get that protection until it was finished at the end of July. This team had it running months earlier, which is exactly what you would want a security team to do.

Then Microsoft changed something.

Microsoft moved where these rules live. On 1 July, the existing Agent 365 real-time protection settings moved to a new policies screen, and rules set to block stopped enforcing that day. To keep blocking, somebody had to redefine the rule in the new place. Nothing carried it across, and nothing raised an error when it stopped (Microsoft Learn, 1 July 2026).

On 27 July, Microsoft announced that real-time protection for agents was generally available. The team read that the way anyone would, as confirmation that what they had been running was now fully supported. Nobody went back to check whether their own rule had survived the change, because nothing suggested it had not.

Somebody asked the question five weeks later.

When did this rule last block anything?

Nobody could say. There had been no alerts from it in weeks, and everyone had been treating that as good news.

So, a question. Can anything in that folder tell the difference between a control that is working and a control that quietly stopped?

Start with what these controls actually do. AI agents now run inside companies doing work people used to do, and they reach real systems to do it. Some of what they can do is fenced off by a rule.

There was a second change on the same date, and it hit a different set of organizations. Until 1 July, security capabilities for agents built in Copilot Studio and Microsoft Foundry were covered through Defender for Cloud Apps and Defender for Cloud respectively. Microsoft moved them behind Agent 365, a separate licence, and said so plainly: these capabilities are no longer covered by existing Defender for Cloud Apps or Defender for Cloud licences. Organizations without an eligible Agent 365 licence could no longer use the affected agent-security capabilities: agent discovery, security posture, threat detection, real-time protection, and investigation and response. All on the same day. In affected tenants, the Defender portal could instead show a message about the licence required.

Two failures, one date. The migration caught the organizations that were already paying. The licence caught the ones who thought they were.

Derk van der Woude, a Microsoft Security MVP, described it two weeks beforehand better than the documentation did. "This is a silent enforcement gap, Block rules don't error out, they just stop enforcing." (Medium, June 2026)

On its own that is a migration note. The same platform does the same thing twice more.

Security teams run searches to check what agents have been doing. Those searches now read from a new place, and the new place only works with the right licence. Without it, the search comes back empty. No error, no warning. An empty result looks exactly like a clean environment.

Then the activity records themselves. Microsoft's guidance from 10 August says somebody has to be assigned that licence, and that owning it without giving it to anyone is not enough. Without that, the agent sends its records, the system replies that everything went fine, and the records are discarded. The agent runs. The message says success. Nothing arrives.

Three places, one pattern. Each one says fine and gives you nothing.

Nothing about normal operation would surface it. No alerts looks like a quiet month. An empty search looks like a clean environment. Both are what everyone is hoping to see, which is why neither gets a second look.

When a system goes down, somebody finds out. A monitor fires, a user calls the service desk, a ticket opens. None of that happened here, because nobody in the building did anything. No engineer switched a rule off. No architect signed a change. There is no ticket, because nothing happened that would produce one. It changed at a supplier's licence boundary, and everything the team held about that rule stayed accurate throughout. The screenshot, the settings, the approval. Every one of them still proved exactly what it always proved, which is that somebody set something up once.

That is Governance Debt arriving. The work that got deferred was never the setup. It was deciding how you would know the control still ran, and then doing that on a schedule. Skipping it costs nothing you can see. The agents keep working, the dashboards stay quiet, and the rule is still sitting there in the console. Then an auditor pulls a sample, or something goes wrong and you go looking for what the agent did. That is when you learn the rule has not been stopping anything, and that there are no records for the period you are being asked about. The gap is not a few days. It runs back to the day the licence changed, which for anyone affected is 1 July.

The rules here are older and plainer than people expect. NIST asks whether a control is working as intended and producing the result you wanted (NIST SP 800-53, control assessment). Producing the result is the part a screenshot cannot show. In the UK, a firm must tell the FCA about anything the regulator would reasonably expect to hear about, and the Handbook’s guidance names a significant failure in a firm’s systems or controls as one of those things. Elsewhere in the same section, when the FCA sets out how to judge whether a breach is significant, one of the four factors listed is whether there were delays in identifying or rectifying it. A control that fails without complaining is one you find late, and finding it late is the kind of thing that gets weighed.

Microsoft has already written the sentence that settles the technical question. In its own guide, step four tells you to check the data actually arrived, because a success message is not proof that it did (Microsoft Learn, 10 August 2026).

A control that is set up and a control that is working are two different things. The only way to tell them apart is to send something through and look at what comes out.

The fix takes an afternoon. Pick a control you have signed off on. Decide what one transaction it should stop or record. Do this with a test account and a destination you own, and use harmless stand-in data rather than a real credential or a real customer record, because the point is to test the fence, not to climb it. Run it on purpose. Then go and check the far end. For a rule that blocks, try the thing and confirm it was refused. For a search, create something you know happened and confirm it shows up, rather than treating an empty screen as an answer. While you are there, check the things the control quietly depends on, because in this case the break was a licence that nobody had assigned to a person. Write down the date, what you sent, and what came back. That is a different kind of proof from a screenshot, because it shows a result rather than a setting. One test does not prove the control worked all quarter. It proves you have a way of finding out when it stops.

Then write down the dates when a supplier can change how your controls behave. Renewals are the obvious one, but the July change came from a licence being repackaged, not from one expiring. Migrations to a new screen, changes to what a licence includes, and old features being retired all belong on that list. Any of them can change what a control does without anyone at your company deciding anything. The Authorization Coverage Lifecycle makes this point about your own agents: approval at launch is not authorization over time. July added the version nobody had written down. The thing that changed was not yours, and you were not set up to see it change.

The rule is still in the console. It still says block. What that proves is that somebody set it up, on a day that is now some months behind you.

In your organization, when did anyone last confirm that an agent control did the thing it is supposed to do?

The team described at the start is constructed. No organization, date or quotation is attributed to it. Every product, regulatory and source claim is cited above.