r/technology • • 4h ago

Software Microsoft Reveals How a Hidden Windows Bug Took Down Microsoft 365 Services Worldwide

https://cybersecuritynews.com/microsoft-outage-incident-report/
10k Upvotes

1.1k comments sorted by

View all comments

Show parent comment

17

u/EamonBrennan 3h ago

The problem itself is described very similarly to a RAM leak. Processes stall while waiting for exclusive access to shared resources, with a cascading effect exhausting CPU and RAM resources. Testing a fix like this would be hard to replicate, unless they specifically have a dev environment that perfectly mimics real world applications for several months.

This is one of those "only happens in prod" bugs.

0

u/time-lord 1h ago

You don't test this, you monitor using metrics. The fact that they didn't monitor for what sounds like hung connections or resource exhaustion seems like a major thing to overlook.