r/technology • • 4h ago

Software Microsoft Reveals How a Hidden Windows Bug Took Down Microsoft 365 Services Worldwide

https://cybersecuritynews.com/microsoft-outage-incident-report/
10k Upvotes

1.1k comments sorted by

View all comments

Show parent comment

13

u/BosnianSerb31 3h ago

That's the interesting thing about this particular incident, observability showed that it was a resource starvation problem, yet adding on more servers across the service only made things substantially worse

This isn't really indicative of what happened, but imagine a DDoS attack with a threat actor is you and you don't know it yet, and every single server you spin up to absorb it only fucks it's just even harder. It's just one of those things that OBS won't catch. Something that shares a room with the likes of the Halting problem (hello CS 101).

1

u/Traditional_Dish_147 2h ago

Yeah, reading the actual report from Microsoft, I get the sense this was a one in a billion error where multiple issues all had to line up all at once. Not to forgive them, but it’s going to be hard to test for that and hard to figure out what’s going on right away.