Security News > 2022 > July > Massive Microsoft 365 outage caused by faulty ECS deployment
In a preliminary post-incident report, Microsoft has revealed that this week's 5-hour-long Microsoft 365 worldwide outage was triggered by a faulty Enterprise Configuration Service deployment that led to cascading failures and availability impact across multiple regions.
ECS is an internal central configuration repository designed to enable Microsoft services to make wide-scope dynamic changes across multiple services and features, as well as targeted ones such as specific configurations per tenant or user.
What initially started like a minor Microsoft Teams outage ended up expanding downstream to multiple Microsoft 365 services with Teams integration that also leverage ECS, including Exchange Online, Windows 365, and Office Online.
As a result, users worldwide began reporting that they could not use Microsoft Teams and multiple Microsoft 365 services or features.
"A deployment in the ECS service contained a code defect that affected backward compatibility with services that leverage ECS. The net result was that for services that utilize ECS it would return incorrect configurations to all its partners," the company explained.
As a result of this incident, Microsoft says they're working on improving the resiliency of the Microsoft Teams service to fail back to a cached ECS configuration version in the event of a future ECS failure.
News URL
Related news
- Microsoft Azure outage takes down services across North America (source)
- Microsoft 365 anti-phishing feature can be bypassed with CSS (source)
- Microsoft 365 anti-phishing alert “erased” with one simple trick (source)
- Windows Server August updates fix Microsoft 365 Defender issue (source)
- Microsoft Fixes ASCII Smuggling Flaw That Enabled Data Theft from Microsoft 365 Copilot (source)
- Copilot for Microsoft 365 might boost productivity if you survive the compliance minefield (source)
- Microsoft fixes bug crashing Microsoft 365 apps when typing (source)