The provider post-mortems from 2025 and 2026 describe one failure again and again: a change reached every region or edge site before anyone could see what it did. In the six incidents below where the provider’s records let us compare, its first update to customers then arrived 23 to 72 minutes after the start of impact that the provider later reported. This post goes through four 2025 incidents as the providers published them, compares the update times, puts the cost surveys in context, and lists what an independent check needs to do.

Reliability is not getting worse across the board. Uptime Institute reports that outage rates per site fell for a fifth consecutive year, but that external infrastructure failures are more prominent in publicly reported outages, and that third-party providers (cloud and internet giants, telecoms and colocation companies) accounted for about two-thirds of the publicly reported outages it has tracked over nine years (Uptime Institute Annual Outage Analysis 2026, May 13, 2026). In New Relic’s 2026 survey of 2,575 engineering and IT leaders and practitioners, third-party and cloud provider failures were the most cited cause of high-impact outages (New Relic 2026 Observability Forecast).

Four 2025 incidents, as the providers reported them

Times are as each provider published them. Where a provider reports in Pacific time, that is noted.

January 8–11, 2025: Azure East US 2 (about 50 hours)

Between 22:31 UTC on January 8 and 00:44 UTC on January 11, Microsoft’s post-incident review says, a networking configuration change in a single availability zone of East US 2 caused connectivity problems, timeouts and resource allocation failures across multiple Azure services. The root cause it describes: during an operation to enable a feature, replicas of the Azure PubSub service were recycled in parallel instead of one at a time, three metadata partitions lost quorum, and about 60% of network configurations stopped reaching the networking agents on hosts. Services configured as zonally redundant but using VNet integration could be affected across several zones.

Two details matter for monitoring. Microsoft’s timeline lists targeted notices to the earliest affected services at 23:43 UTC and a status page update describing the wider networking impact at 02:37 UTC on January 9. And the review calls the advice given during the incident to fail over “an inappropriate communication.” Provider advice written before the scope is known can be wrong, so verify against your own user path.

April 16, 2025: Spotify (3 hours 25 minutes)

Between 12:20 and 15:45 UTC the outage affected the majority of Spotify’s users worldwide, with most traffic disrupted everywhere except Asia Pacific. In its incident report, published May 9, 2025, Spotify says it changed the order of its Envoy proxy filters, judged the change low risk and applied it to all regions at the same time. That triggered a bug in one of its own filters and crashed every Envoy instance simultaneously. The restart, combined with client retry logic, produced a load spike that exposed a second problem: Envoy’s maximum heap size was set above the Kubernetes memory limit, so Kubernetes kept shutting instances down and restarting them. Spotify mitigated it by adding perimeter capacity. Asia Pacific stayed up because lower traffic at that hour meant memory never reached the limit.

June 12, 2025: Google Cloud (global API errors)

Google’s incident report describes a policy change with unintended blank fields, inserted into regional Spanner tables at about 10:45 AM PDT and replicated globally within seconds. It hit code added on May 29 to Service Control, the component that checks authorization and quota on API requests. That code had no error handling and no feature flag, and the failing path was never exercised during its region by region rollout, so Service Control crashed in a loop in every region. Google’s incident page runs from 10:51 to 18:18 PDT; its mini incident report gives a core duration of 3 hours, with every region except us-central1 mitigated by 12:48 PDT. Google Cloud, Google Workspace and Google Security Operations products returned increased 503 errors on external API requests.

Google also reports that its first incident post went up about an hour after the crashes began, because the Cloud Service Health infrastructure was itself down, and that some customers’ monitoring running on Google Cloud failed too. Cloudflare reported a 2 hour 28 minute outage of its own the same day, after an outage at a third-party cloud provider took down storage behind its Workers KV service. Its post-mortem does not name the provider.

July 14, 2025: Cloudflare 1.1.1.1 (62 minutes)

Cloudflare’s 1.1.1.1 public DNS resolver was unavailable worldwide from 21:52 to 22:54 UTC. Cloudflare’s post-mortem traces it to a configuration error made on June 6 at 17:38 UTC, 38 days earlier: a release preparing a topology for a future Data Localization Suite service accidentally included the 1.1.1.1 prefixes. It sat dormant because it changed nothing in production. A second change on July 14 added an offline test location to the same service, triggered a global refresh and withdrew all 1.1.1.1 prefixes. The release was peer-reviewed by multiple engineers but, Cloudflare says, did not go through canary deployments, because the legacy system it touched does not use staged rollouts.

Cloudflare’s own alerts fired at 22:01 UTC, nine minutes after traffic dropped, and the fix was deployed at 22:20. Impact depended on how users reached the resolver: DNS over HTTPS through cloudflare-dns.com, which uses different IP addresses, was mostly unaffected.

The common thread: a change that reached everywhere at once

The providers describe the same shape in their own words:

  • Spotify applied a change it judged low risk to all regions at the same time.
  • Google’s policy data was replicated globally within seconds, and the code that crashed was not behind a feature flag.
  • Cloudflare’s July 2025 change reached every data center without canary deployments. Its resilience plan says of the November and December 2025 outages: “In both events, a wrong configuration took down our network in seconds.”
  • Microsoft’s Azure Front Door rollout waits about a minute between stages, but the crash surfaced about five minutes after deployment, when the change had already reached most edge sites.

ThousandEyes reported in January 2025 that configuration changes were behind many 2024 outages, and that cloud providers’ share of the ISP and cloud outages it tracks rose from 17% in 2023 to 27% in 2024. Its conclusion, from a vendor that sells monitoring: because testing every scenario is impractical, “monitoring the service as a whole provides the best opportunity to mitigate disruptions and user impact” (ThousandEyes Internet Report, January 17, 2025).

How long providers took to tell customers

A status page publishes a diagnosis, and a diagnosis takes people time. The table compares each provider’s own start of impact with its first update to customers, using only timestamps from the provider’s review or status history.

IncidentStart of impact (provider’s report)First update to customersGap
Azure East US 2, Jan 8, 202522:31 UTC23:43 UTC targeted Azure Service Health notices; status page updated with the wider impact at 02:37 UTC on Jan 972 min (status page: 4 h 6 min)
Google Cloud, Jun 12, 202510:51 PDT11:46 PDT, Cloud Service Health55 min
AWS us-east-1, Oct 19, 202511:48 PM PDT12:11 AM PDT, Health Dashboard23 min
Azure Front Door, Oct 29, 202515:41 UTC16:18 UTC, public status page (Microsoft’s monitoring alerted at 15:48)37 min
AWS us-east-1, May 7, 20264:20 PM PDT5:25 PM PDT, Health Dashboard65 min
AWS CloudFront, Jul 16, 202612:45 AM PDT1:44 AM PDT, Health Dashboard59 min

The start times are ones the providers worked out afterwards, so nobody could see them live, and the gaps are not a finding of negligence. They show the window in which your own check is the only signal you control. AWS entries come from the public Health Dashboard history as read on October 17, 2026. For why the status page itself can lag or go dark, see Your Provider’s Status Page Is the Last Place You’ll Learn You’re Down.

What these outages cost, and for whom

Four recent studies, with the population each one covers. They are not comparable with each other, and none measures a typical small website.

SourceWho was surveyedFinding
Splunk and Oxford Economics, May 20262,000 executives at Global 2000 companies, 20 countries$600 billion a year in unplanned downtime across those companies, 50% more than the $400 billion in the 2024 edition (53 countries, 9% of profits). Average $15,000 per minute and a 3.4% average share price drop after an incident.
EMA Research for BigPanda, 2024400+ IT professionals in North America, EMEA and APACUnplanned IT downtime averaged $14,056 per minute, $23,750 for large enterprises.
New Relic Observability Forecast, 20262,575 engineering and IT leaders and practitioners, 24 countries, surveyed April and May 2026Mean cost of a high-impact outage $1.85 million per hour. Mean time to detect 41 minutes, and 42% of organizations still learn of disruptions through manual checks and customer complaints.
Uptime Institute, 2026 report on its 2025 surveyRespondents to Uptime’s 2025 Annual Survey57% said their most recent major outage cost more than $100,000, and about 1 in 5 said more than $1 million.

Splunk, BigPanda and New Relic sell observability or incident software, and all four studies describe large organizations. For your own site, our downtime cost calculator estimates lost revenue and lost working time from your annual revenue, the headcount affected and the length of the outage.

What an independent check needs to do

  1. Run outside the provider you depend on. Google reports that some customers’ monitoring running on Google Cloud failed during the June 2025 incident, leaving them without a signal. The provider’s own tools can be impaired as well: during the October 2025 AWS event, creating support cases and signing in with IAM users failed for part of the outage, and during the October 2025 Azure Front Door incident customers could not open support cases in the portal or by phone. Do not make your alerts depend on logging in to the affected provider.
  2. Check the URL your users load. In the July 2026 CloudFront incident, AWS says only distributions using VPC Origins were affected and other origin types were not, so a check against a different endpoint would have passed. In May 2026 AWS described one data center in one availability zone, while Coinbase said failures reached several zones and that its design tolerated only a single-zone outage. What users experienced is the measurement that matters.
  3. Check from more than one place. Spotify’s Asia Pacific users were unaffected while most others were down, Azure’s January 2025 incident was confined to one region, and during the 1.1.1.1 outage the path users took to resolve DNS decided whether they saw it. A check from a single location can pass while users elsewhere fail, or fail while they are fine.
  4. Set the check interval to the delay you can tolerate. The gaps above are 23 to 72 minutes. A check that runs every 5 minutes, or every 30 seconds on a paid plan, can flag a failure users can see within about one interval (longer if you require several locations to fail first), without waiting for the status page.

Where Site Qwality fits

Site Qwality runs HTTP checks from the locations you choose on AWS, Google Cloud and Microsoft Azure, with Virginia (AWS) as the default; the locations page has the live list. The free plan checks each monitor from one location at 5-minute intervals. Paid uptime plans can select any number of locations and check as often as every 30 seconds, and each location counts as one check toward your monthly volume (see pricing). With several locations, you set how many must pass for the monitor to count as up. Alerts can go to email, SMS, Slack, Microsoft Teams, Discord, Telegram, PagerDuty and webhooks, among others (see the FAQ), and a status page (one is free, on a Site Qwality subdomain) lets you tell your own customers what is happening. More on the approach: multi-location monitoring.

Two limits apply here. Browser checks, TLS certificate checks, domain expiry checks and DNS record checks run from Virginia (AWS) only, so multiple locations apply to HTTP checks. And our platform itself, meaning storage, scheduling and alerting, runs in a single AWS region, us-east-1, the region in two of the incidents above. HTTP checks run from the locations you choose, but every result is recorded and every alert is sent from there, as our FAQ says. Start free, no credit card.

Sources

Provider reports and status histories, read October 17, 2026:

Studies and surveys:

News coverage used only for effects the providers did not describe: