Cascading Failure

A cascading failure happens when one small failure triggers a chain reaction across the system. A single component breaks, and the impact spreads quickly through dependent systems. What starts as a minor issue turns into a much larger outage.

For example, a slow database causes request timeouts. Applications retry aggressively, multiplying the load. The increased traffic exhausts connection pools and brings down related services, even though they were functioning normally moments before. The failure cascades beyond the original problem.

Cascading failures happen in tightly coupled systems with poor isolation, no circuit breakers, no rate limits, no bulkheads between components. They are dangerous because the original cause is often hidden by the larger breakdown. By the time teams respond, multiple services are down and root cause analysis becomes difficult.

Preventing cascading failures requires designing for isolation and graceful degradation. Circuit breakers stop unhealthy dependencies from being called. Rate limits prevent retry storms. Timeouts prevent slow operations from blocking resources. Systems should degrade gracefully rather than assume everything will always work.

A cascading failure happens when a fault in one component spreads to the systems that depend on it, turning a local problem into a wide outage. Each failure puts pressure on the next dependency. Cascades are most likely in tightly interconnected systems such as cloud platforms and microservice architectures, where many services share the same underlying infrastructure.

So what? Software now depends so heavily on other services that one provider going down can take many unrelated products with it. That raises questions about the risk of so many companies relying on so few providers.

Examples: In February 2017, a typing mistake made while fixing a billing problem took down part of AWS for several hours, along with the many services built on it. In December 2021, AWS had three major outages in one month, affecting devices and services for an estimated millions of users. The July 2024 CrowdStrike update spread from one security product to airlines, banks and hospitals worldwide.