What is the Circuit Breaker Pattern?
The Circuit Breaker Pattern prevents an application from repeatedly trying to execute an operation that is likely to fail. Like an electrical circuit breaker, it "trips" after a threshold of failures and stops all calls to the failing service for a period, allowing it time to recover. This prevents cascading failures across a distributed system.
Circuit Breaker States
- Closed: Normal operation — requests pass through. Failures are counted. If failures exceed the threshold, transition to Open.
- Open: All requests are immediately rejected without calling the service. After a timeout, transition to Half-Open.
- Half-Open: A limited number of test requests are allowed through. If they succeed, transition to Closed. If they fail, transition back to Open.
TypeScript Circuit Breaker Implementation
enum CircuitState {
CLOSED = "CLOSED",
OPEN = "OPEN",
HALF_OPEN = "HALF_OPEN",
}
interface CircuitBreakerOptions {
failureThreshold: number; // Number of failures before opening
successThreshold: number; // Successes in half-open before closing
timeout: number; // Time in ms before trying half-open
monitorInterval?: number; // Health check interval
}
class CircuitBreaker {
private state: CircuitState = CircuitState.CLOSED;
private failureCount = 0;
private successCount = 0;
private lastFailureTime: number = 0;
private readonly options: CircuitBreakerOptions;
constructor(
private readonly operation: () => Promise,
private readonly fallback?: () => Promise,
options?: Partial
) {
this.options = {
failureThreshold: options?.failureThreshold ?? 5,
successThreshold: options?.successThreshold ?? 3,
timeout: options?.timeout ?? 30000,
monitorInterval: options?.monitorInterval ?? 10000,
};
}
async execute(): Promise {
if (this.state === CircuitState.OPEN) {
if (Date.now() - this.lastFailureTime >= this.options.timeout) {
this.state = CircuitState.HALF_OPEN;
this.successCount = 0;
console.log("Circuit breaker: OPEN -> HALF_OPEN");
} else {
// Circuit is open — use fallback or throw
if (this.fallback) {
return this.fallback();
}
throw new CircuitBreakerOpenError("Circuit breaker is open");
}
}
try {
const result = await this.operation();
this.onSuccess();
return result;
} catch (error) {
this.onFailure();
if (this.fallback && this.state === CircuitState.OPEN) {
return this.fallback();
}
throw error;
}
}
private onSuccess(): void {
if (this.state === CircuitState.HALF_OPEN) {
this.successCount++;
if (this.successCount >= this.options.successThreshold) {
this.state = CircuitState.CLOSED;
this.failureCount = 0;
console.log("Circuit breaker: HALF_OPEN -> CLOSED");
}
} else {
this.failureCount = 0;
}
}
private onFailure(): void {
this.failureCount++;
this.lastFailureTime = Date.now();
if (this.state === CircuitState.HALF_OPEN) {
this.state = CircuitState.OPEN;
console.log("Circuit breaker: HALF_OPEN -> OPEN");
} else if (this.failureCount >= this.options.failureThreshold) {
this.state = CircuitState.OPEN;
console.log("Circuit breaker: CLOSED -> OPEN");
}
}
getState(): CircuitState {
return this.state;
}
getStats(): { state: string; failures: number; successes: number } {
return {
state: this.state,
failures: this.failureCount,
successes: this.successCount,
};
}
}
class CircuitBreakerOpenError extends Error {
constructor(message: string) {
super(message);
this.name = "CircuitBreakerOpenError";
}
} Using the Circuit Breaker
// Wrap external service calls with circuit breakers
class PaymentClient {
private circuitBreaker: CircuitBreaker;
constructor(private readonly baseUrl: string) {
this.circuitBreaker = new CircuitBreaker(
() => this.callPaymentService(),
() => this.fallbackResponse(), // Fallback when circuit is open
{
failureThreshold: 3,
successThreshold: 2,
timeout: 60000, // Try again after 1 minute
}
);
}
async processPayment(amount: number): Promise {
return this.circuitBreaker.execute();
}
private async callPaymentService(): Promise {
const response = await fetch(`${this.baseUrl}/api/payments`, {
method: "POST",
body: JSON.stringify({ amount: 100 }),
signal: AbortSignal.timeout(5000),
});
if (!response.ok) {
throw new Error(`Payment service returned ${response.status}`);
}
return response.json();
}
private async fallbackResponse(): Promise {
// Queue the payment for later processing
return {
status: "queued",
message: "Payment service is temporarily unavailable. Your payment has been queued.",
};
}
}
// Circuit breaker with retry
class ResilientHttpClient {
async fetchWithResilience(
url: string,
options: {
retries?: number;
retryDelay?: number;
circuitBreaker?: CircuitBreaker;
timeout?: number;
} = {}
): Promise {
const { retries = 3, retryDelay = 1000, timeout = 5000 } = options;
const operation = async (): Promise => {
let lastError: Error | null = null;
for (let attempt = 0; attempt <= retries; attempt++) {
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(timeout),
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
return response.json();
} catch (error) {
lastError = error as Error;
if (attempt < retries) {
// Exponential backoff with jitter
const delay = retryDelay * Math.pow(2, attempt) * (0.5 + Math.random() * 0.5);
await new Promise(resolve => setTimeout(resolve, delay));
}
}
}
throw lastError!;
};
if (options.circuitBreaker) {
return options.circuitBreaker.execute();
}
return operation();
}
} Bulkhead Pattern
The Bulkhead Pattern isolates failures by partitioning resources. Like bulkheads on a ship that prevent water from flooding the entire vessel, this pattern limits the impact of a failing component. Combine it with circuit breakers for comprehensive resilience.
// Bulkhead: Limit concurrent calls to each service
class Bulkhead {
private active = 0;
private queue: Array<{ resolve: () => void; reject: (err: Error) => void }> = [];
constructor(
private readonly maxConcurrent: number,
private readonly maxQueue: number = 100
) {}
async execute(fn: () => Promise): Promise {
if (this.active >= this.maxConcurrent) {
if (this.queue.length >= this.maxQueue) {
throw new Error("Bulkhead queue is full");
}
await new Promise((resolve, reject) => {
this.queue.push({ resolve, reject });
});
}
this.active++;
try {
return await fn();
} finally {
this.active--;
if (this.queue.length > 0) {
const next = this.queue.shift()!;
next.resolve();
}
}
}
}
// Usage: separate bulkheads for each downstream service
const paymentBulkhead = new Bulkhead(10); // Max 10 concurrent payment calls
const inventoryBulkhead = new Bulkhead(20); // Max 20 concurrent inventory calls
// A payment service failure cannot exhaust all connections
await paymentBulkhead.execute(() => paymentClient.charge(amount));
await inventoryBulkhead.execute(() => inventoryClient.check(items)); Resilience Strategy Checklist
- Timeouts: Always set timeouts on external calls — never wait indefinitely
- Retries: Use exponential backoff with jitter — do not retry immediately in a tight loop
- Circuit breakers: Wrap all external service calls with circuit breakers to prevent cascading failures
- Bulkheads: Isolate resource pools per dependency so one failure does not exhaust all resources
- Fallbacks: Provide degraded functionality when a dependency is unavailable (cached data, default values, queuing)