Caching Strategies

Why Caching Matters

Caching is the practice of storing copies of frequently accessed data in a high-speed storage layer so that future requests for that data can be served faster. In system design, caching is one of the most effective tools for improving performance and reducing load on downstream systems.

Consider a typical web application: every page load might require 10-20 database queries. If your site has 10,000 concurrent users, that means 100,000-200,000 database queries per page load cycle. A well-designed caching layer can reduce database load by 80-95%, turning what would be a database bottleneck into a fast, responsive experience.

Caching Patterns

How your application interacts with the cache and the database is defined by the caching pattern you choose. Each pattern has different trade-offs in terms of consistency, complexity, and performance.

1. Cache-Aside (Lazy Loading)

In the cache-aside pattern, the application is responsible for reading from and writing to both the cache and the database. The cache does not interact with the database directly.

Read flow: Check the cache first. If the data is found (cache hit), return it. If not (cache miss), read from the database, store the result in the cache, then return it.

Write flow: Write to the database and then invalidate (delete) the cache entry so the next read fetches fresh data.

// Cache-Aside Pattern Implementation
class UserService {
  private cache: Redis;
  private db: Database;

  async getUser(userId: string): Promise<User> {
    // Step 1: Check the cache
    const cached = await this.cache.get(`user:${userId}`);
    if (cached) {
      return JSON.parse(cached); // Cache hit
    }

    // Step 2: Cache miss - read from database
    const user = await this.db.query('SELECT * FROM users WHERE id = $1', [userId]);

    // Step 3: Store in cache with TTL
    await this.cache.setex(`user:${userId}`, 3600, JSON.stringify(user));

    return user;
  }

  async updateUser(userId: string, data: Partial<User>): Promise<void> {
    // Step 1: Update the database
    await this.db.query('UPDATE users SET name = $1 WHERE id = $2', [data.name, userId]);

    // Step 2: Invalidate the cache
    await this.cache.del(`user:${userId}`);
    // Next read will fetch fresh data from DB and repopulate cache
  }
}

2. Read-Through

In the read-through pattern, the cache sits between the application and the database. The application only interacts with the cache. When there is a cache miss, the cache itself is responsible for loading the data from the database.

This is similar to cache-aside but the key difference is that the data loading logic lives in the cache layer rather than in the application code. This simplifies the application and centralizes the caching logic.

3. Write-Through

In the write-through pattern, every write goes to the cache first, and the cache immediately writes the data to the database before returning success to the client. This ensures the cache and database are always in sync.

• Pros: Strong consistency between cache and database, no stale data.
• Cons: Higher write latency (every write hits both cache and database), cache may contain data that is never read.

4. Write-Behind (Write-Back)

In the write-behind pattern, the application writes data to the cache, and the cache asynchronously writes to the database in the background. This dramatically reduces write latency because the application does not wait for the database write.

// Write-Behind Pattern (conceptual implementation)
class WriteBehindCache {
  private cache: Map<string, any> = new Map();
  private writeQueue: Array<{ key: string; value: any }> = [];
  private flushInterval: number = 1000; // Flush every second

  constructor(private db: Database) {
    // Background flush to database
    setInterval(() => this.flushToDatabase(), this.flushInterval);
  }

  async write(key: string, value: any): Promise<void> {
    // Write to cache immediately (fast)
    this.cache.set(key, value);

    // Queue for async database write
    this.writeQueue.push({ key, value });
    // Returns immediately - does NOT wait for DB write
  }

  async read(key: string): Promise<any> {
    return this.cache.get(key); // Always read from cache
  }

  private async flushToDatabase(): Promise<void> {
    const batch = [...this.writeQueue];
    this.writeQueue = [];

    if (batch.length === 0) return;

    // Batch write to database for efficiency
    await this.db.batchInsert(batch);
  }
}

Cache Eviction Policies

Caches have limited memory, so when the cache is full and a new item needs to be stored, the cache must decide which existing item to remove. This decision is governed by the eviction policy.

LRU is the default choice for most systems because it works well in practice: recently accessed items tend to be accessed again soon. Redis uses an approximated LRU algorithm by sampling a small number of keys and evicting the least recently used among them.

Redis vs Memcached

Redis and Memcached are the two most popular in-memory caching systems. While they serve similar purposes, they have significant differences that affect which one you should choose.

Choose Redis when you need rich data structures, persistence, pub/sub, or built-in replication. Choose Memcached when you need a dead-simple, multi-threaded cache for large-scale key-value storage and do not need advanced features.

CDN Caching

A Content Delivery Network (CDN) caches content at edge locations around the world, serving requests from the location closest to the user. CDNs are essential for reducing latency for global users and offloading traffic from origin servers.

• Static assets: Images, CSS, JavaScript files, fonts, and videos are perfect CDN candidates.
• Dynamic content: API responses can also be cached at the CDN with short TTLs (e.g., 5-30 seconds) for significant performance gains.
• Cache headers: Control CDN behavior using Cache-Control, ETag, and Vary headers.

// Setting cache headers for CDN
app.get('/api/products', async (req, res) => {
  const products = await getProducts();

  // Cache in CDN for 60 seconds, allow stale for 300 seconds while revalidating
  res.set('Cache-Control', 'public, max-age=60, stale-while-revalidate=300');
  res.set('CDN-Cache-Control', 'max-age=300'); // CDN-specific header (Cloudflare)

  res.json(products);
});

app.get('/api/user/profile', async (req, res) => {
  const profile = await getUserProfile(req.userId);

  // Private data - do NOT cache in CDN
  res.set('Cache-Control', 'private, no-store');

  res.json(profile);
});

Cache Invalidation Strategies

Phil Karlton famously said: "There are only two hard things in Computer Science: cache invalidation and naming things." Cache invalidation is the process of removing or updating stale data in the cache when the underlying data changes.

1. Time-Based Expiration (TTL)

Set a time-to-live on each cache entry. After the TTL expires, the entry is automatically removed. This is the simplest approach but can serve stale data until the TTL expires.

2. Event-Based Invalidation

When data changes in the database, publish an event that triggers cache invalidation. This provides near-real-time consistency but requires an event system.

3. Version-Based Invalidation

Include a version number in the cache key. When data changes, increment the version. Old cache entries naturally become orphaned and eventually expire.

// Event-based cache invalidation with Redis Pub/Sub
class CacheInvalidator {
  private redis: Redis;
  private subscriber: Redis;

  constructor() {
    this.redis = new Redis();
    this.subscriber = new Redis();

    // Listen for invalidation events
    this.subscriber.subscribe('cache-invalidation');
    this.subscriber.on('message', (channel, message) => {
      const { pattern } = JSON.parse(message);
      this.invalidatePattern(pattern);
    });
  }

  async invalidatePattern(pattern: string): Promise<void> {
    // Find all keys matching the pattern
    const keys = await this.redis.keys(pattern);
    if (keys.length > 0) {
      await this.redis.del(...keys);
    }
  }

  // Called when data changes
  async onDataChange(entityType: string, entityId: string): Promise<void> {
    // Publish invalidation event to all cache instances
    await this.redis.publish('cache-invalidation', JSON.stringify({
      pattern: `${entityType}:${entityId}:*`,
    }));
  }
}