Latency vs throughput optimization
- Latency: optimize round trips, protocol overhead, queueing, and geography.
- Throughput: optimize bandwidth, window sizes, parallelism, compression, and congestion control.
- Many systems trade one for the other (e.g., batching increases throughput but adds latency).
CDN concept
- A CDN caches content at edge locations closer to users to reduce latency and origin load.
- Good for static assets and cacheable responses; needs cache keys/invalidation strategy.
Edge networking
- Push compute and caching closer to users (edge POPs).
- Improves latency and resiliency; adds complexity (deployment, observability, data locality).
Caching strategies
- Client/browser caching, CDN caching, reverse proxy caching, application caches.
- Control with HTTP cache headers, TTLs, and explicit invalidation where necessary.
Bottlenecks in distributed systems
- Hot partitions, limited DB connections, noisy neighbors, throttling, and cross-zone hops.
- Measure end-to-end, then optimize the dominant contributor (don’t guess).