Why your app is slow in Sydney and fast in Mumbai
Your servers are in Mumbai. Your app loads in 200 ms for you and 1.8 seconds for a user in Sydney. Nothing is broken. No query is slow. You cannot fix this with a bigger server, more replicas, or better code.
You are fighting the speed of light, and you are going to lose. Mumbai to Sydney is about 10,000 km. Light in fibre covers that in roughly 50 ms one way — and a real request needs several round trips for DNS, the TLS handshake, and finally the data. That is your floor before your application does a single useful thing.
A CDN — Content Delivery Network — is how you stop fighting the distance and start deleting it.
The analogy: warehouses, not one big shop
Imagine you sell one product from a single warehouse in Mumbai. Every order ships from there. A customer in Sydney waits two weeks, no matter how fast your packing team is. Hiring more packers does not help — the delay is the distance.
So you rent small warehouses in Sydney, London, and São Paulo, and you keep copies of your bestsellers in each one. Now the Sydney customer gets their order in a day, from a shelf 20 km away. The Mumbai warehouse still exists, but it only handles the rare item nobody stocked locally.
That is a CDN exactly. The small warehouses are edge locations (also called points of presence, or PoPs). The bestsellers are your static assets. The Mumbai warehouse is your origin server.
How a CDN request actually works
Here is the path, step by step:
- A user requests
image.jpg. DNS resolves your CDN hostname to the nearest edge location, not to your origin. - The edge checks its local cache. If the file is there — a cache hit — it returns it immediately. Typically 10–30 ms.
- If it is not there — a cache miss — the edge fetches it from your origin once, stores a copy, and returns it. That first user pays the full distance penalty.
- Every subsequent user in that region gets the cached copy. One slow request, thousands of fast ones.
The important consequence: your origin's traffic drops off a cliff. A site serving 50,000 requests per second might send 500 of them to the origin. Everything else is answered by machines you do not run, in cities you have never visited.
What belongs on a CDN — and what does not
The classic answer is "static assets": images, CSS, JavaScript bundles, fonts, video segments, PDFs. Anything identical for every user.
The better answer, and the one that gets you credit in an interview, is broader. Modern CDNs also cache:
- API responses that are the same for everyone — a product catalogue, a public leaderboard, exchange rates. Cache for 30 seconds and you have removed 99% of that endpoint's load.
- Personalised pages at the edge, using edge compute to assemble a shell from cache and fill in the user-specific part.
What must never be cached: anything authenticated or user-specific without a per-user key. Cache one user's account page at a shared edge and you will serve it to the next person who asks. This is a real incident category, not a hypothetical — it is why Cache-Control: private exists.
The two headers that control everything
Cache-Control is how the origin tells the edge what it may do:
max-age=31536000— cache for a year. Correct for a versioned asset likeapp.a3f9c1.js.no-store— never cache. Correct for a bank balance.private— the browser may cache it, the shared CDN may not.stale-while-revalidate=60— serve the stale copy instantly, refresh in the background. Users never wait for a refresh.
ETag handles the follow-up question: "has this changed?" The edge sends the ETag it holds, and the origin answers 304 Not Modified with no body if it still matches. You pay for a round trip but not for the bytes.
Want to master this with video lessons and real case studies? This topic is covered in depth in my Udemy course System Design Fundamentals for Interviews — 5.5 hours, rated 4.8★, built from real interview questions.
The mess: the invalidation trap
Here is the failure every team hits once. You deploy a fix to styles.css with max-age=86400. The fix is live on your origin. Your users see the broken version for the next 24 hours, because every edge in the world is confidently serving yesterday's copy.
You can call the CDN's purge API, but purging propagates across hundreds of PoPs and is not instant. And under pressure, someone will purge everything, which sends every edge to your origin at once — a self-inflicted stampede on the server you were protecting.
The fix: never invalidate, rename
Do not fight the cache. Change the URL instead.
Build your assets with a content hash in the filename: styles.a3f9c1.css. When the content changes, the hash changes, so the filename changes, so it is a completely new URL that no edge has ever seen. The old file expires quietly on its own.
This lets you set max-age to a year on every asset with no risk at all. The only file that needs a short TTL is the HTML that references them — and HTML is small.
The general principle is worth remembering beyond CDNs: immutable content with changing names is easier to cache than mutable content with fixed names. The same idea shows up again in caching strategies at the application layer.
Quick check
Your CDN hit rate is 40%. Traffic is mostly product images that never change. What is the most likely cause?
(Think about it before reading on.)
Your cache key is too specific. Almost always this is query strings: image.jpg?utm_source=twitter and image.jpg?utm_source=email are cached as two separate objects even though the bytes are identical. Strip marketing parameters from the cache key and the hit rate jumps. The second suspect is a short or missing max-age, which evicts objects before they can be reused.
How to talk about CDNs in an interview
Do not say "we'll add a CDN" and move on — that is a checkbox, not an argument. Say this instead:
"Users are global and our origin is in one region, so latency is bound by distance, not by compute. I'd put a CDN in front of static assets and public API responses. Assets get content-hashed filenames and a one-year max-age, so I never need to invalidate — I just deploy a new URL. HTML gets a short TTL with stale-while-revalidate. Anything authenticated is marked private so it never lands in a shared cache. That drops origin traffic by roughly two orders of magnitude and cuts p95 latency for distant users from seconds to tens of milliseconds."
That answer shows you understand the constraint (distance), the mechanism (edge caching), the operational trap (invalidation), and the security trap (caching private data). It takes 30 seconds.
One more number worth having ready: from the estimation walkthrough, a photo app serving ~7 GB/s at peak is not a "maybe CDN" system. At that egress, the CDN is the only affordable option — origin bandwidth alone would dominate your bill.
Key Takeaways
- A CDN removes distance, which is the one latency cost you cannot optimise away in code.
- Cache hits are served from an edge near the user; misses fetch from origin once and populate the edge.
Cache-ControlandETagcontrol caching;privatekeeps user data out of shared caches.- Never rely on invalidation — use content-hashed filenames so new content means a new URL.
- Low hit rates are usually a cache-key problem, most often query strings.
Next Steps
A CDN protects your read path. But what protects your write path when 50,000 users hit submit in the same second? That is what message queues are for — the component that lets a system absorb a spike instead of collapsing under it.