DEV Community: Krishanu Das

Designing a Notification System for an E-commerce App

Krishanu Das — Fri, 30 Oct 2026 09:17:10 +0000

Every e-commerce app sends notifications: your order shipped, your payment failed, a flash sale started. This post designs one end to end, with a focus on the architecture.

The problem

An online shopping app has about 10 million customers. When something happens (an order is placed or shipped, a payment fails, a price drops), the customer gets a notification by push, email or SMS. Some are urgent (a login code), some are marketing (a flash sale), and users choose which kinds and channels they want.

Requirements

Question	Answer
Types	Critical (OTP, password reset, login alert, payment failed), order updates (confirmed, shipped, delivered, cancelled, refund), marketing (flash sale, price drop, back in stock, abandoned cart)
User choice	Users choose kinds and channels. Critical cannot be turned off. At least one channel is always on
"Delivered"	The provider confirms the notification reached the device or inbox
History	Users see their last 90 days
Speed	OTP within 5 s; payment failed and order updates within 30 s; marketing within minutes (a 5M-user campaign finishes within 30 min)
Failures	Critical messages must not be lost or sent twice; a stale marketing offer is worse than a missed one

Out of scope: the services that trigger notifications, and the providers (APNs/FCM, an email provider, an SMS gateway).

Capacity estimation

Inputs: 10M users; about 50% have everything on (about 4 notifications a day), 30% transactional only (1), 15% custom (2), 5% almost muted (0.2); a flash-sale campaign of 5M users within 30 minutes; peak is about 10x the average; channel opt-in is push 60%, email 90%, SMS 25%, so each notification becomes about 1.75 provider calls.

Step	Calculation	Result
Per day	5M x 4 + 3M x 1 + 1.5M x 2 + 0.5M x 0.2	26.1M notifications/day
Average	26.1M / 86,400	about 300 notifications/s
Normal peak	300 x 10	about 3,000 notifications/s
Campaign	5M / (30 x 60 s)	about 2,800 notifications/s
Combined peak	3,000 + 2,800	about 5,800 notifications/s
Provider calls	5,800 x 1.75	about 10,150 calls/s (push 3,480, email 5,220, SMS 1,450)
DB writes	5,800 x (1 insert + 1.75 status updates)	about 16,000 writes/s (average about 830)
Storage, 90 days	26.1M x 300 B x 90	about 0.7 TB

Notifications, provider calls and database writes are different numbers, and the campaign is added on top of the normal peak.

What this tells us: 0.7 TB fits on one machine, but 16,000 writes/s at peak does not suit a single relational database. The write path drives the architecture.

Architecture

Calling services (orders, payments, marketing)
        |  POST /notifications  (request_id, user_id, type, expires_at, ...)
        v
+-----------------------------+
| Notification API            |  1. duplicate check  --> Redis (request_id, NX EX)
|  validate, reply 202        |  2. settings lookup  --> settings cache / table
|                             |  3. save "accepted"  --> Notifications store
|                             |  4. enqueue          --> topic below
+-----------------------------+
        |
        v
+--------------------------------------------------------------+
| Queues: one topic per channel x priority (9 topics)          |
|            critical        orders         marketing          |
|   push     push-critical   push-orders    push-marketing     |
|   email    email-critical  email-orders   email-marketing    |
|   sms      sms-critical    sms-orders     sms-marketing      |
+--------------------------------------------------------------+
        |
        v
+--------------------------------------------------------------+
| Worker groups (their own workers per topic)                  |
|   1. drop if past expires_at                                 |
|   2. own rate limiter (token bucket) toward the provider     |
|   3. call the provider; retry with backoff                   |
+--------------------------------------------------------------+
        |                    |                      |
        v                    v                      v
   APNs / FCM          Email provider          SMS gateway
   (a backup provider is kept for critical messages)
        |
        +--> status: sent / delivered / failed --> Notifications store
        +--> retries exhausted ------------------> Dead-letter topic

Recovery job: re-queues records still "accepted" after about a minute

The front door

Validate the request.
Duplicate check: SET request:<id> NX EX 86400 in Redis. A repeat gets nil and is dropped.
Settings lookup: which channels the user has on, and whether marketing is allowed (cached).
Save the record with status accepted.
Enqueue on the right topic.
Reply 202 Accepted.

We reply "accepted", not "sent". Sending happens in the background, so the caller is never slowed by a slow or failing provider.

Why save before queueing?

Order	Crash in the gap	Result
Queue, then save	Message queued, no record	It is sent but untracked
Save, then queue	Record exists, nothing queued	Never sent, unless something notices

So we save first, and a recovery job re-queues any record still accepted after about a minute. The worst case is a duplicate, never a silent loss.

Idempotency

The caller creates a request_id once per event ("order 9001 shipped") and reuses it on every retry. Redis is the fast check, and the same request_id is stored on the record as the durable safety net, because Redis can lose keys on a restart.

Data model

Store	Holds	Why
Notifications (Cassandra or DynamoDB)	id, `user_id`, type, template, status, timestamps, `expires_at`, `request_id`	High write rate. Partition by `user_id`, sorted by `created_at`, so "this user's latest 50" is one partition read. 90-day TTL
Request IDs (Redis)	`request:<id>` with an expiry	Fast duplicate check
User settings (small table, cached)	channels on, marketing on	About 10M rows, read often, changed rarely

Why not just Postgres? The average (about 830 writes/s) is easy, but the peak and campaign burst (about 16,000/s) would need sharding and careful tuning. Cassandra and DynamoDB are built for heavy writes, and our reads are a simple key lookup. We give up joins, flexible queries and multi-row transactions. TTL deletion isn't instant (DynamoDB can take days), so the history query also filters by date.

API

POST /notifications
{
  "request_id": "...",     // idempotency key, same on every retry of an event
  "user_id": "...",
  "type": "otp | order_update | marketing",
  "order_id": "...",
  "template_id": "...",
  "expires_at": "..."      // never send late
}

202 Accepted
{ "notification_id": "...", "status": "accepted" }

Plus GET and PUT /users/{id}/settings, and GET /users/{id}/notifications?limit=50&cursor=....

Queues and workers

One topic per channel and priority. Kafka has no priority queue, and neither does standard SQS, so priority means separate topics with separate workers.
Partitions give parallelism, topics give priority. Separate partitions inside one topic do not protect an OTP from a campaign, because the same worker group serves them. Critical topics need their own workers.
Partition sizing: partitions >= calls/s / calls per worker, rounded up (you can add partitions later but never remove them). With an assumed 100 calls/s per worker, email at about 5,220 calls/s needs about 53 workers, so use 64 partitions on the marketing topic.
Partition key: user_id, where one user's messages must stay in order.
Provider limits: if the email provider allows 4,000 calls/s and we need 5,220, workers run their own token bucket and send at most 4,000/s. The extras wait in the queue; they are not dropped. Part of the allowed rate is reserved for critical messages.
Expiry: a worker checks expires_at before sending. A stale OTP or an old offer is dropped and marked expired.
Delivery guarantee: commit the offset after the provider call, which gives at-least-once. Workers must tolerate a rare duplicate.

Failure handling

	One call times out	Whole provider is down	Expired	Retries exhausted
OTP	Retry quickly, until `expires_at`	Backup provider, else another enabled channel	Drop	Mark failed, alert
Payment failed / order updates	Retry with backoff	Backup provider or another channel	Rarely applies	Dead-letter topic
Marketing	Give up (at most one retry)	Wait in the queue until recovery	Drop	Drop

For critical messages a loss hurts and a duplicate is harmless; for marketing a miss is cheap and a duplicate annoys at scale.

Backoff and dead letters: wait longer between retries (1 s, 2 s, 4 s), cap the attempts, then park the message in a dead-letter topic.
Circuit breaker: after many failures in a row, stop calling a provider for a short while and test it occasionally.
A timeout is ambiguous: we don't know if the provider got the call. Push services have collapse ids and keys that replace an earlier message, but support varies, so we don't rely on them.

Trade-offs

Decision	Chose	Gave up
Database	Cassandra / DynamoDB	Joins, flexible queries, multi-row transactions
Duplicates vs loss	OTP and orders may duplicate; marketing may be missed	A perfect "exactly once"
Queues	Topics per channel x priority, reserved capacity for critical	More queues and workers; idle reserved capacity
Save before queue	A possible duplicate after a crash	Simplicity (needs a recovery job)
Provider pacing	Our own limiter, extras wait	Marketing can be delayed

Designing a Rate Limiter for an SMS API: The Mistakes I Made Along the Way

Krishanu Das — Sun, 11 Oct 2026 16:11:07 +0000

I'm working through system design as an AI engineer, and I'm writing up each problem as I go. This one is a rate limiter. It looked like the easy one. It caught me out in three places.

The problem

Picture a public API that sends SMS messages, something like Twilio. Thousands of companies call it from their servers, on free, paid and enterprise plans.

Two things keep going wrong. A buggy script on a free account loops forever and slows the API down for everyone else. And some customers send bursts to squeeze out more than they pay for.

So we need a rate limiter: a cap on how many requests a client can make in a time window. Go over the cap and the API answers with HTTP 429 Too Many Requests.

Step 1: Ask before designing

I wrote my questions down first. These are the answers I settled on:

Everything is limited together (sending and status checks), with one limit per client.
The client is the API key, not the IP address. Lots of customers can share one IP, and one customer can use many.
Over the limit gets a 429 with a Retry-After header.
Free gets 60 requests per minute, paid gets 600, enterprise gets 6,000. About 90% of active accounts are free, 9% are paid and 1% are enterprise.
100,000 accounts in total, and 20,000 of them active in the peak minute.
Many API servers behind a load balancer, and the check can only add a few milliseconds.

The question I almost skipped was "who is the client?" Nearly everything else in the design hangs off the answer.

Step 2: Estimate the worst case

I assumed every active account sits exactly at its limit.

Tier	Active accounts	Requests/min	Requests/sec
Free	18,000	1,080,000	18,000
Paid	1,800	1,080,000	18,000
Enterprise	200	1,200,000	20,000
Total	20,000	3,360,000	56,000

That's 56,000 requests per second, and the limiter checks every one of them, including the ones it ends up rejecting. A disk-based database can't take that on the hot path, so the counters have to live in memory.

One more number. Paid and enterprise accounts are only 10% of accounts, but they're 68% of the traffic. I didn't think that mattered until the very end.

Step 3: Where does the count live?

My first answer was that the limiter should "make sure the requests across servers add up to 60". That's just the requirement said again. It doesn't tell you how.

Here's what actually goes wrong. If every server keeps its own count, the load balancer spreads one customer's requests across, say, 3 servers, and each server only sees a third of them. With a local limit of 60 on each, that customer can send 180 requests a minute and nobody complains.

So the count has to live in one place that all the servers share. I'd use Redis, because it's in memory and handles this kind of load.

There are two kinds of data here, and I'd keep them apart:

Config: key X is on the free tier, so the limit is 60 a minute. This almost never changes, so each gateway can cache it in memory and refresh it every minute or so.
State: key X has used 43 so far. This changes on every request, so it lives in Redis.

I'll come back to why that split matters.

The race condition I missed

Say a key has used 59 of its 60 and two requests arrive at the same moment on two different servers. Each server reads the count (59), sees it's under 60, lets the request through and writes 60. Both got through, when only one should have.

My fix was to keep the servers in sync with each other. That doesn't work. They'd have to message each other on every request, which is slow and messy, and there's still a gap between "check" and "update" where both can decide at the same instant.

The real fix is to stop doing three steps (read, compare, write) and make it one step inside the store. Redis can add 1 to a counter and hand back the new value as a single operation. Server 1 gets 60 and goes ahead. Server 2 gets 61 and gets a 429. It's the same idea as the atomic claim that makes sure only one worker picks up a job from a message queue.

Step 4: Choosing the algorithm

Fixed window. Put the current minute in the key (X:12:00), increment it on every request, and let it expire. It's simple and cheap, one number per key.

But look at the boundary. A client sends 60 requests at 12:00:59 and another 60 at 12:01:00. That's 120 requests in about 2 seconds, and nothing gets blocked.

My first reaction was "that's fine, the window is whatever the platform defines". That's wrong. The limit exists to protect the backend, and the backend was sized for 60 a minute. If all 18,000 free accounts did this at once, it would see about 112,000 requests a second, double what it was built for.

Token bucket. Each client has a bucket that holds up to a set number of tokens. Every request takes one, and an empty bucket means a 429. Tokens drip back in at a steady rate.

The nice part is that you don't need a timer. You store two numbers per key, the tokens left and the time you last saw it, and you work out the refill only when a request shows up:

tokens_now = min(capacity, tokens_saved + (now - last_seen) x refill_rate)

Here's a small example I worked by hand (capacity 3, refill 1 token a second, bucket starts full):

Time	Tokens available	Result	Tokens left
t=0	3	Allowed	2
t=0	2	Allowed	1
t=0	1	Allowed	0
t=0	0	429	0
t=1	1	Allowed	0
t=5	3 (4 would have built up, capped at 3)	Allowed	2
t=6	3 (2 saved plus 1 refilled)	Allowed	2

I went with the token bucket because there's no boundary to exploit. What I give up is simplicity: two numbers per key instead of one, and the refill maths has to run as one atomic step, since it reads and writes two values.

Tier	Bucket size	Refill
Free	60	1 token/sec
Paid	600	10 tokens/sec
Enterprise	6,000	100 tokens/sec

One thing to know: a bucket the size of a full minute lets a client burst a whole minute's worth at once. If that's too generous, make the bucket smaller. That's a product decision more than a technical one.

The check itself

Redis runs a Lua script as one uninterrupted step, so the whole check is atomic:

local capacity = tonumber(ARGV[1])
local rate     = tonumber(ARGV[2])          -- tokens per second

local t   = redis.call('TIME')              -- Redis's own clock
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)

local data   = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(data[1])
local ts     = tonumber(data[2])
if tokens == nil then tokens = capacity; ts = now end

tokens = math.min(capacity, tokens + math.max(0, now - ts) / 1000 * rate)

local allowed, retry_after = 0, 0
if tokens >= 1 then
  tokens = tokens - 1
  allowed = 1
else
  retry_after = math.ceil((1 - tokens) / rate)
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', KEYS[1], math.ceil(capacity / rate) * 2)
return { allowed, math.floor(tokens), retry_after }

Two details. It uses Redis's clock, not each gateway's, because different machines have slightly different clocks and I don't want them disagreeing about "now". And idle keys expire on their own, so inactive accounts don't eat memory. A key that expired just starts with a full bucket, which is the right behaviour anyway.

The full picture

What the 429 should say

My first answer was "tell them the calls for the minute are over". That's wrong for a token bucket, because there is no minute. Tokens come back continuously, so the useful thing to tell a client is how long until one token exists:

HTTP/1.1 429 Too Many Requests
Retry-After: 1
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0

For a free key that refills 1 token a second, an empty bucket means Retry-After: 1. If it refilled 1 token every 2 seconds, it would be 2. Successful responses carry the X-RateLimit headers too, so a well-behaved client can slow down before it hits the wall.

Scaling it

56,000 checks a second is within what a single Redis node can do for simple commands, but Lua scripts cost more, so I'd want headroom. I'd spread keys across several Redis nodes with consistent hashing on the API key, so adding a node only moves about 1/N of the keys.

And a key that moves, or a counter that gets lost, isn't a disaster. The client just starts with a full bucket. Worst case, one extra burst.

Replication is asynchronous, so after a failover the last few updates can be lost and a client might briefly get a bit more than its limit. For a rate limiter I'm happy with that. It picks availability over perfect accuracy.

When the limiter itself breaks

If Redis is unreachable, the limiter can't check counts, and it has to pick a default.

Fail open: let requests through. Customers keep working, but nobody is limited.
Fail closed: reject requests. The backend is safe, but every customer gets errors.

My first answer was to split by tier: fail open for paying customers, fail closed for free ones. Then I gave a reason that pointed the opposite way ("I'd rather give paying customers an error"), which is fail closed. I'd mixed the two up.

Even after fixing that, the split doesn't hold up against the numbers. Paid and enterprise are 68% of the traffic, so failing open for them leaves two-thirds of everything unchecked. Failing closed for them means errors for the customers who pay. Failing closed for everyone means a protective component takes down the whole product.

What I'd do instead is fail open for everyone, with a safety net. If a gateway can't reach Redis, it switches to a local in-memory bucket using the tier limit divided by the number of gateways. It's rough, but it still stops a runaway script, and it raises an alert straight away.

This only works because each gateway already has the tier config cached. If tier info lived in Redis too, it would disappear in exactly the outage where I need it. That's the reason config and state live in different places.

What I took away

Ask who the client is before anything else. The rest of the design hangs off it.
A counter that each server keeps by itself isn't a limit. The count has to be shared.
"Read, check, write" is a race. Make it a single atomic step inside the store.
A fixed window leaks at the boundary. Check an algorithm against what the limit is protecting, not just against its own definition.
Your 429 response has to match your algorithm. With a token bucket, tell the client how many seconds to wait, not that the minute is over.
Decide what happens when the limiter fails before it happens, and check the decision against your own traffic numbers.

Next I'm going to try the order notifications for something like Flipkart, and this time I'll attempt the whole design on my own before I look anything up.

Designing a URL Shortener From First Principles (and the Hashing Trap I Fell Into)

Krishanu Das — Tue, 22 Sep 2026 07:14:46 +0000

I'm working through system design as an AI engineer, and I'm writing up each problem as I go. This one is the classic: design a URL shortener like bit.ly. It looks simple, but it has one design decision that catches people out, and I walked right into it.

The two operations

Shorten: submit a long URL, get back a short one
Redirect: visit the short URL, get sent to the original

POST /urls { "longUrl": "...", "expiresAt": "..." } → 201 { "shortUrl": "sho.rt/21" }
GET /{shortCode} → 302 redirect to longUrl

Step 1: Estimate before designing

I anchored on a user base and derived everything else from it, rather than guessing RPS directly.

Reads (clicks):

100M total users, 50M active per day
~5 clicks per active user per day → 250M clicks/day
250M ÷ 86,400 seconds ≈ 2,900 RPS

Writes (creating links):
Creating a link is much rarer than clicking one. My first estimate had more creates than clicks, which contradicted the read-heavy intuition, so I redid it:

~5M users create links on a given day, ~5 links each → 25M creates/day
25M ÷ 86,400 ≈ 290 RPS

That's roughly a 10:1 read/write ratio, and it shapes everything below: the design effort belongs on the read path.

Storage:
Each record holds the long URL (~100 bytes), short code (~10), creator ID (~8–16), created timestamp (~8), expiry (~8) and a click counter (~8). That rounds to ~150 bytes.

25M records/day × 365 days ≈ 9.1B records/year
9.1B × 150 bytes ≈ 1.37 TB/year

1.37 TB a year fits on a single well-provisioned database. Storage isn't the bottleneck here, so there's no reason to shard just because the problem "sounds like scale." The numbers should drive that call, not the vibe.

Latency target: 100–200 ms. A redirect is a lookup with no heavy computation in the path, so it should feel instant.

Step 2: The architecture

flowchart LR
    U[User / Browser] --> LB[Load Balancer]
    LB --> A1[App Server 1]
    LB --> A2[App Server 2]
    LB --> A3[App Server N]
    A1 & A2 & A3 --> C[("Redis cache: shortCode to longUrl")]
    A1 & A2 & A3 --> DB[("Database: urls table + sequence")]

Stateless app servers sit behind a load balancer. A cache holds hot redirects, and a single relational database stores the records and generates IDs.

Step 3: The hard part is generating the short code

This is the actual design problem. There were three options:

flowchart TD
    Q{"How do we generate the short code?"}
    Q --> H[Hash the long URL]
    Q --> R[Random string]
    Q --> K[Counter + Base62]
    H --> H1["Same URL always gives the same code: breaks per-user expiry and analytics"]
    R --> R1["Can collide: needs retry when the DB rejects a duplicate"]
    K --> K1["Unique by construction: needs one shared counter"]

The hashing trap

My first instinct was to hash the long URL. It feels natural, since hashing turns a long input into a short output.

Then consider this case. User A shortens example.com/page in January for a 30-day campaign and wants their own click analytics. User B shortens the same URL in July with a different expiry.

Hashing is deterministic: the same input always gives the same output. Both users would get the same short code, and therefore one shared record. Whose expiry wins? Whose clicks are whose?

The real requirement is that each shorten request is its own record, because ownership, expiry and analytics belong to the request, not to the URL. So the short code has to be generated independently of the URL's content.

Random vs counter

Random strings mostly work, but they can collide. The database's unique constraint catches a collision, and you retry. As the table grows into billions of rows, collisions become less rare (the birthday problem).

A counter encoded in Base62 can't collide at all. Take an ever-increasing number and encode it with [0-9a-zA-Z]:

125 = 2 × 62 + 1  →  "21"

No two counter values are the same, so no two codes are the same. There's no retry logic needed.

How long does it last? With 7 characters, 62⁷ ≈ 3.5 trillion codes. At 9.1B links a year, that's over 380 years of headroom.

The distributed counter problem

There are many app servers. If each one keeps its own counter in memory, two servers can hand out the same number at the same moment.

There are two standard fixes:

Let the database own the counter (an auto-increment column or a Postgres SEQUENCE). The database guarantees uniqueness even under concurrent load.
Hand out ranges. Each server grabs a block of, say, 1,000 IDs at a time and assigns them locally. This reduces contention on the shared counter, at the cost of some wasted IDs if a server crashes mid-block.

I went with option 1. At ~290 creates/sec, a single database sequence handles the load comfortably, so ranges would solve a problem this system doesn't have. Match the complexity to the numbers you actually calculated.

sequenceDiagram
    participant C as Client
    participant S as App Server
    participant DB as Database
    C->>S: POST /urls (longUrl, expiresAt)
    S->>DB: nextval(url_seq)
    DB-->>S: 125
    S->>S: base62(125) = "21"
    S->>DB: INSERT shortCode, longUrl, createdBy, createdAt, expiresAt
    DB-->>S: OK
    S-->>C: 201 shortUrl = sho.rt/21

Step 4: Make the read path fast

With a 10:1 read/write ratio, redirects are where performance matters.

Index: the short code is the primary key, so lookups are indexed automatically.
Cache: hot shortCode → longUrl mappings live in Redis. The cache key is just the short code, because a redirect is public and identical for everyone.

sequenceDiagram
    participant C as Client
    participant S as App Server
    participant R as Redis
    participant DB as Database
    C->>S: GET /21
    S->>R: GET 21
    alt cache hit
        R-->>S: longUrl
    else cache miss
        R-->>S: null
        S->>DB: SELECT longUrl WHERE shortCode = 21
        DB-->>S: longUrl
        S->>R: SET 21 longUrl with TTL
    end
    S-->>C: 302 redirect to longUrl

Why 302 and not 301? Browsers cache a 301 (permanent) redirect, so repeat clicks never reach the server and the click counts would be wrong. A 302 keeps every click visible.

What I'd revisit next

Guessable codes: Sequential codes mean anyone can walk through 21, 22, 23... and find other people's links. If that matters, shuffle or obfuscate the ID before encoding it.
Expiry cleanup: Expired links need a background job to purge them, and the cache TTL shouldn't outlive the link's own expiry.
Cache eviction: Decide on a policy (LRU is the usual choice) and how big the cache needs to be.

Takeaways

Derive the numbers, don't guess them. Total users, then active users, then per-user rate, then RPS. The same method works for storage.
Sanity-check your estimates against each other. My first write estimate exceeded my read estimate in a read-heavy system, and that contradiction was the signal to redo it.
Not every system needs to shard. 1.37 TB a year fits on one box.
Deterministic isn't always what you want. Hashing is ideal when identical inputs should map together. Here they shouldn't.
Match solution complexity to real scale. Use a database sequence at 290 writes/sec, and switch to ID ranges when the numbers say so.

This is part of a series where I learn system design in public. Next up: designing a rate limiter.