DEV Community

The Byte Mechanics
The Byte Mechanics

Posted on Edited on Originally published at Medium

Stop Starting With Microservices: The Path Real Systems Actually Follow

#architecture #backend #microservices #systemdesign

Originally published on Medium.

A new project kicks off. Someone opens a blank page and says, "Let's build it with microservices." Everyone nods. It sounds modern. It sounds scalable.

But here's the surprise: most high-scale systems don't start with microservices. They grow into them, and only when the system demands it.

This article walks through that growth one stage at a time. At each stage you'll see what breaks, why it breaks, and what teams do about it. If you'd rather watch, there's a 10-minute animated version of this article:

1. Start with a monolith

Every system starts somewhere, and the first goal isn't scale. It's correctness, speed of development and clarity. That's why most successful systems begin as a monolith.

A monolith is one application that holds every feature:

  • One backend, the code that runs on the server.
  • One deployment, so you ship it as a single unit.
  • Usually one database, where all the data lives.

Inside, each feature is a module, a separate area of the code. Modules call each other directly, as ordinary function calls with no network in between. That keeps two things easy:

  • Transactions are simple. A transaction is a group of changes that succeed or fail together, and a single database handles it for you.
  • Debugging is easy. One process, one log and one stack trace.

This leads to the first rule of architecture:

Don't add distributed complexity, meaning spreading work across many machines, until the system forces you to.

2. When the monolith feels heavy, fix coupling first

As features and engineers pile up, the monolith starts to feel heavy:

  • Builds slow down.
  • Deployments get risky.
  • A small change breaks something unrelated.
  • Teams collide in the same repository.

Many teams jump straight to microservices here. That's a mistake, because the real problem isn't the monolith. It's coupling: parts that depend too tightly on each other's insides.

The fix is a modular monolith. It still ships as one unit, but inside it's split into clean modules:

  • Each module has a defined interface, the official doorway other code must use.
  • Each module owns its data.
  • No module reaches into another's internals.
  • Each module follows a domain boundary, a business area like orders or payments.

This step matters more than it looks. If you split tightly coupled code into services, the coupling doesn't disappear. It just moves onto the network, where it's slower, harder to debug and can fail in new ways. Clean module boundaries now become clean service boundaries later.

3. When traffic grows, split into microservices

In a monolith, scaling means scaling everything. You add more running copies of the whole app, called instances. If payments get heavy traffic, the entire system gets more instances, including all the parts that were fine.

At small scale that's fine. At large scale it's inefficient and expensive, because different parts behave differently:

  • Read-heavy parts mostly fetch data.
  • Write-heavy parts mostly save data.
  • CPU-heavy parts need lots of processing power.
  • I/O-heavy parts spend their time waiting on disks or networks.

This is usually the moment to move to microservices: small, separate applications, each owning one domain. Each one can scale, deploy and fail on its own.

That solves scaling. But there's a catch. Before, everything lived in one process. Now everything lives on the network. Every section below is a consequence of that one sentence.

4. Services need to find each other: service discovery and the API gateway

Services now run on different machines, containers or pods. (A container is a packaged, isolated box for running an app, and a pod is a common way of grouping containers, for example in Kubernetes.) These instances restart, autoscale and move around, so hardcoding addresses is impossible.

The fix is service discovery. A registry tracks every available instance, and services ask it where to send each request.

The next problem is which service the app on your phone should call. With one backend there was one API. Now there are many. You don't want login checks or traffic limits copied into every service, and you don't want the frontend knowing your internal layout.

So you add an API gateway, a single entry point that handles:

  • Routing each request to the right service
  • Authentication, checking who you are
  • Logging
  • Throttling, limiting how fast a client can call
  • Sometimes combining responses from several services

5. The database becomes the bottleneck

As traffic keeps climbing, the first thing to struggle is usually the database, not the services. Services scale horizontally: you just add more machines. Databases are much harder.

The symptoms are familiar:

  • Queries slow down.
  • CPU spikes.
  • Locks appear. A lock is a hold on data while it's being changed, which makes everyone else wait.
  • Write delays grow.

Fix 1: read replicas. These are read-only copies of the database. Heavy queries move there, so reads are separated from writes. That works for a while, until the primary, the main copy that accepts writes, becomes the bottleneck itself.

Fix 2: sharding. The data is split across multiple machines, usually:

  • By user
  • By tenant, meaning a customer organization
  • By hash, a formula that assigns each record to a bucket

6. Transactions across services: the Saga pattern

In a monolith, one transaction can safely update many tables. That's an ACID transaction: a guarantee that a group of changes happens completely or not at all, and stays correct even when others are working at the same time.

Now one request, like create an order, may touch the order, payment, inventory and notification services, each with its own database. There's no easy way to wrap all of them in one ACID transaction. Forcing strong consistency, where everyone sees the same correct data at the same moment, usually hurts performance and reliability.

So high-scale systems accept eventual consistency: data may be briefly out of sync, but it catches up. They get there with three tools:

  • Events, to tell other services what happened
  • Retries, to recover from temporary failures
  • Compensation, which means undoing an earlier step

The common pattern that combines them is the Saga. Each step succeeds independently. If a later step fails, earlier steps are retried or undone. For example, if stock can't be reserved, the payment is refunded and the order is cancelled.

It's more scalable, but it's also more complex. That trade-off comes up at every stage.

7. Stop waiting: event-driven design

With a few services, synchronous calls work fine. Synchronous means the caller waits for an answer, usually over HTTP. As services multiply, these call chains become slow and fragile:

  • One slow service delays everything.
  • One failure breaks the whole flow.
  • Traffic spikes overload the services further down the line.

So high-scale systems switch to messaging. An event is a small message saying something happened. Instead of calling a service directly, you publish the event to a queue or stream, an ordered line of messages waiting to be processed. Consumers, the services that read events, handle them independently.

What you gain:

  • Services become loosely coupled.
  • Spikes are absorbed by the queue.
  • Failures can be retried.
  • Work happens asynchronously, so no one waits.

That's why large systems lean on Kafka, a popular event-streaming platform, as well as queues and pub-sub, where one sender reaches many listeners.

8. Retries are dangerous without idempotency

In distributed systems, failures are normal. Messages get retried. Requests time out, giving up after waiting too long. A consumer can crash after doing the work but before acknowledging it, that is, before telling the queue "I'm done."

So the same request may run more than once. For payments, orders, wallet updates or inventory changes, that's dangerous. Picture a customer who pays \$20 once and gets charged \$40.

The answer is idempotency: the same request produces the same result, even if it runs multiple times. Systems achieve it with two tools:

  • Idempotency keys (or unique request IDs), labels attached to each request
  • Deduplication tables, a record of which requests have already been handled

When a request arrives, the consumer checks its key. If the key isn't in the table, it processes the request and records the key. If the key is already there, it skips the work and returns the earlier result.

At scale, idempotency isn't an optimization. It's a requirement.

9. Designing to fail safely: rate limits and backpressure

At high scale, traffic isn't steady. Requests can suddenly spike, and a system that tries to handle everything at once can crash. So it has to control the flow:

  • Rate limiting restricts how much traffic is allowed in.
  • Backpressure slows down producers, the senders, when consumers can't keep up.
  • Queues buffer short spikes.
  • Dead-letter queues capture messages that keep failing, so they can be inspected instead of blocking everything else.

Sometimes the right move is to reject a request with a polite "try again later" (HTTP 429) instead of crashing the whole service.

High-scale systems aren't designed to never fail. They're designed to fail safely.

10. Observability and the CAP trade-off

In a monolith, logs are usually enough. In a distributed system, one request can pass through many services, and you need three more tools:

  • Tracing follows one request's path, so you can see which service is slow.
  • Metrics are numbers tracked over time, so you can see where load is growing.
  • Alerts tell you about problems before users complain.

Together these make up observability, the ability to see inside a running system. At this scale it has to be part of the design, not an afterthought.

Then comes the biggest trade-off, known as CAP:

  • Consistency: everyone sees the same latest data.
  • Availability: the system always answers.
  • Partition tolerance: the system keeps working when the network splits parts apart.

Networks do split in distributed systems, so partition tolerance isn't optional. When a split happens, you have to choose: keep answering, even with slightly stale data (availability), or refuse until the data is correct (consistency). You can't have all three, and different parts of the same product choose differently:

  • Payments and wallets favor consistency. A wrong balance is worse than a short wait.
  • Feeds and search favor availability. A slightly stale feed beats an error page.
  • Analytics accepts eventual consistency. Numbers that catch up in a minute are fine.

The path, in five steps

Most successful systems follow the same path:

  1. Start simple.
  2. Add structure.
  3. Split when needed.
  4. Distribute when required.
  5. Optimize when scale demands it.

Three takeaways

1. Start with a monolith. Don't add distributed complexity until the system forces you to.

2. Fix coupling first. Use a modular monolith before you split into services. Otherwise you'll just move the coupling onto the network.

3. Microservices aren't a starting point. They're a response to scale, and every stage brings new problems to solve: discovery, data, transactions, retries, traffic and visibility.

Good architecture isn't the most advanced design. It's the right design for today, plus knowing when it's time to evolve.

▶️ Watch the animated version: Stop Starting With Microservices (YouTube)

Next up: why one bad request can crash every user on a server, and how processes and threads decide the blast radius. Watch Processes vs Threads.

If this helped, follow The Byte Mechanics here for a new visual deep dive every week.

About: The Byte Mechanics turns how software really works into short animated explainers: system design, backend and operating systems, one diagram at a time.

Top comments (0)