Home /
System Design /
Why Microservices Need Service Discovery

Why Microservices Need Service Discovery

A monolith doesn’t need to ask where its own functions live – they’re all compiled into the same process, at the same address, forever. The moment you split that monolith into services running as independent, replicated, frequently-restarted processes, that assumption disappears. Every call from one service to another becomes a network call to a location that isn’t fixed and isn’t guaranteed to still be correct by the time the call is made.

Service discovery is the piece of infrastructure that answers a deceptively simple question on every single request: “which instance of this service should I talk to, right now?” It sounds like plumbing. In practice, it’s one of the load-bearing pieces of any microservice architecture – get it wrong and everything built on top of it inherits the problem.

In a static system, addresses are configuration. In a dynamic system, addresses are a query.

1. What Problem Is Service Discovery Actually Solving?

In a microservice architecture, instances are not static. Containers restart. Auto-scalers add and remove replicas based on load. Deployments roll instances over one at a time. Cloud infrastructure reassigns IP addresses. None of this is a failure – it’s the system working as designed. But it means the set of “valid addresses for the Payments service” is different from one minute to the next.

Hardcoding IP addresses or hostnames into configuration files works right up until the first restart, the first autoscale event, or the first deploy – at which point every caller holding a stale address starts failing, usually all at once, usually in production. Service discovery replaces a fixed answer with a live one: instead of asking “what is Payments’ address,” the system asks “what is Payments’ address as of right now,” every time it needs to know.

2. The Two Halves of the Problem: Registration and Discovery

Every service discovery system, regardless of vendor or implementation, is built from two cooperating halves.

2.1 Registration

When an instance starts up, it announces itself to a central registry: here is my service name, my host and port, and whatever metadata matters (version, region, weight). When the instance shuts down – cleanly or otherwise – that entry needs to disappear too, or the registry starts lying.

2.2 Discovery

When a caller needs to reach a service, it asks the registry (directly or indirectly) for a current, healthy set of addresses, and picks one – often with a load-balancing strategy like round robin, least connections, or weighted routing layered on top.

Everything else in this article is really about the details, trade-offs, and failure modes hiding inside those two verbs: registering and discovering.

3. Client-Side vs. Server-Side Discovery

There are two broad architectural styles for wiring these two halves together, and the choice has real consequences for where complexity lives in your system.

Figure 2 – Client-side discovery puts routing logic in every caller; server-side discovery centralizes it behind a router.

In client-side discovery, the calling service queries the registry directly and applies its own load-balancing logic before making the call. Netflix’s early Eureka-plus-Ribbon stack is the textbook example. It cuts out a network hop, which helps latency, but it also means every client needs a discovery-aware library, in every language your organization uses, kept in sync as the registry’s API evolves.

In server-side discovery, the caller just sends its request to a well-known load balancer or router, which is the only thing that talks to the registry. Kubernetes Services and most cloud load balancers work this way. Clients stay simple – they don’t even know discovery is happening – at the cost of an extra network hop and a router that now needs to scale and stay available alongside everything else.

Neither is strictly better. Client-side discovery tends to win in polyglot, high-throughput systems willing to invest in shared client libraries. Server-side discovery tends to win when you want application code to stay ignorant of infrastructure – which is exactly the trade-off Kubernetes makes for you by default.

4. Keeping the Registry Honest: Heartbeats and Health Checks

A registry is only useful if it reflects reality, and reality changes constantly. The mechanism most systems use to keep the registry honest is the heartbeat: each registered instance periodically tells the registry “I’m still here,” and if those heartbeats stop, the registry assumes the instance is gone and removes it.

Figure 3 – The full lifecycle: register on startup, heartbeat while alive, get evicted on silence.

The timing here is a genuine trade-off, not a detail to skim past. A short heartbeat interval and an aggressive timeout mean the registry notices failures quickly, but a slow network blip can cause healthy instances to be evicted anyway – a false positive that can cascade if enough instances get pulled from rotation at once. A longer timeout is more tolerant of noise, but it means traffic keeps flowing to a dead instance for longer, and every request during that window fails.

Most production systems also separate two kinds of health checks that are easy to conflate: liveness (is the process running at all?) and readiness (is it currently able to serve traffic correctly – warmed caches, open DB connections, no active overload?). An instance can be alive but not ready, and routing traffic to it during that gap causes exactly the kind of intermittent errors that are miserable to debug after the fact.

5. Self-Registration vs. Third-Party Registration

Who is responsible for telling the registry an instance exists? There are two patterns, and they push complexity into different places.

Pattern	How it works	Trade-off
Self-registration	The service instance itself calls the registry's API on startup and sends heartbeats while running.	Simple to reason about, but couples every service to the registry's client library and makes the service responsible for its own deregistration on shutdown.
Third-party registration	A separate component (a "registrar") watches the platform - container orchestrator, VM scheduler - and registers or removes instances on the service's behalf.	Services stay completely unaware of discovery, but you now depend on the registrar being reliable and timely; Kubernetes' own control loop is a well-known example.

Kubernetes is worth calling out specifically here because it’s the default most teams encounter first: kubelet and the control plane handle registration automatically based on pod status, so application code never has to think about announcing itself at all. That’s a strong argument for third-party registration when you’re already running on an orchestrator that offers it.

6. DNS-Based Discovery: The Oldest Trick That Still Works

Before dedicated registries like Consul or Eureka became common, DNS was – and still often is – the simplest form of service discovery available. A service name resolves to one or more IP addresses, and clients just use normal DNS resolution to find something to call.

DNS-based discovery is attractive because every language and platform already knows how to do a DNS lookup – there’s no client library to install. The catch is TTL and caching: DNS resolvers and OS-level caches were built for a world where addresses change rarely, so they often cache results far longer than a fast-moving service topology can tolerate, which can mean traffic keeps hitting instances that no longer exist. Kubernetes’ internal DNS (CoreDNS) works around this by keeping TTLs very short and integrating tightly with its own registry, which is part of why it feels more responsive than plain DNS discovery used to.

7. Service Mesh: Discovery Without the Application Noticing

As systems grow, more and more logic tends to accumulate around each individual service call: discovery, retries, timeouts, circuit breaking, mutual TLS, observability. A service mesh pulls all of that out of the application and into a sidecar proxy that runs alongside every instance.

Figure 4 – Sidecars talk to each other; the control plane keeps every sidecar’s routing table current.

In this model, the application still just makes a normal-looking call to “the payments service.” It never touches the registry directly. Its sidecar intercepts that call, consults routing information pushed down from the mesh’s control plane, and forwards the request to a healthy instance – transparently applying load balancing and retries along the way. Tools like Istio, Linkerd, and Consul Connect all follow this shape.

The appeal is real: application code gets simpler and more uniform across languages, because none of it needs a discovery client anymore. The cost is also real – you’re now running and operating an entire additional distributed system (the mesh’s control and data planes) whose failure modes become your application’s failure modes too. This is usually a trade worth making at real scale, and usually not worth the operational overhead for a handful of services.

8. What Goes Wrong Without It - and With It Done Badly

It’s worth being explicit about the failure modes, because “we added service discovery” is not automatically the same as “we solved the problem.”

Stale entries - an instance crashes ungracefully (no clean deregistration) and lingers in the registry until a health check eventually catches it; every request routed there in the meantime fails.
Registry as a single point of failure - if the registry itself goes down and every lookup depends on it synchronously, the whole system's ability to route traffic goes down with it, even though every actual service instance is healthy.
Split-brain during network partitions - in a multi-node registry (Consul, etcd-backed systems), a partition can leave two halves of the cluster each believing they hold the authoritative view, registering conflicting or duplicate state.
Thundering herd on registry restart - every instance re-registering and re-subscribing at once after a registry restart or leader election can itself look like a traffic spike against the registry.
Overly aggressive eviction - heartbeat timeouts tuned too tight turn ordinary network jitter into false-positive removals, pulling healthy capacity out of rotation right when it's needed most.

None of these are arguments against service discovery – they’re arguments for treating the registry itself as critical infrastructure: run it as a highly available cluster, give clients a local cache so a registry blip doesn’t take down routing entirely, and choose timeouts deliberately rather than by default.

9. Picking a Consistency Model for Your Registry

Service registries are themselves distributed systems, and they inherit the same CAP-theorem trade-offs as any other. This matters more than it might seem, because it determines what happens during a network partition.

Model	Behavior during a partition	Examples
CP (consistency-favoring)	May reject reads/writes rather than risk serving stale or conflicting data; can briefly make discovery unavailable.	etcd, Zookeeper-backed registries
AP (availability-favoring)	Keeps answering discovery queries even if some nodes disagree briefly, favoring "an answer" over "the perfectly correct answer."	Eureka, Consul (tunable)

For most service discovery use cases, availability tends to win out over strict consistency: it’s usually better to route a request to a slightly stale list of instances (and let a retry or health check clean it up) than to have discovery itself become unavailable during a partition. This is a big part of why Eureka was explicitly designed AP-first, and why Consul lets you tune the trade-off rather than forcing one choice.

10. A Production-Shaped Setup

Put together, a discovery layer that’s been through a few incidents tends to converge on a similar shape: instances register automatically via the platform, heartbeats and separate readiness checks feed a highly-available registry cluster, and every caller – whether directly or through a sidecar – keeps a short-lived local cache of the last known-good set of addresses so a registry hiccup degrades gracefully instead of taking every call down with it.

STARTUP                             STEADY STATE
Instance boots                      Caller needs Payments
-> registers (host, port,           -> check local cache (fresh?)
    version, metadata)                  -> yes: use cached list
-> begins heartbeat loop                -> no: query registry
-> begins readiness checks              -> cache the result
                                    -> pick instance (LB policy)
                                    -> call it, retry on failure

This isn’t the only correct shape, but the underlying principle generalizes: discovery should make the common case fast (serve from cache, avoid a registry round trip on every call) and the failure case survivable (fall back gracefully instead of cascading).

11. If You're Asked This in an Interview

“How do services find each other in your architecture?” is a common systems-design question, and “we have a service registry” is an incomplete answer on its own. A stronger answer walks through the same ground covered here:

Whether discovery is client-side, server-side, or hidden inside a service mesh sidecar, and why that fits the system.
How instances register – self-registration or a third-party registrar tied to the orchestrator.
How the registry tells live instances from dead ones: heartbeat intervals, liveness vs. readiness checks.
What happens to in-flight traffic when an instance dies ungracefully, before the registry notices.
Whether the registry favors consistency or availability during a network partition, and why that’s the right choice here.
How the registry itself avoids being a single point of failure – clustering, client-side caching, graceful degradation.
How DNS or a mesh control plane fits in, if at all.

As with most infrastructure questions, the strongest signal isn’t naming a tool – it’s being able to describe what breaks, and how the design limits the blast radius when it does.

The Bigger Lesson

Service discovery looks like a small piece of glue code: look up an address, make a call. But it’s actually where a distributed system’s honesty lives. Every other reliability mechanism – retries, circuit breakers, load balancing, canary deploys – assumes that the list of “currently valid instances” it’s working from is roughly correct. Service discovery is the thing that keeps that list roughly correct, continuously, while the ground underneath it keeps shifting.

A microservice architecture doesn’t remove the complexity of knowing where things are – it just moves that complexity from compile time to runtime.

Get service discovery right, and that complexity becomes invisible: services simply find each other, scale up and down, and recover from failure without anyone noticing. Get it wrong, and it becomes the thing every other outage traces back to.

Final Takeaways

Static addresses don't survive dynamic infrastructure - service discovery replaces configuration with a live query.
Every discovery system is built from two halves: registration and discovery - most real problems trace back to one of the two being sloppy.
Client-side discovery centralizes routing logic in every caller; server-side discovery centralizes it behind a router. Neither is universally better.
Heartbeats and health-check timeouts are a genuine trade-off between fast failure detection and tolerance for noise.
Separate liveness from readiness - an instance can be running without being safe to route traffic to.
Self-registration couples services to the registry client; third-party registration (as in Kubernetes) keeps services unaware of discovery entirely.
DNS-based discovery is simple but fights against caching and TTLs designed for a slower-moving world.
A service mesh moves discovery into a sidecar, simplifying application code at the cost of running another distributed system.
Treat the registry itself as critical infrastructure: cluster it, cache around it, and choose a consistency model deliberately.
The goal isn't a registry that's never wrong - it's a system that stays correct enough, fast enough, and degrades gracefully when it isn't.

Still have questions about service discovery, system design, or building scalable architectures? Get in touch with CodeKerdos to discuss your learning goals and find the right way to take your technical skills further.