5. Service registration and the catalog

The catalog answers “what exists and where”. This chapter is about getting things into it correctly.

A service definition

/etc/consul.d/orders.hcl

service {
  id      = "orders-1"
  name    = "orders"
  port    = 8080
  address = "10.0.1.11"
  tags    = ["v2", "primary", "eu-west"]

  meta = {
    version     = "1.4.2"
    team        = "commerce"
    protocol    = "http"
    git_sha     = "a3f9c21"
  }

  check {
    id       = "orders-health"
    name     = "HTTP health on port 8080"
    http     = "http://localhost:8080/healthz"
    interval = "10s"
    timeout  = "2s"
    deregister_critical_service_after = "5m"
  }
}

The two name fields do different jobs and confusing them causes a specific bug:

name is what you look up. Every instance of the same service shares it.

id must be unique on the node. If you run two instances of orders on one machine and give them the same id, the second registration overwrites the first — silently, with no error, and you have half the capacity you think you have.

# two instances on one node: same name, different ids and ports
service { id = "orders-1"  name = "orders"  port = 8080 }
service { id = "orders-2"  name = "orders"  port = 8081 }

id defaults to name, which is why the collision is easy to hit.

address is optional and defaults to the agent’s address. Set it when the service is reachable at somewhere other than the node’s own IP — a container with its own address, or a service on a different interface.

Tags and metadata

Both attach information to a service. They are used differently and the difference matters.

Tags are filterable, including through DNS:

dig @127.0.0.1 -p 8600 primary.orders.service.consul
dig @127.0.0.1 -p 8600 v2.orders.service.consul
curl "http://localhost:8500/v1/health/service/orders?tag=primary&passing"

<tag>.<service>.service.consul is a first-class DNS form, which makes tags the mechanism for the classic pattern:

# a PostgreSQL primary and its replicas
service { name = "postgres" tags = ["primary"] port = 5432 }
service { name = "postgres" tags = ["replica"] port = 5432 }

dig @127.0.0.1 -p 8600 primary.postgres.service.consul   # writes
dig @127.0.0.1 -p 8600 replica.postgres.service.consul   # reads

That is genuinely useful and it is also a trap: a tag is static configuration, not a live fact. If a replica is promoted, nothing changes the tag. The correct version of this pattern uses a health check that asks the database which role it currently has, exactly as chapter 9 of the HAProxy guide does with Patroni’s /primary endpoint — the check decides, not the tag.

Metadata is arbitrary key-value data, not filterable through DNS but available in the HTTP API and usable in prepared queries and mesh configuration. Use it for things you want to read, not things you want to route on: version, team, commit hash, cost centre.

The four ways to register

1. Configuration files

cp orders.hcl /etc/consul.d/
consul reload

Best for services that are always on a machine — a database, a static fleet. Survives agent restarts, is version-controllable, and is what configuration management tools produce.

2. The agent HTTP API

curl -X PUT http://localhost:8500/v1/agent/service/register \
  -d '{
    "ID": "orders-1",
    "Name": "orders",
    "Port": 8080,
    "Tags": ["v2"],
    "Check": {"HTTP": "http://localhost:8080/healthz", "Interval": "10s"}
  }'

Best for services that register themselves at startup. The natural pattern is to register on boot and deregister on graceful shutdown.

Registrations made this way do not survive an agent restart unless the agent’s data_dir persists them, which it does by default — but they are still the agent’s local state, not something in your source control. That is the trade.

3. Automatically, by an orchestrator

Nomad registers its allocations. The Kubernetes catalog sync mirrors Services. ECS has an integration. This is the right answer whenever you have one, because the orchestrator already knows when a workload starts, stops and moves, and the registration cannot drift from reality.

4. Directly into the catalog

curl -X PUT http://localhost:8500/v1/catalog/register -d '{...}'

Almost always wrong. This writes into the catalog without an agent owning it, which means no anti-entropy, no local health checks, and nothing to clean it up. It exists for external services — a managed database, a third-party API — that have no machine to run an agent on:

curl -X PUT http://localhost:8500/v1/catalog/register -d '{
  "Node": "external-rds",
  "Address": "orders-db.abc123.eu-west-1.rds.amazonaws.com",
  "NodeMeta": {"external-node": "true", "external-probe": "true"},
  "Service": {
    "Name": "orders-db",
    "Port": 5432
  }
}'

The external-node metadata is what consul-esm (External Service Monitor) looks for — a separate process that health-checks external services, since no agent is there to do it. Without ESM, an externally registered service is never checked and is permanently “healthy”, which is worse than not registering it.

Anti-entropy, and the deletion that undeletes

Every agent reconciles its local state with the catalog, continuously.

   agent's local services  ──sync every ~60s and on change──►  catalog

   in agent, not in catalog      → registered
   in catalog for this node,
     not in agent                → DEREGISTERED

# looks like it worked
curl -X PUT http://localhost:8500/v1/agent/service/deregister/orders-1

# ...and 60 seconds later it is back, because this still exists:
#   /etc/consul.d/orders.hcl

The agent is authoritative for its own node. The catalog reflects agents; it is not edited independently. To remove a service permanently, remove it from wherever the agent learns about it:

rm /etc/consul.d/orders.hcl
consul reload

The same mechanism is what makes recovery automatic: a server that loses its catalog and restores from a snapshot has agents re-assert their services within a minute without anyone doing anything.

Querying the catalog

# the raw catalog — every instance, healthy or not
curl http://localhost:8500/v1/catalog/service/orders

# the health endpoint — the one you almost always want
curl "http://localhost:8500/v1/health/service/orders?passing"

# what services exist at all
curl http://localhost:8500/v1/catalog/services

# what is on one node
curl http://localhost:8500/v1/catalog/node/web-1

# filtered
curl "http://localhost:8500/v1/health/service/orders?filter=Service.Meta.version==\"1.4.2\""

/v1/catalog/service/X and /v1/health/service/X are different and the difference bites. Catalog returns every registered instance regardless of health. Health returns instances with their check status, and ?passing filters to the healthy ones.

Use the health endpoint with ?passing. Using the catalog endpoint gives you a list that includes the dead instances, which is precisely the problem you adopted Consul to solve.

The filter parameter is a small expression language and it is worth knowing:

   Service.Meta.version == "1.4.2"
   "primary" in Service.Tags
   Checks.Status == "passing"
   Node.Meta.rack == "r3"
   Service.Port > 8000 and "eu" in Service.Tags

The command line

consul catalog services
consul catalog services -tags
consul catalog nodes -detailed
consul catalog datacenters

consul services register orders.hcl
consul services deregister orders.hcl
consul services deregister -id=orders-1

Service defaults and protocol

One configuration entry is worth setting even without the mesh:

Kind     = "service-defaults"
Name     = "orders"
Protocol = "http"

consul config write orders-defaults.hcl

Declaring the protocol is what unlocks layer-7 features — HTTP-aware routing, path-based splitting, per-request retries, and useful metrics from the sidecar. Without it, Consul treats the service as opaque TCP and chapter 11’s traffic management does nothing.

http2 and grpc are the other values worth knowing; grpc in particular changes how Envoy reports status, because a gRPC failure lives in a trailer rather than in the HTTP status.

A registration that will not embarrass you

service {
  id      = "orders-${NODE}-8080"
  name    = "orders"
  port    = 8080
  tags    = ["v2"]
  meta    = {
    version  = "1.4.2"
    protocol = "http"
    team     = "commerce"
  }

  # readiness — should this receive traffic?
  check {
    id                                = "orders-ready"
    name                              = "HTTP /ready"
    http                              = "http://localhost:8080/ready"
    interval                          = "10s"
    timeout                           = "2s"
    success_before_passing            = 2
    failures_before_critical          = 3
    deregister_critical_service_after = "10m"
  }
}

Five things in that block that the minimal version lacks: a unique id, a version in metadata, a /ready endpoint rather than /healthz, hysteresis so one bad probe does not remove the instance, and automatic deregistration of an instance that has been dead for ten minutes. Chapter 6 is about the last three.

What to take from this chapter

• name is what you look up; id must be unique per node. Two instances sharing an id means the second silently replaces the first.
• Tags are filterable, including through DNS (primary.postgres.service.consul); metadata is for reading, not routing.
• A tag is static configuration, not a live fact. Let a health check decide the primary, not a tag nobody updates.
• Register from the orchestrator when you have one, from config files for static services, from the API for self-registering ones, and directly into the catalog only for external services — which then need consul-esm or they are never checked.
• Anti-entropy re-adds anything you deregistered through the API but left in a config file. The agent is authoritative for its node.
• Use /v1/health/service/X?passing, not /v1/catalog/service/X. The catalog includes the dead ones.
• Declare Protocol in a service-defaults entry or every layer-7 feature is unavailable.

Next: health checks — the six kinds, hysteresis, and the difference between “is it alive” and “should it receive traffic”.