nginx: "Host Not Found in Upstream" — DNS at Startup vs Runtime

nginx resolves proxy_pass hostnames at CONFIG LOAD, not per request. If DNS isn't ready (container boot races, dynamic upstreams), startup fails. The fix is runtime resolution via a resolver directive + variables.

What you'll see

Root causes

Static hostname resolved at startup (and the backend isn't up)

A literal hostname in proxy_pass resolves during config load. Compose/K8s boot races: nginx starts before the app DNS name exists → emerg error → crash → restart loop.

Dynamic IPs with a static config

Docker/K8s service IPs change between restarts; nginx cached the old one at load and keeps proxying to a dead address, or fails to boot when the name doesn't resolve at all.

Fix it

  1. Move to runtime DNS resolution (resolver + variable)
    # resolver 127.0.0.11 valid=10s;   (Docker embedded DNS)
    # set $backend_upstream http://app:8080;  proxy_pass $backend_upstream;   # variables make nginx resolve per-request
  2. Compose: order the startup
    # nginx: depends_on: app: condition: service_healthy  — plus the resolver pattern for restarts
  3. Static IPs as a deliberate alternative (kube-apiserver style setups)
    # proxy_pass http://10.0.0.5:8080;  — no name, no DNS, no race; trades away dynamic re-pointing
  4. Validate and reload once resolvable
    nginx -t && systemctl reload nginx   # containers: the crash loop itself retries until the name exists

Field note

The variable form isn't just for boot races: it also makes nginx re-resolve every valid= seconds, surviving backend IP changes without reloads. When you need upstream health/weight features (multiple backends, failover), upstream blocks with a static hostname resurrect the startup-DNS problem — resolver+variable or static IPs are the escape hatches.

Common questions

Why does nginx need a variable to do something obvious?

Design choice: literal hostnames are resolved once at load for speed and predictability. The variable form explicitly opts into runtime resolution — that's the switch you're flipping.

What's 127.0.0.11?

Docker's embedded DNS server on the default bridge networks: it resolves service names (compose services, container links) and forwards the rest upstream. In host networks use the system resolver instead.

Ship it right the first time

Our most-documented failures, packaged as ready-to-ship starter kits: Docker, Kubernetes, and Terraform.

Browse the template store →

One-time. Yours to modify. Instant download from the NinjaOps template store.