"Too many open files" — ulimit, systemd, and Container Limits

Each process has an open-file-descriptor ceiling. When a service hits it, sockets and files fail to open. There are three different ceilings — shell ulimit, systemd LimitNOFILE, container/kernel — and you must raise the right one.

What you'll see

Root causes

Process file-descriptor limit reached

ulimit -n (soft default often 1024). Long-running daemons with sockets, logs, and DB connections accumulate FDs. Check live: ls /proc/$(pidof nginx | awk '{print $1}')/fd | wc -l vs cat /proc/<pid>/limits.

A real FD leak

Missing close() on error paths in your app, or connections pooled without eviction. Raising limits without measuring is postponing the crash.

The limit you raised isn't the limit it runs under

Shell ulimit changes don't apply to systemd services (LimitNOFILE) or containers (often 1048576 kernel-side but app defaults lower). Raise the limit in the context that starts the process.

Fix it

  1. Measure: current FDs vs the effective limits
    pid=$(pidof <service>); ls /proc/$pid/fd | wc -l; grep 'open files' /proc/$pid/limits
  2. Raise it where the service is actually launched
    # systemd: add to unit -> [Service] LimitNOFILE=65535 ; then: systemctl daemon-reload && systemctl restart <svc>
  3. For shells/supervisors, raise soft to the hard limit
    ulimit -Sn $(ulimit -Hn)   # or set * soft nofile 65535 in /etc/security/limits.conf
  4. If FDs still climb over days: fix the leak
    ls -l /proc/$pid/fd | awk '{print $NF}' | sort | uniq -c | sort -rn | head   # what kinds of handles leak (sockets vs files vs pipes)

Field note

Docker: docker run --ulimit nofile=65535:65535 (compose: ulimits: nofile: {soft: 65535, hard: 65535}). A restart 'fixing' it temporarily is the signature of a leak — measure /proc/<pid>/fd growth before celebrating.

Common questions

I set ulimit -n 65535 but the service still hits 1024. Why?

You raised it in your shell, but the service is started by systemd (use LimitNOFILE in the unit), a container runtime (use --ulimit), or an init that resets limits. Check /proc/<pid>/limits for the truth, then raise it in whichever system launches the process.

How high should I set the limit?

High enough for expected FDs (sockets×concurrency + files + logs) with headroom — 65535 is a common sane default. If you're near it in steady state, either your workload grew or you have a leak; measure first.

Related fixes

Ship it right the first time

Our most-documented failures, packaged as ready-to-ship starter kits: Docker, Kubernetes, and Terraform.

Browse the template store →

One-time. Yours to modify. Instant download from the NinjaOps template store.