bash: fork: retry: Resource Temporarily Unavailable

The kernel refused to create a new process: nproc limit (RLIMIT_NPROC) or the system's pid_max is exhausted. Find what's spawning before raising limits — runaway spawning is the usual root cause.

What you'll see

Root causes

Process limit hit (ulimit -u / cgroup pids.max)

Per-user RLIMIT_NPROC or a container cgroup pids ceiling. In containers it's usually the cgroup: cat /sys/fs/cgroup/pids.max vs current. Docker: --pids-limit; Kubernetes: pod-level pid limits.

System-wide pid exhaustion or fork bomb

pid_max reached (check /proc/sys/kernel/pid_max vs actual pid count). A runaway script or actual fork bomb fills the table; the fix is stopping the spawner, not just raising the ceiling.

Fix it

  1. Count what exists vs the ceiling (from another shell or the host)
    ps -e --no-headers | wc -l ; cat /proc/sys/kernel/pid_max ; ulimit -u
  2. Find the top spawner
    ps -eLf | awk '{print $1}' | sort | uniq -c | sort -rn | head   # or ps axo ppid --no-headers | sort | uniq -c | sort -rn | head
  3. Stop the runaway, then raise limits if the workload is legitimate
    # ulimit -u 4096 (user) ; container: --pids-limit=-1 or a sane value ; sysctl kernel.pid_max=N for system headroom
  4. Persist the fix where it will survive
    # /etc/security/limits.conf for users ; compose/k8s pod spec for containers (pids limit is a spec field, not a flag)

Field note

The retry in the message means bash backed off and tried again — a box that still answers this error is healthier than it looks; one answering 'No such process' style cascades is worse. Raising limits without finding the spawner just delays the next outage at a higher number. ps-based attribution first, ceiling changes second.

Common questions

Why can't I run any commands to investigate?

Every command forks. Use another shell if one survives, an SSH session that's already authenticated, or host-level tools (docker exec, nsenter) that don't need the exhausted table.

What's a sane pid limit for containers?

Enough headroom for the workload's real needs plus burst: typically a few hundred for app containers, set via --pids-limit or the pod spec. Unlimited leaves the container able to starve the whole node.

Related fixes

Ship it right the first time

Our most-documented failures, packaged as ready-to-ship starter kits: Docker, Kubernetes, and Terraform.

Browse the template store →

One-time. Yours to modify. Instant download from the NinjaOps template store.