Expired-looking x509 errors, 'token invalid', or nodes failing to join with intact certs is the classic symptom of clock skew. Certificates are time contracts.
VMs resuming from pause, hosts with blocked NTP egress, or chrony/ntpd stopped — the clock free-runs and drifts.
Restored VMs wake up in the past or future; skew exceeds the TLS validity window and every cert check fails.
timedatectl # 'System clock synchronized: yes'?sudo journalctl -u kubelet | grep -i 'x509' | tail -3sudo systemctl enable --now chronyd && chronyc trackingsudo systemctl restart kubelet # and containerd if needed'Certificate is not yet valid' means the host clock is AHEAD; 'has expired' means it's BEHIND. That one word tells you which direction the skew went before you even check chrony.
x509 certs are valid only within a time window: a node clocked even a few minutes off makes valid certs look expired (or not-yet-valid). The API server rejects the node's client cert, kubelet fails auth — a cascade from a time sync problem.
Sync time (chrony/ntp) on every node — and on VM hosts, since nested clock drift propagates. Check kubectl get nodes for the resulting NotReady after sync: the cert error clears without rotation once clocks agree.
Kustomize base with probes, PDBs, and zero-downtime rollouts already wired.
Kubernetes Production Blueprints — $27 →One-time. Yours to modify. Instant download from the NinjaOps template store.
One short email when new fixes and production templates drop. No spam, unsubscribe anytime.
Zero-trust access and device security for your whole team — covers the hardening steps above.
Sentry — Catch the error behind this class of bug — exact line, stack and user context.
We earn a commission if you buy through our links — it never costs you extra. More vetted tools on our picks hub · comparing clouds? DigitalOcean vs Vultr and vs AWS · full deals: DigitalOcean · Vultr · NordLayer · Semrush