Server troubleshooting · Updated
Docker container keeps restarting? Break the restart loop
You deployed an app to a VPS, but its container keeps showing Restarting. A restart policy brings the process back; it does not fix the cause. Find out why the process keeps stopping before changing configuration, investigating dependencies or adding memory.
1. Confirm it is a restart loop and record the state
docker ps -a --format 'table {{.Names}}\t{{.Status}}\t{{.Image}}'
docker inspect --format 'status={{.State.Status}} exit={{.State.ExitCode}} oom={{.State.OOMKilled}} finished={{.State.FinishedAt}} restarts={{.RestartCount}}' YOUR_CONTAINER
docker events --since 30m --until "$(date +%s)" --filter container=YOUR_CONTAINER --filter event=die --filter event=start
Check the service and image are the ones you intended to deploy, and record the incident time and latest deployment. A restarts count that grows each time you run the command, or die/start pairs repeating in docker events, confirms a loop. FinishedAt is in UTC.
What the number in Restarting (N) tells you
The number in brackets is the code from the last exit. It narrows the next check; the full meaning and what to run for each code are in the Docker exit codes guide:
| Code | Short meaning | Details |
|---|---|---|
| 0 | Command finished; a service may have gone to the background | #exit-code-0 |
| 1 | Application error | #exit-code-1 |
| 125 / 126 / 127 | Docker couldn't run it / not executable / not found | #exit-code-125, #exit-code-126, #exit-code-127 |
| 137 | SIGKILL; not proof of OOM | #exit-code-137 |
| 2 / 132 / 255 | Usage or app error / illegal instruction / exit(-1) or failed ssh | #exit-code-2, #exit-code-132, #exit-code-255 |
| 139 / 143 | Segfault / stopped with SIGTERM | #exit-code-139, #exit-code-143 |
2. Check the restart policy and Docker's backoff
docker inspect --format 'policy={{.HostConfig.RestartPolicy.Name}} max_retries={{.HostConfig.RestartPolicy.MaximumRetryCount}}' YOUR_CONTAINER
| Policy | What Docker does |
|---|---|
no (default) | Never restarts the container |
on-failure[:N] | Restarts only after a non-zero exit, optionally at most N times; then the container stays Exited |
always | Restarts whenever it stops; after a manual stop, only when the daemon or the container is restarted |
unless-stopped | Like always, but a container you stopped stays stopped after a daemon restart |
Docker waits longer before each new attempt: the delay starts at 100 ms and doubles, up to a maximum of one minute. Once a container stays up for at least 10 seconds, the delay resets. That is why a crashing container spends more and more time in Restarting and only shows Up for a moment. In Compose the policy is the service's restart: key.
Changing the policy does not repair a failing startup command; it only changes how often you see the failure. Don't combine Docker restart policies with a host process manager such as systemd for the same container. See Docker's restart policy documentation.
3. Read a bounded window around the failure
docker logs --since 15m --tail 200 --timestamps YOUR_CONTAINER
Look for the first error before a stop, not just the final line; in a loop the same failure repeats. Empty output does not prove the app is healthy: it may log to files, or its logging driver may not expose that output. Check the application's logging configuration and startup records.
- Database connection refused: check the dependency, address and port (section 4). Inside a container, localhost is not another service.
- Permission denied or missing file: check mount paths, ownership, runtime user and startup arguments. Do not make everything writable.
- Configuration or migration failure: compare recent changes and establish rollback and data compatibility before editing production configuration.
4. Dependencies that aren't ready yet
A common loop: the app starts before its database, cache or API is ready, fails its first connection, exits, and gets restarted until the dependency happens to be up.
docker compose ps
docker compose logs --since 15m --timestamps YOUR_DEPENDENCY
- In Compose,
depends_onon its own only waits for the dependency's container to start, not to be ready.condition: service_healthywaits for the dependency's healthcheck to pass, so it only helps if that service has a healthcheck. - Compare the address the app uses with the dependency's service name and port on the same network.
- If the app gives up after one failed connection, a short retry in the app is usually a better fix than relying on restarts.
5. Healthchecks: what they do and don't restart
docker inspect --format '{{json .State.Health}}' YOUR_CONTAINER
This shows the health status and the last few check results with their output. On a standalone Docker Engine, an unhealthy container is only marked; Docker does not restart it because of the healthcheck. If unhealthy containers are being restarted, something else is doing it (Swarm, an orchestrator, or a helper container), and that tool's logs are where to look.
A healthcheck that is too strict, or has no start period for a slow-starting app, can keep dependents waiting on service_healthy or trigger that external tool again and again. Check the check's own output before blaming the app.
6. Resources: memory, disk and ports
docker inspect --format 'memory_limit_bytes={{.HostConfig.Memory}}' YOUR_CONTAINER
docker stats --no-stream YOUR_CONTAINER
df -h
- Memory:
OOMKilled=trueis evidence that Docker recorded an out-of-memory kill; a SIGKILL alone is not. Stats are a current snapshot and do not reconstruct the earlier peak. See Exit code 137 and OOMKilled. - Disk: a full filesystem makes databases and apps fail at startup. Follow disk space and inode checks before deleting files.
- Ports: "port is already allocated" means another process or container holds the port;
sudo ss -ltnpshows which.
7. Turn evidence into a next step, not an unreviewed change
Write down the symptom, evidence, possible cause and smallest next check. For production, define the impact, maintenance window and rollback first. Data cleanup, container deletion and broader permissions are not universal fixes.
In OpsMate (itops.sh), you can give the AI a task like this:
The AI is mainly for troubleshooting. Commands that fail the safety check are blocked and never run. Low-risk fixes run automatically by default, and every step is logged. Each time you ask the AI for help, recent terminal output is sent to the cloud AI along with your message, so remove passwords, tokens and customer information first. Verify AI conclusions: a log excerpt alone cannot guarantee an accurate diagnosis.
Illustrative example: database not ready after a reboot
This is an illustrative example, not a customer incident: after a host reboot, the app container shows Restarting (1) and its restart count climbs, while the database container is Up. The app's first log line after each start is a refused connection to the database, and the database's log shows it was still initializing at those times. Compose uses depends_on without a health condition. Together these support a "dependency not ready" hypothesis. The options, a healthcheck with condition: service_healthy or a connection retry in the app, are changes to approve, followed by a check that the restart count stops growing.
Choose the next check
- Need help interpreting the output? See AI and SSH commands in one workspace.
- Logs mention a full filesystem? Follow disk space and inode checks before deleting files.
- Container stays up but users get 502 from Nginx? Find the cause of an Nginx 502 from the error.log line.
- Have a code but not a cause? See Docker container exits immediately? Check why it exited from the exit code.
- Want to follow up away from your desk? Read the Telegram alert and investigation workflow, including its credential requirements.
FAQ
Why does my Docker container keep restarting?
A restart policy brings the process back but doesn't fix the cause: the main process keeps stopping after each start. First confirm the loop: RestartCount in docker inspect grows each time you check, or die / start pairs repeat in docker events. Then read docker logs --since 15m --timestamps and look for the first error before a stop. Common causes are dependencies that aren't ready, permission or file problems, bad configuration, and memory, disk or port problems.
Why does a restarting container spend longer and longer in "Restarting"?
That's Docker's backoff: the delay before each new attempt starts at 100 ms and doubles, up to a maximum of one minute, and it resets once the container stays up for at least 10 seconds. So a crashing container only shows Up for a moment. Changing the restart policy doesn't repair a failing startup command; it only changes how often you see the failure. Don't combine it with systemd for the same container.
I use depends_on in Compose, so why does my app still restart until the database is up?
On its own, depends_on only waits for the dependency's container to start, not to be ready. condition: service_healthy waits for the dependency's healthcheck to pass, so it only helps if that service has a healthcheck. If the app gives up after one failed connection, a short retry in the app is usually a better fix than relying on restarts. Also check the address: inside a container, localhost is not another service.
Does Docker restart a container when its healthcheck reports unhealthy?
Not on a standalone Docker Engine. An unhealthy container is only marked; Docker does not restart it because of the healthcheck. If unhealthy containers are being restarted, something else is doing it, such as Swarm, an orchestrator or a helper container, and that tool's logs are where to look. docker inspect --format '{{json .State.Health}}' YOUR_CONTAINER shows the last few check results with their output.
Need help interpreting the evidence?
OpsMate (itops.sh) puts your SSH terminal and AI on one page, and you review the evidence. Each time you ask the AI for help, recent terminal output is sent to the cloud AI along with your message, so remove sensitive information first.
Start free Desktop with local credentials