Run an AI Agent 24/7 on a VPS

Getting an agent to run under systemd takes half an hour. The hard part is noticing when it is green but dead. Monitoring and safety lessons from an always-on agent.

Getting an agent to run on a VPS takes about thirty minutes. The hard part comes weeks later: noticing that the process is alive but doing nothing. This piece covers the minimal setup, then the traps we actually hit with our own always-on bot, and the monitoring design we use now.

The short version

• Let an LLM API do the heavy inference. The VPS just orchestrates, and 2 GB of RAM is plenty.
• The LLM only decides. Actual changes are made by a few pre-approved scripts.
• Run the monitor on a different host, and have it send an OK every day.

1. Picking a VPS

Prices as shown on official pages on 2026-10-16:

Ad disclosure: some of these providers run affiliate programs we participate in, or plan to. We picked them on price and region only.

2. Minimal setup

sudo adduser --disabled-password agent
sudo ufw allow OpenSSH && sudo ufw enable

[Service]
User=agent
EnvironmentFile=/home/agent/app/.env
ExecStart=/usr/bin/node dist/main.js
Restart=on-failure
WatchdogSec=120

WatchdogSec only helps if your app actually sends WATCHDOG=1 pings.

3. The LLM decides, scripts act

Our monitor agent reads logs every 10 minutes. It can only run a few named scripts, and each one follows the same steps: back up, write to tmp, validate, swap atomically with mv, and restore on failure. Our first version had the notification bot token in the system prompt. Now a notify script holds it, and the model only knows the script’s name. Logs contain untrusted text, so keys never share a context with them.

Letting agents read production without writing to it: permission design.

4. Green but dead

• systemctl is-active and /health only prove the parent process exists. We add CPU monitoring from outside, a tick watchdog on the work loop, and a readiness check.
• “No restarts” is not health. Our mysterious self-healing every few days was unattended-upgrades rebooting a box that had been broken the whole time.
• Missing from the journal does not mean it never fired. The root log level was WARNING, so every info line was dropped.
• A monitor can block itself. Logging at the moment of detection took a handler lock and froze the watchdog thread. The detection path now does a raw os.write(2, …) and nothing else.

5. Four monitoring invariants

1. Run the monitor on a different host.
2. Send an OK every day, so silence itself is the alert.
3. Keep detection separate from notification.
4. Give “intentionally paused” its own label (inert), separate from ok and error.

Add a ?simulate=fail switch so you can exercise the alert path while production is healthy.

Pitfalls beyond these four: monitoring pitfalls.

6. Secrets

• Never put tokens in curl argv, because they show up in ps. Use curl --config - and pass them on stdin.
• An empty stdin piped into wrangler secret put reported “Success” and stored an empty secret. Read the value into a variable and test [ -n "$s" ] before piping it.
• Agent memory and config files should record where a key lives and what it is called, never the key itself.

A deny rule still leaves paths where .env gets read, through grep -r or a script. The leak paths we reproduced with a dummy key, and how to close them, are in Keep Secrets Away from Claude Code.

7. Check that it is really stopped

One automation we thought we had stopped kept running for months. All we had done was hide its output. Before calling something stopped, check timers, crontab, enabled units, and whether its output files are still getting new timestamps.

Quiet accidents where something you thought was stopped or only a test ran in production are collected in Deploy and Migration Accidents.