Troubleshooting

Paste into Claude Code, Codex, Cursor or your favourite coding agent. View Markdown

All commands are for Linux. Run docker compose commands from the deploy/ directory.

Reading the logs

docker compose ps                         # are all containers up?
docker compose logs -f --tail=200 api     # or watcher, sweeper, payouts, caddy
docker compose logs --since=1h watcher | grep -i -E 'warn|error'

Each log line is JSON with a msg and a level. Internal API errors are logged with the request path and a request_id; the client only sees internal error.

Install and Docker

Containers can't resolve DNS, or npm ci hangs during docker build. This is common with Tailscale or a local DNS stub (nameserver 100.100.100.100 or 127.0.0.53), which containers can't reach. Give Docker real resolvers:
```
echo '{ "dns": ["1.1.1.1", "9.9.9.9"] }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
```
For a one-off build, pass --network host to docker build instead.
npm hangs on hosts with broken IPv6: export NODE_OPTIONS="--dns-result-order=ipv4first --no-network-family-autoselection", or pass it to docker build as --build-arg NODE_OPTIONS=....
go mod download fails with permission errors under ~/go/pkg/mod: point the module cache somewhere writable, e.g. go env -w GOMODCACHE=/var/tmp/$USER-gomodcache.
Published ports don't answer, but docker compose exec caddy wget -qO- http://localhost/healthz works: the firewall is blocking host-to-container forwarding. Check sudo iptables -L DOCKER-USER and whether ufw or Tailscale rules drop forwarded traffic.
permission denied on the Docker socket: run sudo usermod -aG docker $USER, then log out and back in.
Caddy can't get a certificate: the DNS name must point at this server, and ports 80 and 443 must be reachable from the internet. Check docker compose logs caddy.

"Too many attempts": login is limited to 10 attempts a minute per IP and per email. Wait a minute.
Two-factor codes rejected: check the server clock (timedatectl; it should say "System clock synchronized: yes"). Each code works once, so wait for the next code. After five wrong codes, checks are locked for 15 minutes.
Lost two-factor device: there is no reset button, on purpose. Someone with server access can clear it from the database, after which the user signs in with their password and enrols again:
```
docker compose exec postgres psql -U coldtill coldtill -c \
  "UPDATE users SET totp_enabled = false, totp_secret = NULL WHERE email = 'user@example.com'"
```

Deposits

A deposit isn't showing up:
1. Check the network under Nodes & networks: the last processed block should be close to the chain head. If not, the nodes are failing or rate limiting you. Add your own node.
2. Check that the token is enabled on that network, and that the customer used the right network.
3. Look for watcher warnings mentioning the address or transaction.
A deposit is stuck "confirming": it needs the network's confirmation count. On Bitcoin a block can take an hour or more.
"evm watcher catching up after downtime" warning: the watcher was down for more than 10,000 blocks. It catches up block by block; that can take a while, but nothing is skipped.
A Bitcoin deposit was marked orphaned: every node reported the transaction gone (replaced or dropped). The customer should pay again.

Sweeps

Nothing is being swept: sweeps run when an address holds more than the activation minimum, then on the amount, address-count or time triggers. Check the hot wallet has gas, and look for sweeper errors.
"insufficient funds" in the sweeper log: top up the EVM or Solana hot wallet with native coin.
A wallet is left unswept after a sweep: the token's transfer failed inside the contract (for example a paused or blocklisted token). The funds stay at the deposit address, still locked to your cold wallet.
A Solana sweep failed after 10 minutes of backoff: see the error in the sweeper log; usually the hot wallet lacks SOL, or the RPC node is rate limiting.

Payouts

Stuck in pending: is the payouts container running and did it unlock the hot wallet? Check docker compose logs payouts for passphrase errors.
Stuck in initiated: EVM payouts are re-signed with higher fees automatically. Check the hot wallet has enough native coin for the higher fee.
needs-review: see Payouts → Needs review.
No approval email: check the COLDTILL_SMTP_* settings and the api log. On testnet, codes are printed to the api log instead.

Webhooks

All deliveries fail: on mainnet the endpoint must be a public https URL; private addresses are refused. Redirects aren't followed. Use Send test to check.
Signature mismatch: verify against the raw body bytes, before JSON parsing. Frameworks that parse the body first (for example Express with express.json()) change it.
Timestamp outside the window: your server clock is off. Enable time sync with sudo timedatectl set-ntp true.

Auto-convert

No swap options at checkout: auto-convert is off for the project, the install is in testnet mode, or the provider couldn't quote that amount (very small amounts often can't be swapped).
A conversion shows mismatch: read the reason in Conversions. The payment wasn't credited. Check the delivery transaction on a block explorer and contact the provider if funds didn't arrive.

Database

docker compose exec postgres psql -U coldtill coldtill

Migrations run automatically at start-up. If the api container exits with a migration error, restore the last backup before retrying, and report the error.