Hetzner from zero to cell

You have a domain, an SSH public key, and an empty Hetzner project. There is no server yet. The goal is not merely to get containers running. The goal is to create a host that another operator can rebuild from notes after the original machine is gone.

This guide uses Hetzner Cloud, Ubuntu 24.04, Docker Engine, Compose, Caddy, and OpenTofu. Package names and service commands change across distributions, so check current vendor guidance before copying a command.

Route map

Choose region and size
  -> create SSH key and firewall
  -> provision host and disk
  -> publish DNS
  -> harden access
  -> install container runtime
  -> install deployment runner and secret adapter
  -> deploy by image digest
  -> verify
  -> back up and rehearse rollback

Working vocabulary

A cell is one hostname, one login boundary, one application stack, and one set of state. It is not a replica behind a load balancer.

Cloud-init is first-boot configuration supplied to the server. It is useful for packages and host policy, but it must not contain long-lived secrets.

Provider backup is Hetzner's server-level backup. It complements an application-aware database dump. It does not replace one.

Bootstrap credential is the narrow secret that lets the host fetch its runtime secrets or register a deployment runner.

1. Decide where state will live

Select a region near the users, subject to legal and organizational data-location rules. A two-vCPU, four-gigabyte x86-64 VM is a reasonable starting point for a private agent application whose model calls leave the host. Confirm current server types and prices. Product names change.

The VM's primary disk is often simplest. A separate volume still shares the provider failure domain. In either case, enable deletion protection and keep logical Postgres dumps in another account or provider.

2. Create the network boundary

Upload the administrator's Ed25519 public key. Create a cloud firewall before creating the server:

• Allow TCP 22 only from known administrator CIDRs.
• Allow TCP 80 from anywhere for certificate issuance and redirects.
• Allow TCP and UDP 443 from anywhere. UDP 443 permits HTTP/3.
• Deny every other inbound port.

An OpenTofu input can make the intended boundary reviewable:

ssh_source_cidrs = ["203.0.113.10/32"]
hostname         = "chat.example.net"
enable_backups   = true

Provider tokens belong in the calling environment or a CI secret store. Keep state files outside Git because infrastructure state can contain identifiers and derived sensitive data even when the module avoids application secrets.

3. Provision without embedding secrets

Create an Ubuntu 24.04 x86-64 server with the firewall, SSH key, protection settings, and optional provider backup attached. Cloud-init may install Docker, the Compose plugin, rclone, fail2ban, unattended security updates, and a small swap file. It may create a dedicated deploy user and the application directories.

Do not put an LLM key, database password, secret-broker token, GitHub runner token, or Cloudflare token in cloud-init. Cloud-init data and logs tend to persist in more places than operators expect.

Wait for cloud-init to finish before debugging missing packages. Check its status and logs through the distribution's supported commands. Then verify the Docker daemon and Compose plugin versions. Do not assume the distribution package and Docker's upstream repository install identical versions.

4. Publish DNS and confirm the route

Create an A record for the application hostname pointing to the server IPv4 address. Add AAAA only if the host and firewall are correctly configured for IPv6. A stale or half-configured IPv6 record can make the site fail for some users while IPv4 tests pass.

Confirm resolution from more than the host itself:

dig +short A chat.example.net
dig +short AAAA chat.example.net

Caddy can obtain the TLS certificate once DNS resolves and ports 80 and 443 reach the server. Check certificate issuance logs. Repeated retries may trigger certificate authority limits, so fix DNS and firewall causes before restarting in a loop.

5. Harden SSH and host access

First confirm key-based login in a second terminal. Only then disable password authentication and direct root login according to the current OpenSSH configuration layout. Some images use included configuration fragments, so editing one familiar file may not change the effective policy.

Run the deployment service as a dedicated user. Docker access is effectively root access because the user can mount host paths into a container. Treat the runner and its repositories as production privileged. Never run pull-request code on this runner.

6. Prepare state and deployment directories

Give each cell a unique root such as /opt/agent/instances/example. Keep release manifests, backup files, generated metrics, and runtime scratch paths beneath a predictable layout. Preserve the Compose project name when migrating an existing cell, since changing it selects different named volumes.

Install a repository-scoped deployment runner with a short-lived registration token. It connects outbound, so GitHub needs no SSH access. Install the secret adapter and one owner-only bootstrap credential.

The host should receive a deployment bundle, not the source tree. The bundle contains Compose, validated instance configuration, and operational scripts. CI should publish the application images first and pass their immutable digests to the deploy step.

7. Bring up the cell

Materialize secrets into a temporary runtime file. Pull the exact image digests. Start Postgres, wait for health, run migrations once, then start the worker, sync service, cache, and Caddy. The exact dependency graph may differ, but migrations must not race and public traffic must not reach a half-migrated application.

Check all layers:

1. Every expected container is healthy.
2. Postgres accepts a simple query.
3. The unauthenticated public page redirects to login.
4. An unauthenticated sync or cache route is denied.
5. A signed-in browser can create data and see it in a second browser.
6. A host reboot preserves that data.
7. A network scan shows only SSH, HTTP, and HTTPS.

8. Back up before calling it done

Create a compressed Postgres dump and snapshot any non-rebuildable Redis data. Put the cell ID in filenames and remote prefixes, copy them off-host, and report the last successful remote copy.

Restore onto a disposable replacement server. Stop writers, restore Postgres, remove stale replicas, restore the applicable Redis data, and let derived caches rebuild. Confirm that the restored cell cannot accidentally consume another cell's backup without an explicit override.

9. Plan rollback before the first upgrade

Record current backend and web digests atomically. Keep several prior manifests. A failed health check can restore previous images. Database migrations are different. If a migration is not backward compatible, image rollback may make the old application unable to read the new schema.

Take a pre-deploy dump. Prefer additive, backward-compatible migrations. When rollback requires data restoration, state the recovery command and expected data-loss window before deployment.

Common setup failures

• An unrestricted SSH rule exposes administration to the Internet.
• A stale IPv6 record creates intermittent failures.
• Changing the Compose project selects empty volumes.
• Terraform variables can copy secrets into state.
• A local-only backup disappears with the VM.

Field card

• Region and server architecture match latency and image requirements.
• SSH key login works before password login is disabled.
• Cloud firewall exposes only 22, 80, and 443 as intended.
• DNS A and optional AAAA records resolve correctly.
• Cloud-init and infrastructure state contain no application secrets.
• Docker and Compose versions are checked against current documentation.
• The deploy user and self-hosted runner are treated as root-equivalent.
• The host pulls digest-pinned images and does not build source.
• Public, internal, reboot, backup, and restore checks have passed.
• Previous image digests and the database recovery command are recorded.

Previous: Chapter 31, "One box, done well".

Next chapter

Chapter 33, "Containers, Caddy, and clear ports", defines the service and ingress contract inside the cell.