Cells, not a cluster

A second team asks for the assistant. Copying the first login onto the same installation would let both teams read the same conversations because the application still uses one local identity. Adding another web container behind a load balancer would not change that. The correct unit is another installation.

That installation is a deployment cell.

Shape of the idea

Cell alpha                    Cell beta
alpha.example.net             beta.example.net
login alpha                   login beta
Postgres alpha                Postgres beta
Redis alpha                   Redis beta
backups/alpha                 backups/beta
telemetry instance=alpha      telemetry instance=beta

Shared:
source repository, CI pipeline, image digests, dashboard definition,
shared non-tenant provider credentials where policy permits

There is no cross-cell request path and no shared application database.

Terminology that matters

A deployment cell is an independently deployable, independently recoverable application instance for one person or trusted group.

A horizontal replica is another copy of a service that handles traffic for the same logical application and shared state.

A tenant is an authorization boundary inside one application. Cells can separate tenants physically, but they do not create application-level tenant authorization.

A shard router maps users or requests to the correct data partition. A DNS bookmark or manually chosen hostname is not a transparent shard router.

A blast radius is the users and data affected by one fault or mistaken operation.

Cell isolation is operational, not magical

Each cell needs unique public and state identity:

• hostname and application origin
• login and session signing secret
• Compose project and instance root
• Postgres, Redis, and cache volumes
• mini-app hosting project or namespace
• backup prefix
• deployment runner label
• telemetry label

A committed instance file can hold these public values. Validate it as data instead of sourcing arbitrary shell syntax. Reject duplicate identifiers, relative paths, unknown keys, command substitutions, and image references without digests.

Secrets split into shared and instance namespaces. An LLM provider key may be shared if policy allows. Database passwords, login hashes, cache administration keys, and session secrets must be cell-specific. The host should read only its own instance namespace plus the approved shared production namespace.

An instance inventory might contain:

INSTANCE_ID=alpha
APP_ADDRESS=alpha.example.net
COMPOSE_PROJECT_NAME=agent-alpha
INSTANCE_ROOT=/opt/agent/instances/alpha
RUNNER_LABEL=agent-alpha

None of these values is secret. They still need review because one typo can point a deployment at the wrong volumes or runner.

Why these are not replicas

Two cells do not make the service highly available. If cell alpha's VM fails, traffic cannot move to beta because beta has different credentials, database contents, caches, and public identity. Recovery means restoring alpha onto a replacement host or redirecting its hostname after that state is ready.

Horizontal replicas solve another problem. They run the same stateless service against shared or replicated state, usually behind a load balancer. They require coordination for sessions, jobs, migrations, WebSockets, cache invalidation, and leader-only work. They may increase availability and throughput, but they also keep users inside one authorization domain unless the application enforces tenant boundaries.

Cells reduce blast radius. A bad migration, full disk, leaked login, or overloaded analytics store in alpha need not affect beta. They also reduce shared-state coupling. The cost is duplicated infrastructure and repeated operations.

One pipeline, many targets

CI should build images once. A deployment workflow selects a committed instance ID, resolves its runner label, and sends the same backend and web digests to that cell. Use a concurrency key that includes the instance ID. Alpha and beta may deploy at the same time, while two alpha deployments serialize.

Backups include the instance ID in filenames and remote directories. Restore rejects a different cell's backup unless the operator supplies an explicit cross-instance override. Metrics and log streams carry instance_id, and a shared dashboard filters on that label. Copying dashboards per cell creates configuration drift without adding isolation.

Infrastructure code can create one host, firewall, and DNS record per module instance. It should not put secret-provider tokens or runner registration tokens in OpenTofu state or cloud-init. Run those short-lived bootstrap steps after provisioning.

Decisions and rejected approaches

Use one VM per cell. Multiple cells on one host would save money, but they would still share the kernel, Docker administrator, disk, public ports, and host failure. Caddy could route several hostnames, yet the operational isolation claim would become weaker and harder to explain.

Keep one database per cell. A shared database with a tenant_id column can scale better, but only after every query, mutation, background task, cache key, export, and backup path enforces tenant authorization. That is an application architecture change, not a deployment shortcut.

Keep the UI on the cell origin. Hosting it centrally at the edge appears cheaper, but session cookies, service-worker scope, WebSocket authentication, Content Security Policy, and push routes would cross origins.

Do not add Kubernetes only because there are several cells. A cluster scheduler can run them, but it also creates a shared control plane and shared failure modes. The cell boundary can later map to namespaces, accounts, or clusters if there is a measured reason.

When the cell model stops scaling

Cells work well when there are a few installations, each has modest traffic, users can choose a stable hostname, and restore time is acceptable.

The approach strains when routine work grows linearly with cell count. Warning signs include dozens of runner registrations, slow fleet-wide security patches, repeated certificate or DNS incidents, inconsistent configuration, backup drills that cannot finish inside the required interval, and idle VMs dominating cost.

It also stops fitting when users need:

• one global login and automatic routing
• private data boundaries inside a shared organization
• cross-cell search, collaboration, or reporting
• zero-downtime host failure recovery
• elastic traffic handling for one tenant
• transactional work spanning tenants
• uniform policy enforced faster than per-cell rollout allows

At that point, choose deliberately. A managed container platform can automate placement while preserving one database per cell. A control plane can provision cells and track versions. Larger tenants may get dedicated cells while smaller tenants use a properly authorized multi-tenant service. Stateless horizontal replicas and a highly available database become appropriate when one logical tenant exceeds one host.

Do not wait for total failure before changing models. Track deploy duration, operator time per cell, restore-test coverage, resource utilization, and fleet version skew. Those numbers identify the ceiling.

Failure modes to rehearse

• A copied instance file retains the original Compose project and mounts another cell's volumes.
• A shared backup directory overwrites similarly named dumps.
• A wildcard runner label sends a deployment to the wrong host.
• Shared secrets accidentally include an instance login.
• Alerts omit the instance label, leaving the operator unsure which host is failing.
• A fleet update changes every cell at once and defeats blast-radius reduction.

Roll out risky changes to one disposable or low-impact cell, then expand.

Cell readiness checklist

• One cell maps to one person or explicitly trusted group.
• Hostname, Compose project, root path, runner, and app namespace are unique.
• Database, login, and session secrets are instance-specific.
• Backups and restores enforce instance identity.
• Logs, metrics, alerts, and release records include instance_id.
• CI builds once and deployments target committed inventory.
• A cell failure has a documented replacement-host recovery path.
• Nobody describes multiple cells as high availability.
• Fleet size, version skew, restore coverage, and operator time are measured.
• The triggers for a control plane or multi-tenant redesign are written down.

Previous: Chapter 34, "Build once, deploy by digest".

Next chapter

Chapter 36, "Secrets that never enter Git", gives each cell a narrow runtime secret path.