The Server Lived in My Head. This Week I Wrote It Down.

My ledger has been live at go.sohag.pro for a few weeks now. Real HTTPS, real Postgres, real deploys on every push to main. And there is a thing I have been not thinking about, the way you do not think about a smoke alarm with a dead battery.

The server was set up by hand. One evening, over SSH, as root, I created the users, wrote the systemd unit, configured nginx, installed Postgres, ran certbot, and tuned the memory so the 1 GB box would not fall over. Then I wrote most of it down in a text file so I would not forget. Most of it.

Here is the uncomfortable question I finally asked myself: if that box died tonight, how would I rebuild it? And the honest answer was, from that text file and from memory, at whatever hour it happened, hoping I did not miss the one chmod that matters. That is not a disaster recovery plan. That is a disaster with a paragraph of notes.

Week 10 was supposed to be "Docker and Terraform for AWS." I did the Docker part, dropped the Terraform part, and spent the real effort turning the server that already exists into code that can rebuild it. This is post 10 of 12.

The Docker part, quickly

The Dockerfile had been a skeleton since week one, with a comment that literally said "fleshed out in Week 10." So I fleshed it out. Multi-stage build, distroless base, non-root, build cache mounts, and the binary stripped down. The point of a multi-stage build is that the heavy toolchain never ships. You compile in a fat image and copy only the finished binary into a tiny one.

The final image is about 10 MB, and there is a make image-size target that fails the build if it ever creeps past 20 MB, so the budget is enforced, not aspirational. There is also a docker compose up that brings up the whole local world at once: the app, Postgres, Jaeger for traces, and Prometheus scraping the metrics. Clone, one command, and you have the full stack running locally.

But here is the thing I want to be honest about: none of that Docker work touches production. My server does not run Docker. It runs the bare binary.

Why production is not a container

This surprises people, so let me defend it. The box has 1 GB of RAM, and it is shared: it also serves my portfolio site. On a machine that small, a container runtime is pure overhead for no benefit I would actually use. The app is already a single static Go binary. It already runs under systemd with a genuinely strict sandbox: no new privileges, a read-only filesystem, no access to devices, memory that cannot be both writable and executable. That is strong isolation at zero memory cost.

Docker earns its place for local development, where "works on my machine" is a real problem worth solving, and for the load-test stack in CI. It does not earn a daemon on my production box. So the image is a dev and CI artifact, and I wrote that decision down in an ADR instead of letting the plan file quietly imply otherwise. Which brings me to the AWS I did not build.

The Terraform I deleted before writing it

The original plan said Terraform modules for AWS: a VPC, ECS Fargate, RDS, a load balancer, the works. I had already decided weeks ago to deploy to a plain VPS instead, and it has worked great. So writing Terraform now would mean building infrastructure code for a cloud I will never provision, which then slowly drifts away from the real box because nobody runs it. That is worse than no code. It is code that lies.

Terraform is for creating cloud resources. I do not have cloud resources to create. I have one server that already exists and needs to be configured correctly and reproducibly. That is a different job, and it has a different tool.

Turning the runbook into a playbook

The tool for "configure an existing machine, the same way, every time" is Ansible. So I took my prose runbook, the one with the gaps, and rewrote it as an Ansible playbook: a set of roles that each describe one slice of the server as a desired state.

• base: the firewall (only SSH, HTTP, HTTPS open), fail2ban, automatic security updates.

• postgres: Postgres 16, a dedicated database and role, the memory tuning for a 1 GB box, and a daily backup.

• app: the two users, the app directory, the root-owned environment file, and a sudoers rule that lets the deploy user restart exactly one service and nothing else.

• systemd: the hardened service unit.

• nginx: the site config that proxies to the app.

• tls: certbot for the certificate.

The difference between the text file and the playbook is not just format. A runbook is a list of commands I ran once. A playbook is a description of how the box should be, that I can run again and again, and each run only changes what has drifted. Run it on a fresh machine and it builds the whole thing. Run it on the live box and it reports "nothing to change," which is its own kind of proof that the code and reality agree.

There is one detail I am a little proud of, because it is the kind of thing a runbook cannot capture but code can enforce.

The safeguard that a text file could never hold

Remember that the box is shared. My ledger and my portfolio live on the same nginx. And the portfolio uses HSTS preload, which is a browser rule that says, roughly: every subdomain here must always serve valid HTTPS, forever, no exceptions, no "click to proceed anyway." Which means if I ever push a broken nginx config while fiddling with the ledger's site, I do not just break the ledger. I can take down the portfolio too, with no override button for visitors.

A prose runbook can say "be careful with nginx." Code can actually be careful. So the nginx role does this:

   write new vhost to sites-available (never overwrite the cert block)
        |
        v
   link it into sites-enabled
        |
        v
   run `nginx -t`  (validate the WHOLE config)
        |
   +----+-----------------------------+
   | passes                           | fails
   v                                  v
   reload nginx (graceful, not        remove the link again,
   a restart, so live connections     abort with a clear error,
   are never dropped)                 leaving the working config live
        |
        v
   assert the portfolio still returns 200

If the new config does not validate, the playbook tears its own change back out and stops, so the box is never left holding a broken config that some later reload would activate. And after every change, it makes an HTTP request to the portfolio and refuses to call itself successful unless that other, unrelated site is still answering 200. The reload is graceful, never a restart, because a restart drops in-flight connections. These are the safeguards I would tell a careful human to remember. Now they are not something to remember. They just happen.

Keeping the secrets out of a public repo

One catch: the repo is public, and a playbook that configures my server naturally wants to know my server's address, my database password, and my deploy key. None of that can be committed.

So the split is simple. The roles and the playbook are generic and tracked: they describe the shape of the box with no secrets in them. The actual inventory (the server's address) and the vault (the password, the key) are gitignored, exactly like the runbook always was. What gets committed are example files that show the shape without the contents. Anyone can read exactly how my box is built. Nobody can read where it is or how to log in.

What this week was really about

The Docker image is nice. The one-command local stack is genuinely useful. But the thing that changed how I feel about this project is smaller and quieter: the server is no longer a thing that only exists as a running process and a half-remembered evening. It is described, in code, in the repo, in a form I could hand to a stranger, or to myself at 2am after the box died.

That is the actual product of infrastructure-as-code. Not automation for its own sake. It is that the knowledge stops living in one person's head, where it decays, and starts living in a file, where it does not.

The repo, playbook and all, is at github.com/sohag-pro/go-ledger, and the reasoning is in ADR-013. Next week is the last real build week: tightening the deploy pipeline and the observability around it, before the launch-and-retrospective finale.