Search for a command to run...
No SSH, no checklist, no remembering the one command I always forget. Just git push, and production updates itself. This is exactly how that pipeline is wired, and how you can build the same one for a plain server.
WhoAmI => notes.sohag.pro/author
No comments yet. Be the first to comment.
Kicking off a 12-week build of an open-source, production-grade payment ledger in Go: double-entry accounting, Postgres, gRPC, OpenTelemetry, and a real deployment.
You send 100 dollars, your friend abroad receives 92 euros, and a few cents seem to vanish into thin air. This week I built the currency exchange machinery that makes those cents go exactly where they should, and never anywhere else.
I had a solid list of reasons my life wasn't moving faster, until a grainy old lecture pointed out the one name missing from it
The finale isn't a victory lap. It's the story of the control I shipped that did nothing, the footgun still sitting in my demo, and the handful of things I'd keep exactly as they are.
How do you hold a large payment for a second pair of eyes without ever letting the unapproved money touch a balance, and how do you stream that decision to the outside world without standing up a broker?
How do you show the total under a parent account when the whole system refuses to store a balance? A recursive query, a trigger that refuses to draw a circle, and a rule about what actually has to sum to zero.
Every time I wanted to change an FX rate I had to edit a file on the server and restart the app. So I moved rates and markup into a live admin API, and then audited it hard enough to find the bug that quietly undid the whole thing.
I was sitting in a cafe last week, fixed a small thing in the ledger, and typed git push. Then I closed my laptop and finished my coffee. Somewhere in that couple of minutes, without me touching anything else, my tests ran, a fresh binary got built, it was copied to my server, swapped into place, the service restarted, and the live site was checked to make sure it came back healthy. By the time I looked again, the fix was in production.
That is the whole promise of a deploy pipeline: the boring, error-prone, easy-to-forget steps between "I wrote the code" and "it is running for real" happen the same way every single time, because a machine does them, not a tired human at 11pm.
People assume this needs a big cloud setup. It does not. My whole thing is GitHub Actions and one small VPS. No Kubernetes, no container registry, no cloud console. This is a special edition, a bonus post between the numbered weeks, walking through every step of that pipeline, and then how you can build the same one.
The first thing to understand is that the same workflow file does two different jobs depending on how the code arrived. Open a pull request, and it only checks. Push to main, and it checks and ships.
open a pull request push to main
------------------- ------------
lint lint
test + coverage gate test + coverage gate
smoke load test smoke load test
(that is all: prove it is good) coverage badge
DEPLOY <-- the extra part
So the pull request is a safety net: nothing merges unless the checks are green, but nothing ships either. The deploy only happens on the real thing, a push to main, and only after the checks that matter pass. That guard is written right into the job: the deploy needs: [lint, test] and only runs if the event is a push to the main branch. A red test is not a warning you can click past. It is a locked door.
Three jobs run in parallel to prove the code is good:
Lint runs golangci-lint, pinned to an exact version so the rules do not silently change under me.
Test runs the whole suite with the race detector on, then enforces a coverage gate. The gate is not "some coverage exists," it is a real floor, and generated code is stripped out first so the number reflects the code I actually wrote.
Smoke load boots the full stack with Docker Compose, waits for it to be healthy, installs k6, and fires a short burst of real traffic at it. This catches the class of bug that only shows up under concurrent load, not in a single unit test.
These run on every pull request and every push. They are the "is this code safe" half. Now the interesting half.
Here is what actually happens when the deploy job runs. It is deliberately simple, and every step is defensive.
GitHub Actions runner my VPS (/opt/go-ledger)
--------------------- -----------------------
1. build static linux binary
(CGO off, stripped, ~10 MB)
|
2. write the SSH key + pinned
host key from secrets
|
3. scp binary -------------------------> go-ledger.new
|
4. ssh in and run the swap: -----------> mv go-ledger go-ledger.prev
mv go-ledger.new go-ledger
sudo systemctl restart go-ledger
|
5. curl https://go.sohag.pro/healthz
(retries; fails loud if the app
does not come back)
Walk through why each step is shaped the way it is.
Build in CI, not on the server. The runner compiles a static linux/amd64 binary with cgo disabled and the debug symbols stripped. My server never needs a Go toolchain, a compiler, or git. It only ever receives a finished binary. That keeps the box tiny and its attack surface small.
Ship the new binary next to the old one, do not overwrite. The binary lands as go-ledger.new, not on top of the running go-ledger. You cannot cleanly overwrite a file that is currently executing anyway, and staging it separately means the swap can be a single, fast rename.
The swap keeps the previous binary. This is the part I like most:
mv go-ledger go-ledger.prev # keep the version that currently works
mv go-ledger.new go-ledger # promote the new one
sudo systemctl restart go-ledger # start it
If a deploy ever ships something broken, the last known-good binary is sitting right there as go-ledger.prev. Rolling back is copying one file over another and restarting, no rebuild, no waiting for CI. That is a thirty-second recovery instead of a panic.
The restart uses one exact, restricted command. The deploy user cannot do anything on that box except run sudo systemctl restart go-ledger, and nothing else. If the CI key ever leaked, the worst someone could do is restart my one service. They cannot read files, install packages, or touch the other site on the box. That restriction is a single sudoers line, and it is the difference between "a leaked key is annoying" and "a leaked key is a breach."
The last step is proof, not hope. After the restart the runner curls the real public health endpoint, with retries, and the whole run fails loudly if the app does not come back. A deploy that leaves the site down is a failed deploy, and I hear about it, rather than finding out from a user.
Two things here are quietly important.
The SSH connection pins the server's host key. GitHub does not just trust whatever server answers on that address; it is given the exact expected host key up front, and if the machine that answers presents a different one, the connection fails instead of being silently accepted. So a man-in-the-middle between GitHub and my VPS breaks the deploy rather than hijacking it. (I care about this one personally: earlier in this project my laptop's known-hosts file got overwritten with a fake key mid-session, so "pin the key" is not theoretical to me.)
And every sensitive value (the private key, the host, the user, the pinned host key) lives in GitHub Actions secrets, never in the repo. The workflow file is public. The credentials it uses are not.
You do not need my code for this. The pattern is four ingredients, and it works for any single-binary service on any plain Linux box.
1. A locked-down deploy user on the server. Create a user that exists only to receive deploys. Give it key-only SSH (no password), and a single sudoers rule letting it restart exactly your one service:
deploy ALL=(root) NOPASSWD: /usr/bin/systemctl restart your-service
That is the whole privilege it gets.
2. A dedicated SSH keypair. Generate a keypair used only for deploys. The public half goes in that deploy user's authorized_keys; the private half goes into a GitHub secret. It is not your personal key.
3. Four secrets in the GitHub repo. The private key, the server host, the deploy username, and the server's pinned host key (grab it with ssh-keyscan your-host). The workflow reads these; nobody reading your public repo sees them.
4. The workflow itself. One job that builds your binary, one that, only on a push to your main branch and only after tests pass, copies the binary over, swaps it with a keep-the-previous rename, restarts via that restricted sudo, and health-checks the result. The shape is exactly the diagram above.
That is it. There is no step five. The thing that makes people reach for heavier tooling, "how do I get my code onto a server safely and repeatably," is genuinely solved by scp, a rename, a restricted restart, and a health check, as long as you are disciplined about the key and the rollback.
I want to be clear that this is not a toy version of a "real" pipeline that I will replace later. For one service on one box, this is the real pipeline. Container registries, orchestrators, and blue-green rollouts solve problems I do not have yet: fleets of servers, zero-downtime at scale, many teams shipping at once. Adding them now would be cost with no benefit, the same reasoning that kept my production box a bare binary instead of a container.
The test is not "does it look like what a big company runs." The test is "when I push a fix from a cafe, does it reach production, safely, without me thinking about it." This does. That is the entire job.
The full workflow is in the repo at github.com/sohag-pro/go-ledger, in .github/workflows/deploy.yml, and the reasoning behind the VPS choice is in ADR-013. The numbered series continues with the last build week and the launch.