Field Notes · Operations
In Praise of the Boring Deploy
There is a certain kind of engineer who wants deployment to be exciting. I am not that engineer. The proudest deploys of my career are the ones nobody noticed: no war room, no held breath, no message in the channel that begins with "so, small issue."
A boring deploy is not an accident. It is the sum of a few unglamorous habits.
Ship the same way every time
If deploying is a sequence of steps a human remembers, then deploying is a sequence of steps a human will eventually forget. Put it in a pipeline. The pipeline runs the tests, builds the artifact, and rolls it out identically whether it is a Tuesday afternoon or a Friday you should not be deploying on anyway.
Make rolling back cheaper than rolling forward
When something does go wrong, and it will, the question that matters is how fast you can get back to safety. If your answer involves a hotfix, a review, and a fresh build, your answer is too slow.
- Keep the previous version one command away.
- Make config changes reversible, not just code.
- Practice the rollback before you need it.
Watch the right things
A green deploy that quietly doubles your error rate is not a success; it is a delayed incident. The deploy is not done when the new version is running. It is done when the graphs say the new version is behaving.
When systems fall, customers notice. When they don't, nobody does, and that, quietly, is the work.
None of this is clever. That is the point. Cleverness is a cost you pay again every time you deploy. Boring is a cost you pay once.
Also in the notebook
Building a Tunnel I Didn't Need
I use ngrok and I like it. I built my own tunnel anyway, to find out what it was doing underneath, and most of what I learned came from the parts that broke.
Do You Use Kafka, or Just Pay for It?
Most teams reach for a distributed log and then use it as a work queue. Here are the questions that tell you which one you actually have, before the invoice does.