Guides

Thirty-three walkthroughs that finish the job. Every one has numbered steps with a way to check each has worked, says near the top what it does not achieve, and ends with a checklist you can run before you call it done.

Starting from nothing, with a new server? The order that works is: secure itclose the portspoint a domain at itput a proxy in frontrun your first applicationback it upfind out when it breaks. About a day, and everything after that is easier.

Nine of these go much deeper than the rest. Each follows one thing through its entire path — stage by stage, in the order it actually happens — shows the command that inspects each stage, ends with a worked diagnosis rather than a summary, and carries a table of the widely repeated advice on its subject that has quietly expired.

  • Running PostgreSQL Properly — the eight stages between a client sending COMMIT and data you could still restore a year later, and why the one fsync that matters is not the one you would guess
  • Kubernetes, Honestly — the seven stages between kubectl apply and a running container, and why nothing in the middle is ever sent anywhere
  • etcd, Honestly — the six stages between the member that answered and the restore that hands your data back on a clock that has been wound backwards
  • The Life of a Certificate — the six stages between an ACME account and a certificate your server is actually serving, and why a renewal that succeeds proves less than you think
  • The Half of Patching That Happens After the File Is Correct — the seven stages between a security advisory and a machine you can vouch for, and why the tool that is supposed to tell you what needs restarting says nothing about the processes it cannot name
  • The Life of a DNS Zone — the six stages between a change you made and a change that is over, and why four of the six clocks that decide when it is finished belong to somebody else
  • Building a Container Image — the seven stages between the file you wrote and an image somebody else has to run, and why a green build log is a statement about the builder rather than about the image
  • Restoring a Linux Server — the seven stages between the archive you kept and a working machine, and why almost everything a restore loses was in the backup all along
  • The Life of an Outbound Email — the nine stages between an application calling sendmail and a person deciding they wanted it, and why every mechanism that judges the message authenticates an address somebody else chose

They take half an hour rather than the usual ten minutes, and they are the ones to read when you want to stop guessing at a class of problem for good. There are twenty more of them in Learn, on the mechanisms underneath.

What are you trying to do?

If you want to…Read
Find your way around a terminal at allCommand Line Basics
Set up a brand new server without leaving it openSecuring a New Server
Find out why a fresh server is listening on port 25 on every interfaceSecuring a New Server
Close everything except the ports you meant to openLinux Firewalls
Give three people their own logins, and take one away laterSSH Access for a Small Team
Make a service harder to abuse once it is exposedHardening a Public-Facing Service
Find out which processes are still running the old library after a security updateThe Half of Patching That Happens After the File Is Correct
Work out why apt list --upgradable never reaches zeroThe Half of Patching That Happens After the File Is Correct
Prove to a scanner or an auditor that a fix is actually installedThe Half of Patching That Happens After the File Is Correct
Buy a domain and point it at your serverGetting a Domain and Its DNS Right
Work out why a DNS change has not taken effect yetThe Life of a DNS Zone
Change nameservers, or move DNS provider, without an outageThe Life of a DNS Zone
Work out why a domain is dead for some people and fine for othersThe Life of a DNS Zone
Get your first application online, end to endSelf-Hosting Your First Application
Run several things on one server, each with HTTPSA Reverse Proxy with Automatic TLS
Get a TLS certificate and keep it renewingThe Life of a Certificate
Work out why a renewal succeeded and the old certificate is still being servedThe Life of a Certificate
Work out why HTTPS works in a browser and fails from an API clientThe Life of a Certificate
Understand images, volumes and container networksDocker Basics
Build an image of your own, rather than adopt somebody else’sBuilding a Container Image
Work out why a build went green and the image is brokenBuilding a Container Image
Work out why a container will not start as the user your Dockerfile namesBuilding a Container Image
Speed up a build that rebuilds everything every timeBuilding a Container Image
Work out why docker history sizes do not add upBuilding a Container Image
Understand what actually happens when you run kubectl applyKubernetes, Honestly
Work out why a Pod is Pending, or running but getting no trafficKubernetes, Honestly
Work out why a node keeps going NotReady after an upgradeKubernetes, Honestly
Understand what etcd is doing underneath a control planeetcd, Honestly
Restore etcd without quietly breaking every controlleretcd, Honestly
Work out why etcd says its disk is slow when the disk is fineetcd, Honestly
Stop an etcd database growing until it refuses writesetcd, Honestly
Serve a website and stop guessing at the configNginx
Run PHP with a database behind itA LEMP Stack
Put a database somewhere it will not be found by scannersPutting a Database Somewhere Sensible
Run PostgreSQL so that you could still restore it in a yearRunning PostgreSQL Properly
Work out why a database stalls every few minutesRunning PostgreSQL Properly
Have somewhere to break things before customers see themA Staging Environment on One Server
Set up backups that run without youAutomated Backups
Be certain you can get your data backRestoring a Linux Server
Work out which of two identical disks your fstab is actually reachingThe Life of a Block Device
Clone a VM without the copy and the original fighting over one filesystem UUIDThe Life of a Block Device
Work out why a backup disk mounted cleanly and the data is staleThe Life of a Block Device
Restore a backup and find out what did not come backRestoring a Linux Server
Work out why a restored file has the right mode bits and still will not workRestoring a Linux Server
Work out why restored files are owned by a number instead of a nameRestoring a Linux Server
Work out why a restored directory is hundreds of times bigger than the originalRestoring a Linux Server
Restore a database copy that starts cleanly and is missing dataRestoring a Linux Server
Work out what two machines share after a restore that neither of them can seeRestoring a Linux Server
Find out something died before your users tell youMonitoring Without a Full Stack
Put something on a schedule and be able to tell later whether it ranThe Life of a Scheduled Job
Work out why a scheduled job exits 0 every night and produces nothingThe Life of a Scheduled Job
Work out why a timer is enabled, green, and has never once firedThe Life of a Scheduled Job
Work out why a script works when you run it and fails from a timerThe Life of a Scheduled Job
Work out why a job set to run every fifteen minutes runs every fortyThe Life of a Scheduled Job
Work out why systemctl list-timers does not list the timer you just wroteThe Life of a Scheduled Job
Work out why it diedReading Logs
Work out why the log does not say why it diedThe Life of a Log Line
Work out why journalctl -p err shows fewer errors on the day everything brokeThe Life of a Log Line
Work out why a stack trace arrives as eight separate log entriesThe Life of a Log Line
Work out why grep finds the start of a log line and not the endThe Life of a Log Line
Work out whether the program named on a log line actually wrote itThe Life of a Log Line
Work out why a log line is inside your time window and --since will not return itThe Life of a Log Line
Search logs from several machines in one placeCentralising Logs
Work out why the fields you filter on locally are not there after shippingThe Life of a Log Line
Send password resets that do not land in spamSending Email From Your Server
Work out why a message left your server, produced no error, and never arrivedThe Life of an Outbound Email
Work out why SPF, DKIM and DMARC all pass and mail still lands in spamThe Life of an Outbound Email
Work out why a message that passes SPF and DKIM is rejected for DMARCThe Life of an Outbound Email
Find out who actually receives the bounces for mail your application sendsThe Life of an Outbound Email
Work out why your mail broke the day it went through a mailing listThe Life of an Outbound Email
Work out why mail from your own server fails SPF for your own domainThe Life of an Outbound Email
Change relay providers without a week of mail going to spamThe Life of an Outbound Email
Move everything to a new machine without a gapServer Migration
Take something you installed with apt and containerise itMoving a Service Into a Container
Survive the machine itself dyingA Second Machine
Protect a laptop that might get stolenDisk Encryption on a Laptop
Keep files and media on a machine at homeA Home File and Media Server
Reach your own machines from anywhere, privatelyA WireGuard VPN, Set Up Properly

Start here

Securing it

  • Securing a New Server — key-only SSH, a non-root user, automatic security updates
  • Linux Firewalls — ufw, firewalld and nftables, and the Docker rule that bypasses all of them
  • SSH Access for a Small Team — individual logins, scoped sudo, and offboarding that actually removes access
  • Hardening a Public-Facing Service — the layer after the basics: systemd sandboxing, rate limiting, and assuming the first layer failed
  • The Half of Patching That Happens After the File Is Correct — the long one. All seven stages between a security advisory and a machine you can vouch for: the version number that was designed not to move, what the word security means to apt — which is a string in a Release file and not a property of a fix — the automation your cloud image is already running and what it is already not doing, the process still mapping the deleted library that the restart tool found and did not mention, what a restart does not restart and the code on the box no package owns, the reboot flag that is never written on most servers and what livepatching actually removes, and proving one CVE fixed offline from the changelog — plus the one-question test for any other page on the subject

Getting an application online

  • Getting a Domain and Its DNS Right — registrar versus nameservers, the records that matter, and the TTL that decides how a migration goes
  • The Life of a DNS Zone — the long one. All six stages between a change you made and a change that is over: the three things a registry actually stores and why your zone’s own copy of them is decorative, a serial that cannot be moved backwards and secondaries that will never converge if you try, three authoritative servers giving two different answers with no error anywhere, the DS record a stranger publishes about a key of yours that nothing ever checks, the six clocks of which only two are yours — and the section every other page on the subject stops one short of, which is the one that tells you when it is over
  • Self-Hosting Your First Application — the whole path: compose file, TLS, reboot, backups, alerts
  • A Reverse Proxy with Automatic TLS — several applications behind one domain, with certificates that renew themselves
  • Docker Basics — images, volumes and networks, without the hand-waving
  • Building a Container Image — the long one, and the first page here that builds an image rather than adopting one. All seven stages between the file you wrote and an image somebody else has to run: the builder that solves a graph rather than running your file, and the two stages of yours that never executed; the forty bytes it transferred where the old builder sent sixty-seven megabytes; the base image that chose your shell, your passwd file and your certificate store; sixteen instructions that produced five layers, three of them the same empty blob; a cache miss that does not cascade down the file; the USER string that four different parties may have to resolve, at four different times; the linter that exits 1 while your pipeline exits 0 on the same findings — and the handover to adopting an image, which is a harder problem than it looks
  • Kubernetes, Honestly — the long one. All seven stages between kubectl apply and a running container: the discovery cache that hides a resource for six hours, the API server’s real order of authentication, admission and validation, the controller pattern every operator you will ever meet also uses, the one field the scheduler writes, the kubelet and the cgroup driver it no longer takes from its own config, the condition that names the hand-off to the container runtime, and the EndpointSlice controller that decides when traffic arrives — plus the test that tells you whether any other page on the subject was written from a running cluster
  • etcd, Honestly — the long one, and the page the one above hands off to. All six stages between the member that answered your write and the restore that gives it back: which member you are actually talking to and why a follower answers anyway, the one fsync whose metric turns out not to measure the disk, the revision that gets issued and the three other numbers people mistake it for, what a linearizable read costs and what a serializable one hides, why compaction never shrinks the file and why deleting keys under a quota alarm makes it worse — and the restore that hands your data back on a clock that has been wound backwards, together with the two flags that stop it silently breaking every controller you have
  • Nginx — serving a site, virtual hosts and the configuration people get wrong
  • The Life of a Certificate — the long one. All six stages between an ACME account and a certificate your server is actually serving: the two private keys and which failures belong to which, the profile that now decides your certificate’s lifetime, the authorisation that outlives the challenge inside it, what the certificate authority discards from your request, the reload nobody tests and the one command that does test it, and who decides when your certificate should be replaced — plus why the dry run everybody recommends skips the exact step that breaks
  • A LEMP Stack — Nginx, MariaDB and PHP, including the socket paths and permissions

Data, and somewhere to test

  • Putting a Database Somewhere Sensible — PostgreSQL with a scoped role, no public port, and a dump you have restored from
  • Running PostgreSQL Properly — the long one. All eight stages between a client sending COMMIT and data you could still restore a year later: connections and pooling, shared buffers, the WAL and the full pages you did not expect, the one fsync the client actually waits for, archiving and replication slots, checkpoints, vacuum and transaction ID wraparound, and the restore itself — plus a table of the widely repeated tuning advice that has quietly expired
  • A Staging Environment on One Server — a copy you can break, that cannot email your customers

Keeping it running

  • Automated Backups — what to back up, how often, and testing the restore
  • Restoring a Linux Server — the long one, and the other half of the page above it. All seven stages between the archive and a working machine: what the archive actually recorded and what it left you to decide, the names in it against the numbers on the new box, the account that ran the restore and its umask, the metadata sitting beside the bytes, links and holes and the things that are not files at all, what a filesystem walk could never see, and the other copy that now exists — plus the one command that compares a restore against the thing it was restored from, and why diff exits 0 on a tree that has lost its ACLs, its extended attributes and its file capabilities
  • Monitoring Without a Full Observability Stack — knowing a service died before your users do
  • Reading Logs — journalctl, log files, and finding the line that matters
  • Centralising Logs — when one machine’s journal is no longer enough, and the label mistake that ruins it
  • Sending Email From Your Server — the relay, the three DNS records, and an honest account of running a real mail server
  • The Life of an Outbound Email — the long one, and the page above it read as a diagnosis rather than a build. All nine stages between an application calling sendmail and a person deciding they wanted it: the envelope sender your application never set and that appears in no header, the SPF check that authenticates a domain your relay owns, the three of ten DNS lookups your one include: has already spent, the mailing-list footer that breaks a signature where a whitespace change does not, the Subject: header that verifies as dkim=pass and is not the one displayed, a message that passes SPF and passes DKIM and is rejected for DMARC anyway, the 250 that means the receiver has taken responsibility and not that anybody will read it, the detailed bounce report delivered to a mailbox at your relay — and the one number that decides whether your next message arrives, which is a percentage of strangers pressing a button and lives in somebody else’s dashboard

Moving it, and surviving a failure

Your own machine, not a server

Where to go next

These pages tell you what to do. When you want to know why it works that way, Learn explains the mechanisms underneath — and Commands is the reference for every individual tool the steps above use.