By the end of this you will have nightly snapshots that each look like a complete copy while costing almost nothing in disk space, running on a schedule, logged where you can see them, and — the part most guides skip — actually tested by restoring something.
The tools are rsync and a systemd timer. Nothing to install.
1. Decide what actually matters
Backing up everything is slow and makes restores harder to think about. On a typical server the list is short:
/home— or whichever directories hold work you cannot recreate/etc— configuration, small and enormously valuable when rebuilding/var/www,/srv— site content- Database dumps, not database files — see the warning further down
You do not need /usr, /bin or installed packages. Those come back with a package manager, and a list of what was installed is far smaller than the software itself:
dpkg --get-selections > /backup/packages.txt # Debian, Ubuntu
dnf repoquery --userinstalled > /backup/packages.txt # Fedora, RHEL2. Prepare the destination
A backup on the same disk as the original is not a backup. It protects against your mistakes and nothing else — not a failed drive, not a lost machine. Use a separate disk at minimum, and somewhere else entirely if the data matters.
lsblk -f # find the disk
sudo mkdir -p /backup
sudo mount /dev/sdb1 /backup # test it first
# Then make it permanent — by UUID, with nofail
UUID=3e7a9c12-... /backup ext4 defaults,nofail 0 2nofail matters here: without it, a backup drive that fails or is unplugged stops the machine booting. Disks and mounting covers this properly.
One requirement specific to this approach: snapshots and the destination must be on the same filesystem, because the space saving comes from hard links, which cannot cross filesystems. Keeping everything under /backup satisfies that.
3. The backup script
Save as /usr/local/bin/backup.sh:
#!/bin/bash
set -euo pipefail
SOURCES=(/home /etc /var/www)
DEST=/backup
DATE=$(date +%F_%H%M)
KEEP=14
# Refuse to run if the backup disk is not mounted
mountpoint -q "$DEST" || { echo "ERROR: $DEST is not mounted"; exit 1; }
echo "=== Backup started $(date) ==="
rsync -aH --delete \
--exclude='*.tmp' \
--exclude='.cache' \
--exclude='node_modules' \
--link-dest="$DEST/latest" \
"${SOURCES[@]}" "$DEST/$DATE/"
# Point 'latest' at this run
ln -sfn "$DEST/$DATE" "$DEST/latest"
# Prune old snapshots
cd "$DEST"
ls -1d 20* 2>/dev/null | sort -r | tail -n +$((KEEP + 1)) | while read -r old; do
echo "Removing old snapshot: $old"
rm -rf "${DEST:?}/$old"
done
echo "=== Backup finished $(date) ==="
df -h "$DEST" | tail -1sudo chmod 750 /usr/local/bin/backup.shSeveral details in there are deliberate:
-H— preserve hard links.-adoes not include it, and without it the snapshots stop sharing data. It is one of five things-aleaves out, and rsync says so itself:archive mode is -rlptgoD (no -A,-X,-U,-N,-H).--sparseis a sixth that is not even in that list. For the backup itself only-Hmatters; for the restore,-A,-Xand--sparseare the difference between getting your ACLs, extended attributes and file capabilities back and not — see Restoring a Linux Server. See also ln.--link-dest— the whole trick: unchanged files become hard links to the previous snapshot instead of new copies.ln -sfn— thenstops the symlink being created inside the directory it already points to.${DEST:?}— makes therm -rffail loudly if the variable is somehow empty, rather than deleting from the root.
Run it by hand once, with --dry-run added to the rsync line, and read the output before trusting it to a schedule.
4. Databases need dumps, not files
Worth its own section because it is the most common way a backup turns out to be worthless.
Copying a running database’s files with rsync gives you a snapshot of a moment nobody recorded. The files change while they are being read. A complete copy of a PostgreSQL data directory is usually crash-consistent: it replays its write-ahead log, logs database system is ready to accept connections, and serves a database that is entirely valid — just not the one that existed at any single instant, and two copies taken seconds apart come up at different log positions with both reporting success. The copy that genuinely will not start is the incomplete one an exclude list makes: drop pg_wal to save space and you get PANIC: could not locate a valid checkpoint record. SQLite in WAL mode, which most self-hosted applications run, is quieter still — back up app.db without its -wal sidecar and the restored file passes PRAGMA integrity_check having lost every transaction since the last checkpoint. So dump first and back up the dump, not because the copy will fail but because it will succeed:
# Add near the top of the script, before the rsync
mkdir -p /var/backups/db
mysqldump --single-transaction --all-databases \
| gzip > /var/backups/db/mysql-$(date +%F).sql.gz
sudo -u postgres pg_dumpall \
| gzip > /var/backups/db/postgres-$(date +%F).sql.gzThen include /var/backups/db in SOURCES. Credentials belong in ~/.my.cnf with mode 600, never on the command line where ps exposes them to every user on the machine.
5. Schedule it with a systemd timer
A timer over cron here for three concrete reasons: output goes to the journal automatically, a missed run can be caught up, and failures are visible in systemctl --failed. All three are keyed on the job exiting non-zero or not running at all, and none of them helps with the failure this guide is most exposed to — a backup that runs, exits 0, and copies nothing because a glob was empty or a source path moved. For that, the timestamp file in step 6 is not the simpler option; it is the only one.
/etc/systemd/system/backup.service:
[Unit]
Description=Nightly rsync snapshot backup
RequiresMountsFor=/backup
[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh
Nice=19
IOSchedulingClass=idle/etc/systemd/system/backup.timer:
[Unit]
Description=Run the backup nightly
[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=15m
Persistent=true
[Install]
WantedBy=timers.targetsudo systemctl daemon-reload
sudo systemctl enable --now backup.timer
systemctl list-timers backup.timer
sudo systemctl start backup.service # run it once, now, to test
journalctl -u backup.service -fEnable the timer, not the service. Enabling the service would try to run the backup at every boot. Persistent=true catches up one run missed while the machine was off — not one per missed run — and the Nice and IOSchedulingClass lines keep it from starving anything else; see processes. RequiresMountsFor is weaker than it sounds: it creates a dependency on whichever mount unit currently covers that path, so if nothing is mounted at /backup it resolves to -.mount, the root filesystem, which is always active, and the job runs normally — writing your backup to the system disk. The mountpoint -q check at the top of the script is not belt and braces; it is the only one of the two that does the job.
6. Notice when it breaks
A backup that has been failing quietly for six months is the worst outcome here — worse than none, because you thought you were covered.
journalctl -u backup.service --since "7 days ago"
systemctl --failed
ls -la /backup/ # do the dates look recent
du -sh /backup/* | tail -5 # do the sizes look saneFor an alert on failure, add a drop-in that runs on error:
# In backup.service
OnFailure=backup-failed@%n.service…with a matching backup-failed@.service that sends a mail or a webhook. That covers a run that fails. It does not cover a run that succeeds and copies nothing, and it cannot fire at all for a timer that never fired — so add the check that catches both: have the script write a timestamp file on success, and alert on its age rather than on any failure. find /backup/.last-success -mmin +1500 printing anything means last night did not happen, whatever the journal says. The Life of a Scheduled Job is why that check has to be the one you rely on.
7. Test the restore — this is the actual step
Everything above is preparation. A backup you have never restored from is a hypothesis, and the first restore is not the moment to discover that the excludes were too aggressive, the database dump was empty, or the permissions did not survive.
# Restore one file to a scratch location and compare
mkdir -p /tmp/restore-test
rsync -aH /backup/latest/home/kevin/notes.txt /tmp/restore-test/
diff /home/kevin/notes.txt /tmp/restore-test/notes.txt && echo "identical"
# Restore a whole directory somewhere safe
rsync -aH /backup/latest/var/www/ /tmp/restore-test/www/
# Check a database dump is real, not an empty file
zcat /backup/latest/var/backups/db/mysql-*.sql.gz | head -20
zcat /backup/latest/var/backups/db/mysql-*.sql.gz | wc -lThose checks answer a question about bytes, and most of what a restore loses is not a byte. Run them against a tree restored with -aH and diff -r exits 0 on a tree that has lost its ACL, its extended attribute and its file capability, and that has grown from 36K to 257M because the sparse files were written out in full. Nothing is corrupted — rsync -aH does not carry those things, and diff cannot see them. So compare with a tool that can, and restore with the flags that carry them:
# the same comparison, with a tool that can see attributes
rsync -n -i -aHAX --checksum /backup/latest/var/www/ /tmp/restore-test/www/
# and restore with the flags that carry them
rsync -aHAX --sparse /backup/latest/var/www/ /tmp/restore-test/www/-n changes nothing; read which letter appears in each line rather than how many lines there are, because each column is a different attribute with a different owner. And the letters are only six of the seven things a restore decides — the rest is the account that ran it, what was never a file, and who else on the network now thinks this machine is the machine it was restored from. That is a page of its own: Restoring a Linux Server, which is this step 7 taken all the way.
Note the trailing slash on the source in that second command — without it you get /tmp/restore-test/www/www/. Restoring over live data is where a slash mistake costs you something, so restore to a scratch directory and move things into place deliberately.
Put a restore test in the calendar every few months. It takes ten minutes and is the only thing that turns a backup into an actual guarantee.
8. Get a copy off the machine
Local snapshots handle deleted files and bad edits. They do nothing about fire, theft, a failed controller, or ransomware that encrypts every mounted disk. The usual guidance is 3-2-1: three copies, on two kinds of media, one of them somewhere else.
# Push last night's snapshot to another machine
rsync -aHP --delete /backup/latest/ backupserver:/remote/backup/thishost/Set that up with keys as described in ssh, and give the remote account only what it needs. For anything sensitive going to storage you do not control, restic or borg are the better answer — they add encryption, deduplication and integrity checking that plain rsync does not have.
Common mistakes
| Mistake | What happens |
|---|---|
| Backup on the same disk | Protects against nothing except your own rm |
| No mount check | The system disk quietly fills with backups of itself |
Forgetting -H | Every snapshot is a full copy; the disk fills in a fortnight |
--link-dest across filesystems | Hard links silently become copies |
Restoring with -aH rather than -aHAX | ACLs, extended attributes and file capabilities are dropped, and every command still exits 0. Restoring a Linux Server |
| Copying live database files | A complete copy restores perfectly — as a valid database at a moment nobody recorded. An incomplete one, made by an exclude list, is the one that will not start |
| Never testing a restore | You find out during the emergency |
| No monitoring | Six months of silent failure |
Only --delete, no snapshots | Yesterday’s accidental deletion is propagated tonight |
Related
- Restoring a Linux Server — the long one, and the other half of step 7: the seven stages between this backup and a working machine, and why
diffcannot see most of what a restore loses. - The Life of a Scheduled Job — the long one on step 5 and step 6: the eight stages between the timer you wrote and the person who needed last night’s snapshot, and why a backup that ran and copied nothing is green on every instrument here.
- rsync — the trailing slash,
--delete, and--link-destin detail. - systemctl — units, timers and reading the journal.
- cron — the alternative, and an honest comparison.
- Disks and mounting — mounting the backup disk so it cannot break the boot.
- ln — the hard links that make snapshots nearly free.
- Disk space — why
dureports odd figures across snapshots.