The first round of server security is about keeping people out: key-only SSH, a non-root user, automatic updates and a firewall. Do that first; this guide assumes it is done.
The second round assumes the first one failed. If an attacker gets code execution inside your application, what can they reach? On a default setup the answer is usually “most of the machine”, and it does not have to be.
What this does not do. None of it fixes a vulnerable application — an SQL injection that reads your customer table reads it just as well from inside a sandbox, because the application is supposed to be able to reach the database. This is damage limitation for the step after a compromise: stopping a foothold in one service from becoming the whole machine. It is also not a substitute for updates, which remain the single highest-value thing you can do. And it will break things the first time you apply it, which is why every step below has a way to check.
Step 1: give the service its own user, and nothing else
A service account should not be able to log in, should own only what it needs to write, and should not be in any group that grants power elsewhere.
sudo useradd --system --no-create-home --shell /usr/sbin/nologin myapp
sudo chown -R root:myapp /opt/myapp # app can read, not modify itself
sudo chmod -R 750 /opt/myapp
sudo install -d -o myapp -g myapp -m 750 /var/lib/myapp # the one writable placeThe ownership pattern matters: the service should not own its own code. If it does, anyone who compromises it can modify the application and survive a restart. Root owns the files, the service group reads them.
Verify: sudo -u myapp touch /opt/myapp/test fails with permission denied, and id myapp shows no supplementary groups you did not intend — docker in particular is equivalent to root.
Step 2: use the sandboxing you already have
This is the part most people never touch. systemd can confine a service using the same kernel features containers use — namespaces, capability bounding, seccomp — without any container at all. It is a dozen lines in the unit file.
# /etc/systemd/system/myapp.service
[Service]
User=myapp
Group=myapp
ExecStart=/opt/myapp/bin/server
# filesystem
ProtectSystem=strict # everything read-only except...
ProtectHome=true # /home, /root, /run/user invisible
PrivateTmp=true # its own /tmp, gone at stop
ReadWritePaths=/var/lib/myapp # ...this
# kernel and devices
PrivateDevices=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
ProtectControlGroups=true
ProtectClock=true
# privileges
NoNewPrivileges=true
CapabilityBoundingSet=
RestrictSUIDSGID=true
LockPersonality=true
MemoryDenyWriteExecute=true
RestrictRealtime=true
RestrictNamespaces=true
# network
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
IPAddressDeny=any
IPAddressAllow=localhost
# syscalls
SystemCallFilter=@system-service
SystemCallArchitectures=nativeThe two doing the most work are ProtectSystem=strict, which makes the entire filesystem read-only except what you name in ReadWritePaths, and CapabilityBoundingSet= (empty), which removes every capability the process could otherwise acquire.
MemoryDenyWriteExecute=true blocks a common exploitation technique, and breaks anything with a JIT — most JVM, .NET and JavaScript runtimes. Leave it out for those rather than wondering why the service will not start.
If the service needs to bind a port below 1024, do not give it root. Give it the one capability:
AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICEVerify: systemd-analyze security myapp.service scores the unit from 0 to 10 and lists exactly what is still exposed. A default unit scores around 9 (“UNSAFE”); the block above should land under 3. Work down the list it prints — it is the best security checklist on the machine, and it is generated from your actual configuration rather than from an article.
Step 3: make updates happen without you
Unattended security updates are worth more than everything else on this page combined, because the overwhelming majority of compromises are of known vulnerabilities with patches available.
# Debian and Ubuntu
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
# Fedora, Rocky, Alma
sudo dnf install dnf-automatic
sudo systemctl enable --now dnf-automatic.timerOn the Fedora side, two settings need changing before that timer is worth enabling. dnf-automatic ships with apply_updates = no and upgrade_type = default, so enabling the timer as it stands downloads packages nightly and installs none of them — while flipping only apply_updates would install everything rather than security fixes. Set both:
# /etc/dnf/automatic.conf
[commands]
upgrade_type = security # shipped default: default — meaning everything
apply_updates = yes # shipped default: no — meaning nothingThen handle the part people skip: a patched library does not protect a process that is still running the old one. Install needrestart so you find out, and consider automatic reboots in a window you choose. Know what needrestart will not tell you, though. On a test machine that had just taken a real OpenSSL security upgrade it found five processes holding the deleted library and printed “No services need to be restarted.” — because the report it produces is a list of names, and it had not managed to name any of them. The Half of Patching That Happens After the File Is Correct has the one command that shows both of the lists the tool keeps, and what the difference between them means.
# /etc/apt/apt.conf.d/50unattended-upgrades
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";
Unattended-Upgrade::Mail "you@example.com";And remember that containers are not covered by any of this. Your host updates itself; the image you pinned three months ago does not. Something has to pull new images, and if nothing does, containerising a service made its patching worse rather than better.
Verify: sudo unattended-upgrades --dry-run --debug shows what it would install — but not what it will skip, and it says so itself: The list of kept packages can't be calculated in dry-run mode. The kept packages are the interesting ones. /var/log/unattended-upgrades/ is weaker evidence than it looks, too, because All upgrades installed is logged when there was nothing to install in the first place. The check worth doing is apt list --upgradable a week later: if that count is not falling, something is being held back, and neither the dry run nor the log will have told you.
Step 4: rate-limit at the proxy, not in the application
Credential stuffing and scraping are handled far more cheaply one layer out, before a request costs you a database query.
# nginx: 10 requests/second per IP, allowing short bursts
limit_req_zone $binary_remote_addr zone=general:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
server {
location / {
limit_req zone=general burst=20 nodelay;
proxy_pass http://127.0.0.1:8080;
}
location /login {
limit_req zone=login burst=5;
proxy_pass http://127.0.0.1:8080;
}
}A tighter limit on the login path than on everything else is the shape you want — a legitimate user logs in once, a script tries thousands of times.
The trap: if your proxy sits behind another proxy or a CDN, $binary_remote_addr is the upstream’s address, and you will rate-limit everyone as though they were one person. Configure the real client address properly first, and while you are there, only trust the forwarded header from addresses you control — otherwise anyone can set it to anything.
Verify: for i in $(seq 1 50); do curl -s -o /dev/null -w "%{http_code}\n" https://example.com/login; done should start returning 503. Then check the access log shows your real address rather than 127.0.0.1.
Step 5: ban the obvious offenders
fail2ban watches logs and adds firewall rules for addresses that repeatedly fail. On a key-only SSH setup it stops very little that matters — the failures were never going to succeed — but it does cut the log noise substantially, which makes real events visible.
# /etc/fail2ban/jail.local
[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 5
ignoreip = 127.0.0.1/8 10.0.0.0/8
[sshd]
enabled = true
[nginx-http-auth]
enabled = truePut your own networks in ignoreip, and make sure you have another way into the machine — a console through your provider, or a second address. Banning yourself is the standard fail2ban experience, and it is much less funny without console access.
Verify: sudo fail2ban-client status sshd shows the jail active and lists banned addresses after a day. sudo fail2ban-client set sshd unbanip 1.2.3.4 is the command to remember before you need it.
Step 6: reduce what is running at all
Every listening port is a way in, and servers accumulate them.
sudo ss -tulpn | grep LISTEN # what is listening, and what owns it
systemctl list-units --type=service --state=running
systemd-analyze blame | head -20 # often reveals things you forgotAnything listening on 0.0.0.0 that does not need to be should be bound to 127.0.0.1 instead. Databases especially: a database reachable from the internet is a database that will be found, and “but it has a password” is how most of the ransom notes start.
Remember the Docker exception: a published container port bypasses ufw and firewalld entirely. ss tells you the truth; the firewall’s own status does not.
Verify: from another machine, nmap -Pn your.server.address. The open ports should be exactly the ones you meant. Scan your own server only.
Step 7: know when something changes
Hardening without noticing is only half of it. Three cheap signals:
- Successful SSH logins — alert on every one. On a server nobody logs into daily, that alert is nearly always you, and the one time it is not, you want to know within seconds.
- Service restarts you did not perform — a crash loop is often the first visible sign of something being probed.
- Outbound connections — a compromised service usually needs to phone home.
IPAddressAllowabove is one way to stop it; noticing is the other.
journalctl -u ssh --since today | grep 'Accepted'
journalctl -p err -b --no-pager | tail -20
sudo lsof -i -P -n | grep ESTABLISHED # who is talking to whom, right nowVerify: log in from somewhere unusual and confirm the alert reaches you. An untested alert is not an alert.
Before you call it done
- Service runs as its own user, with
nologin, owning only its data directory systemd-analyze securityscore under 4, and you understand what is left- Unattended upgrades installed and verified in the log a week later
- Something updating container images too, if you run any
- Rate limits at the proxy, tighter on login than elsewhere, with the real client IP correct
- fail2ban configured with your own networks excluded, and console access confirmed
- Nothing listening publicly that does not need to be — checked with
ss, not with the firewall’s status - An alert on successful SSH logins that you have tested
- Backups you have restored from — because the honest recovery from a compromise is rebuild and restore, not clean up
Related reading
- The Half of Patching That Happens After the File Is Correct — the long one, and the other half of the same afternoon: this page assumes the patch worked, and that one is about finding out whether it did
- Securing a new server — the first layer, which comes before this one
- Linux firewalls — including why Docker ignores yours
- systemd explained — what those unit directives are doing
- Users, groups and root — the memberships that quietly grant power
- lsof and fuser — finding what is actually listening