A new server on a public IP starts getting login attempts within minutes. Not because anyone has noticed you — because the entire IPv4 space is scanned continuously, and an unhardened SSH port is found automatically.

The good news is that almost every real compromise of a small server comes down to one of four things: a password that could be guessed, an unpatched service, an exposed port that should never have been open, or credentials left in a file. The first four steps below close all four. Everything after them is genuine improvement with sharply diminishing returns.

Commands are for Ubuntu and Debian. On RHEL, AlmaLinux or Rocky, substitute dnf for apt and firewalld for ufw.

Before you start: open a second SSH session and leave it connected. Several steps below can lock you out, and an open session is the difference between fixing a typo and filing a support ticket for console access.

Step 1 — SSH keys, and turn off passwords

This is the single highest-value change on the list. A password can be brute-forced; a 256-bit key cannot.

On your own machine:

bash
ssh-keygen -t ed25519 -C "you@example.com"
ssh-copy-id you@your-server-ip

Ed25519 rather than RSA — shorter, faster, and at least as strong.

Test that the key works before continuing. Open a new terminal and log in. If it asks for a password, stop and fix it, because the next command removes your other way in.

Then edit /etc/ssh/sshd_config:

text
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
MaxAuthTries 3
bash
sudo sshd -t && sudo systemctl reload ssh

The sshd -t is not optional politeness — it validates the config, and reloading a broken sshd config on a remote machine is how people lose access to servers.

Create an ordinary user with sudo first, if you have not:

bash
sudo adduser deploy
sudo usermod -aG sudo deploy

Root login disabled plus keys only removes the entire category of brute-force attack. Every one of those thousands of daily attempts now fails at the first step.

Step 2 — A firewall that denies by default

The rule is: deny everything, then allow the ports you actually serve. Not the other way round.

bash
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

Allow SSH before you enable it. Enabling a default-deny firewall without an SSH rule disconnects you immediately, and the server is then unreachable.

Then check what is actually listening:

bash
sudo ss -tulpn | grep LISTEN

Every line is a service reachable by someone. Databases are the usual finding — MySQL and PostgreSQL should be bound to 127.0.0.1, not 0.0.0.0. An exposed database with a weak password is the second most common way a small server is lost, and it does not require anyone to break anything.

If you can restrict SSH to a known IP range, do — it is the strongest single control available, and it makes everything in step 4 redundant.

Step 3 — Automatic security updates

Unpatched software is how servers get compromised without anyone attacking you specifically. A public vulnerability plus a scan is enough.

bash
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Confirm in /etc/apt/apt.conf.d/50unattended-upgrades that the security origin is enabled. Set:

text
Unattended-Upgrade::Automatic-Reboot "false";

unless you genuinely want unscheduled reboots. Then track pending kernel updates yourself — needrestart will tell you when a reboot is actually required.

Security updates only, automatically; feature updates on your own schedule. Automating everything is how a minor version bump takes down production at 3am. If none of this sounds like something you want to own, or take a managed VPS and let somebody else own the patching.

Step 4 — Fail2ban

With key-only SSH, brute force cannot succeed — but it still costs you log noise and CPU. Fail2ban watches the logs and bans repeat offenders at the firewall.

bash
sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit jail.local:

ini
[DEFAULT]
bantime  = 1h
findtime = 10m
maxretry = 5

[sshd]
enabled = true

[recidive]
enabled  = true
bantime  = 1w
findtime = 1d

Always edit jail.local, never jail.conf — the latter is replaced on package upgrade.

The recidive jail is the one most guides omit and the one that does the most work: it watches Fail2ban's own log and applies a week-long ban to anything that keeps coming back. Enable jails for every internet-facing service, not just SSH — the web server and mail matter too.

bash
sudo fail2ban-client status sshd

Step 5 — Backups you have actually restored

This is not hardening, and it is the step that decides how bad your worst day is. A ransomware event or a bad rm is survivable with backups and terminal without them.

Three rules:

  1. Off the machine. A backup on the server it backs up dies with it.
  2. Versioned. If your backup mirrors the current state, it faithfully mirrors the encrypted state too, an hour after the incident.
  3. Tested. An untested backup is a belief, not a backup.

restic or borg for versioned, deduplicated, encrypted backups to object storage. Whatever you choose, put a calendar reminder to restore one to a scratch server every quarter. The failure mode is not backups that do not run — it is backups that run for two years into a bucket nobody can decrypt.

Step 6 — Reduce what is running

Every installed service is a way in. On a web server, ask what each of these is for and remove what is not needed: a mail transfer agent listening publicly, an FTP daemon, a DNS resolver open to the internet, a stale database from a project two years ago, the sample apps some panels install.

bash
sudo systemctl list-units --type=service --state=running

The most secure service is one that is not installed.

Step 7 — Web server and TLS

  • TLS 1.2 and 1.3 only. Everything below is deprecated and exploitable.
  • HSTS once you are confident https works everywhere — it is hard to undo, so be sure first.
  • Remove version banners. ServerTokens Prod on Apache, server_tokens off on Nginx. Minor, but it removes you from searches for a specific vulnerable version.
  • Rate-limit login endpoints at the web-server layer, not just in the application.
  • Set the security headers: X-Content-Type-Options, X-Frame-Options, Referrer-Policy, and a Content-Security-Policy if the app can take one.

Step 8 — Permissions and secrets

The finding that shows up in almost every audit is a .env file readable by the web server, or worse, reachable over http.

  • Application files owned by a deploy user, not by the web server user. The web server needs read; it does not need write on its own code.
  • Secrets at 600, owned by the user that reads them.
  • Verify from outside that config files are not served:
bash
curl -sI https://yoursite.com/.env
curl -sI https://yoursite.com/.git/config

Both should be 404 or 403. An exposed .git directory hands over your entire source history, credentials included, and it is astonishingly common.

  • No shared logins. One account per person, keys per person. When somebody leaves you remove a key, not change a password everyone knows.

Step 9 — Know when something changes

Prevention fails eventually. Detection is what shortens the gap between compromise and discovery, which is usually measured in months.

  • Uptime and certificate-expiry monitoring, external to the server.
  • Log shipping off the box. An attacker's first move is the local logs.
  • File integrity monitoring — AIDE or Tripwire — on a web server, where the usual symptom is new PHP files appearing in an uploads directory.
  • An alert on disk filling. Unglamorous, and it takes down more servers than attackers do.

The 15-minute version

If you do nothing else on a new server:

bash
# 1. non-root user with sudo
sudo adduser deploy && sudo usermod -aG sudo deploy

# 2. copy your key up, then TEST it in a second terminal
ssh-copy-id deploy@server

# 3. keys only, no root
sudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/'         /etc/ssh/sshd_config
sudo sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sshd -t && sudo systemctl reload ssh

# 4. default-deny firewall
sudo ufw default deny incoming && sudo ufw allow OpenSSH \
  && sudo ufw allow 80,443/tcp && sudo ufw enable

# 5. automatic security updates
sudo apt install -y unattended-upgrades fail2ban

# 6. what is still listening?
sudo ss -tulpn | grep LISTEN

That is the majority of the practical benefit, and it is a quarter of an hour.

What this does not cover

Be honest about the boundary. This hardens the server. It does nothing about:

  • Application vulnerabilities. An outdated WordPress plugin is exploited over port 443, which you have deliberately left open.
  • Supply chain. A compromised dependency runs with your application's permissions.
  • Stolen credentials. Keys taken from a developer's laptop are valid keys.

Server hardening is one layer. Keeping the application patched is another, and on a typical web server it is the layer that is actually breached. If maintaining both is not a good use of your time, or take a managed VPS and let somebody else own the patching.

Frequently asked questions

Should I change the SSH port from 22? It cuts log noise considerably and stops nothing determined — port scans find services regardless. Do it if the noise bothers you, but do not count it as a security control, and never do it instead of key-only authentication.

Is Fail2ban still worth it with key-only SSH? Yes, for two reasons: it cuts CPU and log volume from constant attempts, and its non-SSH jails protect the web and mail services, where credentials are still guessable.

How often should I patch? Security updates automatically, daily. Feature and kernel updates on a schedule you choose, with a reboot window. Watch needrestart so you know when a pending kernel update actually requires the reboot.

Do I need this on shared hosting? No — the host does it, and you cannot do most of it anyway. This applies from a VPS upwards. Which tier you need is a separate question: see shared vs VPS vs dedicated.

Is a managed server actually more secure? In practice, usually, because patching becomes somebody's job rather than somebody's intention. The most common cause of a compromised small-business server is an unmanaged VPS that was set up correctly and then never touched again.

What is the first thing to check if I think I have been compromised? Do not reboot — it destroys evidence in memory. Check last and lastb for logins, ss -tulpn for unexpected listeners, and look for recently modified files (find /var/www -mtime -7). Then rebuild from a known-good backup rather than cleaning in place. You cannot prove you removed everything.