Linux Server Hardening

Last update:

Most of the time I enjoy System Administration tasks, but there is a number of steps that I take on every new instance that I create that are boring repetitive and easy to get wrong. It usually takes me a few minutes to recollect all commands, then a few more minutes to make sure that I'm doing it right. This guide is for the future me and anyone who may come here.

These are the very steps that I used on the server hosting this blog.

Create new Users

When you start an instance on providers like Linode or DigitalOcean, you usually end up with a very unsecure server. Usually these have the root account enabled, or maybe root is the only account available. No firewall, no SSH keypair setup.

The first thing that you want when configuring a brand new Linux Server is to get rid of the root user as soon as possible. AWS does a better job at this, providing you with a non-root user for all new instances, accepting only password-less login.

I usually create two users: one to be used to perform administration tasks, like installing packages and updates, setting up system services and crontabs, the other only needs to have access to its home directory, to host and run the actual application server and is unable to use sudo.

As the root user, execute:

useradd -m -s $(which bash) admin
useradd -m -s $(which bash) app

This will create two users named admin and app, their home directories (/home/admin and /home/app) and set bash as the default shell for both.

To allow the admin user to perform operations as root, we have to change the /etc/sudoers file. Open it with visudo and add this line at the bottom:


If that looks like magic, as many things in the IT, it's just a matter of understanding the syntax. Each line in the sudoers file is defined as:

User Host=(Otherusers) [Options] Commands

It means that User must be allowed to execute Commands on Host, changing its user to one of Otherusers. So for example:

deploy localhost=(root) /bin/ls

This means that deploy can execute ls as root on localhost. All placeholders in the line can be replaced with ALL, so it's easy to understand where we're going. The NOPASSWD: option means that the user is not required to type the password to change it's identity.

SSH Keys

The second most important step that you don't want to forget is to configure the authorized_keys file for your new users. This file lists the public keys that that allows someone to ssh as that user. First, if you don't have any, create your ssh key (on your development machine, not on the server):

ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa [-N password]

Two things to note here:

  • You may want to encrypt your private key with a password, which is great, but it may make your life harder when using automated tools.
  • It is nowadays recommended to use Ed25519 instead of RSA, which is supposed to provide superior security with shorter keys. Being a relatively new format, not every tool supports that kind of keys yet. I learned this the hard way, when after spending hours trying to setup a cluster of servers I couldn't find the reason why capistrano refused to deploy, and it was because it only supports RSA keys.

Now that you have a shiny SSH key, create the authorized_keys file on the server for both new users (still as root):

mkdir /home/{admin,app}/.ssh
touch /home/{admin,app}/.ssh/authorized_keys
chown -R admin: /home/admin/.ssh    # admin is owner of its .ssh directory 
chown -R app: /home/app/.ssh        # app is owner of its .ssh directory       

On your development machine you will have an ~/.ssh/ file. Take its content and paste it into both `authorizedkeys` files.

We're done for this first part. Double check that you can actually login with both users:

ssh admin@my.ip.adrr.ess
ssh app@my.ip.adrr.ess

And most importantly that you can execute sudo as admin.

Disable root access and password logins

If everything looks correct, it's now time to prevent SSH access to the root user. Using the newly created admin user, edit the SSH server configuration:

sudo vi /etc/ssh/sshd_config

Ensure that the following settings are present and uncommented (can appear anywhere in the file):

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

The last two lines ensure that you cannot use SSH with a password and that authentication using the SSH key has been enabled. Time to restart the SSH daemon. Depending on your system, this may be different:

sudo service sshd reload

Before exiting the shell, try to login again on another terminal. If you leave this shell and you got something wrong, you're locked out of your own server.

Use a firewall

If you are lucky like me to have an Ubuntu server you should have ufw installed. Ufw is just a very simple wrapper around iptables, and if you cannot use it, you may have to learn to use iptables.

A firewall will make sure that most ports on your Server will appear closed to the scary internet, with a few, chosen, exceptions. First of all you want to make sure that you can still access SSH.

sudo ufw allow ssh

Ufw is smart enough to understand that when you write ssh, what you really mean is port 22. Watch out: enabling ssh on the firewall doesn't mean that anyone can access your server. It only means that anyone can try to access your server, but only those with the correct SSH key will succeed. You may want to further limit access by limiting the range of IPs that can connect to port 22 (make sure that your IP is static!):

sudo ufw allow ssh from my.static.ip.address
You made it this far! If you are enjoying the article, do not miss out the next!
* I promise to keep your email safe and to not send spam

Install Fail2Ban

With all these precautions in place, you are pretty much safe from intruders. This doesn't mean that attackers will not try. In 5 days of this server existence I have already received 3 attacks, and it's probably only going to get worse and worse. Making sure that you keep your system up to date with latest security patches is another important step.

Fail2Ban is a nice piece of software that will scan your log files looking for a number of failed login attempts in a short period of time and will automatically blacklist IPs and simply drop off their connection on subsequent attempts.

After a defined amount of time, the IPs will be "freed" and the same process will repeat. This is to make sure that if it was you trying to authenticate, you are not completely cut out of your machine.

sudo apt install fail2ban

Fail2Ban has many features and can inspect much more than SSH logs. While the default values may work in most cases, make sure to read the documentation to exploit its full powers.

Change the SSH port

Another effective solution in reducing the number of attacks to your server is using a non standard SSH port. You see, when someone wants to attack your server, the first thing he/she will try is to hammer on port 22, where SSH is usually located. When facing a closed port 22, most attackers will just change target instead of sniffing all possible port number for an SSH.

Besides, even if the hacker was to find your alternative SSH port, that would still mean that you are not a simple, naive target, because you took your time in hardening your machine, making you even less valuable in his/her eyes.

For the final port number you can use anything between 1024 and 65535, just make sure to not collide with any other services you have installed like MySQL (3306), PostgreSQL (5432) or really anything else.

Open your ssh configuration file again:

sudo vi /etc/ssh/sshd_config

Find the line that says Port 22, which can be commented - in that case, uncomment it -, and change it to your new port. Commonly used alternatives are 2222, 2022, 2200, but a random number like 37194 is more appropriate.

Very important step: if you enabled ufw, or any other firewall, make sure to allow the new SSH port, or you will be locked out again:

sudo ufw allow 37194   # change with your port number
sudo ufw reload

At this point you can reload the SSH daemon:

sudo service sshd reload

You may be disconnected from the remote session, which is a good time to test whether the new configuration is in place:

ssh -p 37194 admin@my.server.ip.address

If everything is correct, you may also remove the old port 22 rule from the firewall:

sudo ufw deny 22


There is much more that I would like to write here, but my fingers hurt... I'm still adapting to write all of this words in a single day, but I promise that I'll come back and complete this.

In the mean time, is there anything that I missed? I'd really like to know your opinions and concerns. I'm talking with you, IndieWeb people!