Hardening an Ubuntu Server

Quick steps to secure your node.

Thank you for your support and kind messages! It really energizes us to keep creating the best crypto guides. Use cointr.ee to find our donation addresses and share your message. 🙏

🤖 Prerequisites

Ubuntu Server or Ubuntu Desktop installed
SSH server installed
a SSH client or terminal window access

In case you need to install SSH server, refer to:

How to install SSH server on Ubuntu

In case you need a SSH client for your operating system, refer to:

How to Connect to an SSH Server from Windows, macOS, or LinuxHow-To Geek

🧙‍♂️ Creating a Non-root User with sudo Privileges

Make a habit of logging to your server using a non-root account. This will prevent the accidental deletion of files if you make a mistake. For instance, the command rm can wipe your entire server if run incorrectly using by a root user.

🔥 Tip: Do NOT routinely use the root account. Use su or sudo, always.

SSH to your server

ssh username@server.public.ip.address
# example
# ssh myUsername@77.22.161.10

Create a new user called cardano

useradd -m -s /bin/bash cardano

Set the password for cardano user

passwd cardano

Add cardano to the sudo group

usermod -aG sudo cardano

🔏 Disabling SSH Password Authentication and Using SSH Keys Only

The basic rules of hardening SSH are:

No password for SSH access (use private key)
Don't allow root to SSH (the appropriate users should SSH in, then su or sudo)
Use sudo for users so commands are logged
Log unauthorized login attempts (and consider software to block/ban users who try to access your server too many times, like fail2ban)
Lock down SSH to only the ip range your require (if you feel like it)

Create a new SSH key pair on your local machine. Run this on your local machine. You will be asked to type a file name in which to save the key. This will be your keyname.

Your choice of ED25519 or RSA public key algorithm.

ssh-keygen -t ed25519

ssh-keygen -t rsa -b 4096

Transfer the public key to your remote node. Update the keyname.

ssh-copy-id -i $HOME/.ssh/<keyname>.pub cardano@server.public.ip.address

ssh cardano@server.public.ip.address

Disable root login and password based login. Edit the /etc/ssh/sshd_config file

sudo nano /etc/ssh/sshd_config

Locate PubkeyAuthentication and update to yes. Delete the #, if needed.

PubkeyAuthentication yes

Locate PasswordAuthentication and update to no

PasswordAuthentication no

Locate PermitRootLogin and update to prohibit-password

PermitRootLogin prohibit-password

Locate PermitEmptyPasswords and update to no

PermitEmptyPasswords no

Optional: Locate Port and customize it to your random port number.

Use a random port # from 1024 thru 49141. Check for possible conflicts.

Port <port number>

Validate the syntax of your new SSH configuration.

sudo sshd -t

If no errors with the syntax validation, restart the SSH process.

sudo systemctl restart sshd

Verify the login still works

ssh cardano@server.public.ip.address

ssh cardano@server.public.ip.address -p <custom port number>

Alternatively, add the -p <port#> flag if you used a custom SSH port.

ssh -i <path to your SSH_key_name.pub> cardano@server.public.ip.address

Optional: Make logging in easier by updating your local ssh config.

To simplify the ssh command needed to log in to your server, consider updating your local $HOME/.ssh/config file:

Host cardano-server
  User cardano
  HostName <server.public.ip.address>
  Port <custom port number>

This will allow you to log in with ssh cardano-server rather than needing to pass through all ssh parameters explicitly.

🤖 Updating Your System

It's critically important to keep your system up-to-date with the latest patches to prevent intruders from accessing your system.

sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get autoremove
sudo apt-get autoclean

Enable automatic updates so you don't have to manually install them.

sudo apt-get install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

By default when enabled, the unattended-upgrades service only installs security updates automatically. For details on configuring unattended upgrades, see How to Setup & Configure Unattended Upgrades on Ubuntu 20.04, for example.

🧸 Disabling the root Account

System admins should not frequently log in as root in order to maintain server security. Instead, you can use sudo execute that require low-level privileges.

# To disable the root account, simply use the -l option.
sudo passwd -l root

# If for some valid reason you need to re-enable the account, simply use the -u option.
sudo passwd -u root

🛠️ Configuring Two Factor Authentication for SSH

SSH, the secure shell, is often used to access remote Linux systems. Because we often use it to connect with computers containing important data, it’s recommended to add another security layer. Here comes the two factor authentication (2FA).

sudo apt install libpam-google-authenticator -y

To make SSH use the Google Authenticator PAM module, edit the /etc/pam.d/sshd file:

sudo nano /etc/pam.d/sshd

Add the follow line:

auth required pam_google_authenticator.so

Now you need to restart the sshd daemon using:

sudo systemctl restart sshd.service

Modify /etc/ssh/sshd_config

sudo nano /etc/ssh/sshd_config

Locate ChallengeResponseAuthentication and update to yes

ChallengeResponseAuthentication yes

Locate UsePAM and update to yes

UsePAM yes

Save the file and exit.

Run the google-authenticator command.

google-authenticator

It will ask you a series of questions, here is a recommended configuration:

Make tokens “time-base”": yes
Update the .google_authenticator file: yes
Disallow multiple uses: yes
Increase the original generation time limit: no
Enable rate-limiting: yes

You may have noticed the giant QR code that appeared during the process, underneath are your emergency scratch codes to be used if you don’t have access to your phone: write them down on paper and keep them in a safe place.

Now, open Google Authenticator on your phone and add your secret key to make two factor authentication work.

Note: If you are enabling 2FA on a remote machine that you access over SSH you need to follow steps 2 and 3 of this tutorial to make 2FA work.

🧩 Securing Shared Memory

One of the first things you should do is secure the shared memory used on the system. If you're unaware, shared memory can be used in an attack against a running service. Because of this, secure that portion of system memory.

To learn more about secure shared memory, read this techrepublic.com article.

One exceptional case

There may be a reason for you needing to have that memory space mounted in read/write mode (such as a specific server application like **Chrome **that requires such access to the shared memory or standard applications like Google Chrome). In this case, use the following line for the fstab file with instructions below.

none /run/shm tmpfs rw,noexec,nosuid,nodev 0 0

The above line will mount the shared memory with read/write access but without permission to execute programs, change the UID of running programs, or to create block or character devices in the namespace. This a net security improvement over default settings.

Use with caution

With some trial and error, you may discover some applications(like Chrome) do not work with shared memory in read-only mode. For the highest security and if compatible with your applications, it is a worthwhile endeavor to implement this secure shared memory setting.

Source: techrepublic.com

Edit /etc/fstab

sudo nano /etc/fstab

Insert the following line to the bottom of the file and save/close.

tmpfs	/run/shm	tmpfs	ro,noexec,nosuid	0 0

Reboot the node in order for changes to take effect.

sudo reboot

⛓️ Installing fail2ban

Fail2ban is an intrusion-prevention system that monitors log files and searches for particular patterns that correspond to a failed login attempt. If a certain number of failed logins are detected from a specific IP address (within a specified amount of time), fail2ban blocks access from that IP address.

sudo apt-get install fail2ban -y

Edit a config file that monitors SSH logins.

sudo nano /etc/fail2ban/jail.local

Add the following lines to the bottom of the file.

🔥 Whitelisting IP address tip: The ignoreip parameter accepts IP addresses, IP ranges or DNS hosts that you can specify to be allowed to connect. This is where you want to specify your local machine, local IP range or local domain, separated by spaces.

# Exampleignoreip = 192.168.1.0/24 127.0.0.1/8

[sshd]
enabled = true
port = <22 or your random port number>
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
# whitelisted IP addresses
ignoreip = <list of whitelisted IP address, your local daily laptop/pc>

Save/close file.

Restart fail2ban for settings to take effect.

sudo systemctl restart fail2ban

🧱 Configuring Your Firewall

The standard UFW firewall can be used to control network access to your node.

With any new installation, UFW is disabled by default. Enable UFW. Assuming that you use the default profile allowing outgoing traffic and denying incoming traffic, open the following ports to incoming traffic:

Port 22 (or your random port #) TCP for SSH connection
Port 123 UDP for chrony ntp
Port 6000 TCP for Cardano network traffic

If you install a Grafana dashboard for monitoring your Cardano stake pool, then also open the following ports in UFW:

Port 3000 TCP for Grafana web server
Port 9100 TCP for Prometheus Node Exporter data
Port 12798 TCP for Cardano Node data for Prometheus

# By default, deny all incoming traffic and allow outgoing traffic.
sudo ufw default deny incoming
sudo ufw default allow outgoing
# Allow ssh access
sudo ufw allow ssh #<port 22 or your random ssh port number>/tcp
# Allow cardano-node p2p port
sudo ufw allow 6000/tcp
# Enable firewall
sudo ufw enable

# Verify status
sudo ufw status numbered

Do not expose Grafana (port 3000) and Prometheus endpoint (port 9100 and 12798) to the public internet as this invites a new attack surface!

Better idea - SSH tunnel to Grafana server

Setup a SSH tunnel with the following command:

ssh -L 3000:localhost:3000 <user>@<your-server-ip-or-dns>

Alternatively, If using Putty for SSHing, you can configure the tunnel as follows. Make sure to click "Add" and save your new profile settings.

Now you can access the Grafana server from your local machine's browser by visiting http://localhost:3000

Only open the following ports on nodes behind a network firewall. This is not required if using the above SSH tunnel method.

🔥 It is dangerous to open these ports on a VPS/cloud node.

# Allow grafana web server port
sudo ufw allow 3000/tcp
# Allow prometheus endpoint port
sudo ufw allow 9100/tcp
# Allow prometheus cardano-node metric data port
sudo ufw allow 12798/tcp

Confirm the settings are in effect.

     To                         Action      From
     --                         ------      ----
[ 1] 22/tcp                     ALLOW IN    Anywhere
[ 2] 3000/tcp                   ALLOW IN    Anywhere
[ 3] 6000/tcp                   ALLOW IN    Anywhere
[ 4] 22/tcp (v6)                ALLOW IN    Anywhere (v6)
[ 5] 3000/tcp (v6)              ALLOW IN    Anywhere (v6)
[ 6] 6000/tcp (v6)              ALLOW IN    Anywhere (v6)

[ Optional but recommended ] Whitelisting (or permitting connections from a specific IP) can be setup via the following command.

sudo ufw allow from <your local daily laptop/pc>
# Example
# sudo ufw allow from 192.168.50.22

🎊 Port Forwarding Tip: You'll need to forward and open ports to your validator. Verify it's working with https://www.yougetsignal.com/tools/open-ports/ or https://canyouseeme.org/ .

🧱 Additional Hardening Rules for a Block-producing Node

Only your Relay Node(s) should be permitted access to your Block Producer Node.

sudo ufw allow proto tcp from <RELAY NODE IP> to any port <BLOCK PRODUCER PORT>
# Example
# sudo ufw allow proto tcp from 18.58.3.31 to any port 6000

🧱 Additional Hardening Rules for Relay Nodes

In order to protect your Relay Node(s) from a novel "DoS/Syn" attack, Michael Fazio created iptables entry which restricts connections to a given destination port to 5 connections from the same IP.

Replace <RELAY NODE PORT> with your public relay port, replace the 5 with your preferred connection limit.

sudo iptables -I INPUT -p tcp -m tcp --dport <RELAY NODE PORT> --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 5 --connlimit-mask 32 --connlimit-saddr -j REJECT --reject-with tcp-reset

Set the connection limit high enough so that your internal relay/block producer node topology remains functional.

🔥 iptables rules applied via terminal are not reboot-resistant!

You can check you current connections with a sorted list. Change the relay node port number, if needed.

sudo netstat -enp | grep ":6000" | awk {'print $5'} | cut -d ':' -f 1 | sort | uniq -c | sort

DDoS/Syn attacks can be complex and can take down a node. A single iptables rule may not be sufficient to protect or mitigate against more modern attacks. Moreover, iptables rules added via the terminal are forgotten if the machine or the iptables service is restarted.

Carden Pool [CRPL] provides a script that configures and deploys iptables rules specifically designed to protect from various DDoS attack vectors, ensuring the persistence of these rules even after reboots.

Further information can be found in the Carden Pool GitHub repository.

🔭 Verifying Listening Ports

If you want to maintain a secure server, you should validate the listening network ports every once in a while. This will provide you essential information about your network.