Hardening Linux Server Setup

in IT on Linux, Hardening Last modified at:
Hardening Linux Server Setup

The system administrator is responsible for security of the Linux servers. In this post, I will provide Linux server hardening tips for default installation of Linux system.
Hacker Noon©

For more information : See the document, written by ANSSI, the French National Information Security Agency, presents the “Configuration recommendations of a GNU/LINUX system”

  1. Disk Partitions
  2. Disable IPv6 Networking - Turn Off IPv6
  3. Use Secure Shell (SSH)
  4. Avoid Using FTP, Telnet, and Rlogin / Rsh services on Linux
  5. Keep system updated
    1. Applying Debian security updates only
    2. Updating all packages (OS and other installed packages)
  6. No accounts with empty Passwords
  7. Make sure no non-root accounts have UID set to 0
  8. Disable root Login
  9. IPtable, fail2ban, IPset and GeoIP filtered access
    1. IPtable
      1. Viewing current configuration
      2. Delete rules
      3. Save after reboot
      4. Basic commands
      5. Persist at reboot
    2. IPset
      1. IPset install
      2. Listing / adding / removing IP addresses from our blacklist
      3. Persist at reboot
    3. GeoIP
      1. Install and configure GeoIP
      2. Direct download GeoIP CVS format database
    4. Secured acces now with iptable, ipset and GeoIP
      1. iptables to allow traffic from private network and one country only
    5. fail2ban

Disk Partitions

It’s important to have different partitions to obtain higher data security in case if any disaster happens. By creating different partitions, data can be separated and grouped. When an unexpected accident occurs, only data of that partition will be damaged, while the data on other partitions survived. Make sure you must have following separate partitions and sure that third party applications should be installed on separate file systems under /opt.

    /
    /boot
    /usr
    /var
    /home
    /tmp
    /opt

Disable IPv6 Networking - Turn Off IPv6

echo 'net.ipv6.conf.all.disable_ipv6 = 1' > /etc/sysctl.d/90-disable-ipv6.conf
sysctl -p -f /etc/sysctl.d/90-disable-ipv6.conf

Note: The first one created a file /etc/sysctl.d/90-disable-ipv6.conf which contains the configuration setting to disable IPv6 and the second one applies the change.

Use Secure Shell (SSH)

Telnet and rlogin protocols uses plain text, not encrypted format which is the security breaches. SSH is a secure protocol that use encryption technology during communication with server. Never login directly as root unless necessary. Use sudo to execute commands. sudo are specified in /etc/sudoers file.

See Hardening SSH Server Setup post for more information.

Avoid Using FTP, Telnet, and Rlogin / Rsh services on Linux

Under most network configurations, user names, passwords, FTP / telnet / rsh commands and transferred files can be captured by anyone on the same network using a packet sniffer.

sudo apt-get --purge remove xinetd nis yp-tools tftpd atftpd tftpd-hpa telnetd rsh-server rsh-redone-server

Keep system updated

Applying Debian security updates only

sudo grep security /etc/apt/sources.list | tee /etc/apt/security.sources.list
sudo apt update
sudo apt upgrade -o Dir::Etc::SourceList=/etc/apt/security.sources.list

Press “Y” and “Enter”. Security updates will be installed and your system will be up to date.

To check and apply updates at a later stage do this:

sudo apt update
sudo apt upgrade -o Dir::Etc::SourceList=/etc/apt/security.sources.list

Updating all packages (OS and other installed packages)

sudo apt update && sudo apt upgrade

No accounts with empty Passwords

Type the following command to identify account whith empty Passwords

sudo awk -F: '($2 == "") {print}' /etc/shadow

and lock all empty password accounts

sudo passwd -l <accountName>

Make sure no non-root accounts have UID set to 0

Only root account have UID 0 with full permissions to access the system. Type the following command to display all accounts with UID set to 0:

awk -F: '($3 == "0") {print}' /etc/passwd

You should only see one line as follows:

root:x:0:0:root:/root:/bin/bash

If you see other lines, delete them or make sure other accounts are authorized by you to use UID 0.

Disable root Login

Never ever login as root user. You should use sudo to execute root level commands as and when required. sudo does greatly enhances the security of the system without sharing root password with other users and admins. sudo provides simple auditing and tracking features too.

To disable, you can remove the password of the account or lock it down, or even do both of them:

  1. Remove the root password:
    sudo passwd -d root
  2. Lock the account:
    sudo passwd -l root
  3. Change root User’s Shell
sudo vi /etc/passwd

Change the line:

root:x:0:0:root:/root:/bin/bash

to

root:x:0:0:root:/root:/sbin/nologin

From now on, when root user logs in, he/she will get the message This account is currently not available. This is the default message, but, you can change it and set a custom message in the the file /etc/nologin.txt. This method is only effective with programs that require a shell for user login, otherwise, sudo, ftp and email clients can access the root account.

IPtable, fail2ban, IPset and GeoIP filtered access

IPtable

iptables is an utility program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as different Netfilter modules) and the chains and rules it stores. Different kernel modules and programs are currently used for different protocols; iptables applies to IPv4, ip6tables to IPv6, arptables to ARP, and ebtables to Ethernet frames.

iptables requires elevated privileges to operate and must be executed by user root, otherwise it fails to function. On most Linux systems, iptables is installed as /usr/sbin/iptables.

iptables let’s you configure default policies for chains in the filter table, where INPUT, FORWARD and OUTPUT, are the main ones (or at least the most used). Users can even define new chains.

These aforementioned chains, are better explained in this graph that comes from Wikipedia.

SSH, CryptCheck, screenshot Packet flow in Netfilter and General Networking

Viewing current configuration

sudo iptables -L

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

You may prefer to use iptables -L -vn to get more information, and to see ports as numbers instead of its names.

Delete rules

  • Delete all rules
sudo iptables -F
  • Delete spécific rules
sudo iptables -L --line-numbers

Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination
1    ACCEPT     all  --  192.168.0.4          anywhere
2    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
3    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
4    ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh

Notes: For deleted the record number 3 type the command below

sudo iptables -D INPUT 3

Save after reboot

sudo /sbin/iptables-save

Or if you want deleted all rule and save

sudo iptables -F
sudo /sbin/iptables-save

Basic commands

  1. Block from an IP
    sudo iptables -A INPUT -s 11.22.33.44 -j DROP
  2. If you want to block only on an specific NIC
    sudo iptables -A INPUT -s 11.22.33.44 -i eth0 -j DROP
  3. Or an specific port
    sudo iptables -A INPUT -s 11.22.33.44 -p tcp -dport 22 -j DROP
  4. Using a Network and not only one IP
    sudo iptables -A INPUT -s 11.22.33.0/24 -j DROP
  5. Block traffic from a specific MAC address Suppose you want to bloc traffic some a MAC address instead of an IP address. This is handy if a DHCP server is changing the IP of the maching you want to protect from.
    sudo iptables -A INPUT -m mac --mac-source 00:11:2f:8f:f8:f8 -j DROP
  6. Block incoming traffic to a port Suppose we need to block port 21 for incoming traffic
    sudo iptables -A INPUT -p tcp --destination-port 21 -j DROP
  7. Block outgoing traffic to a port If you want to forbid outgoing traffic to port 25, this is useful, in the case you are running a Linux firewall for your office, and you want to stop virus from sending emails.
    sudo iptables -A FORWARD -p tcp --dport 25 -j DROP

Persist at reboot

In /etc/network/interfaces add below face eth0 inet dhcp this lines :

pre-up iptables-restore < /etc/iptables.up.rules

So at eatch boot le system load the file /etc/iptables.up.rules

IPset

IP sets are a framework inside the Linux kernel, which can be administered by the ipset utility. Depending on the type, an IP set may store IP addresses, networks, (TCP/UDP) port numbers, MAC addresses, interface names or combinations of them in a way, which ensures lightning speed when matching an entry against a set. If you want to :

  • store multiple IP addresses or port numbers and match against the collection by iptables at one swoop;
  • dynamically update iptables rules against IP addresses or ports without performance penalty;
  • express complex IP address and ports based rulesets with one single iptables rule and benefit from the speed of IP sets

then ipset may be the proper tool for you.

IPset install

sudo apt update
sudo apt install ipset

Listing / adding / removing IP addresses from our blacklist

  • To list current IP addresses in our list, do:
sudo ipset list
  • To add an IP address, simply do:
sudo ipset add blacklist <IP-ADDRESS_OR_CIDR-IF-NETHASH>
  • To remove an IP address, do:
sudo ipset del blacklist <IP-ADDRESS_OR_CIDR-IF-NETHASH>
  • To remove ALL IP addresses in a list, use the flush command
sudo ipset flush blacklist
  • Querying to see if IP address is in list

You can use the test command to check if an IP address is in a specific list:

sudo ipset test blacklist <IP-ADDRESS>
  • Deleting a list You might want to delete a list. To do so you can use the ‘destroy’ command:
sudo ipset destroy blacklist

Persist at reboot

In /etc/network/interfaces add below face eth0 inet dhcp this lines :

pre-up ipset restore -! < /etc/ipset.up.rules

So at eatch boot le system load the file /etc/ipset.up.rules

GeoIP

Install and configure GeoIP

  1. Create an free account on GeoIP maxmind to access at the GeoIP database
  2. Install geoipupdate debian package
    sudo apt install geoipupdate
  3. Create a licence key on https://www.maxmind.com/en/accounts/<user_id>/license-key
  4. Create a config file GeoIP.conf in /etc/
    sudo touch /etc/GeoIP.conf

    And add this line :

     # file: "GeoIP.conf"
 # Please see https://dev.maxmind.com/geoip/updating-databases?lang=en for
 # instructions on setting up geoipupdate, including information on how to
 # download a pre-filled GeoIP.conf file.

 # Replace YOUR_ACCOUNT_ID_HERE and YOUR_LICENSE_KEY_HERE with an active account
 # ID and license key combination associated with your MaxMind account. These
 # are available from https://www.maxmind.com/en/my_license_key.

 # "AccountID" is from your MaxMind account.
 AccountID <user_id>

 # "LicenseKey" is from your MaxMind account
 LicenseKey <LicenseKey>

 # Enter the edition IDs of the databases you would like to update.
 # Multiple edition IDs are separated by spaces.
 #
 # Include one or more of the following edition IDs:
 # * GeoLite2-ASN - GeoLite 2 ASN
 # * GeoLite2-City - GeoLite 2 City
 # * GeoLite2-Country - GeoLite2 Country
 EditionIDs GeoLite2-Country GeoLite2-City GeoLite2-ASN
  5. Automatic Updates for GeoIP2 MaxMind provides the GeoIP Update program, which performs automatic updates for both GeoIP2 and GeoIP Legacy binary databases. Please follow the instructions below.
  • Step 1 – Install GeoIP Update The latest release may be downloaded from GitHub Releases. See here for installation instructions.

    Note: If you are using an older version of GeoIP Update, you may need to upgrade to GeoIP Update 4.x or later version. The 4.x and later versions meet our requirement for using TLS 1.2 or greater for all requests to our servers to keep your data secure.

sudo apt update
sudo apt install geoipupdate
  • Step2 – Run GeoIP
sudo geoipupdate -v

geoipupdate version 
Using config file /etc/GeoIP.conf
Using database directory /var/lib/GeoIP
Acquired lock file lock (/var/lib/GeoIP/.geoipupdate.lock)
Calculated MD5 sum for /var/lib/GeoIP/GeoLite2-Country.mmdb: 211201c65b48d0c8145597223b402c82
Performing update request to https://updates.maxmind.com/geoip/databases/GeoLite2-Country/update?db_md5=211201c65b48d0c8145597223b402c82
No new updates available for GeoLite2-Country
Acquired lock file lock (/var/lib/GeoIP/.geoipupdate.lock)
Calculated MD5 sum for /var/lib/GeoIP/GeoLite2-City.mmdb: 466dbb4c731f70860cc638752bdfb8ae
Performing update request to https://updates.maxmind.com/geoip/databases/GeoLite2-City/update?db_md5=466dbb4c731f70860cc638752bdfb8ae
No new updates available for GeoLite2-City
Acquired lock file lock (/var/lib/GeoIP/.geoipupdate.lock)
Calculated MD5 sum for /var/lib/GeoIP/GeoLite2-ASN.mmdb: cd850e751ba984f94f54b1acdee56cbd
Performing update request to https://updates.maxmind.com/geoip/databases/GeoLite2-ASN/update?db_md5=cd850e751ba984f94f54b1acdee56cbd
No new updates available for GeoLite2-ASN

ls -la /var/lib/GeoIP

total 65008
drwxr-xr-x  2 root root     4096 May  8 09:07 .
drwxr-xr-x 30 root root     4096 May  7 19:30 ..
-rw-------  1 root root        0 May  7 19:35 .geoipupdate.lock
-rw-r--r--  1 root root  8492226 May  7 19:35 GeoLite2-ASN.mmdb
-rw-r--r--  1 root root 51616375 May  7 19:35 GeoLite2-City.mmdb
-rw-r--r--  1 root root  6444374 May  7 19:35 GeoLite2-Country.mmdb
  • Step 3 – Run GeoIP Update Run geoipupdate. To fully automate this process on Linux or Unix, use a crontab file like:
###############
##  GeoIP    ##
###############
0 0 * * 7 		sudo -u root /usr/bin/geoipupdate

Notes: This cron expression would run every week at 00:00 on Sunday.

Direct download GeoIP CVS format database

You can direct download GeoIP CVS format database whith command line below :

cd /tmp
curl -O -J -L -u "<accountID>":"<licenseKEY>" 'https://download.maxmind.com/geoip/databases/GeoLite2-Country-CSV/download?suffix=zip'
curl -O -J -L -u "<accountID>":"<licenseKEY>" 'https://download.maxmind.com/geoip/databases/GeoLite2-ASN-CSV/download?suffix=zip'
curl -O -J -L -u "<accountID>":"<licenseKEY>" 'https://download.maxmind.com/geoip/databases/GeoLite2-City-CSV/download?suffix=zip'

Note: For more information, visit maxmind.com developer portal.

Secured acces now with iptable, ipset and GeoIP

Now we know iptable. And ipset & GeoIP are installed. So we can secure outcoming and incoming network traffic from our server.

iptables to allow traffic from private network and one country only

  1. create a hashmap with ipset to allow private IP to access your host
  2. create a hashmap with ipset to allow only french IP to access your host
  3. loading this rules in iptable
  4. And persist rules for each system reboot
  5. Add a cron entry for the regular update of the hash table

The scripts below (ux_ipset_update.sh ) do it automatically

#file: "ux_ipset_update.sh"
#!/bin/bash
#-------------------------------------------------------------------
#~ @(#) Name : ux_ipset_update.sh
#~ @(#) Desc : Update the ipset that iptables references for allowing based on country.
# Takes 2 parameters: ipset name (no spaces), country name e.g. 'Australia', 'france' ...
# iptables should have an existing '--match-set' rule e.g
#  $ iptables -I INPUT -p tcp --dport 22 -m set --match-set australia4 src -j ACCEPT
#  $ ip6tables -I INPUT -p tcp --dport 22 -m set --match-set australia6 src -j ACCEPT
#~ @(#) version : 1.1
# Auteur : hello@di-marco.net
# Date : 2024-05-08
#-------------------------------------------------------------------
# Version history
# V1.0 [2020-02-01]
# V1.1 [2024-05-08]
#-------------------------------------------------------------------
#~ Usage : ux_ipset_update.sh whitelist <country>
#-------------------------------------------------------------------

SRC=$(dirname $0)

CHANGELOG="/var/log/ipset_changes.log"
IPSET="/usr/sbin/ipset"

IPSET_UP_RULES="/etc/ipset.up.rules"
IPTABLES_UP_RULES="/etc/iptables.up.rules"

accountID="<accountID>"
licenseKEY="<LicenseKEY>"

if [ $# -ne 2 ]; then
    echo "Usage $0 <ipset_name> <country>"
    exit
fi

SETNAME=$1
COUNTRY=$2
TMPDIR=/tmp/$$_tmp
GEOIP_FILE=${TMPDIR}/$$_geoip.zip
CIDR_FILE=${TMPDIR}/$$_${SETNAME}_CIDR.zone
mkdir -p $TMPDIR

function fail {
    echo "FAILED: "$1
        cleanup
    exit 1
}

function cleanup {
	rm -rf $TMPDIR
}

cd $TMPDIR
[ $? -ne 0 ] && fail "Problem using tmp directory $TMPDIR"

curlout=`curl -J -L -u "${accountID}":"${licenseKEY}" 'https://download.maxmind.com/geoip/databases/GeoLite2-Country-CSV/download?suffix=zip' -o $GEOIP_FILE 2>&1`
[ $? -ne 0 ] && fail "Problem downloading GeoIP database: $curlout"

unzip -q $GEOIP_FILE
cd GeoLite2-Country-CSV*

GEOID=`egrep -i "${COUNTRY}" GeoLite2-Country-Locations-en.csv | cut -f1 -d,`
[ $? -ne 0 ] && fail "$COUNTRY not found in GeoIP file"

for IPvX in 4 6; do

	SET=${SETNAME}${IPvX}
	grep $GEOID GeoLite2-Country-Blocks-IPv${IPvX}.csv | cut -f1 -d, > $CIDR_FILE

	if [ "$IPvX" -eq "4" ]; then
		$IPSET -exist create ${SET} hash:net family inet
		$IPSET create ${SET}-tmp hash:net family inet
	else
		$IPSET -exist create ${SET} hash:net family inet6
		$IPSET create ${SET}-tmp hash:net family inet6
	fi
	[ $? -ne 0 ] && fail "Problem creating tmp ipset"

	while read line; do
		echo "$IPSET add ${SET}-tmp $line;"
		$IPSET add ${SET}-tmp $line;
		[ $? -ne 0 ] && fail "Problem updating tmp ipset"
	done < $CIDR_FILE

	# Check for changes
	if [ -n $CHANGELOG ]; then
		$IPSET save ${SET} > ${TMPDIR}/$$_${SET}.ipset
		$IPSET save ${SET}-tmp > ${TMPDIR}/$$_${SET}-tmp.ipset
		echo `date`": Processing $COUNTRY..." >> $CHANGELOG
		echo `wc -l $CIDR_FILE | cut -f1 -d' '`" blocks loaded" >> $CHANGELOG
		diffout=`diff ${TMPDIR}/$$_${SET}.ipset ${TMPDIR}/$$_${SET}-tmp.ipset`
		if [ $? -ne 0 ]; then 
			echo $diffout >> $CHANGELOG
		else
			echo "No changes" >> $CHANGELOG
		fi
		echo >> $CHANGELOG
	fi

	$IPSET swap ${SET}-tmp $SET
	[ $? -ne 0 ] && fail "Problem updating live ipset"

	$IPSET destroy ${SET}-tmp
	[ $? -ne 0 ] && fail "Problem destroying tmp ipset"
done

cleanup

# Add local IPv4 and IPv6 address
/usr/sbin/ipset add whitelist4 127.0.0.0/8
/usr/sbin/ipset add whitelist4 10.0.0.0/8
/usr/sbin/ipset add whitelist4 172.16.0.0/12
/usr/sbin/ipset add whitelist4 192.168.0.0/16
/usr/sbin/ipset add whitelist6 ::/128
/usr/sbin/ipset add whitelist6 fc00::/7

# Update rules and persit at reboot
if [ -f ${IPSET_UP_RULES} ];then
  cp ${IPSET_UP_RULES} ${IPSET_UP_RULES}.save
  $IPSET save > ${IPSET_UP_RULES}
else 
  $IPSET save > ${IPSET_UP_RULES}
fi

if [ -f ${IPTABLES_UP_RULES} ];then
  cp ${IPTABLES_UP_RULES} ${IPTABLES_UP_RULES}.save
else 
  touch ${IPTABLES_UP_RULES}
fi
cat <<EOF >${IPTABLES_UP_RULES}
# Generated by $0 on $(date)
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:LOGGING - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m set ! --match-set ${1}4 src -j DROP
-A INPUT -m set ! --match-set ${1}6 src -j DROP
COMMIT
# Completed on $(date)
EOF

exit 0

Note: In script os_ipset_update.sh

  • <LicenseKey> is your GeoIP maxmind.com licence key
  • Only French public IP are allow
  • All private IP are allow
  • And localhost are allow

To fully automate this process on Linux or Unix, use a crontab file like.

###############
##  GeoIP    ##
###############
0 23 * * 6		sudo -u root /<path>/os_ipset_update.sh <hashmap_name> <contry_name>

Notes: run every week at 23:00 on Saturday for example.

fail2ban

See part on fail2ban in Protect against DDOS and brute force attacks with fail2ban post.

Share it :