Intrusion prevention software Fail2ban is typically used to block IP addresses making bogus login attempts. These often come in the form of so-called “brute force” password guesses against SSH or website login pages.
Fail2ban simply monitors server log files for dodgy requests and when it sees a match, it adds the requester to the server’s firewall rules. All future requests from the originating IP address are then silently dropped.
You can also configure Fail2ban to match requests for specific file types. For example, if you have lines which include /wp-login.php cluttering up the logs of your static website, you can stop responding to future requests from any IP address making those requests.
These are not legitimate requests if you aren’t using PHP, they are from vulnerability scanners testing you. No need to be polite about it, just silently drop them.
Prerequisites
If it is not already obvious, WordPress and other PHP-based CMS users should not blindly copy and paste the commands in this tutorial or you will lock yourself and everyone else out of your server. You can still benefit and adapt the commands below to block your unused file types, but make certain you do not include .php in your filters or it’s game over for you.
With that disclaimer out of the way, it is best to switch to an interactive root shell now, otherwise you will need to prepend sudo to all the following commands, so switch using this command:
sudo -i
The most specific package you need is fail2ban so install it using your system’s package manager and let it figure out dependencies. On an Ubuntu Server machine, try this:
apt update && apt install fail2ban
If you are happy with the proposed changes, accept them and wait for fail2ban and its dependencies to install.
Usage with Nginx
(Apache users, skip ahead to the next section)
Fail2ban requires two new files to make this work, a .conf file defining the filters to use as Python regular expressions, and a custom jail (configuration) file.
First create the filter file. Use this command to create one called “404-pests”:
nano /etc/fail2ban/filter.d/404-pests.conf
Into the file, paste the following three lines, which are suitable for static sites (once again, adapt if using PHP, ASP, or other server-side webapp frameworks):
[Definition]
failregex = ^<HOST> -.*"(GET|POST|HEAD).*(\.php|\.asp|\.cgi|\.dat)
ignoreregex =
Before continuing, you need to know the name and location of your website’s default log file. Ubuntu Server users can find it in the /var/log/nginx/ directory by default. Try this command to list the directory contents if you do not already know the log file name:
ls /var/log/nginx/
If you have not made any customizations, it is probably named access.log.
Here is how to test, even if you think you have it correct (changing “access.log” to match your actual log file):
fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/404-pests.conf
You should see results from the fail2ban-regex utility showing most lines missed (these are normal requests) and some matches (these are the vulnerability scanners requesting PHP and other scripting pages).
After successfully testing and confirming the log file location and regex matches, create the second file with the fail2ban jail.
nano /etc/fail2ban/jail.d/404-pests.local
Paste the following seven lines into the new jail file:
[404-pests]
enabled = true
port = http,https
filter = 404-pests
logpath = /var/log/nginx/*access*.log
maxretry = 1
bantime = -1 ;banned until reboot
Here I have set the bantime variable to -1, which means “until a reboot” (the text after the semicolon is a comment). The value is normally expressed in seconds and you can change this to whatever you prefer. Try setting it to “30” during testing, and lock yourself out of the server for only 30 seconds at a time. Once everything works as expected increase the time.
Now skip ahead to the “Final Steps” section below.
Usage with Apache HTTP Server
Fail2ban requires two new files to make this work, a .conf file defining the filters to use as Python regular expressions, and a custom jail (configuration) file.
First create the filter file. Use this command to create one called “404-pests”:
nano /etc/fail2ban/filter.d/404-pests.conf
Into the file, paste the following three lines, which are suitable for static sites (once again, adapt if using PHP, ASP, or other server-side webapp frameworks):
[Definition]
failregex = ^<HOST> -.*"(GET|POST|HEAD).*(\.php|\.asp|\.cgi|\.dat)
ignoreregex =
Before continuing, you need to know the name and location of your website’s default log file. Ubuntu Server users can find it in the /var/log/apache2/ directory by default. Try this command to list the directory contents if you do not already know the log file name:
ls /var/log/apache2/
If you have not made any customizations, it is probably named access.log.
Here is how to test, even if you think you have it correct (changing “access.log” to match your actual log file):
fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/404-pests.conf
You should see results from the fail2ban-regex utility showing most lines missed (these are normal requests) and some matches (these are the vulnerability scanners requesting PHP and other scripting pages).
After successfully testing and confirming the log file location and regex matches, create the second file with the fail2ban jail.
nano /etc/fail2ban/jail.d/404-pests.local
Paste the following seven lines into the new jail file:
[404-pests]
enabled = true
port = http,https
filter = 404-pests
logpath = /var/log/apache2/*access*.log
maxretry = 1
bantime = -1 ;banned until reboot
Here I have set the bantime variable to -1, which means “until a reboot” (the text after the semicolon is a comment). The value is normally expressed in seconds and you can change this to whatever you prefer. Try setting it to “30” during testing, and lock yourself out of the server for only 30 seconds at a time. Once everything works as expected increase the time.
Final Steps
Nginx and Apache HTTP Server users, now it is time to enable the new rules and check that everything works as expected.
Restart the Fail2ban process to enable the new rules:
systemctl restart fail2ban
If you get errors here, go back and continue re-testing with the fail2ban-regex utility until you find the correct log file. The usage is fail2ban-regex /path/to/logfile /path/to/filter Assuming no errors after the restart, check the process status with this command:
systemctl status fail2ban
You should see a “active (running) since…” message if everything worked and you are all done. Press “q” to exit the status report.