6

I am configuring Fail2Ban on my Ubuntu web server to prevent it from being a victim of DoS / DDoS. I don't want to use Cloudflare because I have to route my DNS over and use their SSl cert.

Currently, I found a script online that checks for more than 1 HTTP HEAD request per second, or more than 1 request to xmlrpc.php per second. I don't think it's sufficient protection, as these aren't the only kinds of requests that people can employ to execute a DDoS attack.

I'm looking at restricting the number of GET / POST requests a given IP can make in a short window, but I'm not sure how I should set the restriction, since big pages that load a lot of Javascript, CSS or images will make a lot of GET requests in a short amount of time. Should I be looking at limiting GET / POST requests, or should I be looking at something else? Why?

John Doe
  • 375
  • 2
  • 4
  • 20

3 Answers3

3

It can be hard to see the difference of good or bad guys, just by checking rate of requests per second. You would need to run the script on Your environment to see how many requests from a single IP address per 5 minutes (example) is "normal" for Your website, before making the final decision.

Once you have figured out the normal rate, it should be possible to count GET and/or POST (depending on your logfile analysis) with your script.

It is though possible to find other suspicious activities in the logfiles to filter, like scanning for scripts or executables etc. (GET/POST that "hopefully" causes an error in a well configured web server ;-) )

I have used this external fail-2-ban-link on my own systems.

Ingvar J
  • 531
  • 2
  • 7
2

A good setup is to check multiple connections within 1 second. I believe that 100 connections in 1 second is something very suspicious and should be blocked.

As you use Apache, I recommend also using mod_evasive along with fail2ban.

Using mod_evasive.

Using fail2ban.

Tom
  • 309
1

Is your concern real DoS or just malicious crawlers? By which I mean crawlers that will scrape your site for contact information, try to find security flaws or try to spread spam, but there goal is not to create denial of service. However they can create a problematic load that can effectively result in DoS. These are the ones you usually have to concern yourself with on a small web page.

With these kind of bots I have found, implementing a rate level on the application level works very good. This also has the effect that it will work with good crawlers that just crawl to aggressively. By responding with 429 - Too many requests if they are implemented correctly they will slow down but not stop indexing your site.

You can due this in nginx like so, I recommend applying the limit only to locations that run PHP (or any other language) that is usually responsible for the most CPU, RAM and disk usage and will cause your server to stop working. You can usually service a lot of static files without running into issues. Finding the correct values for just the PHP files is also a lot easier usually.

# This will limit requests to 2 per second,
# you should run some tests here to make sure this is a value
# low enough to ensure an individual user will not be able to cause DoS
# while not blocking legitimate requests.
# The limit of 2/s will probably be to low if you have a lot of ajax requests.
limit_req_zone $binary_remote_addr zone=php_req_limit:10m rate=2r/s;

This is needed for limiting the total number of parallel requests.


limit_conn_zone $server_name zone=conn_limit:10m;


server {
    # This will limit the number of concurrent requests to 100.
    # This means an ip cannot open more than 100 simultaneous connections.
    # If you have a lot of static resources css files, jss, images
    # or you make a lot of AJAX calls you might need a higher value here.
    limit_conn conn_limit 100;


# Apply the limit to the place that runs your code.
# For example the place where you proxy your requests to fpm.
location ~ [^/]\.php(/|$) {

    # This applies the limit.
    # The burst parameter regulates by how many requests the limit can
    # be overstepped on the first interaction.
    # This means even tough the limit is set to 2 request per second
    # on the first interaction you can actually make up to 22 requests.
    # When the user than waits for a second after all requests have
    # been processed the buffer becomes empty again and he can burst again.
    # This means a normal user (with a webbrowser) that goes to a page
    # stays there for a few seconds and then continues to another
    # page will have up to 22 requests per page load.
    # But an aggressive crawler will not be able to constantly make 22 requests.

    limit_req zone=php_req_limit burst=20 nodelay;

    [...]
}


# This will make your server to respond with 403 instead of 503
# when the limit is exceeded. This is important so that legit
# crawlers will know to slow down.
listen       443 ssl http2;
listen       [::]:443 ssl http2;

[...]



}

You can also combine this approach with fail2ban by blocking ips if they trigger to many 429 (so they refuse to slow down) in a time frame. That way you give legitimate bots (like search crawlers) a chance to adjust their behavior but you also make sure that malicious bots get blocked the most efficient way (on the TCP level).

Gellweiler
  • 136