Awhile back, a long while, I started to get into hosting, this is when I first thought, “Everyone has a website, so I need one!” Not the right mentality, mind you, but that was then! It wasn’t too soon until I started to shy away from shared hosting plans into an un-managed dedicated server environment.
Fast forward about to about three years ago, when I started to get into dedicated servers quite heavily. This is when I began to host gaming servers for friends, and boy, let me tell you, it was a learning experience. Before I continue, let me say that I am by no means an expert on this subject, but I like to think I have more knowledge than the Average Joe when it comes to dedicated servers. So please, take this article with a grain of salt, and by all means, give me some feedback on what you would do, and why.
If you’ve ever hosted a popular gaming server (or website), you have more than likely experienced things like DDoS attacks, general web-based attacks, brute-forcing, etc… Granted, DDoS attacks are harder to pin down and usually boil down to your network speed and overall hardware, I found myself searching the web with, “Am I under a DDoS attack?” quite frequently. Keep in mind, I’m speaking from a single-server, non-load balanced system.
I’ve become an avid fan of CentOS. I know you probably have your own preferred OS, but they’re generally the same (I may be shunned for saying that), at least in their purpose. I can respect the disagreements this may pose, but remember, we’re all different. Anyhow, back on topic! It wasn’t long until I ditched the GUI’s like Plesk or CPanel in favor of my own know-how via command line. To this day I refuse to use any control panel if I can help it, I’m just faster typing than I am with clicking.
Dealing with Brutes
I found myself needing to deal with those pesky WordPress brutes that just won’t quit. There are a couple of ways you can do this, but I chose the easiest way, which was using WP Fail2Ban, Fail2Ban and ModSecurity.
ModSecurity: ModSec for short! It worked right out of the box, and honestly, I couldn’t ask for more. I’d seen this was in use on a client’s server awhile back, before I even started with WebDev, and have used it every since then. It works wonders for dealing with XMLRPC attacks and in studying these attacks; most of them don’t even include an XML body, and if that happens, ModSecurity just drops them immediately.
I heard about Fail2Ban a while back when I was playing a game with some friends. The first priority I had with Fail2Ban was to stop the flood of SSH attempts to my server, so I set it up to auto-ban users who attempted (and failed) to login via SSH and set a generous ban time of six hours and a max retry of three. This is why I love Fail2Ban: I can set the amount of times a user should be able to retry an action, and ban them for whatever time I set.
Fail2Ban wasn’t so straight forward to install. There is a little bit of configuration involved, so whip out your favorite text editor and start typing! Make sure you read through the jail.conf file and get a full understanding on how it works because that’s a whole other topic! On top of all that, make a copy of jail.conf and call it jail.local–edit the local file only!!!
Second priority was to handle FTP brutes, which was pretty simple since I’m using vsftp. I just enabled the directive in my jail.local. I mean, the configuration was already there–I just set it to enabled, and again, set a generous six hour ban time.
Last but not least, was WordPress. I really didn’t want a huge overhead on the site for banning people, extra database info, etc.. Since I was using Fail2Ban already, the use of WP Fail2Ban was a no-brainer. With this, I wanted to a bit more strict on the retry amounts, but more relaxed on ban times since I didn’t want to auto-ban myself for six hours should I forget a password. On top of that, I wanted to know when someone was trying to hit my WordPress installation. If I get more than one report/email from a specific IP address, I ban them permanently (more on that later). So here’s a snippet of my jail.local in case you need to use it:
[WordpressLogins] enabled = true filter = wordpress #action = iptables-multiport[name=NoAuthFailures, port="http,https"] action = %(action_mwl)s logpath = /var/log/messages bantime = 900 maxretry = 2
Notice in the config I’m setting the action. This action I’ve set will email me when an IP is banned with their IP address, whois information, and how many attempts they made. This way, I check my email every day, if I see an IP address that looks familiar, or has the exact same octets, and I just ban them permanently. Since this is a personal site, I usually look to see if they’re in the same CIDR range, if so, I consider that entire network hostile, and ban the entire range. If you don’t know what CIDR is, check this post out on Digital Ocean titled, “Understanding IP Addresses, Subnets, and CIDR Notation for Networking.”
When to Permanently Ban
This is really a case-by-case basis, and I realize that one site may get more hits than another, so take this information as needed. Also note that simply banning an IP address, or even an IP range, will not permanently stop these threats–though most of the time they just give up if you have an auto-ban system in place, which is why I love Fail2Ban. It does not stop them from just bouncing across a VPN or proxy and trying again. It’s up to you to be proactive in reporting these cases to the ISP’s listed in the whois database–even if you get no response, it cannot hurt. I prefer to selectively email ISPs and server hosts of potential abuses, but if you want to auto-send emails to parties listed in the abuse contacts, again Fail2Ban can do this for you. Here is a quick how-to on Stack Overflow.
When I perma-ban an IP from my server, I want to make 100% sure that it’s for a good reason, so I look for a pattern of IPs in my emails. If the same IP shows up more than once in my email, I consider that IP a threat, and just stop it from accessing the site. Of course, this could be at your own discretion, but if an IP has too many login attempts, I really don’t want that person hitting my site anymore.
Networks are more of a pain to track down. Usually I’ll get emails of attempt from an IP like 1.2.3.4, and a few more from another IP like 1.2.5.7, so I do a little research into those IPs. I use ip-lookup.net to see if multiple IPs fall within the same netrange. It’s super simple: Type in the IP and click the ‘IP owner info (whois)’. Or, if you have whois installed on your server, just run a whois on the IP there.
My rule is that if an IP shows up with more than six to nine attempts, or if an entire network shows up an outrageous amount of times (usually thirty to forty), I either ban the IP or the entire network. Granted, there are other factors that come into play, such as country of origin, the page they were requesting, and more, but most of my bans are from other countries such as China, Russia, and Ukraine.
Here is a great reference on NixCraft that I constantly look at when I need to ban an IP; it’s one of those things I can’t always remember, so having a reference saved works well for me.
So that’s it. How do you guys deal with this? What do you think of this method? I would especially love to hear from you CSF guys since I’ve wanted to get into it, but have yet to do so!
Good stuff. But my first impression is, this doesn’t scale. That is, once you get past having to do this for a couple sites you’re going to be consumed by chasing these ghosts. How many clients REALLY want to pay for that?
The easy – and cheaper? – route seems to be two way auth, or something similar, yes? That is, rather than try to stop the attacks, why not just completely prevent them by making them (close to) impossible?
Finally, there’s a browser plugin called WOT (Web Of Trust) that tells you – via crowdsourced data collection – which sites are safe and which might not be. It would seem to me, a WP-centric implementation of the same concept at it applies to these type of security threats is a no brainer. Safety in numbers, right?
The methods I use are definitely time consuming, but I find it fun to track these guys down. I currently only do this for two sites, both of which are on just one server.
Though, I agree, if the flood of connections increased, I would have to find a more automated solution. Truth be told, I used to have a captcha on login, and a plugin called Limit Login Attempts. I just wanted to get rid of the PHP overhead ( remove the plugins ), and find an alternative. Fail2Ban gave me that option, so I stuck with it ever since.
Man, I liked your approach but I agree with @MF Shishock above, it seems a lot of work. Maybe for learning purposes, it is great, but for a simple client site, it seems too much chasing.
Fought this for a long time with Fail2Ban, Wordfence, Sucuri, etc.. Finally setup ModSecurity with the Comodo ruleset and my problems dropped to almost zero overnight.
Just want to leave a huge “THANK YOU”! I host a small private server with RStudio-Server running on it and had to constantly deal with SSH brute force attacks.