Last week many web hosts and website owners reported they were under a brute force attack, which was launched on a massive scale affecting thousands of sites and a multitude of web hosts. As a result of the attack, many WordPress sites were hacked, while others loaded slow and/or returned errors. As a directory specialised in monitoring WordPress hosting providers’ performance, we were very interested to see how various hosts responded to this massive attack and we would like to share our observations with you as we believe that kind of situations show the true quality of the provider and it could only help a user choose from the multitude of such. We based our analysis on the official statements made by many hosts such as Hostgator, Fatcow, SiteGround, Inmotionhosting,WPengine, Zippykid, GreenGeeks and others, which became public on various Internet sources.
What is a brute force attack and what this particular one involved?
Brute force attack is not an unusual event for web hosts. In its essence it is trying to exploit weak passwords to hack websites. The current attack against WordPress sites was targeting weak admin passwords through the wp—login.php form. More massive than a normal brute force attack, this one used a hundred thousand IPs (or more) to make multiple queries to various WordPress sites, attempting to guess their admin passwords. During that process, the load on the host servers significantly increased and the WordPress admin backend started loading really slow or returning server errors.
As a summary, the problem for the WordPress hosts was not only to try and save their hosted sites from being hacked, but also to make sure their servers don’t get overloaded by the attack. Now the questions are whether the providers managed to fight those two problems successfully and how long it took them to do so.
Overview of hosts’ actions:
There were several strategies taken by hosts, which focused on just one or several of the actions explained below.
Stop the hack actions:
1. Change of admin passwords
Almost all hosts asked their clients to change their WordPress admin passwords. Many users would forget to change the default passwords or would use easy to remember ones, which are also easy to be hacked. The brute force attack is looking to exploit exactly that kind of weak passwords, so the most natural thing is to change the weak passwords with stronger ones. Strong password is a combination of letters, numbers and special characters and is usually 8 or more symbols.
As stated, many hosts asked their clients to change the passwords – GoDaddy for example did that. There was one host, which impressed us with the proactive approach it took as it did not wait for the client to change the passes, but instead identified the WP instances with weak passwords and changed them on behalf of the users. We like that approach for two reasons: first, many customers would fail to respond to the request for change due to various reasons and second, even if they choose to respond, they would delay in doing so hence this leaves plenty of time for the attackers to hack the sites and take over the whole server.
2. WordPress Plugins
There are various WordPress plugins that are quite helpful in protecting a single installation from being hacked through a brute force attack. Matt Mullenberg the CEO of Automatic, official owner of WordPress software, recommended the use of plugins in his official statement on the topic. We also noticed that many users who understood about the problem, started installing that kind of plugins, which is a reasonable user-level move. However, a host urging its clients to install such plugins is not really addressing the problem of the server load and overall hack possibility on a global level, simply because the bruteforce attempts would still reach the server, still try to hack into the WordPress instance and server load would still be generated on a massive scale.
3. .htaccess files
Many hosts added restrictions in the .htaccess files to deny all (or after a certain number) login attempts to the WordPress backend. That seemed to be all some hosts did, unfortunately. The bad news is that while these restrictions are valid, the admin cannot access his own site because of this “fix” and should also wait till the whole attack is over.
Inmotionhosting seems to be using that kind of approach to restrict the attack. They say in their blog: “These rules will detect a brute force attempt to get into a customer’s WordPress website, and once there are 5 failed logins within 30 seconds, further access to the WordPress admin dashboard will be denied for the next 15 minutes.” They also told the users how to allow access of their own IP to the backend in the meantime.
WPengine also said that “…we install Limit Login Attempts across the board on all of our customer sites over at WP Engine.”
Fatcow did that as well based on what was stated in their official blog.
Although this fix obviously protects the WordPress site from being hacked, it does not decrease the load on the server. The reason is that the attack continues and the server has to process the queries each hit makes, which results in a server overload and slow to not accessible websites. That is why we go to the next set of actions that decrease the load.
Decrease Server load actions:
Obviously, all hosts agreed that they should do something on a server level as this is the best way to protect their servers and all hosted customers. Here is what they tried:
1. Third party solutions such as CloudFlare and Sucuri Cloudproxy
Some hosts relied on their partners and third-party software providers for shielding from the attack. Hosts using either CloudFlare or Sucuri could benefit from the hack-prevention solutions provided by those. Neither CloudFlare nor Sucuri explained in details what solution they provided for this particular attack maybe due to security reasons, as the more hackers know, easier it is to hack, but it seemed to be effective. The only drawback is that although the host might offer that service to their users for free as most do, the client does not necessary enable the CDN correctly and hence a small portion of the hosting clients are protected thanks to the fixes provided by those. Also, the traffic through CloudFlare is always accessed by www.YOURdomain.com. In this case, that’s good but it is possible to have a brute force by IP where CloudFlare will be of no help.
GreenGeeks is an example of a company that seems to mainly rely on CloudFlare solutions as a way to stop the server load due to the attack, based on the official company blog.
2. Custom fixes
Other hosts tried to write their own solutions. Some mentioned writing special rules for mod_security module of Apache, which however, opened the question whether those rules were well thought-out to differentiate legitimate human traffic from traffic by bots.
Liquidweb seemed to apply a custom mod_security rule, “which immediately filters incoming HTTP requests, which assists against taxing server resources.” They also explain brute force tactics in their knowledgebase based on limiting IPs after a certain number of failed login tries.
Unfortunately, some hosts that actually applied custom fixes did not report on doing it or did not share the details on how they did it, again to prevent hackers from penetrating their defences. Despite that, it became clear that the key winning custom tactics seemed to be channeling the bad traffic away from the host server.
SiteGround is an example of a WordPress hosting provider that managed to stop the attack, decrease the load and were the first to do it. Fatcow also managed to channel traffic away, but about 2 days later as announced in their blog.
How effective were hosts in their handling?
Based on the official statements and chronology of events, SiteGround proved to be the host with most adequate and fast reaction that practically stopped all negative effects of the attack. They were proactive in preventing hacks and also wrote custom, in-house solutions to decrease the server load first. Their CEO said: “I will not go into details in terms of what we did, cause chances are some of those hackers running those same botnets would read it and will try to outsmart us, but the facts show that for the past 12 hours we have blocked more than 15 million brute force attempts (That’s A LOT!) towards our clients and our servers are not experiencing any load issues.”
After initially using the .htaccess approach and stopping all login, Fatcow seemed to manage to address the situation one day or more after SiteGround. As stated in their blog: “Our breakthrough happened on Thursday, as our team looked through data on the web and data in our logs. We found a difference between the way the attack accesses WordPress and legitimate customers access WordPress. Thursday afternoon, we rolled that change out to our edge servers (before the traffic even reaches the web server that might be hosting your site) to drop any traffic that didn’t look legitimate.”
WPengine has posted a message, which fails to explain what they do but only that they take care of their customers, on Friday. They seem to be effective judging by customer reactions, but their reaction was slow and hence, we cannot really praise their efforts under these circumstances.
A very unclear message from Hostgator so it is hard to say what they did or didn’t do, but as stated in their blog by Friday, they were unable to decrease the server load by then.
Zippykid, a managed WordPress host, seemed to have handled the problem by Saturday, but without any details on the tactics it used. It seems however, they applied one of those rough to the client solutions: “There is a very small possibility that the system thinks of you or your users as an attacker, when that happens, you’ll see a message like the one below… As the message says, just relax, and try again in a minute or so. We’re sorry about this, but we’ll make our detection algorithms better over time.” It is a bit surprising that Zippykid is advertising to be a specialized WordPress host doing nothing else but WordPress, yet they were not very fast crafting a solution, neither they seemed to do it themselves (they thanked another host – Site5 for sharing information that helped them address the issue).
Godaddy one of the biggest and most well known brand, asked their clients to change the passwords and applied a strategy of blacklisting IPs from which the attack came. This is their official statement: “Our Security team continues to identify these attacks, down to the IP address, and block anything that looks malicious. Additionally, we’ve installed new features on every single one of our thousands of servers to block these bad actors more quickly.”
Hosts are often used to relying on ready solutions provided by their partners or patches that used to work earlier in a different situation they faced. The problem is hackers get smarter and so should hosts. Even when you choose to use old tactics you should do so smartly, or you should consider custom solutions under the new circumstances as shows the example of SiteGround, which applied gentle to the user fixes to prevent hacking and custom in-house patches that resulted in them being the first to decrease the load on the servers.