This site was comprehensively hacked in February 2016. This is the story of what happened, and how I solved the problem.
Firstly a few words on me. I am pretty web-savvy! I have put together around 30 websites in my time, starting life using cold-fusion and DreamWeaver back in the last millennium, and since have used Joomla, WordPress, Drupal and many other authoring and CMS systems. My sites have never been hacked before, and maybe that meant that I had become a little lax – more on that later
Also by way of introduction for the non-techies out there, I would like to give you a brief description of the main components that I will be mentioning throughout this blog.
- A web server is simply a processor, storage and memory attached to the internet
- Sitting on that will be an operating system, typically Linux or Windows
- Sitting on top of that will be an authoring environment – typically WordPress, Joomla or Drupal – these can also be called CMSs(Content Management Systems)
- On top of that is a theme – a theme gives an overall look and feel to the site, and can be written by the author, or bought off the shelf and modified
- On top of that will be Plugins that give functionality that you don’t want to write yourself – shopping baskets, moving banners, security modules, webmaster tracking tools, and specialist content production systems are all examples of functionality that you might want to service via a plugin rather than write yourself.
OK basic definitions over, what is a hack?
A hack is anything where the server is under the control, even for a short while of someone who isn’t authorised to be there
We’ve all heard of usernames, passwords, credit card details being hacked, but that is a very small part of the hacking world
So if it’s not credit card details that the hackers are after, why would they want to hack your server?
To do whatever they want!!
Now this might seem a bit trite to say it that way, but I want you to think about this in the widest context
Let’s say you want to spam the world with your mis-spelt Viagra offering – you don’y use your own server – you use a hacked server
Let’s say you need a repository for dodgy photographs and videos – someone else’s is less traceable than your own
Need a server to launch a denial of service attack? You need 100’s of compromised machines.
Get the picture? A hack is simply using a machine for the hackers benefit – whatever they want.
Lastly, you want the hack to be as invisible as possible. Whilst it makes good TV to have letters dripping off a screen with a home page that says “you have been hacked” most hacks try to be as “silent” as they can be. After all, if the web-master doesn’t know he or she has been hacked, they won’t fix the issue, and the server is under the control of the hacker for longer.
I found it fascinating that they also used some fairly advanced methods to minimise their presence – only send e-mails at night, when I was less likely to notice, and keep the basic responsiveness of the ste as high as possible, so again I didn’t notice
That all said from the time of me putting in the plugin with the vulnerability, to me discovering it, was only 48 hours! When these people find a chink in your armour, they really exploit it!
So what happened to me?
I had updated some plugins as part of the general maintenance of the site. A few days later I was doing a blog, and I noticed that everything seemed to be sluggish, so I started to look at the system logs to see if anything untoward was going on. Oh, yes! An awful lot was going on!!!
I seemed that the previous night “I” had sent out 4400 e-mails selling Viagra, and all of a sudden I had lost nearly all of my disk storage, and seemed to be having 10 times as ,many hits on the web-site – but all going to files I didn’t recognise.
My initial thought on discovery was “PANIC”, followed by “This will be easy to fix, I will go back to some old backups, and fix it, it will only take a couple of hours.
Nope, it took a whole month of pretty intense activity to get back to where I started
I don’t want to give too many details away, as this hack is a new style, and is being investigated by the “White hats”, but what happened looks like this
The plugin had a vulnerability which was found and exploited
That enabled a very simple backs door to be inserted into the site
That back door allowed 6 other doors to be created
Each those back doors resided in different places – the cMS, the Database, Posts, Pages, core files, plugin files, Picture libraries
Each of the backdoors could “Heal” any other back door, so if I fixed backdoor 1 , backdoor 2 would instantly put it back to its hacked state.
Some examples of files that they were storing and accessing on my Server
So, these hackers seem to be quite well read! But there was also a huge volume of particularly nasty images and videos, as well as the methods by which they sent 10’s of 1000’s of spam e-mail messages
Doing a restore was a waste of time because general restores assume that the base infrastructure is fine(ie, the CMS and operating system and non functional tables in the database are all OK
So what would happen and I would do a restore. All seemed fine, but the next day I would find out that yet again I was hacked.
Time for action
I tried other minimally invasive methods of getting the site clean again. None worked because as soon as I fixed one vulnerability, another section of code would re-hack it.
There was only one solution – Nuke the lot
That was a really hard decision to take – years of blogs, photos and videos would need to be re-linked, special functionality that I had forgotten how it had worked would need to be re-created. I simply couldn’t trust anything, even innocuous comment fields contained snippets of code that were part of the vulnerability
I did copy the text of the site to Wordpad, so at least that was saved, but the rest had to go
It took a few days to get the basic functionality back, and another week to get the site back to something worth having, and it will take another 2 months to redo all of the blogs and pages
Obviously, I now have added extra layers of security such as WordFence and iThemes, and they are doing a good job in informing me what’s going on, and adding layers of security, but most crucially, every time I update a plugin, I scan it for issues. I haven’t found any yet, but I am going to be a lot more vigilant
The attacks still continue
I have also added some other layers of security that I won’t talk about here, given that they are still trying to hack me, it’s best to not let them know exactly what I am doing!!
The hackers still attack my site every minute of every day, looking for those backdoors, You can see here that the attacks are now about once every 15 seconds, and they come from all over the world
- Be vigilant – not all upgrades will make your life easier
- Scan for vulnerabilities every time you do an upgrade
- Back up your entire site -database and all – page and post backups are insufficient
- Look for unusual activity and behaviour such as slowing response time
- Put in security software as part of every install
- Don’t wait till you get hacked to do something, do it on every install
For those of you who want to identify whether you are being attacked by the same hackers as me, look for any files of this type
The attacks against the backdoor have this format
where the section after the “?” defines a previous hacker identity