3

I've inherited a web server that's been compromised. Trying to figure out why apache was hanging and the causes of high server load, I found several copies of perlbot 4.5 in /tmp. I'm now trying to figure out how they got in the machine so I can close the hole(s). Ive been looking at various scanners, nessus seems nice, and I ran a scan on the machine and one of the websites hosted. But there are a couple hundred sites, too many for anyone to know the ins & outs of all of them, and I'm new here so I really have no idea what they might be doing. Is scanning each site the best option?

How would you check so many sites on the same machine for issues?

EDITED TO ADD: we are wiping everything and restoring from back ups. Which is good but still leaves us open to the original vulnerability. Scan each site one at a time with Nessus or Metasploit to try to figure out what that vuln is?

EDIT 2: It was phpmyadmin. Even though that would have been something I would have upgraded as soon as I noticed we were running it, I found out the problem specifically by pouring through apache logs. nessus and metaspolit were neat but not helpful. (I may not understand how to fully utilize them though, I just ran basic automated scans).

karmet
  • 209

4 Answers4

7

Backup the sites, rebuild the server, then add back only the files that you know are needed and That do exactly what you expect.

It is actually less work than attempting to clean up a compromised system.

If you are dead-set on finding vulns, patch the server and then run metasploit against it. I'd also watch netstat to see what connections are being made. I'd also disable any accounts that don't belong there. Check what is started at boot for oddities. The list goes on...

http://sectools.org/ has a list of their top 125 security tools, which you can filter somewhat. This may give you some assistance choosing a tool, but for this you are going to have to do your own research.

The last time I needed to use metasploit or nessus was years ago, so I don't remember the specifics of using them anymore.

gWaldo
  • 12,027
1

Given that you have decided to rebuild the server, as a post-mortem you can do the following;

Make a READ-ONLY Image of the bad machine and store securely

Make a copy of the filesystem on the offending server. To do this you will need another drive or mounted volume with sufficient space; Look into the dd tool. How you use it will depend on whether you have a single big root "/" partition, or lots of /var /usr and /lib filesystems etc.

Conduct forensics in a secure sandbox environment

Move the copy of the offending image of the file-system to a secure location, such as a guest ubuntu in a VirtualBox. Specially prepare the sand-box virtualhost. Install tools such as sleuthkit, foremost, clamav, autopsy.

Conduct the post-mortem in a safe and sensible manner

Don't give the sandbox access to the internet or local drives. Delete the sandbox guest after the process is over. Make notes, document changes.

Tom
  • 11,611
0

The first thing to do is take all the sites offline. If your sites are compromised, they are probably being used to host nefarious files and/or to send spam.

The second thing is to alert any transactional banking services that any of the sites use.

The third thing is to alert your user base. These three steps can be done under an hour and there is no excuse if you haven't done so already.

Me_
  • 1
0
find / -mtime -1

Replace -1 with the number of days since when you think the intrusion happened. By finding the files that were modified, you have a fairly good chance of figuring out which site was used to get in. For instance, look for uploaded backdoor scripts in image folders and things of that nature.

cdonner
  • 391