I'm backing up a Linux server and storing it on another server.
I began with a simple
rsync -aPh --del server.example.com:/ /mnt/backup
Then someone pointed out that I shouldn't back up /proc, since you don't want to restore the /proc of one server on another.
Is there anything else that I should/shouldn't include?
For instance, what about /sys?
Both /proc and /sys are virtual filesystems which reflect the state of the system, and allow you to change several runtime parameters (and sometimes do more dangerous things, like directly writing to the memory or to a device). You should never backup or restore them.
In most modern distributions, /dev is dynamically created at boot (it is a memory filesystem filled by udev and friends). There is no point in backing it up, and attempting to restore it is futile. However, if your distribution is configured to use a static /dev, this does not apply (check /proc/mounts, if /dev is a tmpfs it is a memory filesystem).
There are other filesystems you should not back up; usbfs (usually at /proc/bus/usb, if mounted at all), debugfs (supposed to be at /sys/kernel/debug if mounted at all, but some people put it somewhere else; you probably do not have this one), devpts (mounted at /dev/pts), other tmpfs instances (often found at /dev/shm, /var/run, /var/lock, and other places; backing them up and restoring them should be harmless but pointless, as their contents are lost on shutdown), and any remote filesystems or magic automounter directories (attempting to backup or restore them could end up in disaster, as you could end up backing up/restoring to a different machine). You should also be careful with /media and /mnt, as external devices (like a CD you forgot in the drive) could be found there, but you might also have used them on purpose to mount something which should be backed up.
Note that, other than mostly harmless tmpfs instances, network filesystems/automounters, and removable media, the filesystems you should not back up are all descendents of /dev, /proc, or /sys. If you have no network filesystems (or automounters), and no removable media, excluding /sys and /proc and rebooting after a restore (to wipe the tmpfs instances) should be enough.
This really depends on how you are going to restore your system. If you will rebuild then you only need the configuration/data files for your services (eg: /etc, /opt, /var, /home)
If you are after a full system restore, then it you could omit /proc, /boot & /dev. Then you can install the minimum OS from your boot media and then restore your system via your backup.
Of course, the best backup is one that has been tested and verified.
So omit what you don't think you need, try to restore in a VM and verify you can get back your system using this data.
Some of the special files in /proc and /sys confuse rsync. You don't want to back up mounted network filesystems either, usually. Sparse files can also cause problems.
Add -x to limit it to one file system. That avoids all the network file systems and /proc etc. However you then need to run one rsync for each file system you have mounted.
Add -S to handle sparse files sensibly.
/boot, /dev and /proc are quite useless to backup -- though, if you know what you're doing you can backup /boot.
I would also not backup /lib, /media, /mnt, /sbin, /bin, /srv, /sys, or /tmp.
/usr is optional, depending on whether you have anything in /usr worth backup up. If I were you I"d worry most about backing up user's $HOMEs, /var, and /etc (for configuration files).
Again though, this really all depends on the type of backup you want to do. Is this a web server? Is this a personal computer? Is this a shell server with tons of directories under /home?
You could achieve a total backup using sfdisk and dd.
To backup the partition scheme of each hard drive, you'd use sfdisk like this:
sfdisk -d /dev/sda > parttable_sda.part
To backup each partition you could use dd, like so:
dd if=/dev/sda1 of=devsda1.img
Where /dev/sda1 is unmounted, such as with a live-CD boot.
(keep in mind you're going to need to have a a lot of free space to write this file; so you may want to write it to an external media) Do that for each partition, one at a time, and back everything up.
Then, to restore on another computer you can do:
sfdisk /dev/sda < parttable_sda.part
dd if=devsda1.img of=/dev/sda1 # do this for each partition
As pointed by this great community:
/dev /proc /sys /tmp /run /media /lost+found /boot ( /boot is optional see other comments)
For reference my final rsync command (under Arch with external media mounted in '/run/media/fred/INTENSO/' and backing up to a folder named 'fred') is:
$ sudo rsync -Pazhmxv --exclude /run/media --exclude /dev --exclude /lost+found --exclude /tmp --exclude /proc --exclude /boot --exclude /sys / /run/media/fred/INTENSO/fred/.
(excluded files could also be specified with curly brackets (--exclude={/dev,/proc}) under Bash or with a text file (--exclude-from='excude.txt')).
-P: show progress -a: archive mode -z: compress during transfer -h: output numbers in a human-readable format -m: prune empty directories -x: limit to one file-system -v: verbose
Instead of excluding, I usually only backup what I want. Including: /home /etc /var (except /var/log)
Basically, pseudo-filesystems (/proc, /sys, /dev/shm...) need not be backed up.
I'm on a Ubuntu 18.04 machine and I have these excluded:
/dev/
/proc/
/sys/
/tmp/
/run/
/mnt/
/media/
/lost+found/
/cdrom/
/swapfile
Also, specificially for my setup, I exclude these:
/home <-- Backed up separately
/backup <-- Mount point for backup disks
/data <-- Mount point for data disks, which are backed up off-site
/scratch <-- Mount point for volatile fast SSD scratch disk
I generally make a habit of backing up everything on a system, even the stuff that I know for certain is useless to back up. It's simpler to set up and you can be 100% sure that you're going to get everything you do need included in the backup.
I'm using an Ubuntu linux box as a testing server for website development, and for hosting a documentation wiki. Each night a crontab dumps the MySQL database into /var/www, and then all of /var/www is zipped up and replicated to the backup server. It's not ideal, but it's enough. I had to rebuild the server at one point, and all I really missed were the Apache and Samba configuration files.
I would assume you don't have Linux on a virtual machine. If at all possible, I would urge considering moving to virtualisation. Backups on vm level are a whole new level of consistency and ease of use. There exist free virtualisation tools, so you don't necessarily have to invest into VmWare or other expensive monster tool.
Question: Which directories should I exclude when backing up a server?
Here is a script I use often, from a Ubuntu 16.04 LTS laptop to a Ubuntu 16.04 LTS server. It clearly shows which directories should be skipped while making a full backup:
echo "EMPTYING TRASH"
rm -rf ~/.local/share/Trash/* >/dev/null 2>&1
echo "DELETING OLD LOGS"
sudo rm -f /var/tmp/* >/dev/null 2>&1
sudo rm -f /var/log/*.gz >/dev/null 2>&1
sudo rm -f /var/log/kern* >/dev/null 2>&1
sudo rm -f /var/log/messages* >/dev/null 2>&1
echo "DELETING CHROMIUM CACHE"
rm -rf /home/pi/.cache/chromium/Default/Cache/* >/dev/null 2>&1
echo "====================================================================="
echo " BEGINNING RSYNC from PAV root to PRIME5:/mnt/full/pav"
echo "====================================================================="
time sudo rsync -aAXv \
/ \
--bwlimit=500 \
--delete \
--delete-excluded \
--ignore-errors \
--exclude="/dev/*" \
--exclude="/proc/*" \
--exclude="/sys/*" \
--exclude="/tmp/*" \
--exclude="/run/*" \
--exclude="/mnt/*" \
--exclude="/media/*" \
--exclude="/lost+found" \
abc@prime5:/mnt/full/pav
echo "====================================================================="
df -h
Note the exclusion of /mnt - which is where each Ubuntu system has a full-time backup drive mounted for cron-based rsync self-backups 4 times a day. These drives are mounted by entries in fstab and are always present. To include them in a backup to another system would be duplicative.
Similarly, /media is where USB drives get mounted. They are backed up separately.