Try using mirrordir. With an appropriate script, it seems to be the ideal solution for you. It only updates the files which have changed, (modified, created, or deleted,) but also has the capability to preserve old files. I'm not sure how that function works, but it shouldn't be hard. Here's the script I use: (Edited somewhat for clarity. Hope I didn't cause problems with the edits)
#! /bin/bash
logfile="/home/share/Backup-log.txt"
echo "" | unix2dos >> $logfile
echo `date`" /bin/mirror_backup started" | unix2dos >> $logfile
echo ""
echo ""
echo "mirror_backup Automatically archive a list of"
echo " directories to a storage location"
# Mount mirror drive
mount -o remount,rw /mirror
xstatus=$?
if [ $xstatus -ne 0 ]
then
mount -o remount,rw /mirror 2>&1 | unix2dos >> $logfile
echo `date`" Mount failed, aborting /bin/mirror_backup..." 1>&2
echo `date`" Mount failed, aborting /bin/mirror_backup..." | unix2dos >> $logfile
mount -o remount,ro /mirror 2>> /dev/null
exit $xstatus
fi
# Define Source Directories
sourcelist="/home /etc /root"
dest="/mirror"
for dir in $sourcelist
do
if [ ! -d ${dest}${dir} ]
then
mkdir -p ${dest}${dir} 2>&1 | unix2dos >> $logfile
# chown mirror:mirror ${dest}${dir}
fi
done
# Mirror directories
for dir in $sourcelist
do
# Delete old files
echo ""
echo "Deleting old files in "${dest}${dir}
mirrordir --nice 0 --exclude-from /root/exclude-list --only-delete ${dir} ${dest}${dir} 2>> /dev/null
# Run full mirror
echo "Mirroring "${dir}" to "${dest}${dir}
mirrordir --nice 0 --restore-access --access-times --exclude-from /root/exclude-list ${dir} ${dest}${dir} 2>&1 | unix2dos >> $logfile
done
# Perform miscellaneous tasks
report="/home/share/disk-report.txt"
echo "Report generated on "`date` | unix2dos > $report
echo "" | unix2dos >> $report
echo "RAID drive status:" | unix2dos >> $report
cat /proc/mdstat | unix2dos >> $report
echo "" | unix2dos >> $report
echo "Disk usage per slice:" | unix2dos >> $report
df -h | unix2dos >> $report
echo "" | unix2dos >> $report
echo "Disk Usage per User:" | unix2dos >> $report
du -h --max-depth 1 /home | unix2dos >> $report
echo "" | unix2dos >> $report
echo "Disk Usage on Share drive:" | unix2dos >> $report
du -h --max-depth 1 /home/share | unix2dos >> $report
echo "" | unix2dos >> $report
echo "Filesystem Usage Overview:" | unix2dos >> $report
du -h --max-depth 1 / | unix2dos >> $report
echo "" | unix2dos >> $report
echo "Report Complete" | unix2dos >> $report
echo ""
echo "mirror_backup complete."
# Unmount Mirror Drive
mount -o remount,ro /mirror 2>&1 | unix2dos 2>> $logfile
echo `date`" /bin/mirror_backup completed successfully" | unix2dos >> $logfile
exit 0
With no changes to commit (second run-through, for example) it takes about 5-7 minutes to scan 1.5 TB of files. Of course, it's a lot slower on the first run-through.
By the way, this script was written by me for my use on my personal server at home. While anyone is absolutely free to use or modify it for themselves, I am making absolutely no guarantees or warranties. It's free, so you get what you pay for. Hope it helps, though!
