Setup a Backup for 90k+ photos and desktop file structure

Question

I'm working for an engineer that takes about 500 pictures a month and uses Adobe Lightroom to manage them. He currently has about 2TBs of data that need to be backed up regularly. To setup a backup for him I was thinking about getting 6 1TB hard drives and then setting them up in a Software RAID 1 configuration with Ubuntu Server. Then his windows machine could regularly back up to the Ubuntu Server. But I dont know if this is the best or most effective method. Does anyone have any suggestions or methods to help me figure this out? Right now I think cost would be an issue for him with the 6 Hard drives we would probably order 8 just so we have two backups.

Answer 1

I am thinking this really is overkill for what a large USB drive could handle much better while offering portability for storing offsite.

Considering what you have described of the engineer's usage pattern I am going to guess that he doesn't work with a fresh batch of 2tb every time, or your storage requirements would be much more insane.

I suggest getting a 3Tb External drive ( or two ) and keeping the data on the drive. This would be increasingly useful If the engineer is in the field with a laptop too. Keep a drive on the road and one at home if you want to be paranoid about replication.

Answer 2

If the Ubuntu server is located at the same place as the main data, you don't have a backup. IOW: Make sure you have an offsite backup.

Other than that, there are so many methods to do a backup that you can just take your pick and see what works for you. However, the server method appears to be a bit overkill just for this purpose. Have you considered a NAS instead? Works the same way from the client side but you don't need to do a lot of management with it.

Answer 3

I also think 1 or 2 external HDDs would be sufficient for his needs. And when he fills them up buy another (set). Something like DVD backups but on a way bigger media.

Adobe Photoshop Lightroom also have the option on import to "make a second copy" of the photos to additional destination aka the external HDD while importing them locally or on the network from the camera or CF/SD card. After that he can sync/replicate the 2 external HDDs if he uses more than 1 for backup

Answer 4

I don't have as many large files as you're asking about, but I easily have a few hundred thousand files that I keep. (Many are old and useless, but I'm a packrat...)

For personal backup, and number of schemes using external hard drives and software like rsync or one of the many commercial tools (I use Second Copy) can be devised.

I think the key thing is to come up with a scheme that works for the user of separating active files (ones that might be changed, renamed, whatever) from older, inactive ones (files that you're going to stick in a subdirectory and probably never change again). In my case, I accumulate files every year, saving them in appropriate subdirectories (2011\personal, 2011\pictures, 2011\whatever), and on January 1st, I start a new directory (2012)

In my case, every file I've ever created (going back to the early 90s) fits easily w/in 1TB. I have several copies of all these files on multiple hard drives that I keep in multiple locations. With the growth of hard drives, if I ever run out of space, I'll probably just replace all of my 1TB drives with 2TB.

In your case, I'd also set up some sort of date structure and once I had something like 2TB of older, static data I'd make a few copies of that data on some hard drives and put them aside, available for reference if necessary, but not used day-to-day.