Long term data storage

red-beard · red-beard's Garage

My brother in law had an interesting question yesterday. What is the best media for long term storage of large amounts of data? He writes music and produces his own CDs. He said he might end up with 20GB of data with all of the raw tracks and intermediate data.

This is too big for even a blu-ray disk.

So we have USB hard drives vs. Flash drives

Which would you choose and why? Or is there another tech?

KFC911 · 12-28-2010, 04:33 AM

Regardless of media, I'd suggest at least two backups of the "mission critical" stuff as any single media (whether burned, drives, etc.) is a single point of failure. I've got hundreds of live shows on DAT/CD...I don't consider them permanent archives (...only time will tell).

legion · 12-28-2010, 04:50 AM

I'd look into a RAID 10 SAN.

Every type of media is succeptible to failure and degradation over time. With a RAID 10 disk array, you defeat this by periodically swapping out hard drives and having every bit written to more than one place. This is about the only way to protect from media failure, though you are still screwed if your building burns down.

id10t · 12-28-2010, 05:31 AM

Quote:

Originally Posted by legion

I'd look into a RAID 10 SAN.

Every type of media is succeptible to failure and degradation over time. With a RAID 10 disk array, you defeat this by periodically swapping out hard drives and having every bit written to more than one place. This is about the only way to protect from media failure, though you are still screwed if your building burns down.

+1, although RAID 5 would probably do it. Then set up an identical system and keep it off site and sync everything back and forth on a regular basis to protect against fire, flood, etc.

GH85Carrera · GH85Carrera's Garage

Long term data storage is a pain. It all comes down to how much do you want to spend and how safe do you want to feel.

With only 20 gigs I would suggest multiple flash drives, and a simple dual drive RAID. You might even look into storing that "on-line".

KFC911 · 12-28-2010, 05:38 AM

You guys are offering nice $olution$ but....just a couple of backup USB/Flash drives (backed up and rotated periodically) with one kept offsite at Red's house would work too

.

id10t · 12-28-2010, 06:12 AM

Quote:

Originally Posted by KC911

You guys are offering nice $olution$ but....just a couple of backup USB/Flash drives (backed up and rotated periodically) with one kept offsite at Red's house would work too

.

Not too much $ - it is all about how critical it all is... I build a 2tb (usable) RAID 5 array using an old computer ($25 on c list if you don't already own one) a $40 controller card, 3 1tb drives ($99 each), and Linux (free). Could save some $ by going to RAID 1 instead of 5.

Halm · 12-28-2010, 06:22 AM

Quote:

Originally Posted by id10t

Not too much $ - it is all about how critical it all is... I build a 2tb (usable) RAID 5 array using an old computer ($25 on c list if you don't already own one) a $40 controller card, 3 1tb drives ($99 each), and Linux (free). Could save some $ by going to RAID 1 instead of 5.

RAID 5 needs a hot spare if mission critical.

Also, from what I have read and been told, USB drives are the bottom of the mfg's Q/C barrel: The best go to the SAN mfg's, next to PC makers, and what is left become USB drives. Personally, I have seen too many of these fail and I wouldn't trust long term storage of anything on one of those drives. I much prefer id10t's suggestion to any USB retail device.

legion · 12-28-2010, 06:26 AM

I can confirm. I've had three USB drive failures in the past 5 years.

I've had 0 hard drive failures in the past 15 years.

I've had a few CD-ROM failures, but that is because of physical abuse (scratches).

matthew-s · matthew-s's Garage

I've posted this before, but what about using Amazon S3? It's in the cloud, they are "responsible" for further backups, and its accessible from your PC using a tool like Jungle Disk.

Personally, I don't make it my exclusive point of storage (although I would not be surprised if a number of businesses do this), but it makes for a great off site backup, and it's only a few dollars a month.

legion · 12-28-2010, 06:30 AM

With those services, you are paying for space on someone else's SAN. Not necessarily a bad thing. They will manage the physical location and the hardware for you.

But, if you have privacy issues, these services are not a good thing. There's no guarantee that someone won't steal your data for their profit, use it to embarrass you, or use it to prosecute you.

KFC911 · 12-28-2010, 06:36 AM

I'm guessing that Red-Beard's BIL (musician) is looking for a cheap solution for his "valuable to him" music files. Granted, I've never used USB drives, and even if they're not "the best", using two (both are "backups") with one offsite might still be a better viable low cost, low tech solution for casual backups. Regardless of media, it's critical that one backup copy be kept physically offsite.

matthew-s · matthew-s's Garage

Quote:

Originally Posted by legion

With those services, you are paying for space on someone else's SAN. Not necessarily a bad thing. They will manage the physical location and the hardware for you.

But, if you have privacy issues, these services are not a good thing. There's no guarantee that someone won't steal your data for their profit, use it to embarrass you, or use it to prosecute you.

Very true. I was encrypting the data out there - however it's all family photos that are probably boring by my own family's standards, let alone anyone else. My main criteria was:

1) offsite
2) inherent redundancy / backups

Security is a distant 3rd given my content. I strongly agree that one needs to take a hard look at their own privacy needs.

TechnoViking · TechnoViking's Garage

USB drives are ~$100/Tb. Buy one, back up on it, replace in a couple years.

red-beard · red-beard's Garage

This isn't 20GB of data, it is 20GB per CD he produced.

I suggested a RAID 1 Iomega NAS (only 2 Disk drives), which have 1TB effective (2 x 1TB drives RAID 1). They also make a 4 disk setup which can be RAID 1/5/10.

The 2 drive unit is about $275. I run this for our business with a 1.5 TB USB drive hooked up to it which backs up the main unit every day with a 7 day rotation.

He's buying 16GB and 32GB usb flash drives.

It seems we've outgrown CDs and Tape drives for backup.

KFC911 · 12-28-2010, 07:37 AM

Quote:

Originally Posted by red-beard

This isn't 20GB of data, it is 20GB per CD he produced.... .

Well nevermind me then, except for an "offsite copy being critical" part

azasadny · azasadny's Garage

I have all of our family data (videos, music and photos) on a Windows Home Server (redundant storage) and an "off-site hard drive with all data in my safe...