The future of data storage?
 

I understand that more computers are going to start shipping without DVD drives because optical storage media is supposedly going the way of the floppy disk. Macs latest OS doesn't even support an external floppy drive. I'm afraid in a few years it won't support an external DVD.
I have terabytes of data on DVDs and I'm afraid it won't be accessible in 5 years. What will the ultimate storage solution be?

Solid state drives, think USB thumb drives. I have a 32GB thumb drive, it's unbelievable that so much data can fit on such a small, fast device! Hard drives have ball bearings!

I gave up backing up to DVD years ago.

Had a couple gig of stuff that needed backing up, bought a sleeve of DVD's, started doing the math and it was days of work.

Hard drives were/are dirt cheap so now I buy one or two big drives a year, load em up and stick them on a shelf.

Hard drives were/are dirt cheap so now I buy one or two big drives a year, load em up and stick them on a shelf.

What bothers me is the long-term viability of media.

In the end, does it really matter? Print the photos you really want. As for the rest...I'm almost of the mind that if I haven't accessed it the last couple years, it likely can go away. I have a stack of smaller HDs that I've been thinking I should consolidate, go through the files, etc. Then I think about the scope of it and instead think, "screw it."

I can't think of any affordable form of media that won't decay over time. Even the old yellowing photos printed on photo paper are decaying while stored in shoeboxes across the world. I just took a bunch of old photos out of the musty decaying photo albums and scanned them into digital format and I was shocked as to the amount of decay they have gone through in only 30-40 years. Who knows how long data will remain on magnetic media?

Magnetic media isn't going anywhere any time soon. SSD's still have a few issues to overccome. Cost is only one of them.

I'm worried about the mechanicals of the hard drive functioning when I need it ten years from now, plus all magnetic media seems fragile to me.

Hard drives, the platters specifically, are the most durable option that the average consumer can afford.

hah! I have templates and other things on floppies. Can't get at it. I see that future coming around again.

I would not really worry about that. Hard drives are sealed air tight. Barring a fire or some other catastrophic event I would not lose sleep over it.

Your bigger issue will really be do you have a PC 10 years from now that will have a compatible interface for the HD.

Couldn't you just buy a legacy computer on Ebay for a few bucks to access your data down the road? There are millions of them out there.

There will be (and already is to some extent) a market for old computers simply because they can access certain things and support devices that have been EoL'd and are no longer supported.

Most digital content is junk anyway, if you really think about it. I have some things that matter (portfolio/marketing stuff, scans of all my important documents, photos of all property of any value, etc.) but those exist on my computer's HDD, on a couple of DVDs (one is off-site, one is in a fire safe) and online (Carbonite). Some stuff I have the paper copies of too. I think I'm okay.

My critical media (documents and photos) are online and on local storage. Online storage gives you the 'forever' capability - perhaps to a fault. I don't bother with manual backups anymore. If my local data was lost I can easily restore it with an internet connection. Consumer stuff (music/video) is already available via streaming and as connectivity (access and speed) matures it will be the 'new' storage format. IMO.

I ended up doing just that last year in order to access some floppy disks - got one of those little Macs that looks like a toilet seat (iBook?). Just bought a new Macbook Pro and I'm not getting rid of the old one. All that obsolete software may come in handy some day.

Cloud is the future of storage. Apple's doing it, and you've got services like Dropbox.

Convenient, but kinda scary too: I don't want all my info out there where it is far easier to hack into.

-Z

Maybe I'm weird, but I absolutely hate the idea of losing my data. Plus, I hate the idea of putting my data out there on the internet for a hacker to get access.

So, I do backups the old fashion way. I first do daily/weekly backups of my hard drive to a second hard drive in the computer. This protects from primary hard drive crashes.

To prevent total data loss from fire, once a year I burn the entire backup folder onto double-sided blu-ray discs and put them into the storage unit.

If the house burns down I only lose a year's worth of photo's, music, video's.

The only chance of data theft is someone breaking into my storage unit and grabbing the discs. Then they have to figure out the encryption code and unpack the files onto a hard drive... Good luck on that!

Slow and rather low capacity, but boy do they last.

http://fineartamerica.com/images-med...et-granger.jpg

Do you really think it's easier for someone to hack an Apple (or Google, or Microsoft, or whoever) data repository than a home network? Just curious and I'm not an IT guy, but I'd tend to think that well-funded data repositories with everything to lose reputation-wise take data security very, very seriously and employ lots of cutting-edge technologies and probably have lots of the "best and brightest" constantly shoring it up. Yes, it represents a central target, but I'd think if there's anything in the world that would be well-defended, those sorts of storage places would be it.

I'm more worried about some knucklehead hacking into my WiFi, but (1) I lock it down and keep it secure and (2) anywhere my WiFi signal reaches can also be reached by at least five different caliber weapons from the upstairs of my house, so I'm not too worried.

I think the BIGGEST threat to data security (for me personally at least) is copying crap onto flash drives (or smart phones) and then forgetting about it. I almost had that happen once. Never again. Be VERY careful with those things and use encryption (i.e. BitLocker or similar) if at all possible! Get a phone that can be tracked/wiped/bricked if it EVER goes missing also - and make sure you put at least halfway decent password protection apps on anything sensitive.

Online storage is very safe and is rapidly becoming more accessible for the general public. As PoP mentioned keeping data secure is their top priority, well other than attracting paying customers, their business model relies on keeping stuff locked down. I have been to several Cloud conferences and at the last event it was mentioned that if enterprise is not spending at least $100K/year on security they would better off (safer) keeping data in the Cloud. If it makes people feel better they can always encrypt sensitive docs/archives before uploading to the cloud.

the long-term viability of media is critically important for scientists, engineers, lawyers and many other professionals -- for artists and historians in some cases too

BTW - Google is about to give us a free 5 G of cloud storage -- you can upload everything and let them pick thru it for evil purposes, just like they did with their drive by wifi collection efforts

Due to our enormous storage needs we're testing a companies holographic/laser storage unit. The unit is good for a 100 years and data is accessed in 200 milliseconds or less. The big advantage here is you can write data on a thousand angles in one space.

If you're relying on cloud, you're still relying on rotating media. Its just in someone else computer.

Yes, but the owness is on them to ensure the data is not lost or corrupted. So the liability in the case of data loss is on them.

To those who feel cloud data storage is safe: may I point out the recent hacks into banks and credit card companies where pertinent data was stolen. If they hack into there, they can certainly hack into the cloud.

-Z

why not just use what the NSA uses?

Likewise. I have 3 bankers boxes of VHS tapes (aviation, historical, natgeo type stuff) I thought I would like to attempt to re-do. Too much time, effort & money to transcribe, especially the time involved.

Cheers
JB

Z,
You are right. Just pointing out that you're not getting away from (using) media.

True.

May I also point out that solid state also has a limited life -- it can only do so many reads and writes during its lifetime before memory sectors become unreadable/unwritable. BTW: CD's DVD's, and BD's also have a limited shelf life, before the silicon backing starts to peel, and the data becomes corrupt. All digital media has a limited life expectancy. Most modern storage devices have longer life expectancies, but nonetheless, as far as I know, no digital media lasts forever. At least not yet... And even if a CD were to last forever, in 4-5 generations of removable media, there won't be devices out there which can read that data -- backwards compatibility can only go so far. Same applies to your USB stick -- how are you going to read data off of that media when USB ports have been replaced by the next best interface port?

I know that a few years ago, the brains at IBM were working on a new type of non-magnetic media - one that uses divots on plastic for the 1's and 0's of the digital work. These divots were placed on plastic on the nano-level (ie: very small dots). A series of prongs would sweep over the dots and interpret the data, kind of like a music box does. The benefit: it was not subject to magnetic corruption, and the data could theoretically be kept forever. the drawback -- not ideal for storing in hot environments. Not sure how much they will pursue that technology, as solid state storage seems to be the direction most storage vendors (EMC, IBM, NetApp, Hitachi) are migrating towards. (Still very expensive media...)

Incidentally, I am a SAN/Storage architect for my company, and have been involved in enterprise-level storage for close to 20 years. In my opinion, the best way to protect data is Raid technology, combined with data duplication in an offsite facility. But while this methodology works for a corporation, it can get too costly for the average consumer. Hence the cloud, which typically will offer at least Raid technology, and possibly data duplication.

-Z-man.

Z,
You have been at this a long time. I was with IBM when re-writable optical media was new. (Early 80's). We have replaced all of the SSD due to the failure mode you mentioned. We use only HHD's (our own drives) on the factory floor.

I'm an IT professional and the first thing they taught me at Uni was that it's only a matter of time before you'll loose data.

As Z mentioned the best option right now is raid arrays with secure off site storage. Anyone can duplicate this setup by have multiple redundant hard drives at home with a backup copy stored off site either in a bank safety deposit box or a relatives homes.
Either way you have the problem of security.

Cloud is all the buzz right now but it does have it limitations.

BUT ask yourself... If you lost 100% of your stored data right now, what would happen? Nothing? ;)

I respect your credentials Z. I will offer up that RAID is not a sure thing and I was a victim to a massive RAID failure while supporting the Marines in Korea. I was the late shift on a 24hr watch and had to transfer application servers to the lone functional server HW after 2 drives died simultaneously on a RAID 5 configuration at 3AM...just before the brief.... I became a smoker again for the next 12 hours...LOL.

The Cloud offers massively replicated data with failsafes. Short of a global catastrophe your data will be there...even if you are not. ;)

We all already do. We just don't know it.

They're no doubt backing up all your data and info. already.

So I'm supposed to buy a hard drive and just give those away when people want their photos ?

Spot on.

Funny, as our tech allows ever bigger faster storage it also becomes evermore volatile.

A really big solar flare could wipe out all the data magnetically stored on earth. This includes HDs, tapes, etc...not sure about SSD and DVDs, but imagine the mess it could cause. Pretty much a temporary halt of modern civilization.

I read a few years ago that the Library of Congress is recording 2nd or 3rd backup recording voice and music to vinyl, because all you need is a record needle to play it. Makes sense, I have a wind up gramaphone and a bunch of early 1900's music that I can play.

Try Blu Ray. Much quicker, much larger.
Can't wait till the price of blanks drops down lower though. They're still a bit pricey.

Have been using the cloud for years now but also backing things up on several hard drives and stashing them away. If I need them to back up lost data they only have to work once and thats usually not a problem.

Like others, anything really secure will never go on a cloud where it can be hacked...

I think Verbatim will make a big comeback...

That would be awesome!