Backup vs Archiving
A while back I posted an open question on twitter regarding backing up verses archiving, and it seems that everyone is in agreement regarding the core difference for media professionals. What I have yet to understand is why these same media professionals who understand the concept of long term archival long to use backup practices to achieve this. Either deploying binders full of optical media, shelves full of USB driven external storage, or a huge rack of SAN.
Simply trying to adding to a SAN or a RAID strategy for accommodating an every growing amount of data is NOT the path to archival bliss. Studies by IT experts show that the costs for doing this are deceptive – individual disk drives are cheap but not the infrastructure to support them. Adding rack space, power, and cooling requirements is just impractical. Also according to the Clipper Group Inc., the costs for a Terabyte stored long term on SATA disk versus LTO data tape is about 23, to as much as 290 times more! The issue with trying to grow your storage you will always grow faster than you anticipate.
The best path is to only keep as much data on-line as you need for active and repeat projects. You CANNOT keep it all ONLINE. You see, backing up is what you should be doing for disaster recover. Backup is what covers you if you have a hard drive failure, it what you use if you have a fire, flood, or something else horrible happens to your home or office. Archiving data on the other hand is very much like archiving any other kind of assets. Moving data from a hard drive to a long lived storage medium like that of a LTO tape. Data tape that is stored on the shelf or in an automated storage. The point being that not only do we have records, we want to assure that they are preserved for the future and easily accessible when any time a single file is needed.
You may feel that once your project is finished, you’ll never need to recreate the project is one thing. But to just dump all of the source material (paid or otherwise) is just misguided especially in the era of solid state production. Because of solid state shooting (CF / P2 / SDHC / SxS ) regardless of the format you have to find a stable solution for long term storage. To date there is only one solution that I know of that will service the needs of my fellow media professionals.
The Cache-A Archive Appliance
While I don’t want to turn this subject into a sales pitch, or a product review (I’ll publish a full review soon enough). I do however wish to introduce you to a very solid solution. The main thing that attracts me to the Cache-A, is that it’s a true appliance! I love features like automating the archival process, the fact that it writes to universal compression standard (TAR) and doesn’t manufacture any propriety information (other than the table of contents that written to each tape). There are several “roll-your-own” LTO soultions out there for you to consider, this so far has been the only one that comes on it’s own total solution. (Linux Based Server, Virtual Tape Library, and on board RAID to hold the on-line data). All for about the same cost as a roll your own solution – with out propriety structure. More information on the Cache-A.
This is why I love Brad’s blog. Amazing, informative and well thought out. Great post, Brad!
Thanks Brian – so how are you archiving your projects long term?
I’m not. Lame, I know. I need a good backup solution and the cash to fund it.
Brian, don’t sweat it, I have some 3½ inch floppies in the garage.
I’d hate to format your only copies of Windows 3.1!
It’s cool – I have an extra set of Win 95 3½” so that give you a extra 19.5 Mb!
I’m trying to find a lower cost of entry – and I’m also kicking around the idea of having Powered Production becoming a sort of Iron Mountain for Media guys – just don’t know if people would actually pay for someone to archive their data to an LTO for them.
Is there a market for such?!
Most recently, we have been working with a vendor that has helped me shape the way I’m starting to think about all storage and media, generally speaking. So let’s use the concept of an old analogy – your desk that you write and work on. Think about this – do you work better on a desk that has ALL files, papers, notes, pencils, media, etc., etc., stacked on it where you try to work on the only thing you are focused on right now, or do you try to clear away as much space as possible so that you have an efficient use of space for the work you are focused? Most say the later and if that’s the case, then what’s the difference when we are talking about a project that moves from a physical desktop space to a virtual one. What I mean is why are we continuing to keep data on spinning media or any sort of media that consumes power and resources if we are not using it? Think of your “storage” or “media” like this: 1) How much space do I need to work on the project or projects that have my attention right now? 2) While I’m working on it, since its the most critical, how am I going to back it up and protect it while I work on the completion of it? and 3) Once the project or projects that I’m working on are done, where do I keep an archive of these such that they do not compromise #1 and #2 as well as ensure that I will always have access to these projects? It’s a thought process of workspace (#1), protection of workspace to keep my current project(s) from damage until the finished project(s) (#2) and archiving that does not cost me spinning disk or system performance, yet, allows me to retrieve data from old projects easily and efficiently (#3). Within each of these 3 are tons of topics of discussion. For #1 its everything from sizing the disks you use to match the application performance and even have some protection to optimize efficiency and up-time to proper RAIDing and even spindle counts. For #2, it might be as simple as dumping to DVD’s or some sort of NAS/SAN, but doing this will save your butt. And for #3, its long term retention that is always accessible no matter what changes in technology – tape is the most logical. Great article Brad and something that everyone needs to be thinking about regardless of the data type.
Oooh! Love the article! This is definitely a topic I’ve been thinking about lately and haven’t had the time to put into researching options. Thanks Brad!!! You’re awesome!
Thanks so much! I’m developing a post off-line that will compare the process of building your own archive device – but I’m not sure that I’m going to come anywhere near what the Cache-A prime offers.
I’m also kicking around the idea of diving in and purchasing one for Powered Production (yes – I sell Cache-A product at work). I’m thinking of offering the service of archiving and off-site storage for media guys who cannot make the $7k plunge.
Wow lovely post @DSLRGIRL lool dnt be lie ???really u r also think about it ?