Image Mechanics Off-line Archiving System: The Next Generation
It’s been over a year and a half since first developing the unique archiving system that we have since used to safely and efficiently store the terabytes upon terabytes of image data that our clients consistently generate. In that time, the system has barely changed, and we feel that we can safely say this is still the most economical, practical, and easiest long-term archiving solution out there. (For our purposes, anyway.)
However, in that time we have also finessed certain aspects of the system, and figured that it has matured to the point where it would be a good idea to revisit and show exactly how we currently make it work.
One important aspect of the system that we didn’t cover in much detail last time around is the A/B archive set concept. This is the redundancy part of our system. Every archive disk has a B counterpart containing the exact same data. We make it simple by labeling each archive set, for example, “Archive Disk 30A” and “Archive Disk 30B”. Every time we dump a new shoot on to Archive 30A, we also put it on 30B. The B disk provides a safety net in case the A disk fails, and once that archive set is full, we move the B disk off-site for additional safety (we’ll get to that in a second).
The newest and most useful addition to our system is FirmTek’s recently released SeriTek 5PM five-bay SATA enclosure. Since we already use FirmTek’s two-bay models for backing up data while on set, it made perfect sense to invest in this five-bay monster. This way, we can have the two most recent archive sets live, along with an extra bay available which enables us to quickly slap in the set backup disk after a long day of shooting, start the files copying over, and leave. All without even unpacking the computer we shot to.
Also in our previous entry we mentioned using Extensis Portfolio to track which disk each image is on. This is a step we did away with because even though it was relatively simple, it still wasn’t worth the extra time (again, in our case. This step might prove invaluable to some people). What we now do instead is simply keep a plain text document accessible on a network shared drive containing a list of all the job folder names and which disk they are on. Looking up a job is a simple matter of hitting Cmd-F. And, our standard job folder naming scheme tells us the most important info about a shoot right away: date, photographer’s name, and client or subject.
For those that are curious, the “ImageServerNAS” share that you see in the background is the vestige of a failed archiving experiment. For a few months, we decided to try adding a NAS appliance in to the mix in order to keep the last 90 days worth of shoots live on a server. We quickly determined that the NAS was not suitable for our purposes. First of all, it was slow as molasses. It would take on average one to two hours to copy over the typical shoot. Second of all, and this was the big one for us, Lightroom can’t open a catalog if it’s stored on a network share. We would have to copy the entire shoot over to the machine we were working on if we wanted to browse through it, which just wasn’t practical given how long it would take. The beauty of our current system is how fast we can get to an image in order to process it and have it available for the client to download in mere minutes. We now use the NAS to hold printing and retouching jobs, a task it is much better suited for.
When it comes time to pull out an old job, all that one has to do is open the file list document, find which drive the shoot is residing on, then pull that drive from our customized antistatic pink foam storage drawer. Slap the drive in, and the shoot is live and looking just like it did the day it was shot.
Remember I mentioned taking the “B” disks offsite? We came up with a convenient process for that as well. Pink foam inserts custom cut to fit in a Pelican 1520 case. This case gets stored in a safe place miles away.
We also slip in a printed copy of the Archive Disk File List text document so we have a hard reference for what is on each disk in case of disaster.
Addendum- funnily enough, as I was writing this article, we got an image delivery request from a shoot we tech’d on over three years ago. The shoot resided on Archive Disk 1 (no kidding). But sure enough, the disk mounted without issue, the files (DNGs) opened exactly as they should, and we had the images processed and ready to be downloaded within 15 minutes.
No fooling around, the system works!
Add to:| Digg it | +Google | Slashdot
2 Responses to “Image Mechanics Off-line Archiving System: The Next Generation”
Leave a Reply
You must be logged in to post a comment.
July 18th, 2008 at 11:26 am
Hi there,
I’ve been looking at your site for over a year now and never had the time to get involved alittle more.
I love what you do, I wish we could educate this business to digital a little more!
It’s so frustrated sometimes, people don’t have a clue……..
……
Story:
“a” magazine request the select for the cover, unfortunately they don’t know how to use their FTP
or as they say : We are not allowed to use it………( what do they have it for???)
I send a CD with the retouched cover and the opening to the story inside the magazine.
About a month later ( when their free archiving time was well over the limit of 60 days)
they call me in a panic saying that they lost the CD and they needed the cover right away.
Luckly I just got back from a 2 weeks long vacation.
This magazine is part of one of the biggest publishing companies,
nobody thought to copy those 2 files to one of the 10 computers they have in their
“creative department”.
as bruce use to quote say:” It’s not easy being green!”
mk
July 18th, 2008 at 11:30 am
I forgot to say,
I love your archiving system and I’m building one just the same for a client of mine which I’ll be maintaining.
Hopefully my estimate is close to be approved.
Thank you so much for that!
You guys rock!!!
mk