February 11, 2008
CalDigit
by Jared Picune
Creativecow Leader
Editor/President of Idea Spring Editing, Inc.
Often overlooked, video storage has always been a central and essential part of any edit system. While the type of media may change from one edit bay to the next, the need remains the same; fast reliable storage. Over the years there have been numerous advancements in video storage technology, leaving us with ultra cheap solutions, as well as extraordinarily expensive ones. The task of deciding which to use, can be a daunting experience.
I often think back to my fist edit system. I had an original blue and white Apple G3, running Radius EditDV. For storage, I had bought and external SCSI drive, all of 4GB in capacity. Today 4GBs seems ridiculous, as does the thought of SCSI. I still have the drive today, but it's sitting on a shelf collecting dust. I can't bare to get rid of it, even though it's practically useless.
As time passed my edit systems changed, as did my storage needs. First I tried an internal software RAID system, followed by an external SCSI array. Then it was time to get serious. I needed an extremely large storage capacity that was supper fast. At the time the Xserve RAID was the obvious solution. So I bought one and have been using it ever since. The Xserve RAID, is a beast, in both good ways and bad. It's expensive, loud, and heavy. But it's fast, reliable and sexy.
Today there are so many more options. The problem is they're not all great. Each editor works differently, and has different requirements to fill. There are so many different factors to consider, and options to weigh. I need a storage solution with a large capacity, fast and reliable performance and the ability to expand in the future.
When I first saw the CalDigit HDPro, I have to admit I was awestruck. Crazy fast performance with massive amounts of storage, it was love at first sight. Then reality set it, there had to be a catch. What did this thing cost? Well, it turns out the cost was not the catch. The HDPro is relatively cheap especially compared to what I paid to the Xserve RAID. I could get the HDPro with almost double the capacity of my Xserve RAID (and twice the speed), for nearly a ten thousand dollars less.
So needless to say, I was really excited when I got the opportunity to get my hands on my very own HDPro. Seeing all the specs on paper, and getting results in the edit suite, are two different things. I really wanted to put the HDPro up to the Xserve RAID, and see how they differed. So I did.
Obviously the two units are not exactly the same. So to be fair I set some ground rules and established the obvious differences between the two:
Both systems would start empty. The Xserve RAID with about 2.5TB (RAID 50) and the HDPro about 4TB (RAID 5). Both edit systems (hosts) were comparable. The Xserve RAID was on a dual G5 2.5 GHz with a Kona 2 card, and the HDPro was on at Mac Pro 2.66 GHz using the Multibridge Pro. I would perform all of the same tests (when possible) on both systems. Now, I am a working editor, so I have a ton of work to do beside testing these units. I have to admit that there are times that I just needed to get a project done, and additional files and or changes were added to one or both of the drives. Same version of FCP on both systems. Same video formats and footage for all tests. None of these tests are scientific, I just wanted to test practical things that I do everyday and find the differences between the two systems. A notable difference is the type of drives used in the RAIDs. Both use 7200 rpm drives, but the HDPro uses SATA II drives. Where as the Xserve uses Ultra ATA.
I really wanted to compare the setup of the two systems. But, I have had the Xserve RAID mounted in a rack and setup for nearly four years now. I don't exactly remember the process, however, I do remember having to print out a fairly lengthy PDF with detailed instructions. Nevertheless, I was more than excited, when the HDPro showed up. One thing lead to another and I had the HDPro setup before I even considered comparing the process. With that said, I can tell you that I found the setup for the HDPro to be very easy and quick. The drives are shipped outside the enclosure and all you have to do is slide them in. Then just install the card, attach the cable and you are good to go. The RAID was already setup in RAID 5, which is what I wanted, so I was ready to start editing. With the Xserve RAID, I had to select the RAID configuration I wanted, and wait for it to format. This is a lengthy overnight process.
I was very eager to get results, but before I started any testing I ran the AJA System Test. This software can test drive speed, and is a very usefully utility. The results; the Xserve RAID's best score was 212.2 MB/s, while the HDPro's was 399.9 MB/s. Later on I got results clocking in at over 405 MB/s with the HDPro, I never got better performance out of the Xserve RAID. While I'm not sure what caused the minor difference in speeds, I figure and fluctuation of less than 10 MB/s is insignificant.
And The Testing Begins - Performance:
Just like any project, I had to start by getting video onto they system. I captured 20 minutes of 1080i 10 Bit uncompressed high definition video. The HDPro worked without any issues at all. The Xserve RAID had some issues with dropped frames, however after a restart it captured the 20 minutes of video just fine.
Next I cut the video I had just captured, along with a few other things, into a timeline and let the video play overnight. All material was still 1080i 10 Bit uncompressed video. Fourteen hours later the HDPro was still looping the timeline, the Xserve RAID had dropped frames and stopped playing.
Now, in the four years I had been working with the Xserve RAID, I have seldom had any issues with dropped frames. Although the truth of the matter is, that I rarely work with 1080i 10 Bit uncompressed video. So I wanted to really put these RAIDs to the test.
I called it the "client test" and here's how it works. The client comes in, we edit, client changes his/her mind 87.36 times, they're impatient, I try to make them happy, and figure out what they want. Then I evaluated the experience to see what system things went most smoothly on.
The winner was the HDPro system. While both systems were rather responsive, the HDPro felt faster, hands down. One of the things that I've found over the years, is that the time between hitting the space bar, and the time the video starts to play, is extremely important. Systems using external Firewire drives seam to have an extremely long lag time. Very annoying when trying to work, especially with a client breathing down your neck. The Xserve RAID and the HDPro excel here, and this is one of the most important parts of having a fast RAID. This speed is worth more, than saving a few dollars on a cheaper system, trust me.
I was curious about how the HDPro was able to sustain a higher data rate. Whereas the Xserve RAID had various peaks and valleys. With a little online research I found that CalDigit is using a technology called ASTT (Active Sustained Transfer Technology). Basically it means that the data rate is sustained consistently for optimal transfer rates. So if the RAID sees a big group of large files coming it readies itself ensuing a fast and reliable transfer.
Along with performance comes noise. The Xserve RAID, is one noisy system. I mean the thing is loud. It's white noise, so you get used to it. But let me tell you, you really don't want it in your edit suite. The HDPro is really quite. Now, you can hear it when it's the only thing running, but I can't even hear it over the Mac Pro, and definitely not over the Xserve RAID. I would have no problem putting the HDPro in my edit suite.
You're able to monitor both units temperatures through software. Both ran about the same 31C - 34C on average. The back of the Xserve RAID felt really hot, while the back of the HDPro was just warm. Both had been up and running for over a week when I made this assessment.
For over a week I made the effort to switch back and forth between the Xserve RAID system and the HDPro system. Editing a little here, editing a little there. Frankly they were both rock solid. I enjoyed editing on both very much.
Protection
To be honest there's not a lot that I could do to tell you about protection. The thing is when you know what happens, you know what happens. Both systems have hardware RAID protection capabilities. The Xserve RAID has 1, 3 and 5. While the HDPro has 0, 1, 5, 6, and JBOD. The systems I was using were protected with RAID 5, but really the Xserve RAID was 50. Basically that means two sets of RAID 5 put together with a software based RAID 0. At least with the HDPro in RAID 5, I can lose one drive. But in all my years of editing, I have never lost a drive (knock on wood!).
The nice thing with both of the units, is that they have hot swappable drives. So if anything were to happen, they can easily be replaced. Then within a few hours the RAIDs get rebuilt and everything is as good as new. I know others who have tested this, and since I have plenty of work to get done, I'm gong to trust them. Even though pulling a drive sounds like a ton of fun!
Both systems offer other redundant features for protection as well. I think the way the Xserve RAID does this is sexier than the HDPro. But in the end, both have redundancy and that's important. I feel like the Xserve RAID is two units in one box, while the HDPro is a single unit. It is also important to note that some of these features are options, so if you're going to buy one make sure to look into what you need. I highly recommend spending a little extra on the options, it will pay off in the long run.
The other feature both systems offers is a software configuration and monitoring solution. The Xserve RAID has RAID Admin, and the HDPro has RAID Shield. Both are intuitive and easy to use lightweight applications. The RAIDs can be configured here, and the software will alert you issue should arise. Both can send out emails, which can be very handy. From what I can tell, the only major difference between RAID Admin and RAID Shield, is that the Xserve RAID can be controlled over the internet. The HDPro has to be directly connected to the host. The ability to check up on things remotely is really convenient, but is a feature I rarely need to use. However the addition of VNC software (like Apple's Remote Desktop) makes this possible for the HDPro. I rather spend my time editing than configuring my RAID, so extra software options are not necessary, but it's nice that they thought of them.
Capacity
Obviously capacity is really similar between the Xserve RAID and the HDPro. The big difference is the way that the RAID systems work. The Xserve RAID uses two controllers that take care of 7 (or less) drives each. The HDPro has one controller that takes care of 8 drives. Don't be fooled by the number of drives, The Xserve RAID is using two controllers to achieve it's best performance, the HDPro only needs one.
Interestingly, you don't have to fully populate the Xserve RAID, and don't have to use both controllers. But you're paying for both even if you're not using both. The HDPro can achieve a two controller setup when using a two port card to connect two units. With a software RAID, performance of nearly 800 MB/s can be reached. The Xserve RAID is now capable of running multiple systems off of one computer as well, although I'm not exactly sure what the maximum performance is.
Then there's SAN. The Xserve RAID has XSAN, expensive, but cheap compared to other options. Likewise, CalDigit has designed the HDPro with SAN in mind. CalDigit has developed a switch for the external PCIe interface. This switch costs much less than the fiber switches that can be used with XSAN. This is very exciting because it may be the first solution to bring a SAN environment to some of the smaller shops, who otherwise, might not be able to afford a SAN solution.
Both systems utilize an extremely fast connection to the host. Xserve RAID uses a fiber channel connection. HDPro uses external PCIe or PCI-x. Many people are not that familiar with the external PCI interface. All it's doing is essentially bringing the PCI bus outside the computer. It has bandwidth capacities up to 20 Gbps. This is much faster than fiber.
One major difference between the two systems is portability. The Xserve RAID is heavy. It's really not something that one person can move. Sure it's possible, but let me extend a friendly warning; bend with your knees! The HDPro is no feather weight, but is much lighter and portable. The handles on top are really convenient and make the unit a breeze to move.
The most awesome part of the whole thing is the ExpressCard option that works with my MacBook Pro. All you need to do, is buy the CalDigit ExpressCard, and attach the HDPro to your notebook computer. Now you can edit on the road. Obviously the bus in not nearly as fast on the notebook, but speeds of close to 200 MB/s can be achieved. While this may not be an every day occurrence, it sure is a nice option to have. You can't do this with the Xserve RAID. In the past year I have had two edits that I needed to do on set. I used the CalDigit S2VR, which worked like a charm, but the fact that I can take my entire RAID with me is very appetizing.
Another new and exciting feature is PCI-x. While most new computers are using PCIe, there are numerous systems out there that still use PCI-x. This means that older systems, like my G5 are now capable of running the HDPro. Apple offers both options as well, but the price is higher, and there can be some complications. The early Xserve RAIDs shipped with a different card and cable/controller system than the latter versions. This can greatly complicate things, especially if you have an older system like me.
So I bought the Xserve RAID nearly 4 year ago, because it was the best solution at the time. Today I have many more options. I have been very happy with the Xserve RAID, and it has run without any major issues. And as an editor that is all I want in the end. But today, knowing that I can save a lot of money, get most of the same feature set, improved performance and capacity. I would pick the HDPro over the Xserve RAID in a heartbeat.
Overall the HDPro blew my socks off! Don't get me wrong, the Xserve RAID is a great reliable product. But considering the cost and the performance, the HDPro is a clear winner for any present or future edit suite.
