I'm awful, aren't I.  I start out thinking FW800 is fast, and, after reading a few posts and checking out a few web sites, suddenly even SATA-1 sounds slow, especially if it's software-RAIDed.  O.K., reality check.  I'll be editing this 35Mbps stuff, in HD.  I typically end up with five or six video tracks, and I now understand (from Job) that if I end up with a few different shots, all coming from the same drive, in a layer in my timeline, I might need a lot of bandwidth.  But how much bandwidth?  

How about, since part of my conundrum is that I'll be coming back from a shoot with all my footage on an ExpressCard (or two), and I need a a good backup strategy (since I don't have my footage on a tape anymore), if I go for a two-tier set-up like Job's:

1. that tray-less (easy to use; inexpensive) Wiebetech hard drive enclosure, to back up to (and then put on a shelf).

2. a Firmtek, Sonnet or G-Tech e-SATA set-up, for editing.  Not hardware RAID, but probably still plenty fast for me?

Malcolm

It's not just a speed improvement over Firewire 800 - eSATA is more stable, in my experience.

Your plan sounds like a good set-up. Then you could just software RAID 0 your Media Drives for speed without the stress -- even just a two-drive stripe will give you at least 90-125 MB/sec until the drive is half-full. A three-drive stripe gets faster and so on, up to a five-drive stripe which should yield around 200 MB/sec. Of course, you don't need that, but it's always nice to rule out drive slowness when troubleshooting MC software.

I just sold my Firmtek because I'm setting up an internal RAID, but I still recommend it over any of the others you listed - best bang for your buck IMHO. Get yourself five 1TB drives and you're cooking with gas. Samsung recently released the Spinpoint F1 enterprise-class 1TB drive that only has three platters, so it runs a lot cooler than a five or four-platter (Hitachis & Seagates) and has fewer heads susceptible to failure. I saw them online for $200 a pop today - amazing.

I agree, eSATA is better in almost every way. If you can't afford fiber or SAS, go SATA.

The reason is that eSATA doesn't need a "bridge" chip to convert the drive. In FW, the drive is SATA or IDE/PATA and w convert it to use FW. In eSATA, you are connecting to the drive, no "software layer" between the devices. (not exactly the way it works, but I cna't get into it all here)

Remember that unlike SCSI and Fiber Channel ((which is technically also a SCSI protocol) SATA uses bursts or packets of data, not streams so we need to have substantially more bandwidth that we need. The good news is that the drives and raid hardware are cheap.

I generally spec twice the bandwidth I need for editing, real world. So if you figure 50-60 MB per sec per drive (rough number) and you want 4 streams of SD uncompressed (let's say 110MB per) then I'd try to have near 200MB per sec theoretical throughput, or 4 drives. Can you get by with less? Of course, will you have a better experience with more drives, of course. If budget is a concern, start with a couple and add the same brand and size so you can raid them successfully later. (matching firmware is recommended but not as critical as it used to be)

Now, take all of that and add a grain of salt, cause that was free advice. One thing you can bank on is that having a great dealer or consultant to help you will pay for itself 1000 times over in data security and convenience. Storage is your media, your job and your very life. Don't trust the bargain basement anymore than you'd go to get a liver transplant in a mini-mall.

;)

All very interesting... now what if a person's editiing HD, not SD (if I'm importing Sony XDCAM-EX footage that's HD, shouldn't I just edit in HD?) - - I imagine I'd need a lot more than 200MB/s.

Interesting news about Samsung's new Spinpoint drives... they sound great... will check them out.

Cheers, Malcolm

You definitely don't need to crank up and outfit your system for true HD with that footage - it's just some proprietary codec that gives you a high quality image at 35 mb/sec. But I'm no expert in the ever-expanding universe of pro-sumer high-def, so get a few opinions for your project.

Incidentally, I did speak with a lead tech at Firmtek today. He said that while Firmtek officially recommends the Hitachi Ultrastar A7K1000 for 1TB drives, he's got all Samsung Spinpoints in his enclosure at work and swears by them. He said the Samsung "runs cool and quiet and is faster than the Hitachi."

Good luck.

funckdren:

a high quality image at 35 mb/sec

35 millibit/sec giving a high quality image? Please people its confusing enough as is ;)

Capital M = Mega

Capital B = Byte

undercast b = bit

1 byte equals 8 bits

So, following the info on the Avid website concerning HD streams, found here http://www.avid.com/dnxhd/features.asp 

Where in order of appearance per stream of video without counting audio:

145 Mb/s = 18.1 MB/s for DNxHD145

100 Mb/s = 12.5 MB/s for DVCPro HD

135 Mb/s = 16.9 MB/s for HDCAM

220 Mb/s = 27.5 MB/s for DNxHD220

440 Mb/s = 55 MB/s for HDCAM SR

For more info on video/audio formats and their datarates, you can use the aja datarate calculator, found here http://www.aja.com/html/support_kona2_swd.html

My bad...will watch my capitalization in the future.

Of course, in a thread about eSATA, I think you probably knew I didn't mean 35 millibits per second for a high-quality image, but I was still wrong, nonetheless.  MB vs. Mb vs. mb...it is confusing. Thanks for clearing that up (for a 2nd time -- missed your earlier lexicon).

So Malcolm, maybe I misunderstood: when you wrote "35mbs" in your original post, I'm now assuming you weren't referring to drive speed requirements, you were actually referring to the what's going on in your image from the Sony EX-1 camera -- 35 Mb/sec in HQ mode (Megabits per second, not Megabytes per second). Is that right? If so, then the Megabytes/sec speed you need for storage depends on what resolution you use to bring the footage into MC. That AJA calculator from rinzeschuurman is great.

Bottom line is there are now some HD codecs available to you for that EX-1 material in the 35 MB/sec range, and the eSATA enclosures you were asking about can all be fast enough.

Hi again,

Brian, I don't think you misunderstood me; I think I've not explained myself well.  Even now, I'm not sure I have any solid understanding of the technical reasons why one drive set-up might work better for me than another.  I really just want to know whether, if I'm using footage from an EX-1, I can edit it without video stutter or whatever, in Avid Media Composer, and not have to wait forever for renders, etc.  I now see that it might in good part depend on whether I compress the footage when I bring it into Avid, or edit 1:1 or something like that.  I'm still hoping someone who edits with EX1 footage, with MC, on a Mac, with an external hard-drive (SATA) array, in RAID-5, can describe their workflow.  Or maybe I've already been given all the advice I need - - many of you are editing happily on software-RAID set-ups... in all likelihood, this means I could do the same!

Anyway, I've downloaded the AJA Data Rate Calculator, and I don't see an option of choosing an option of entering DNxHD anywhere.  If the 'Preset' for the Sony EX-1 footage is 1080i 8-bit, and the frame size is 1920X1080... don't I need to be able to choose DNxHD145 for the Compression type? I don't see it as an option.  If there were such an option, this Data Rate Calculator might be very useful to me, as far as figuring out how much hard drive space I need.  

Good to hear that your contact at Firmtek likes the Samsung drives.

cheers, Malcolm

Malcolm,

The rule of thumb for editing Uncompressed 10-bit HD footage is you'll need drives that can sustain at least 200 MB/sec. A five-drive RAID 0 in my Firmtek eSATA enlosure gave me at least 210 MB/sec, often more like 225 MB/sec.

You're right, the Aja calulator does not have DNxHD codecs (why would it, really?), but it has others that are somewhat similar based on rinzeschuurman's calculations above (DVCPro HD for example, though that one's a little more compressed). What I wouldn't do is edit everything in Uncompressed HD -- that's crazy. Use one of those HD codecs to bring your file sizes and data rates down while you're editing, then bump up your picture-locked cut to the best resolution your drives/system can handle. And depending on your client, a DNxHD master may be deliverable.

Hope that helps. Good luck.

Brian, thanks yet again for the good advice.  Can I ask which codec you use for your day-to-day editing? Malcolm

The resolution I would use really depends on the amount of footage I have. The more I've got, the more compressed I go.

Regarding the Avid HD codecs: the show that I'm working on shoots in true HD, but we actually still offline edit in SD because the production company has an army of Avid Meridiens (OS9). Then we bump up to HD for online and broadcast. So, I actually don't use the DNxHD codecs every day, but I just talked to my post-supervisor, and he told me that we deliver the final masters to Discovery Channel in DNxHD 220x. It's considered master quality and easily passes QC there.

And if you look again at the Avid link from rinzeschuurman's post, you'll see that DNxHD 220x is very similar to Uncompressed 8-bit SD. Your drives RAIDed in a port multiplier should easily be able to handle that. So I imagine any among the new crop of DNxHD codecs will likely work for you and look great. Of course, run some tests before you take anybody's advice...

Cheers.

 I also found that the frontpage of www.avid.com now has a link to avid's version of the calculator which will show you how much GB's you need, this one ofcourse including the various DHxHD codecs. I'll add some terms to the lexicon I already wrote in terms of connections to the motherboard of various pc's (PCI, PCI-x, PCI-e, etc.) when i have the time available. The lexicon will change and be edited depending on questions and remarks I find in posts.

thanks for the tip about the calculator, rinzeschuurman... and for keeping us up-to-date...

Malcolm

Just be aware if you do go with an eSATA enclosure that your drive
speeds will ultimately be determined by the card the enclosure connects to.

Firmtek's 2-port SE2 card and Sonnet's 2-port  E2P card look great and
work well, but they are one-lane PCIe cards that cap out at 125
MB/sec. If you need more speed than that, you would require a
multi-lane card like the Sonnet E4P. Seritek does not offer a multi-lane
PCIe as of yet. The E4P is more expensive but worth the security to
know your port-multiplier has as much elbow room as it needs.

funckdren:

but they are one-lane PCIe cards that cap out at 125
MB/sec.

added some info on the PCI variants to the lexicon... have fun