Internal RAID Setup

[Mixmaster5]

Hi there,

 

I'm am looking for advice cocerning my storage solution! 

 

Details which might help giving me advice:

 

- What will the RAID be used for: I'am a video editor/filmmaker

- What is my current setup: 2TB WD green --> aboslute no'go!! (my system: see below)

- What am I looking for; a internal RAID Setup which is fast enough for handling footage in

Premiere Pro & After Effects etc. work. Does it need to be 1000 mb/s no, but it should be aible

to hanlde upcoming 4K footage!

- Why not external; as far as my reserched resulted, it is quite expensive and afraidly windows

doesn't have Thunderbold and USB 3 doens't work over 250mbs.

 

The new RAID System

 

I have absolutly no experience in setting up RAID's, I've read some things about it, but my technical

knowledge is quite limited...

 

Do I need a RAID Card (Motherboard connection are quite limited...) if yes, which?

How many HDDs do I need for this RAID Setup? Which Brand?

Which RAID? 6, right?

Etc...?

 

As said, I don't have experience and I haven't really found a fitting solution in my reserach.

 

It would be incredibly nice, if you would help me to setup a decent RAID Setup from A-Z

 

Your help is much appreciated.

 

Best 

Jay

 

[Sharif]

Check these both out.

 

http://www.youtube.com/watch?v=TuwjadbtUCY This video talks more about to when to use a raid card.

[Vitalius]

@Mixmaster5

You should probably use RAID 10.

Get WD, Seagate, or Hitachi 4TB NAS oriented drives, and you probably don't need a RAID card. RAID 10 requires 4 HDDs minimum. This would be around $640 though in total just for HDDs. You can get smaller drives if you don't need 8TB of usable space. 

By NAS oriented, WD Reds, Seagate NAS, and HItachi NAS drives are what I mean. 7200RPM preferably.

RAID 10 because it has both redundancy and speed. 4TB HDDs because it's cheaper TB/$ if you do that.
NAS drives because they tend to be of higher quality/performance in general. 
No RAID card because they are pointless unless you have a really crappy motherboard (you don't) or weak CPU (you don't).

 

[ForsakenLive]

You should be using an SSD, it's an important boost on speed, and it's great as a scratch disk.

Raid is used to get redundancy, where keeping files safe is important. If you want just speed, get a good SSD, 512GB-1TB ones started to become cheap. A speed gain on mechanical drives is minor, so getting several drives and make a raid 0 makes little sense. For your mass storage, where you back up your files edited/to edit, RAID 1 0 makes a lot of sense, but 50% of the drives will not be available for you to store data on, and you will need at least 4.

[Mixmaster5]

Thanks for all the quick answers.

 

@ sharif: thanks for the links! 

 

@Vitalius: so concering Linus's video: wouldn't RAID 5 (or RAID 50?)be the best option, because actually

I don't care that much about writing speeds and if I understood it right reading speed is fast

in RAID 5. No my motherboard is not crappy but i doesn't have that many SATA Inputs

(3 out of 6 already occupied). How could I attach an extra 3 SATAs? Woudln't it make

sense to have 8x2 TB for a great speed increasement?

 

@forsaken live: yes I thought about that option, but the storage of 1 TB is to less for a project with

RAW (4k) footage. Scratch Drive though should definitly be an SSD.

 

Thanks for further advice.

[Vitalius]

Thanks for all the quick answers.

 

@ sharif: thanks for the links! 

 

@Vitalius: so concering Linus's video: wouldn't RAID 5 (or RAID 50?)be the best option, because actually

I don't care that much about writing speeds and if I understood it right reading speed is fast

in RAID 5. No my motherboard is not crappy but i doesn't have that many SATA Inputs

(3 out of 6 already occupied). How could I attach an extra 3 SATAs? Woudln't it make

sense to have 8x2 TB for a great speed increasement?

 

@forsaken live: yes I thought about that option, but the storage of 1 TB is to less for a project with

RAW (4k) footage. Scratch Drive though should definitly be an SSD.

 

Thanks for further advice.

Eh. It's about 1 HDD speed. RAID 50 would require 6 HDDs minimum. I don't think that's worth it. At all. Plus Software RAID means that Parity RAID relies on the CPU to work. In other words, when writing to the RAID array, your CPU decides how fast you can write, and since you would probably be rendering video, it would slow down your render by a bit too since the RAID would be using the CPU at the same time.

You could get an HBA (Host Bus Adapter) which does nothing but add SATA ports, but that doesn't bypass the CPU part. You could get a RAID controller, but good RAID controllers cost between $150 and $300. That's why I don't consider it worth it.

3 of 6 of your SATA slots are already occupied. One of them is your Boot drive. One is your Scratch disk I assume, and one is the 2TB HDD. If you are replacing the 2TB HDD, that means you have 4 SATA ports available. 4 Drives is minimum for RAID 10, and depending on your OS (I assume Windows), RAID 10 is epic for Read speeds. It doesn't work in Windows though, sadly.

You don't want to use that 2TB Green HDD in your RAID array. It will only cause problems. Very bad problems. Trust me on that. 8 x 2TB isn't going to give you much better read speeds than 4 x 4TB, if any at all. 

I really recommend 4x4TB WD Reds (or other NAS drives) in RAID 10 over 8 x 2TB in RAID 5, 50, 6, or 60. For a lot of reasons that I already covered but will summarize here:

1. Parity RAID is CPU intensive, meaning if you do other CPU intensive tasks like rendering video, it will slow down your renders and RAID array speeds at the same time.
2. You'd need an HBA to get that many HDDs (6 HDDs for RAID 50, and 8 HDDs for RAID 60) which is extra money and wouldn't fix the CPU problem.
3. You'd need a good RAID controller to get that many HDDs and also fix the CPU problem, but that's a bit of money in and of itself. You could get an extra 4TB WD Red for that price or even more.
4. Your current setup already has support for RAID 10 as you have 4 SATA slots available (if I'm right in assuming that the 2TB Green HDD would come out giving you an extra one).
5. RAID 5, 50, 6, or 60 wouldn't be a noticeable advantage in speeds over RAID 10, even if you ignored the CPU problem.

[Mixmaster5]

Thank you for your great, detailed help!

 

Today I had the chance to talk to an industry professional:

 

His opinion was to get a RAID Controller with RAID 5 with

Seagate Barracudas --> empty up to 800mb/s

 

Speed might not be much faster than in RAID 10 but -

as far as I unsterstand it now - I have more Stoarge

instead of the 50% in RAID 10, right?

 

So I would definitly be willing to spend some money

on a RAID controller. Could you recommend a model?

 

 

Best

Janis

[Vitalius]

Thank you for your great, detailed help!

 

Today I had the chance to talk to an industry professional:

 

His opinion was to get a RAID Controller with RAID 5 with

Seagate Barracudas --> empty up to 800mb/s

 

Speed might not be much faster than in RAID 10 but -

as far as I unsterstand it now - I have more Stoarge

instead of the 50% in RAID 10, right?

 

So I would definitly be willing to spend some money

on a RAID controller. Could you recommend a model?

 

 

Best

Janis

Here's a list of good ones to choose from. Make sure it's in IR mode and not IT mode.

 

http://blog.zorinaq.com/?e=10

[wpirobotbuilder]

You should be using an SSD, it's an important boost on speed, and it's great as a scratch disk.

Unfortunately an SSD doesn't make sense for this use case, since the cost would be enormous given the file sizes he'd be working with.

 

@Mixmaster5, do you know how much storage space you will need? Also, will you be storing your footage on this RAID array, or is it only for a single project (the current working one)?

 

if you are working directly on it, then I recommend RAID 10. It's less efficient than RAID 5/50/6/60, but it performs much better. You may not need a RAID controller if you need less than 16 TB of storage, because most onboard chipsets support RAID 10. If you need more than that, or need more than about 400 MB/s write speeds, then you should consider a RAID controller. This is a good option for 4-8 drive configurations.

 

If you are also storing your old projects on it, you will need a lot of storage. You may want to invest in a NAS for archiving projects.

[Mixmaster5]

@Vitalius:

 

Thanks for the list, I'll have a look at it.

 

@wpirobotbuilder:

 

 

It's just that RAID 10 is (in my opinion) kind of a waste of storage...then why not RAID 0;

wouldn't this be a possibility if you have a goof backup system?

 

Thanks for the controller advice; it seems to be a good one, but doesn't support RAID 5.

 

 

Concerning NAS/Backup: I actually wanted to open a separate topic for this, because

I didn't want to make it to complicated, but actually it makes sense to ask it in this

topic (hopefully others "noobs" like me can profit from this topic) ; a RAID system requires proper backup.

 

So lets say I have a RAID 5 with a controller 15 TB brutto 10 TB nette (hope thats corrects), what would be a

good and easy (external) solution for backup and archival? Are the good automated options, because

I don't want all the rubbish on my PC backed up; Actually just footage, pictures and project files. 

(shoudln't be too expensive)

 

Again, thanks for your amazing help, it is really appreciated!

 

Best

[Vitalius]

@Vitalius:

 

Thanks for the list, I'll have a look at it.

 

@wpirobotbuilder:

 

 

It's just that RAID 10 is (in my opinion) kind of a waste of storage...then why not RAID 0;

wouldn't this be a possibility if you have a goof backup system?

 

Thanks for the controller advice; it seems to be a good one, but doesn't support RAID 5.

 

 

Concerning NAS/Backup: I actually wanted to open a separate topic for this, because

I didn't want to make it to complicated, but actually it makes sense to ask it in this

topic (hopefully others "noobs" like me can profit from this topic) ; a RAID system requires proper backup.

 

So lets say I have a RAID 5 with a controller 15 TB brutto 10 TB nette (hope thats corrects), what would be a

good and easy (external) solution for backup and archival? Are the good automated options, because

I don't want all the rubbish on my PC backed up; Actually just footage, pictures and project files. 

(shoudln't be too expensive)

 

Again, thanks for your amazing help, it is really appreciated!

 

Best

Because RAID 10 has redundancy between RAID 5 & 6. Depending on which drives die in RAID 10, you can either lose 1 or 2 without losing data. And for some OS' (which I mentioned earlier, but Windows isn't one of them), RAID 10 is effectively two RAID 0's for Read speeds. But the OS has to have that capability, and Windows doesn't. 

Yes, it would be fine if you had a good backup system. But realize that if you are in the middle of a work day and a drive dies, you probably won't be able to work until you fix the array which means buying the drive (I recommend having spares on hand anyway) then having the RAID array rebuild (takes time). That's why the RAID 1 part of RAID 10 is useful. You'd still be able to work just fine.

I don't know anything about pre-built NAS solutions. I only know about custom NAS setups.

[Novistion]

Go for WD Reds for raid no matter what! Please! http://www.wdc.com/en/products/internal/desktop/ if needed blacks, but don't expect them to last long

[Mixmaster5]

Because RAID 10 has redundancy between RAID 5 & 6. Depending on which drives die in RAID 10, you can either lose 1 or 2 without losing data. And for some OS' (which I mentioned earlier, but Windows isn't one of them), RAID 10 is effectively two RAID 0's for Read speeds. But the OS has to have that capability, and Windows doesn't. 

Yes, it would be fine if you had a good backup system. But realize that if you are in the middle of a work day and a drive dies, you probably won't be able to work until you fix the array which means buying the drive (I recommend having spares on hand anyway) then having the RAID array rebuild (takes time). That's why the RAID 1 part of RAID 10 is useful. You'd still be able to work just fine.

I don't know anything about pre-built NAS solutions. I only know about custom NAS setups.

Well, how do you do backup? With a lot of external HDD's? Is there software around for custom backups?

 

Best