RAID card recommendation

[Yatys 93]

Originally I wanted this: http://www.bhphotovideo.com/c/product/1155175-REG/asus_pike_ii_3108_8i_16pd_sas.html as it's the one mentioned on the options for my motherboard (Z10PE-D16 WS) however I am going to have to import it to the UK and in total I am looking at around £450

 

I have now been looking at these 4 : https://www.xcase.co.uk/raid-h-b-a-cards/l/raid-card.html

 

the max I would be using is 24 SATA 3 WD red pro's so they all seem over kill bar the ASUS card which has a max of 16 drives, how ever when I need more than 12, I will most likely get a second card and go from RAID 6 to 60, but that is way way in the future.

 

I have no idea which card to get, I know they would all work for me but that's it. advice would be appreciated.

[scottyseng]

3 hours ago, Yatys 93 said:

-snip-

What enclosure do you have? You also need to buy the battery backup for the RAID card as well, it's called Cachevault.

 

Out of those RAID cards, I would get the 9361-8i. It has SAS 12Gb/s. Make sure you get the battery for it as well.

 

You can run all of the drives from one RAID controller with a RAID expander. You plug the RAID card into it, and it splits out to even more SAS ports. A single RAID card can usually support 128 devices with expanders (You'd have to check the specifics of that RAID card. I know my LSI MegaRAID 9260-8i supports 128).

 

Do keep in mind that you need to keep the RAID card cool. They run really hot. You're going to want a fan blowing on it.

 

I'm guessing this is for a Windows machine though right? Otherwise you could save money by going with FreeNAS or something like that.

[Yatys 93]

6 hours ago, scottyseng said:

What enclosure do you have? You also need to buy the battery backup for the RAID card as well, it's called Cachevault.

 

Out of those RAID cards, I would get the 9361-8i. It has SAS 12Gb/s. Make sure you get the battery for it as well.

 

You can run all of the drives from one RAID controller with a RAID expander. You plug the RAID card into it, and it splits out to even more SAS ports. A single RAID card can usually support 128 devices with expanders (You'd have to check the specifics of that RAID card. I know my LSI MegaRAID 9260-8i supports 128).

 

Do keep in mind that you need to keep the RAID card cool. They run really hot. You're going to want a fan blowing on it.

 

I'm guessing this is for a Windows machine though right? Otherwise you could save money by going with FreeNAS or something like that.

I don't have an enclosure yet, I'm still in the planning stages. however I am planning on this https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-rm-424-pro-24-bay-hotswap-6gb-sgpio-backplane-single-or-redundant-psu-s-389-00-x-case.html

 

and yes it's going in a windows server, most likely running server 2012 unless 2016 is a lot better for a similar price, if they ever launch it.

 

I assume I can still do RAID 60 with expanders?

[scottyseng]

2 hours ago, Yatys 93 said:

I don't have an enclosure yet, I'm still in the planning stages. however I am planning on this https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-rm-424-pro-24-bay-hotswap-6gb-sgpio-backplane-single-or-redundant-psu-s-389-00-x-case.html

 

and yes it's going in a windows server, most likely running server 2012 unless 2016 is a lot better for a similar price, if they ever launch it.

 

I assume I can still do RAID 60 with expanders?

Ah, you'll need to buy a expander for it since that backplane isn't a proper expander based backplane.

 

Yeah, you can do RAID 60 with the expander, it pretty much just expands the ports the RAID has natively (The cool feature of SAS).

 

Do note that the Red Pros are only rated for up to 16 bay enclosures (At least according to WD). You'd have to look into the Ae, Se, or Re enterprise lineups if you want to properly stick more than 16 of them in one chassis (Why I had to get Re drives and migrate my Reds out of my NAS). You can probably get away with having all 24 drives in, but it's not recommended.

 

I would hold off until 2016 is out...at least I hope it comes out soon. I get it free with Dreamspark (college student) that or you can pick it up off of the reddit thread for $15 (I'm assuming this is for personal use).

 

https://www.reddit.com/r/microsoftsoftwareswap/comments/3wxfh5/h_windows_7_8_81_10_windows_server_office_visio/

[Yatys 93]

On 12/03/2016 at 3:33 PM, scottyseng said:

Ah, you'll need to buy a expander for it since that backplane isn't a proper expander based backplane.

 

Yeah, you can do RAID 60 with the expander, it pretty much just expands the ports the RAID has natively (The cool feature of SAS).

 

Do note that the Red Pros are only rated for up to 16 bay enclosures (At least according to WD). You'd have to look into the Ae, Se, or Re enterprise lineups if you want to properly stick more than 16 of them in one chassis (Why I had to get Re drives and migrate my Reds out of my NAS). You can probably get away with having all 24 drives in, but it's not recommended.

 

I would hold off until 2016 is out...at least I hope it comes out soon. I get it free with Dreamspark (college student) that or you can pick it up off of the reddit thread for $15 (I'm assuming this is for personal use).

 

https://www.reddit.com/r/microsoftsoftwareswap/comments/3wxfh5/h_windows_7_8_81_10_windows_server_office_visio/

any recommendations on a better case? Ideally 24 bays, as well as a place for 2 SSD's for the OS (RAID 1) and 4U size wise for a SSI EEB dual socket board.

 

I didn't realise the red pro's had a 16 drive max, I just knew then were higher performance than the standard ones and that the standard reds have a 8 drive max, the RE's are only £15 more per drive so I'll get them instead.

[scottyseng]

8 hours ago, Yatys 93 said:

any recommendations on a better case? Ideally 24 bays, as well as a place for 2 SSD's for the OS (RAID 1) and 4U size wise for a SSI EEB dual socket board.

 

I didn't realise the red pro's had a 16 drive max, I just knew then were higher performance than the standard ones and that the standard reds have a 8 drive max, the RE's are only £15 more per drive so I'll get them instead.

I like SuperMicro stuff...but it's very expensive. I bought a used SuperMicro chassis for $200...but then exchanged the SAS 3Gb/s backplane for a $300 SAS 6Gb/s backplane. I also couldn't handle the noise of the 80 Plus power supply and bought the 80 Plus Platinum version (Another $180). I also just leave my SSDs just hanging around inside of the chassis...

 

But the chassis you linked / the Norco 24bay 4U are pretty comparable / cheap for the price for a new unit. The SuperMicro stuff costs more because it has a expander based backplane / redundant PSUs / hot swap fans.

 

Yeah, if you want to cry, go check out the Re SAS version..so expensive. The Re or Se would work for you.

[Yatys 93]

On 15/03/2016 at 8:45 AM, scottyseng said:

I like SuperMicro stuff...but it's very expensive. I bought a used SuperMicro chassis for $200...but then exchanged the SAS 3Gb/s backplane for a $300 SAS 6Gb/s backplane. I also couldn't handle the noise of the 80 Plus power supply and bought the 80 Plus Platinum version (Another $180). I also just leave my SSDs just hanging around inside of the chassis...

 

But the chassis you linked / the Norco 24bay 4U are pretty comparable / cheap for the price for a new unit. The SuperMicro stuff costs more because it has a expander based backplane / redundant PSUs / hot swap fans.

 

Yeah, if you want to cry, go check out the Re SAS version..so expensive. The Re or Se would work for you.

is it possible to get the supermico cases as just a case, as I can't seem to find it, also they appear to be like norco and not ship to the UK

[scottyseng]

10 hours ago, Yatys 93 said:

is it possible to get the supermico cases as just a case, as I can't seem to find it, also they appear to be like norco and not ship to the UK

Yes, the SuperServer are SuperMicro's prebuilt lineup, where SuperChassis is their empty case (do note that SuperMicro comes with PSUs / a LSI expander based backplane). As for shipping, I have no clue on that as I'm from the US...you might also look at ebay for used SuperMicro chassis.

 

That being said, There's nothing wrong with Norco either. You just will want to buy a SAS expander down the road to use all of the drives.

[Yatys 93]

On 19/03/2016 at 0:38 PM, scottyseng said:

Yes, the SuperServer are SuperMicro's prebuilt lineup, where SuperChassis is their empty case (do note that SuperMicro comes with PSUs / a LSI expander based backplane). As for shipping, I have no clue on that as I'm from the US...you might also look at ebay for used SuperMicro chassis.

 

That being said, There's nothing wrong with Norco either. You just will want to buy a SAS expander down the road to use all of the drives.

so after shopping round, I have decided to go with either:

 

https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-rm-424-pro-24-bay-hotswap-6gb-sgpio-backplane-single-or-redundant-psu-s-389-00-x-case.html

Or

 

https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-extra-value-424-24-hotswap-bays-coming-soon-239-00-x-case.html

 

mainly as I will save a fair bit on shipping the others from the US as they don't ship here at all. Would it be possible for you to link a recommended cache vault and expander for the 9361? I think I'm going with the 4l over the 8l, as comparing the two I couldn't find where the extra £120 was being used, as the specs appear the same, both 12gb SAS 128 Drives, I assume I missed something but if it's not on the spec sheet will I benefit from it?

 

Also, I would like you opinion on the storage, as my plan at the moment is a 8 drive RAID 6, which if I expand will end up possibly at a 24 drive RAID 60, however I am curious if it would be easier to bump up to 12 drives off the bat and start at 60 instead of changing later?

 

Thanks for all the help!

[leadeater]

1 hour ago, Yatys 93 said:

so after shopping round, I have decided to go with either:

 

https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-rm-424-pro-24-bay-hotswap-6gb-sgpio-backplane-single-or-redundant-psu-s-389-00-x-case.html

Or

 

https://www.xcase.co.uk/4u-rackmount-server-cases/x-case-extra-value-424-24-hotswap-bays-coming-soon-239-00-x-case.html

 

mainly as I will save a fair bit on shipping the others from the US as they don't ship here at all. Would it be possible for you to link a recommended cache vault and expander for the 9361? I think I'm going with the 4l over the 8l, as comparing the two I couldn't find where the extra £120 was being used, as the specs appear the same, both 12gb SAS 128 Drives, I assume I missed something but if it's not on the spec sheet will I benefit from it?

 

Also, I would like you opinion on the storage, as my plan at the moment is a 8 drive RAID 6, which if I expand will end up possibly at a 24 drive RAID 60, however I am curious if it would be easier to bump up to 12 drives off the bat and start at 60 instead of changing later?

 

Thanks for all the help!

 

The only difference between the 4i and the 8i is the number of SAS ports, 1 vs 2, for a total of 4 disks vs 8 disks without an expander. To get the 128 drives per port you must use an expander. Each SAS port on a RAID card is 4 lanes of the rated speed i.e. 4 x 12Gb which directly connect to a disk each when not using an expander. This is why SAS to SATA breakout cables always come with 4 SATA.

 

To use all 24 disks not using an expander you would need 3 dual SAS port RAID cards, very costly. Also to compound this generally speaking you cannot create RAID arrays that span multiple RAID cards so you won't be able to create the 24 disk RAID 60 like you want.

 

The other thing to note is you cannot expand the span of a RAID 60 once it is created so you would need to have 3 RAID 6 arrays from the start, you can sometimes expand the underlying RAID 6's for more space but you cannot add more. Don't count on being able to expand a RAID 60 unless the manual says you can.

 

I would start with an 8 disk RAID 6 then buy 16 more disks, backup then destroy and create the 24 disk RAID 60. The only problem in doing this is having the space to backup the original RAID 6.

[Yatys 93]

11 hours ago, leadeater said:

 

The only difference between the 4i and the 8i is the number of SAS ports, 1 vs 2, for a total of 4 disks vs 8 disks without an expander. To get the 128 drives per port you must use an expander. Each SAS port on a RAID card is 4 lanes of the rated speed i.e. 4 x 12Gb which directly connect to a disk each when not using an expander. This is why SAS to SATA breakout cables always come with 4 SATA.

 

To use all 24 disks not using an expander you would need 3 dual SAS port RAID cards, very costly. Also to compound this generally speaking you cannot create RAID arrays that span multiple RAID cards so you won't be able to create the 24 disk RAID 60 like you want.

 

The other thing to note is you cannot expand the span of a RAID 60 once it is created so you would need to have 3 RAID 6 arrays from the start, you can sometimes expand the underlying RAID 6's for more space but you cannot add more. Don't count on being able to expand a RAID 60 unless the manual says you can.

 

I would start with an 8 disk RAID 6 then buy 16 more disks, backup then destroy and create the 24 disk RAID 60. The only problem in doing this is having the space to backup the original RAID 6.

My plan was 2 RAID 6's not 3 of that makes a difference, and then I was just going to add drives to each RAID 6, or is that not how it works?

 

I do want to get an expander but I can't seem to find one anywhere, does it have to be a native part of the case?

[scottyseng]

14 hours ago, Yatys 93 said:

-snip-

The only difference I can see between the two is that one has hot swap fans, the other doesn't. I honestly would go for the second one and save the cash.

 

9361 Cache Vault:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816118232

 

You need an expander, something like:

SAS 6Gb/s Expander:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816117306&cm_re=SAS_expander-_-16-117-306-_-Product

 

SAS 12Gb/s Expander (I would say to get this one to match your RAID card's 12Gb/s):

http://www.newegg.com/Product/Product.aspx?Item=9SIA19P3WD3492&cm_re=SAS_expander-_-9SIA19P3WD3492-_-Product

 

Do note that two of the SAS ports on the expander are used as inputs (Two SAS from the LSI 8i card), the rest of the ports are outputs (Go out to your server backplanes).

You'll need several converting cables that go from SAS 12Gb/s (Different SAS end from SAS 6Gb/s) to SAS 6Gb/s (What's on your backplane for those cases...please double check though, look at the manual and see if it has SAS 6Gb/s or SAS 12Gb/s ports on the backplane...pretty sure it's SAS 6Gb/s though for that price).

 

Well, either way, you have to destroy the existing RAID6 array to change it to RAID 60. Only RAID 5 and 6 are expandable. If you want RAID60, you have to buy drives to temp hold your data while you destroy the existing array and build a new RAID60 with all drives. I kind of had to put two 4TB Reds in RAID0 as temp storage as I built my larger array (six drive 4TB RAID10).

[dalekphalm]

To help with migrating your data to a larger array, without wasting drives, is to get enough drives to temporarily hold the data, then repurpose them after the fact as a backup storage array for backup and archiving your main storage. We would be talking long term "deep archiving".

[Yatys 93]

24 minutes ago, scottyseng said:

The only difference I can see between the two is that one has hot swap fans, the other doesn't. I honestly would go for the second one and save the cash.

 

9361 Cache Vault:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816118232

 

You need an expander, something like:

SAS 6Gb/s Expander:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816117306&cm_re=SAS_expander-_-16-117-306-_-Product

 

SAS 12Gb/s Expander (I would say to get this one to match your RAID card's 12Gb/s):

http://www.newegg.com/Product/Product.aspx?Item=9SIA19P3WD3492&cm_re=SAS_expander-_-9SIA19P3WD3492-_-Product

 

Do note that two of the SAS ports on the expander are used as inputs (Two SAS from the LSI 8i card), the rest of the ports are outputs (Go out to your server backplanes).

You'll need several converting cables that go from SAS 12Gb/s (Different SAS end from SAS 6Gb/s) to SAS 6Gb/s (What's on your backplane for those cases...please double check though, look at the manual and see if it has SAS 6Gb/s or SAS 12Gb/s ports on the backplane...pretty sure it's SAS 6Gb/s though for that price).

 

Well, either way, you have to destroy the existing RAID6 array to change it to RAID 60. Only RAID 5 and 6 are expandable. If you want RAID60, you have to buy drives to temp hold your data while you destroy the existing array and build a new RAID60 with all drives. I kind of had to put two 4TB Reds in RAID0 as temp storage as I built my larger array (six drive 4TB RAID10).

it seems I though RAID was simpler than it really is, think I will just have one RAID 6 so I can add drives to it as I need them, the only reason I want to use 60 was it is getting a 10gb connection so I wanted to use as much of that as I could.

 

I think they are both SAS 6gb, but in review on the more expensive case says they wish the 12gb backplane from the cheaper one was included. Although I did find this while seeing if the backplane can be upgraded.

 

https://www.xcase.co.uk/x-case-rm-424-pro-with-sas-expander-sgpio-backplane-649-00-x-case.html

[scottyseng]

1 minute ago, Yatys 93 said:

it seems I though RAID was simpler than it really is, think I will just have one RAID 6 so I can add drives to it as I need them, the only reason I want to use 60 was it is getting a 10gb connection so I wanted to use as much of that as I could.

 

I think they are both SAS 6gb, but in review on the more expensive case says they wish the 12gb backplane from the cheaper one was included.

Well, as long as you have good hardware, RAID is pretty simple. With Hardware RAID, it's even easier. Just install LSI MegaRAID storage manager and use that to manage your RAID card, I find the LSI card BIOS absolutely horrible to look at...it's like something from the 90s. haha.

 

Hmm, yeah, you'll need to buy SAS 12Gb/s to SAS 6Gb/s cables to fit the backplane.

 

Well, I think with eight drives, you probably won't be bottlenecking the 10Gb/s Ethernet quite yet. You'll easily hit it if you get 12+ drives in the future. If you want some numbers, for me:

 

Six 4TB WD Reds in RAID10:

550MB/s sequential read and write

 

Four 4TB WD Reds in RAID10:

350MB/s sequential read and write

 

A single 4TB WD Red is roughly 160MB/s vs 180MB/s on the 4TB WD Re SAS drive.

[Yatys 93]

how much would RAID 6 impact the speeds?

 

[scottyseng]

6 minutes ago, Yatys 93 said:

how much would RAID 6 impact the speeds?

Not too much, since you're using a proper hardware RAID card. You also will have the Cachevault as well. It's probably going to be a little bit slower...but it's not going to be much especially if you have that many drives (You'll still be well over 500MB/s probably with eight drives).

[Yatys 93]

2 minutes ago, scottyseng said:

Not too much, since you're using a proper hardware RAID card. You also will have the Cachevault as well. It's probably going to be a little bit slower...but it's not going to be much especially if you have that many drives (You'll still be well over 500MB/s probably with eight drives).

awesome! I assumed I'd get a fair bit slower, hence why I wanted to use 60 over 6.

 

also just to clarify, if I get this case https://www.xcase.co.uk/x-case-rm-424-pro-with-sas-expander-sgpio-backplane-649-00-x-case.html

 

I can use the 9361 4L to connect the backplane to all 24 drives, once all the drives are connected?

 

I'm gonna get this case as the expander is over £400 by its self so it would cost more to get the cheaper case and the expander than just get the more expensive case

[scottyseng]

1 minute ago, Yatys 93 said:

awesome! I assumed I'd get a fair bit slower, hence why I wanted to use 60 over 6.

 

also just to clarify, if I get this case https://www.xcase.co.uk/x-case-rm-424-pro-with-sas-expander-sgpio-backplane-649-00-x-case.html

 

I can use the 9361 4L to connect the backplane to all 24 drives, once all the drives are connected?

Well, RAID60 would be faster, but with that many drives, you wouldn't be able to notice....you'll easily exceed 10Gb/s Ethernet once that thing is fully packed with 24 drives.

 

Yeah, that case would actually be preferred. It has a built in LSI expander chip. You would just need to feed the two SAS cables from the LSI raid (8i) card to it. You can use the 4i to connect all of the drives via one SAS cable, but I'd recommend getting the 8i version to reduce bottlenecks in case.

[Yatys 93]

Just now, scottyseng said:

Well, RAID60 would be faster, but with that many drives, you wouldn't be able to notice....you'll easily exceed 10Gb/s Ethernet once that thing is fully packed with 24 drives.

 

Yeah, that case would actually be preferred. It has a built in LSI expander chip. You would just need to feed the two SAS cables from the LSI raid (8i) card to it. You can use the 4i to connect all of the drives via one SAS cable, but I'd recommend getting the 8i version to reduce bottlenecks in case.

Ok, I will swap out the 4l for the 8l, then I would only need 2 converter cables correct? I feel like such a noob here

[scottyseng]

1 minute ago, Yatys 93 said:

Ok, I will swap out the 4l for the 8l, then I would only need 2 converter cables correct? I feel like such a noob here

Yep, only two. Don't worry, it took me a lot of reading to learn about RAID cards myself. I just bought my LSI 9260-8i two years ago and it's been running fine since.

 

Ah, I just want to make sure...where is this case going to be placed in? Your room? Do note that server chassis aren't quiet...especially when loaded with WD Re drives...I only have two Re drives, and I can tell you, I can tell when it's seeking data. haha. It's also going to push out a lot of heat.

[Yatys 93]

1 minute ago, scottyseng said:

Yep, only two. Don't worry, it took me a lot of reading to learn about RAID cards myself. I just bought my LSI 9260-8i two years ago and it's been running fine since.

 

Ah, I just want to make sure...where is this case going to be placed in? Your room? Do note that server chassis aren't quiet...especially when loaded with WD Re drives...I only have two Re drives, and I can tell you, I can tell when it's seeking data. haha. It's also going to push out a lot of heat.

No they won't

 

At the moment I am just using an extra drive in my gaming tower as my backup, as I am living away from home for my nursing training, once I move back home, I have a couple of techie freinds who all want to move out of their parents house, we are going to rent a house and put a server cabinet in a spare room, then run wires to where they are needed.

 

the only networking gear that will be in my room is this http://www.amazon.co.uk/dp/B00OZCFVVC/ref=wl_it_dp_o_pd_S_ttl?_encoding=UTF8&colid=1O07UI3BDTGZB&coliid=I3R7T5R8YHX72U

 

which will go in a little 4U cabinet under the desk, and incase your curious I do not need anywhere near 28 slots, but I can't find anything better that still has the 10gb port, I only need the one of those too, as the server its self will have the 10gb switch in the room with it, them I'll run cables to both this switch and directly to my gaming tower.

 

I think the list is done for now  http://uk.pcpartpicker.com/p/Qdgmyc

 

Thanks again!

[scottyseng]

3 minutes ago, Yatys 93 said:

-snip-

Ah, that sounds good. Yeah, my NAS makes my room 7-8F hotter than the rest of the house...I have like 7 Be quiet fans in it right now because the original fans it came with sounded like jet engines. I think if my brother moves out, I'm going to use his room as a server room. haha.

 

I would just double check the clearance of the Noctua coolers, they seem kind of large to use on a dual CPU board. Also keep in mind that 4U cases are limited in height internally. I couldn't fit a hyper 212 in mine, but a 92mm cooler did fit (I use the Hyper D92).

 

Everything else looks good though.

[Yatys 93]

Just now, scottyseng said:

Ah, that sounds good. Yeah, my NAS makes my room 7-8F hotter than the rest of the house...I have like 7 Be quiet fans in it right now because the original fans it came with sounded like jet engines. I think if my brother moves out, I'm going to use his room as a server room. haha.

 

I would just double check the clearance of the Noctua coolers, they seem kind of large to use on a dual CPU board. Also keep in mind that 4U cases are limited in height internally. I couldn't fit a hyper 212 in mine, but a 92mm cooler did fit (I use the Hyper D92).

 

Everything else looks good though.

I got the cooler out of the list on there site for server gear, it's listed as 4U compatible, so I hope it fits  and if it doesn't I can email them and hopefully sort something out.

[scottyseng]

1 minute ago, Yatys 93 said:

I got the cooler out of the list on there site for server gear, it's listed as 4U compatible, so I hope it fits  and if it doesn't I can email them and hopefully sort something out.

Ah, it might be 4U compatible, but dual CPU socket clearance is kind of iffy (Especially with push/pull fans). Well, it hope it works out well for you. Hopefully you can post it when it's built and join the LTT Petabyte club. haha.