H200 - SAS expander and PCIe lane advice

[jkirkcaldy]

So a while ago I asked on this forum what the best way of getting more sas/sata ports for my server and the advice given was buy a Dell PERC H200 or a LSI 9211-8i but as far as I can tell they are basically the same thing.

 

So I have done exactly that. I also bought an IBM SAS expander card to give me the 16 ports that I need for my server. So I need to flash the H200 to IT mode and will be following this tutorial.

 

But last night I thought I would plug everything in, that way I could at least see if my server picked it up and it was seeing all the drives and this is where I start to bang my head against the wall.

 

So I am using a Z77-UDH5 motherboard with an i7 3770 (it's my old gaming rig.) And there are already some cards plugged in. The layout of the motherboard is:

 

PCIe x1 - gigabit nic

PCIe x16(8) - 10GBe SFP+

PCIe x1 - Hauppage tuner card

PCIe x1 - SATAx4

PCIe x16(4) - Startech sas

PCI - empty

PCIe x16 (4) - Gigabit nic

 

So my plan was to run it in this config:

 

PCIe x1 - gigabit nic

PCIe x16(8) - 10GBe SFP+

PCIe x1 - Hauppage tuner card

PCIe x1 - empty

PCIe x16(4) - H200

PCI - empty

PCIe x16 (4) - SAS Expander

 

However, the problem I am having is that my H200 or my 10GBe isn't recognised by my system unless it is in the top 16x slot.

I can lose the two gigabit nics and even the tuner card if needs be. My motherboard says that when the last 16x slot is occupied the 2nd slot becomes a x4. Maybe this is causing the issue with it not being recognised by the PC? which would be annoying as I'm pretty sure the SAS expander only needs the PCIe for power.

If push comes to shove I could maybe lose the 10GBe but I don't want to. Also I can't get my drives to show up when connected to the SAS expander. Maybe this is a compatibility issue? I read that a sas expander should just work with any HBA but that doesn't seem to be the case here? Do you think I will just have better luck once my card has been flashed to IT mode?

 

It is an IBM 46M0997 SAS expander card.

 

[leadeater]

9 minutes ago, jkirkcaldy said:

However, the problem I am having is that my H200 or my 10GBe isn't recognised by my system unless it is in the top 16x slot.

I can lose the two gigabit nics and even the tuner card if needs be. My motherboard says that when the last 16x slot is occupied the 2nd slot becomes a x4. Maybe this is causing the issue with it not being recognised by the PC? which would be annoying as I'm pretty sure the SAS expander only needs the PCIe for power.

If push comes to shove I could maybe lose the 10GBe but I don't want to. Also I can't get my drives to show up when connected to the SAS expander. Maybe this is a compatibility issue? I read that a sas expander should just work with any HBA but that doesn't seem to be the case here? Do you think I will just have better luck once my card has been flashed to IT mode?

 

It is an IBM 46M0997 SAS expander card.

Welcome to the pain of non server motherboards and the weird firmware/boot issues that come with it. I've had instances where two cards wouldn't work together at all in the same system but individually work fine in it.

 

Flashing the HBA might actually help. It shouldn't matter which slots the HBA and 10Gb cards go in to since they are all 4x anyway, just what ever actually works.

[jkirkcaldy]

Yeah it's a pain.

 

I'd be tempted to just use my R710 for everything but Plex actually performs much better on the i7 compared to the dual xeons.

 

Maybe I'll flash it tonight and then pull everything out, and put it all back in one by one and see where it breaks. Or if it works at all.

[techmattr]

There was a 100+ page long thread on avsforums 5-6 years ago about media storage builds and about 10-20 of those pages we dove into the compatibility issues we were facing at the time with SAS2008 cards (and other HBAs) and Z77/Z87 boards. This brings back memories... The issue we were facing at the time dealt with running 3x SAS2008 HBAs on desktop boards and found that many of them didn't work even though they were electronically capable and it was a crapshoot finding a board that worked with 3 PCIe slots. We tried board with the exact same PCB but different model numbers and one would work with only 1 HBA and the other with 3 HBAs.... it's most likely a firmware issue as @leadeater suggested but ultimately it just makes sense to buy a server or workstation board that you know will run active PCIe slots 100% of the time.

 

As far as the drives not showing up on the expander.... how do you have it connected? It could be anything from user error to the PCIe slot not being active to bad Chinese eBay cables.

[jkirkcaldy]

I got it all working now. I pulled all cards out of the server and flashed the card with the LSI it mode firmware in the one slot I knew it worked in. After I checked it was working and drivers were being passed through correctly I added the expander and plugged all my drives in. It took a bit of work getting them to all show up at once, and everytime i booted the server I would have to pull the drives and put them back in one by one - turns out this was a driver issue and switching to the correct driver solved this.

 

I then plugged my 10GBe card in and that showed up too. so once I confirmed the the H200, 10GBe and SAS expander cards were working correctly I plugged in the 1GB nic and the TV tuner car and everything works.

 

No idea why it didn't work before but everything is working as expected now.

 

I will eventually swap out this server with some actual server grade hardware, I'm thinking a Supermicro CSE-846 with a couple of Octa-core xeons for an plex server that may actually handle transcoding 4k. I will probably change my disk setup at this point too and go for a real hardware RAID setup but until I can find a spare 2k lying around to buy the server and standardise my disks, this will have to do.