Flashing an LSI 9211-8i RAID Card to IT Mode for ZFS/Software RAID (Tutorial)

[electrostatic]

Also, something I found out while helping @Shaqalac: Apparently, the M1015

is actually a 9210-8i. So if the 9211-8i firmware does not work, you could

try going with the 9210-8i firmware. Come to think of it, if it was my

personal setup, I'd even probably start out with the 9210-8i firmware, as

it appears to me that that would be the "more correct" way to go. I'm not

really sure why so many people seem to be flashing the 9211-8i firmware onto

their M1015 cards when there actually is an IT firmware on LSI's website for

the 9210-8i. Maybe they just don't know that the 9210-8i is the M1015 (as

the 9210-8i is just an OEM controller and isn't sold as-is to end consumers,

so it could just be a matter of recognizeability), or maybe there's a

specific technical reason for it. Feel free to let me know if you come

across anything on that, it would be nice to have more info on that matter.

 

Thanks for pointing that out.  After much Googling, I found a few people who say either 9210-8i or 9211-8i firmwares are fine for IT mode.  Myself, I'm going to start with the 9210-8i IT firmware since all I'm after is the core SATA HBA functionality and don't want to unnecessarily complicate things.

 

As a note specific to FreeNAS users, I also answered my original question about which firmware version (or 'phase' in LSI-speak) to use.    The FreeNAS release notes indicate that the driver for LSI 6Gbps SAS/SATA HBAs currently bundled with FreeNAS is version 16 and that the firmware on the card should always match the driver version.  

 

So, it's the Phase 16 9210-8i IT firmware for me.  I'll let you know how it goes.

 

 

Haha, well, you don't have many posts yet, you get some slack (you

have until you get to 100 posts to backup your data to a Swiss bunker ).

Seriously though: If you make a thread asking for advice, yeah we'll

usually recommend ECC for ZFS and all that (as well as large memory

capacity), since that's what is actually recommended officially.

 

Well, I should clarify that my server is not at any kind of full-on trusted production state.  It will eventually become the One Server to Rule Them All, but I'm still hammering on it, breaking it, learning, forging and re-forging it in the fires of Mount Doom (aka my basement).  Once I'm happy with the overall storage design, and have my ZFS snapshots, replication, backing up and restoring skills sharpened up, then I'll get the server-grade motherboard, CPU and of course, ECC RAM.  It'll be great to finally get rid of all these external backup drives...

[alpenwasser]

- snip - 

So, it's the Phase 16 9210-8i IT firmware for me.  I'll let you know how it goes.

Excellent, looking forward to results.

 

Well, I should clarify that my server is not at any kind of full-on trusted production state.  It will eventually become the One Server to Rule Them All, but I'm still hammering on it, breaking it, learning, forging and re-forging it in the fires of Mount Doom (aka my basement).  Once I'm happy with the overall storage design, and have my ZFS snapshots, replication, backing up and restoring skills sharpened up, then I'll get the server-grade motherboard, CPU and of course, ECC RAM.  It'll be great to finally get rid of all these external backup drives...

Ah, sounds like quite the plan. Feel free to do a build log on your

server when you get around to that.

[looney]

I took the easy route this time and just bought a 9210-8i to company my IT flashed m1015.

[alpenwasser]

I took the easy route this time and just bought a 9210-8i to company my IT flashed m1015.

Interesting. Where'd you manage to find it? I have looked for it a few

times out of curiosity, but haven't found anything (which kinda makes

sense because it's an OEM-only card).

[looney]

Interesting. Where'd you manage to find it? I have looked for it a few

times out of curiosity, but haven't found anything (which kinda makes

sense because it's an OEM-only card).

Found it on a dutch tech newsite/forum (http://www.tweakers.net)

It secondhand advertised as 9210 but I guess ill just see when it comes in.

 

The picture looked like 9210/1015 so im happy

[Shaqalac]

Also, something I found out while helping @Shaqalac: Apparently, the M1015

is actually a 9210-8i. So if the 9211-8i firmware does not work, you could

try going with the 9210-8i firmware. Come to think of it, if it was my

personal setup, I'd even probably start out with the 9210-8i firmware, as

it appears to me that that would be the "more correct" way to go. I'm not

really sure why so many people seem to be flashing the 9211-8i firmware onto

their M1015 cards when there actually is an IT firmware on LSI's website for

the 9210-8i. Maybe they just don't know that the 9210-8i is the M1015 (as

the 9210-8i is just an OEM controller and isn't sold as-is to end consumers,

so it could just be a matter of recognizeability), or maybe there's a

specific technical reason for it. Feel free to let me know if you come

across anything on that, it would be nice to have more info on that matter.

I can't recommend this hard enough. Unfortunately I started out with a bricked card and any 9211-8i firmware wouldn't work, which most likely was before of the card being bricked.

After @alpenwasser's discovery I did managed to flash the card with an older version of a 9210-8i firmware.

I'm not saying you won't be able to flash a M1015 with a 9211-8i, because that's obviously not true with all the threads around the internet.

What I'm going to do is to recommend you to flash it with a 9210-8i firmware.

[electrostatic]

My M1015 arrived early, so I went at it, wiped out BIOS and firmwares and reflashed leaving a bloody trail of voided warranties and violated software license agreements in my wake.

 

Some random early notes that may be of interest:

 

 

I went with this pre-made bundle of firmwares and utilities as it seemed to be frequently updated and had a lot of good discussion around it.

 

My system (briefly) is an in-development FreeNAS machine (non-server grade equipment mostly) with an AMD FX8320, 32GB G.Skill 1866 matched RAM (non ECC, but nerfed down to 1333) and an ASUS M5A99X v2.0 board.  The ZFS pool going on the M1015 is eight WD Reds arranged in a stripe of mirrored pairs.)

 

The M1015 had to be in IT mode using Phase 16 firmware to match the Phase 16 drivers in the current build of FreeNAS (9.2.1.6 beta).  I intended to use the 9210-8i package, and to install the IT BIOS although I understand you can leave it uninstalled for a faster boot.  I wanted to see the boot-time checks and stuff for peace of mind.

 

To avoid the UEFI Kraken, I pulled an old Pentium D (IA64) based Dell desktop out of the closet and used a USB flash drive with freedos and the above mentioned collection on it.  The old dell had no problem recognizing the card.  After wiping the firmware and BIOS out, I chickened out and used the IT firmware and BIOS that came with the bundle, which was the 9211-8i IT version, instead of the 9210-8i I downloaded.

 

Thankfully, everything worked perfectly, the firmware uploader/flasher verified, and had no complaints at all with loading the included 9211-8i IT firmware on the zombie M1015.

 

The bundle came with Phase 15 firmware, so I then upgraded to the 9211-8i Phase 16 IT BIOS and firmware from LSI's site, again without any issues.

 

I installed the newly brainwashed 9211-8i in my FreeNAS machine, hooked up the pool to it and fired it up.  The bare-bones IT BIOS did a little self-test, showed that all the drives were present and working and boot continued as normal.  Once up and running, the ZFS pool was there as if nothing had changed.  Boot logs and dmesg confirmed that the firmware matched the driver and that FreeBSD seemed more concerned with it being an LSI SAS2008 chipset device and didn't seem to know or care which exact model it was.

 

 

Jun 18 22:04:06 atlas kernel: mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfe3c0000-0xfe3c3fff,0xfe380000-0xfe3bffff irq 24 at device 0.0 on pci1

Jun 18 22:04:06 atlas kernel: mps0: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd

Jun 18 22:04:06 atlas kernel: mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>

 

 

This replaces two Highpoint Rocket 640s, and seems to be definitely faster so far, and has resolved false-positive SMART warnings (which is known to be an issue between FreeBSD and Marvell SATA controllers I believe)

 

I did a scrub on the pool (about 2.4TB) which was about 10% faster, and without any errors.  

 

I agree completely that it makes sense to use the 9210-8i package on the M1015 (since that's what it physically is) but it might not make much of a difference for use in IT mode.  The flashing utility seems to do some extensive hardware and firmware verification before installation and I didn't see even a hint of complaint when installing both phase 15 and 16 9211-8i firmware.  If problems do come up, reflashing to the 9210-8i version is the first thing I'm going to do though...

 

Anyway, sorry for the rambling, it's getting late...

[alpenwasser]

- snip -

Thanks for documenting your process, good to know.

[wpirobotbuilder]

Thought this might be useful to someone in the states:

 

Here is an ebay seller who has a few m1015s (the same controller as the 9211-8i) that are already in IT mode, and they're selling for $135.

 

Just in case you can't get the flash working.

[wpirobotbuilder]

Out of curiosity, did you have to have a CPU installed in the motherboard for it to work?

[alpenwasser]

Out of curiosity, did you have to have a CPU installed in the motherboard for it to work?

Yes, the PC was (well, is) fully equipped (it's one of my dad's work PCs).

EDIT:

I never tried it without a CPU, so I'm not sure if a CPU is actually needed

for it to work, but I didn't see the point in disassembling the machine just

to find out. I was just glad I had finally got it to run.

Edited January 5, 2015 by alpenwasser

[wpirobotbuilder]

Yes, the PC was (well, is) fully equipped (it's one of my dad's work PCs).

EDIT:

I never tried it without a CPU, so I'm not sure if a CPU is actually needed

for it to work, but I didn't see the point in disassembling the machine just

to find out. I was just glad I had finally got it to run.

I figure you would need one because most boards don't let you get in without a CPU and memory installed.

 

Is this the board you used? I have a Sandy Bridge pentium I could use to do this.

[wpirobotbuilder]

Also, I was thinking about whether a motherboard's native EFI shell will work. My Z97 MPOWER board will automatically enter an EFI shell if there's no bootable device there.

[alpenwasser]

I figure you would need one because most boards don't let you get in without a CPU and memory installed.

 

Is this the board you used? I have a Sandy Bridge pentium I could use to do this.

 

I'm not sure if it was Revision 3.0, but yes, that is my Go-To board for flashing my LSI controllers.

I'll try to find out the revision when I get home tonight.

 

Also, I was thinking about whether a motherboard's native EFI shell will work. My Z97 MPOWER board will automatically enter an EFI shell if there's no bootable device there.

Overall the process has been pretty finicky, but I'd certainly give it a shot if your board has a native EFI shell. Would have saved me quite a bit of hassle if I could have done that.

[Danny91]

Thanks very much for the guide, used it on two cards i got of ebay with a Asrock extreme 4 with no issues. 

 

Although i will warn people i had issues  with CRC errors and slow transfers using the 20 firmware but once i downgraded to 19 it was amazing. So if anyone runs into weird problems try going to 19 rather than 20.

 

Thanks again

[alpenwasser]

Although i will warn people i had issues  with CRC errors and slow transfers using the 20 firmware but once i downgraded to 19 it was amazing. So if anyone runs into weird problems try going to 19 rather than 20.

Glad you found it useful, and thanks for the info!

[Danny91]

Glad you found it useful, and thanks for the info!

 

No worries just hope it saves some people a headace i went through, with the 20 firmware i was only getting 5MB/s parity and lots of errors between the had drvies and the cards but once i downgraded i get 500-600MB/s calculation

 

Also i dont know if it has been mentioned here before but in windows if you edit the lsi_sas2.inf driver file and ensure these lines are in it then spin up and down will work in windows. Although adding the lines you will invalidate the drivers certificate so you have to say you want to add it anyways. Done this for both of my cards and now my drives spin up and down as needed.

 

 

[shutdown_addreg]

HKR,"ScsiPort","NeedsSystemShutdownNotification",0x00010001,1

HKR, "StorPort", "EnableIdlePowerManagement", 0x00010001, 0x01

[alpenwasser]

- snip -

Excellent, added a link to your post to OP.

[wpirobotbuilder]

No worries just hope it saves some people a headace i went through, with the 20 firmware i was only getting 5MB/s parity and lots of errors between the had drvies and the cards but once i downgraded i get 500-600MB/s calculation

 

Also i dont know if it has been mentioned here before but in windows if you edit the lsi_sas2.inf driver file and ensure these lines are in it then spin up and down will work in windows. Although adding the lines you will invalidate the drivers certificate so you have to say you want to add it anyways. Done this for both of my cards and now my drives spin up and down as needed.

Wow, those are all useful finds!

[wpirobotbuilder]

Btw, it looks like FreeNAS has the sas2flash utility built in to their OS. Below is the output from my machine.

 

You could, in theory, just run:

sas2flash -o -f {firmware_file.bin}

And have it updated. I don't have a spare card lying around to test this with, but one of the guys from the FreeNAS forum confirmed it for me..

[root@freenas] ~# sas2flash -listLSI Corporation SAS2 Flash UtilityVersion 14.00.00.00 (2012.07.04)Copyright (c) 2008-2012 LSI Corporation. All rights reserved        Adapter Selected is a LSI SAS: SAS2008(B2)        Controller Number              : 0        Controller                     : SAS2008(B2)        PCI Address                    : 00:03:00:00        SAS Address                    : 5000ea6-3-2005-7300        NVDATA Version (Default)       : 0d.43.00.00        NVDATA Version (Persistent)    : 0d.43.00.00        Firmware Product ID            : 0x2713 (IR)        Firmware Version               : 13.00.57.00        NVDATA Vendor                  : LSI        NVDATA Product ID              : Undefined        BIOS Version                   : 07.25.00.00        UEFI BSD Version               : N/A        FCODE Version                  : N/A        Board Name                     : PIKE 2008        Board Assembly                 : N/A        Board Tracer Number            : N/A        Finished Processing Commands Successfully.        Exiting SAS2Flash.[root@freenas] ~#

[alpenwasser]

- snip -

That's pretty neat, added it to OP, thanks!

[wpirobotbuilder]

That's pretty neat, added it to OP, thanks!

You might want to add an addendum, because this article says that sas2flsh, not sas2flash allows you to convert between IR and IT firmware. Although that is confusing, since you used sas2flash successfully to convert between firmware types.

[Cheatdeath]

I am going nuts trying to flash my LSI 9211-8 to P16 IT firmware for use with Freenas. I have tried boot-able dos usb method and it gives the "failed to initialize PAL Error" , booting into the efi shell method will not detect my files after mounting drive...see attached image. I am using a Supermicro X10SLL-F motherboard which has a built in EFI shell. I have been able to use another machine to get into a efi shell and view my files but when I attempted "sas2flash.efi -o -e 6" the raid card is not detected. There must be a way to do this via my Supermicro motherboard, please someone help! 

 

Please do not start with I should do a google search or any shit like that I am far beyond that, if however you have located a thread worth looking into please let me know, it would be much appreciated.

 

 

 

[alpenwasser]

I am going nuts trying to flash my LSI 9211-8 to P16 IT firmware for use with Freenas. I have tried boot-able dos usb method and it gives the "failed to initialize PAL Error" , booting into the efi shell method will not detect my files after mounting drive...see attached image. I am using a Supermicro X10SLL-F motherboard which has a built in EFI shell. I have been able to use another machine to get into a efi shell and view my files but when I attempted "sas2flash.efi -o -e 6" the raid card is not detected. There must be a way to do this via my Supermicro motherboard, please someone help! 

 

Please do not start with I should do a google search or any shit like that I am far beyond that, if however you have located a thread worth looking into please let me know, it would be much appreciated.

That is weird.

What filesystem are you using for your flash drive? Can you try printing the current directory

once you've changed location onto the USB drive?

So:

ft0:pwd

Not sure if this command even exists on UEFI shell, and I can't test it out at the moment,

but I'm curious if the USB drive's directory structure isn't detected at all or if it's

something else.

Oh, and no worries: The whole reason this thread exists is because I had huge trouble

figuring this out myself from other sources and needed a ton of trial and error until I

found a process which worked. It's entirely possible that something which is different

in your config from mine might prevent the process from working for you (it's really

fucking finicky ).

EDIT:

What do you get for

sas2flash.efi -listall

on the other machine?

[Cheatdeath]

That is weird.

What filesystem are you using for your flash drive? Can you try printing the current directory

once you've changed location onto the USB drive?

So:

ft0:pwd

Not sure if this command even exists on UEFI shell, and I can't test it out at the moment,

but I'm curious if the USB drive's directory structure isn't detected at all or if it's

something else.

Oh, and no worries: The whole reason this thread exists is because I had huge trouble

figuring this out myself from other sources and needed a ton of trial and error until I

found a process which worked. It's entirely possible that something which is different

in your config from mine might prevent the process from working for you (it's really

fucking finicky ).

EDIT:

What do you get for

sas2flash.efi -listall

on the other machine?

 

 

sas2flash.efi -listall on my new PFsense box ends with "no lsi card detected" something like that, its a new system I just built and it was hopeful because I was able to see my files in the efi shell but the system does not detect the card.... if there is something I can do to make the card show up I think that system has promise.  I have tried everything and searched for days online trying to find another way, I basically have no other hardware left to try with. Having this brand new freenas server built and loaded with drives but no HBA card is driving me nuts, this is my first dedicated server.