electrostatic

My M1015 arrived early, so I went at it, wiped out BIOS and firmwares and reflashed leaving a bloody trail of voided warranties and violated software license agreements in my wake. Some random early notes that may be of interest: I went with this pre-made bundle of firmwares and utilities as it seemed to be frequently updated and had a lot of good discussion around it. My system (briefly) is an in-development FreeNAS machine (non-server grade equipment mostly) with an AMD FX8320, 32GB G.Skill 1866 matched RAM (non ECC, but nerfed down to 1333) and an ASUS M5A99X v2.0 board. The ZFS pool going on the M1015 is eight WD Reds arranged in a stripe of mirrored pairs.) The M1015 had to be in IT mode using Phase 16 firmware to match the Phase 16 drivers in the current build of FreeNAS (9.2.1.6 beta). I intended to use the 9210-8i package, and to install the IT BIOS although I understand you can leave it uninstalled for a faster boot. I wanted to see the boot-time checks and stuff for peace of mind. To avoid the UEFI Kraken, I pulled an old Pentium D (IA64) based Dell desktop out of the closet and used a USB flash drive with freedos and the above mentioned collection on it. The old dell had no problem recognizing the card. After wiping the firmware and BIOS out, I chickened out and used the IT firmware and BIOS that came with the bundle, which was the 9211-8i IT version, instead of the 9210-8i I downloaded. Thankfully, everything worked perfectly, the firmware uploader/flasher verified, and had no complaints at all with loading the included 9211-8i IT firmware on the zombie M1015. The bundle came with Phase 15 firmware, so I then upgraded to the 9211-8i Phase 16 IT BIOS and firmware from LSI's site, again without any issues. I installed the newly brainwashed 9211-8i in my FreeNAS machine, hooked up the pool to it and fired it up. The bare-bones IT BIOS did a little self-test, showed that all the drives were present and working and boot continued as normal. Once up and running, the ZFS pool was there as if nothing had changed. Boot logs and dmesg confirmed that the firmware matched the driver and that FreeBSD seemed more concerned with it being an LSI SAS2008 chipset device and didn't seem to know or care which exact model it was. Jun 18 22:04:06 atlas kernel: mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfe3c0000-0xfe3c3fff,0xfe380000-0xfe3bffff irq 24 at device 0.0 on pci1 Jun 18 22:04:06 atlas kernel: mps0: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd Jun 18 22:04:06 atlas kernel: mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc> This replaces two Highpoint Rocket 640s, and seems to be definitely faster so far, and has resolved false-positive SMART warnings (which is known to be an issue between FreeBSD and Marvell SATA controllers I believe) I did a scrub on the pool (about 2.4TB) which was about 10% faster, and without any errors. I agree completely that it makes sense to use the 9210-8i package on the M1015 (since that's what it physically is) but it might not make much of a difference for use in IT mode. The flashing utility seems to do some extensive hardware and firmware verification before installation and I didn't see even a hint of complaint when installing both phase 15 and 16 9211-8i firmware. If problems do come up, reflashing to the 9210-8i version is the first thing I'm going to do though... Anyway, sorry for the rambling, it's getting late...

Thanks for pointing that out. After much Googling, I found a few people who say either 9210-8i or 9211-8i firmwares are fine for IT mode. Myself, I'm going to start with the 9210-8i IT firmware since all I'm after is the core SATA HBA functionality and don't want to unnecessarily complicate things. As a note specific to FreeNAS users, I also answered my original question about which firmware version (or 'phase' in LSI-speak) to use. The FreeNAS release notes indicate that the driver for LSI 6Gbps SAS/SATA HBAs currently bundled with FreeNAS is version 16 and that the firmware on the card should always match the driver version. So, it's the Phase 16 9210-8i IT firmware for me. I'll let you know how it goes. Well, I should clarify that my server is not at any kind of full-on trusted production state. It will eventually become the One Server to Rule Them All, but I'm still hammering on it, breaking it, learning, forging and re-forging it in the fires of Mount Doom (aka my basement). Once I'm happy with the overall storage design, and have my ZFS snapshots, replication, backing up and restoring skills sharpened up, then I'll get the server-grade motherboard, CPU and of course, ECC RAM. It'll be great to finally get rid of all these external backup drives...

Yeah, you're right, now that I think about it, they're probably adding hundreds of terabytes per minute these days.

My vote is with Samsung, based on the four systems I have using the 830, 840 and 840 EVOs. They make their own flash, their own controller and the Magician software is easily the best I've used for managing desktop SSDs. Plus, their prices are always very competitive....

The data itself is spread out among Google's thousands of datacentres around the world. As for how much? Probably several hundred petabytes? This is a good video: https://www.youtube.com/watch?v=OqQk7kLuaK4

Looks solid, but I would definitely recommend a larger hard drive, something like a 1 or 2 TB WD Blue (or Black if you can afford it) - I've always had great performance from those.

Thanks in advance for the detailed (and illustrated!) guide. My M1015 left Hong Kong yesterday. I'll be using it for FreeNAS, and am recommissioning an older non-UEFI Dell desktop to do the flashing. (I knew that old thing would come in handy some day One question, though. Should I use the most recent firmware package from LSI's site (P19 dated March 2014) or is there a specific older one that should be used? I've been looking around (here and elsewhere) and haven't seen any specific warnings about the current version (yet.) The relevant files mentioned in the guide are all still there and named the same it seems. Thanks again, and it's refreshing to find a forum with this level of detailed, usable info without the resident graybeards automatically assaulting me for not adhering to bank-level IT best practices, ECC RAM, a backup generator and a live failover site in a salt mine in Switzerland for a server to hold my DVD rips and a backup of my Steam library. /rant

Sounds good. I've editied my post to clarify available server vs. OS/Application-reserved storage devices.

No, it's already holding more drives than it was designed for, thanks to the 4x3.5" drive housing that fits in the 5.25" bays. It's a good thing the WD Reds run so quiet and cool.

It's pretty easy, the metal housing thingy the SSDs are mounted in is fastened to the case floor with a couple of screws.

Hey all, just saw this thread and thought I'd show off the FreeNAS monstrosity I've been cobbling together over the last 8 months or so. It started as two old drives and a usb thumb drive in a Windows 7 box and steadily grew into what it is now. Name: ATLAS Usage: Central backup for desktops (two Macs and 3 Win 8.1.1 PCs) Shared working volume for all machines (ie. when renderfarming Blender animations) Software installer repository (ISOs, installers, utils, etc.) Media storage and serving (Plex plug-in / generic DLNA) Consolidate storage in general: death to external HDs. Learning to become one with the ways of ZFS Current Specs: CPU: AMD FX4350 4.2GHz (turbo core off, all other settings default) RAM: Four 8GB G.Skill Ripjaws Series X 1866 kits (left to default 1333) MOBO: ASUS M5A99X R2.0 (95% default settings with all extra onboard features turned off) NIC: Intel Gigabit CT PCIe x1 PSU: Seasonic 550RM 80+ gold CASE: Cooler Master HAF 912 (with Cooler master 4 in 3 drive module) Cooling: Noctua NH-U14S with second 15cm fan, two 120mm Corsair AF Quiet in front, one Noctua P12-1300 top front, Noctua S12PWM rear case and Noctua NF-A14PWM side door Video: some old NVIDIA PCI card (to save on PCIe space) SATA HBAs: 3 x Highpoint Rocket 640L (4xSATA III ports, PCIe2.0 x4) Storage Devices: 4 x WD Red 1TB 2 x WD Red 2TB 2 x WD Red 3TB 2 x WD Green 2TB 1 x WD Green 3TB (Total raw server storage = 21TB) OS, cache and other devices ADATA 8GB USB thumb drive (FreeNAS (FreeBSD) OS boot) Corsair Accelerator 60GB SSD OCZ Synapse Caching SSD (60GB, firmware over-provisioned to 30GB) Storage Configuration I have three ZFS pools configured like this: zpool iostat -v capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- redpool 2.43T 3.91T 0 0 0 0 mirror 691G 2.04T 0 0 0 0 gptid/3b7ce70f-e2de-11e3-5ca0e7d2d - - 0 0 0 0 gptid/3c49195b-e2de-11e3-5ca0e7d2d - - 0 0 0 0 mirror 670G 1.16T 0 0 0 0 gptid/3d134d94-e2de-11e3-5ca0e7d2d - - 0 0 0 0 gptid/3ddcde6c-e2de-11e3-5ca0e7d2d - - 0 0 0 0 mirror 561G 367G 0 0 0 0 gptid/3e8ef650-e2de-11e3-5ca0e7d2d - - 0 0 0 0 gptid/3f499825-e2de-11e3-5ca0e7d2d - - 0 0 0 0 mirror 569G 359G 0 0 0 0 gptid/4009c45b-e2de-11e3-5ca0e7d2d - - 0 0 0 0 gptid/40d0638f-e2de-11e3-5ca0e7d2d - - 0 0 0 0 logs - - - - - - gpt/zil 260K 1.98G 0 0 0 0 -------------------------------------- ----- ----- ----- ----- ----- ----- stripe_pool 3.34T 3.00T 0 0 0 0 gptid/9ea33c3b-da38-11e3-805ca0e7d2d 1.27T 1.45T 0 0 0 0 gptid/9f365f41-da38-11e3-805ca0e7d2d 1.03T 799G 0 0 0 0 gptid/9ffcae67-da38-11e3-805ca0e7d2d 1.04T 787G 0 0 0 0 -------------------------------------- ----- ----- ----- ----- ----- ----- system_ssd 22.6G 30.9G 0 0 0 0 gptid/c79241d9-91b8-11e3-805ca0e7d2d 22.6G 30.9G 0 0 0 0 -------------------------------------- ----- ----- ----- ----- ----- ----- 'redpool' is a ZFS dynamic stripe of mirrors (RAID10, ZFS style) using all the WD Reds in size-matched pairs and is where all my media and daily datasets (ZFS-speak for 'volumes') live 'stripe_pool' is a ZFS dynamic RAID 0 that holds a local rsync mirror of the contents of redpool and datasets for my desktop machines' hourly incrementals Win8 file history and Mac OS X Timemachine. This stripe is a nice example of ZFS's dynamic striping at work: It's made of one 3TB SATA III HD and two 2TB SATA II drives. Traditionally, that would normally be a terrible idea for a stripe, but in my experience here it's been an amazing demonstration of ZFS's dynamic striping at work: when the pool/stripe was created, I could see ZFS measure the size and performance of each member disk. Then as I dumped a few TB of data on it I could watch as the IO operations and data payloads were assigned to each disk based on its capabilities. Sooo, the pool's performance isn't capped by its slowest member device! Pretty awesome stuff. (RAID0 caveats and warnings still apply of course.) 'system_ssd' is FreeNAS' system log store, my home directory and where the plugin jail datasets are. (the old Corsair SSD) A little 2GB partition on the OCZ synapse SSD houses the ZFS intent log store for redpool Pictures Forgive the complete lack of cable management, I am rebalancing the mirrored pairs across the controllers so that no two halves of a mirror vdev are on the same controller. Stack of Drives Thoughts This is by no means finished, I just ordered an IBM Serveraid M1015, so the controller/cable mess will be cleaned up soon. Might not be the 'proper' way to set up a FreeNAS system, but as it is, the system is wicked fast and so far rock solid. Questions and nice comments are welcome! Edited to clarify some details and properly categorize storage device usage.

Performance is about the same as far as I can tell. I haven't been able to try out the AC wireless yet though. I would stick with the ASUS router for a while, though. The firmware in the Linksys is very, very basic at this time. I'm sure it will improve a lot over time, though.

My old OEM Windows 7 Home Premium license has lived on at least four machines before I finally put it in a VM retirement home. All I had to do was call the automated MS code-bot thing, enter the gajillion numbers it asks for, enter the response, tell them that it's only been installed on one machine at a time, and everything's golden. Win 8 is similar, in fact, I had to do it after changing out the CPU and SSD because the OS thought it was on another machine suddenly.

This. ASUS's routers sport hardware like a boss to begin with, their firmware is a mature, well supported fork of WRT, and has features out tha wazoo. Highly recommended.

For unrestricted ventilation, I was pleasantly surprised by the Corsair AF 140 (quiet version) But nothing beats the Noctua NF-S12.