FreeNAS on older HPZ800 with 4TB drives

[Sevilla]

I have an older HPZ800 server currently running Windows 7 Pro 64bit (didn't have a Server license at the time)  on it, along with 2x4TB drives in RAID 1. In order to get the drives detected on this machine at the appropriate size (because the raid controller only saw them as 1.6TBs each) I had to format them in GPT and software RAID 1 them in Windows.

 

I am now looking to transition to FreeNAS. My question is will FreeNAS be able to see my drives as 4TB drives, even though my controller doesn't? It seems Windows was able to get them to work properly, but I am not sure if FreeNAS will be able to apply its own software to see them at their full size.

 

Any ideas/feedback is welcomed and appreciated.

[KirbyTech]

Yes FreeNAS should see them properly. I can't see why it wouldn't. I take it you are upgrading your storage solution?

[Sevilla]

Just now, KirbyTech said:

Yes FreeNAS should see them properly. I can't see why it wouldn't. I take it you are upgrading your storage solution?

Yeah, when I got the HPZ800 (for free at work) along with 48GBs of ECC DDR3 RAM, I bought 2x 4TB drives and popped whatever OS I could at the time, mostly to experiment and get a feel for it in software RAID 1. Half a year later, I host several things through it via VMs but all running form a host OS. Also, have a lot of people depending on it now as a file server. So now I am looking to upgrade from 4TBs to 8TBs of space (Raid 5/Raid Z) so I need to know before I setup ESX and use FreeNAS as a VM if it'll even see the drives because the BIOS on it (latest one HP has released) only sees them at a fraction of their actual size, which is why I had to do a software RAID in Windows.

 

I'm mostly looking to have to avoid buying a RAID card and spend at least 100 bucks to get it, along with extra time I'll need to allocate to begin the update process. If it'll work with the current controller, I'll be happy, but If I have to splurge an extra 100 for a RAID card, I may have to put the upgrade on hold.

[KirbyTech]

If people depend on the server it might be best to spend the money for the RAID card and suck it up. I also would say go with a RAID 6 if you are using 4tb drives or bigger. I am not sure about that exact system but I don't see why it wouldn't see the full drive size unless it is BIOS locked and I can't see that happening. Can you boot into a second drive and see how they show up there without writing data to the drive?

[Sevilla]

I may end up doing that, but with a new 4TB drive. I really don't want to risk the data. Lots of irreplaceable stuff on it already (dissertations/medical records/home movies/etc).

[Chris_Hercules]

     Im JEALOUS. You have a Z800 workstation you are using as a server and want to run FreeNAS on it? Wow, Im looking to replace my XW6600 workstation for a Z800 as my daily driver. I realize that the 800 will actually be pretty perfect for that, but man, its all about perspective. Ive used the 800 and its a solid solid machine.

     Also, a word of advice...If you are running FreeNAS or will be you do want to put an add in card in that machine but you want an HBA (host bus adapter) not a hardware RAID card. A lot of cards can be cross flashed to IT mode from what I understand. Do spend the money on hardware card though. A really popular basic one that can be crossflased is the IDM M1015 It is a branded LSI controller. I think but do not remeber for sure, that the onboard SATA ports are only SATA II 3.0Gbps.

[Sevilla]

5 hours ago, Chris_Hercules said:

     Im JEALOUS. You have a Z800 workstation you are using as a server and want to run FreeNAS on it? Wow, Im looking to replace my XW6600 workstation for a Z800 as my daily driver. I realize that the 800 will actually be pretty perfect for that, but man, its all about perspective. Ive used the 800 and its a solid solid machine.

     Also, a word of advice...If you are running FreeNAS or will be you do want to put an add in card in that machine but you want an HBA (host bus adapter) not a hardware RAID card. A lot of cards can be cross flashed to IT mode from what I understand. Do spend the money on hardware card though. A really popular basic one that can be crossflased is the IDM M1015 It is a branded LSI controller. I think but do not remeber for sure, that the onboard SATA ports are only SATA II 3.0Gbps.

Yeah, it was originally used at my company for heavy CAD engineering. They were donating them cause they were 3+ years old. I got one of them. Then they got rid of another because the board fried (surge fried it) so I took all the RAM, and managed to salvage 16GBs of working ECC RAM and maxed it out.

Also, you're probably right. A friend already told me (and I've been trying not to get it, but I will have to apparently) a raid card set to HBA, specifically the  IBM M1015 flashed to LSI 9211-8i in IT Mode which you mentioned.

 

Oh well, I guess I'll have to spend it.

[F4S4K4N]

10 hours ago, Sevilla said:

Yeah, when I got the HPZ800 (for free at work) along with 48GBs of ECC DDR3 RAM, I bought 2x 4TB drives and popped whatever OS I could at the time, mostly to experiment and get a feel for it in software RAID 1. Half a year later, I host several things through it via VMs but all running form a host OS. Also, have a lot of people depending on it now as a file server. So now I am looking to upgrade from 4TBs to 8TBs of space (Raid 5/Raid Z) so I need to know before I setup ESX and use FreeNAS as a VM if it'll even see the drives because the BIOS on it (latest one HP has released) only sees them at a fraction of their actual size, which is why I had to do a software RAID in Windows.

 

I'm mostly looking to have to avoid buying a RAID card and spend at least 100 bucks to get it, along with extra time I'll need to allocate to begin the update process. If it'll work with the current controller, I'll be happy, but If I have to splurge an extra 100 for a RAID card, I may have to put the upgrade on hold.

Wait so you want to run FreeNAS in an ESX VM? That's generally not a good idea as ZFS wont be able to see any of the hardware, and if it can't see the hardware you loose half of the magic of ZFS. At that point you're better off just running Samba/NFS on Linux in a VM. If VM's are really important i would just install Linux, ZFS, and KVM / VirtualBox on the host. The host in that case could be a file server and VM host. FreeNAS can also run VirtualBox, but it's not supported really. Keep in mind that FreeNAS is just FreeBSD with a fancy UI. Just like how unRAID is just Linux with a fancy UI.

7 hours ago, Chris_Hercules said:

     Im JEALOUS. You have a Z800 workstation you are using as a server and want to run FreeNAS on it? Wow, Im looking to replace my XW6600 workstation for a Z800 as my daily driver. I realize that the 800 will actually be pretty perfect for that, but man, its all about perspective. Ive used the 800 and its a solid solid machine.

     Also, a word of advice...If you are running FreeNAS or will be you do want to put an add in card in that machine but you want an HBA (host bus adapter) not a hardware RAID card. A lot of cards can be cross flashed to IT mode from what I understand. Do spend the money on hardware card though. A really popular basic one that can be crossflased is the IDM M1015 It is a branded LSI controller. I think but do not remeber for sure, that the onboard SATA ports are only SATA II 3.0Gbps.

More specifically you want a SAS HBA. You can still use SATA drives with a SAS HBA but if you have a single drive failure with a SATA HBA you can lock up the entire SATA bus bringing down all of the other drives as well. SAS does not have this issue as each drive is treated as an individual device, where SATA uses a daisy-chain like structure.

[Sevilla]

8 hours ago, F4S4K4N said:

Wait so you want to run FreeNAS in an ESX VM? That's generally not a good idea as ZFS wont be able to see any of the hardware, and if it can't see the hardware you loose half of the magic of ZFS. At that point you're better off just running Samba/NFS on Linux in a VM. If VM's are really important i would just install Linux, ZFS, and KVM / VirtualBox on the host. The host in that case could be a file server and VM host. FreeNAS can also run VirtualBox, but it's not supported really. Keep in mind that FreeNAS is just FreeBSD with a fancy UI. Just like how unRAID is just Linux with a fancy UI.

More specifically you want a SAS HBA. You can still use SATA drives with a SAS HBA but if you have a single drive failure with a SATA HBA you can lock up the entire SATA bus bringing down all of the other drives as well. SAS does not have this issue as each drive is treated as an individual device, where SATA uses a daisy-chain like structure.

Via ESX I can assign the specific controller (in this case the  IBM M1015 flashed to LSI 9211-8i in IT Mode) and it'll directly detect all the drives in non-RAID mode, which will permit it to run in ZFS. That's specifically why the IBM M1015 flashed to LSI 9211-8i has to be flashed so that it behaves like an HBA 6.0gbps SAS / SATA III adapter, hence "IT mode."

 

The friend I mentioned who recommended this has the same card, flashed the same way, and has a server setup the same way as well running FreeNAS in a VM, just dedicating a lot of RAM to it.

 

Realistically, the only plugin I'd use in FreeNAS is probably CrashPlan since I have a subscription and they currently have my entire 4TB raid backed up. Also, the other big reason for using FreeNAS for me is because it has snapshot backups. I have CrashPlan running backup checks every 15 minutes and it's not always optimal, specially if I have a lot of data that needs to be transferred on to the storage RAID at a random point of the day (I rather it run overnight when no one is using the internet).

[F4S4K4N]

2 hours ago, Sevilla said:

Via ESX I can assign the specific controller (in this case the  IBM M1015 flashed to LSI 9211-8i in IT Mode) and it'll directly detect all the drives in non-RAID mode, which will permit it to run in ZFS. That's specifically why the IBM M1015 flashed to LSI 9211-8i has to be flashed so that it behaves like an HBA 6.0gbps SAS / SATA III adapter, hence "IT mode."

 

The friend I mentioned who recommended this has the same card, flashed the same way, and has a server setup the same way as well running FreeNAS in a VM, just dedicating a lot of RAM to it.

 

Realistically, the only plugin I'd use in FreeNAS is probably CrashPlan since I have a subscription and they currently have my entire 4TB raid backed up. Also, the other big reason for using FreeNAS for me is because it has snapshot backups. I have CrashPlan running backup checks every 15 minutes and it's not always optimal, specially if I have a lot of data that needs to be transferred on to the storage RAID at a random point of the day (I rather it run overnight when no one is using the internet).

Are you going to use PCI passthrough on that controller? If not, you're not really seeing the physical drives in the VM. Without PCI passthrough you're basically seeing a virtual disk socket presented to the VM per disk. You don't really get the hardware information ZFS needs to work efficiently in this case.

 

If you are doing PCI passthrough then that should be ok, just keep in mind that you are adding an extra layer to the stack that introduces strange possibilities for failure modes. That why most people wont virtualize a storage system, or a router for that matter. Generally those two things provide core services that need to have as much up-time as possible. However that just depends on how important you're storage server is.

[Sevilla]

8 minutes ago, F4S4K4N said:

Are you going to use PCI passthrough on that controller? If not, you're not really seeing the physical drives in the VM. Without PCI passthrough you're basically seeing a virtual disk socket presented to the VM per disk. You don't really get the hardware information ZFS needs to work efficiently in this case.

 

If you are doing PCI passthrough then that should be ok, just keep in mind that you are adding an extra layer to the stack that introduces strange possibilities for failure modes. That why most people wont virtualize a storage system, or a router for that matter. Generally those two things provide core services that need to have as much up-time as possible. However that just depends on how important you're storage server is.

Yes, it'll be a straight PCI passthrough. Generally speaking I could use the server to strictly use FreeNAS and just use jails for everything else I need, but I'd like to stick with ESX for now. Even if it fails I'd still get a great understanding for how it should operate. Also, realistically, if there is a hardware failure, whether on a VM or not, it'd fail so i'd be screwed either way, this is why I'd use the Crashplan plugin.

Also, if it were to fail, I could try Unraid too, but I guess we'll see how it goes.

[F4S4K4N]

17 minutes ago, Sevilla said:

Yes, it'll be a straight PCI passthrough. Generally speaking I could use the server to strictly use FreeNAS and just use jails for everything else I need, but I'd like to stick with ESX for now. Even if it fails I'd still get a great understanding for how it should operate. Also, realistically, if there is a hardware failure, whether on a VM or not, it'd fail so i'd be screwed either way, this is why I'd use the Crashplan plugin.

Also, if it were to fail, I could try Unraid too, but I guess we'll see how it goes.

Jails only run FreeBSD, so not sure if that matters to you or not but it's something to consider. FreeNAS also has VirtualBox, but like i said it's not officially supported.

 

PCI passthrough should be fine. You can run into timing issues where they hypervisor lags behind the PCI bus on the box, but that usually only happens under heavy load. Most OS's should handle that without any issues unless it gets really bad.

[Sevilla]

6 minutes ago, F4S4K4N said:

Jails only run FreeBSD, so not sure if that matters to you or not but it's something to consider. FreeNAS also has VirtualBox, but like i said it's not officially supported.

 

PCI passthrough should be fine. You can run into timing issues where they hypervisor lags behind the PCI bus on the box, but that usually only happens under heavy load. Most OS's should handle that without any issues unless it gets really bad.

I think it'll be fine, realistically the entire server will mostly be running Plex on CentOS, FreeNAS, couple of VMs (one Linux, one Windows), seedbox, and maybe an encoding box. The reality of everything running under heavy load at once is virtually impossible.

But that's part of the fun right? Running into problems, losing data, spending endless hours fixing a small issue until 4 a.m., but you have to go to work at 6?

That's the life I am choosing, apparently.

[Chris_Hercules]

Quote

Yeah, it was originally used at my company for heavy CAD engineering. They were donating them cause they were 3+ years old. I got one of them. Then they got rid of another because the board fried (surge fried it) so I took all the RAM, and managed to salvage 16GBs of working ECC RAM and maxed it out. Whoa.. I missed something, are you running single or dual proc? with dual procs thats a capacity of 192 gigs of ECC ram..... Holy god. Thats a serious chunk of change just for a company to give away. Kudos to being the right person at the right place at the right time.

 

[Sevilla]

Just now, Chris_Hercules said:

 

Dual Xeon E5620 CPUs. I could add more RAM, though I'd have to buy 8GB ECC sticks. For the time being, all the slots on the board are filled so it's maxed out to 48GBs ECC.

 

The only upgrade I am considering for this machine is upgrading the chips to Hexa cores so I could get 12 cores/24 threads. 8C/16T is great for now, but long term I'll beef it up.