VMware HPE Build with MSA 2062 Need help

[Bear_99]

I'm currently looking to build an affordable VMWARE HPE Build. I have a basic template but not sure if I have everything and I'm new to this.

 

Setting up a small amount of VM's ~5 that has requirements of running on dedicated hosts that is for specific applications to SQL Server Standard Database. Sadly our main vmcluster has ran out of resources to support this. We have enough Vmware licenses to cover this.

 

1st VM: 4 CPU, 32GB, 1TB 

2nd VM: 4 CPU, 32GB, 2TB

3rd VM: Asked that Physical cores should be considered in the design. Requires 24cores 64gb of ram 1TB

4th VM: 8 CPU's 32GB 2TB

5th VM: 8 CPU 32GB 1TB

 

Going by my server spec should have enough to support this many hosts.

 

Going for 3 Servers for a N+1 setup + SAN

Looking to setup 10gig. The confusing part is how do I direct connect all the servers together?

 

3x HPE DL380 Gen 11

• 2x INTEL XEON-SILVER 4514Y 16cores
• 256GB of Ram 8x32
• 1x Broadcom BCM57412 Ethernet 10Gb 2-port SFP+ OCP3 Adapter for HPE
• 1x Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter for HPE
• 1x 480GB Read intensive SSD 
• 1x HPE NS204i-u Gen11 NVMe Hot Plug Boot Optimized Storage Device Raid 1 480x480
• 2x 1000w psu
• 1x HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller

1x HPE MSA 2062 12Gb SAS SFF Storage
22x HPE MSA 1.92TB SAS 12G Read Intensive SFF (2.5in) M2 3yr Wty SSD

 

 

Connections

1x Cisco 9500 16 port 10GB SFP switch - 3 ports taken 1x each server.

1x Cisco 9300 24 Port 1GB Switch - 12 ports taken 4x each server for redundancy. + 4 for ilo = 16

 

HPE MSA SAS 

3x SAS Connector from MSA to each server.

 

I guess my question is direct connecting the 3 servers together? I assume doing it over the 10GB switch would be a bad idea? 

 

[leadeater]

33 minutes ago, Bear_99 said:

Going for 3 Servers for a N+1 setup + SAN

Looking to setup 10gig. The confusing part is how do I direct connect all the servers together?

The MSA 2062 host connection ports are multi protocol and for your setup I would actually recommend 12GB SAS and directly connect them that way, you need a SAS HBA in each server (2 cards recommended). Each controller card in the MSA has 4 host connection ports so 4 + 4 (8 total) giving you capacity of up to 4 directly connected servers with dual path redundancy. This is a typical setup I used to do in a prior job for clients and it works very well. Also using SAS will give you the most bandwidth (4x 12Gb per port) and the lowest latency (not that the 3 options will have much difference but Ethernet will be the highest even with RDMA).

 

For the backend disk there does not need to be any connectivity between the VM host servers, just between the host and the SAN. All HA and vMotion capabilities etc will work.

 

For your network connectivity the it's going to be pretty standard to what you would already know dual path to redundant switches preferable if you have that, existing switches will be suitable so you won't need to buy new switches for this unless you don't have available ports or ability to patch through to them from the rack location you will be putting these servers in to.

 

33 minutes ago, Bear_99 said:

3x HPE DL380 Gen 11

• 2x INTEL XEON-SILVER 4514Y 16cores

Any reason you are going dual socket here and/or Gen11, also Intel? I would actually recommend going with a single socket solution and DL320/DL325 because of that (if enough PCie slots, not checked). Also means you can go with single SAS card and network card (dual socket it's best to have a card per socket). Going single socket avoids NUMA constraints and any performance impacts from that, ESXi will manage that but it's actually better to have 1 CPU with 32 cores than 2 CPUs with 16.

 

If you do go with Intel and Gen 11 I would actually recommend the 6526Y for SQL workloads as operating frequency matters quite a bit for that and it also has more cache, it would obviously cost more. Single socket the 6530 is a good option.

 

33 minutes ago, Bear_99 said:

22x HPE MSA 1.92TB SAS 12G Read Intensive SFF (2.5in) M2 3yr Wty SSD

It probably won't matter much with that many SSDs but do be aware that RI SSDs don't have good write latency when pushed hard in writes so you might not get as good performance as you might expect. Mixed Use (MU) would be more recommended unless you know that your workload will be almost entirely reads.

 

[Bear_99]

I really appreciate your post. very helpful!

 

1 hour ago, leadeater said:

The MSA 2062 host connection ports are multi protocol and for your setup I would actually recommend 12GB SAS and directly connect them that way, you need a SAS HBA in each server (2 cards recommended). Each controller card in the MSA has 4 host connection ports so 4 + 4 (8 total) giving you capacity of up to 4 directly connected servers with dual path redundancy. This is a typical setup I used to do in a prior job for clients and it works very well. Also using SAS will give you the most bandwidth (4x 12Gb per port) and the lowest latency (not that the 3 options will have much difference but Ethernet will be the highest even with RDMA).

I will go with the MSA 2062 12GB SAS SFF Storage.  I will make sure to change my count of connectors. 3 to 6 HPE External 2.0m (6ft) Mini-SAS HD 4x to Mini-SAS HD 4x Cable

 

The template build I got for my server was these two SAS Controller. 
assume the Smart array one is the one I will be connecting to the MSA. Then the MR416ip is the storage controller if i had HDD in the server.

1x HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller

1x HPE MR416ip Gen11 x16 Lanes 8GB Cache PCI SPDM Plugin Storage Controller

I see they offer other ones but I'm unfamilar with any HPE parts.

1 hour ago, leadeater said:

For the backend disk there does not need to be any connectivity between the VM host servers, just between the host and the SAN. All HA and vMotion capabilities etc will work.

 

Ah I see so there's no need to connect each Host server to each other via 10gig nics all i need is the connectors to the SAN. 

1 hour ago, leadeater said:

For your network connectivity the it's going to be pretty standard to what you would already know dual path to redundant switches preferable if you have that, existing switches will be suitable so you won't need to buy new switches for this unless you don't have available ports or ability to patch through to them from the rack location you will be putting these servers in to.

I will be most likely buying new switches as well. I saw these two in my build template that was given to me. So I figured since i need ilo I can get one that is 10GB switch and 1 larger 1GB switch to support ilo+ some redundancy between two nics.

• 1x Broadcom BCM57412 Ethernet 10Gb 2-port SFP+ OCP3 Adapter for HPE
• 1x Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter for HPE
   

1 hour ago, leadeater said:

Any reason you are going dual socket here and/or Gen11, also Intel? I would actually recommend going with a single socket solution and DL320/DL325 because of that (if enough PCie slots, not checked). Also means you can go with single SAS card and network card (dual socket it's best to have a card per socket). Going single socket avoids NUMA constraints and any performance impacts from that, ESXi will manage that but it's actually better to have 1 CPU with 32 cores than 2 CPUs with 16.

I'm open to single socket solution as well. Were a HPE Intel shop and never really tried AMD so my build template given to me is intel.

When I go to check the DL320. I only see these options for SAS controller.  I dont see the option for the HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller

 

 

For the DL380 Gen 11. I see I have this option. I assume I would activate CPU1,CPU2 instead of OCP1,OCP2 if we end up doing dual socket. But I'm leaning towards your 1 socket idea as it might be cheaper. 

 

For DL320 Gen 11 >CPU I probably will go with 6530.

I also dont see a option to add any network cards for the DL320 under OCP or Ethernet PCIe very werid. I can go back and ask if its possible to add it to our build template.

But I assume I would have to drop the 1gb 4-port one and lose reduancy

• 1x Broadcom BCM57412 Ethernet 10Gb 2-port SFP+ OCP3 Adapter for HPE
• 1x Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter for HPE

 

We only have HPE Intel servers in our environment never really tried AMD I guess fear of something different not working properly? But I've been hearing that AMD has been dominating and I'm open to the idea of pushing this.

1 hour ago, leadeater said:

It probably won't matter much with that many SSDs but do be aware that RI SSDs don't have good write latency when pushed hard in writes so you might not get as good performance as you might expect. Mixed Use (MU) would be more recommended unless you know that your workload will be almost entirely reads.

Haha yea I only got the option in the template to pick read intensive drives. I assume i can go back and ask them to change to mixed use 

 

Thanks for your time in reading this long post and your help.

[leadeater]

4 hours ago, Bear_99 said:

3rd VM: Asked that Physical cores should be considered in the design. Requires 24cores 64gb of ram 1TB

Forgot to point this out in my first reply but since you have a requirements for a 24 vCPU VM then you'll ideally not want to go below 24 physical cores per CPU, you can but splitting a VM across sockets can give undesirable results. I'm pretty sure SQL does handle this quite well so long as you correctly configure the VM vCPUs per socket instead of the default setting of 1 core per virtual socket.

 

Just thought worth pointing out since the people requesting that VM are or have some requirements on the underlying physical CPUs so it would likely matter to them about this.

 

1 hour ago, Bear_99 said:

The template build I got for my server was these two SAS Controller. 
assume the Smart array one is the one I will be connecting to the MSA. Then the MR416ip is the storage controller if i had HDD in the server.

1x HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller

 

Correct. The MR416i is a Broadcom Tri-Mode RAID controller (that supports mixed HBA mode) and is used to connect the front drive bays. It'll be included in the build because you have a single 480GB SSD specified in your build configuration. May I ask why? You have the NS204i-u which is a dual M.2 controller card used for OS so I'm not sure what that additional 480GB SSD in the front bays is for?

 

The E208e-p is for connecting to the SAN. The reason why I mentioned typically having two of these is that PCIe cards have to connect to a physical CPU and if you have CPU cores on an opposing socket (dual socket) accessing storage LUNs connected to this PCIe card then that data communication has to go through the socket QPI connection which adds additional latency and if other things are using this could become bandwidth constrained i.e. NICs etc. 

 

The above is why single socket can be advantageous as you don't have to worry about any of those factors and you can half the number of physical PCIe/OCP cards if you attempt to optimize data paths in a dual socket configuration.

 

Also P.S. I'm pretty sure in iLO on Gen10 and Gen 11 Intel servers you can see the QPI bandwidth utilization, you could in older generation. It's a really nice feature.

 

1 hour ago, Bear_99 said:

I'm open to single socket solution as well. Were a HPE Intel shop and never really tried AMD so my build template given to me is intel.

When I go to check the DL320. I only see these options for SAS controller.  I dont see the option for the HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller

I'll just address basically everything below that you wrote here. That's what I was worried about for the DL320, being limited on options due to it being 1U and single socket. I'll have a better look now between what is possible with a DL320 Gen10-11, DL325 Gen10-11 and DL360/365 Gen10-11.

 

Currently we run 16 DL365 Gen10 with dual 7713 CPUs as our VM hosts, for us the 128 cores per server was a big advantage at the time since we got them few years ago now.

 

Anyway one option you have is to run a DL360/365 or DL380/385 with only one socket populated but consideration over that PCIe and OCP slots can be used with a single CPU would need to be checked.

 

1 hour ago, Bear_99 said:

We only have HPE Intel servers in our environment never really tried AMD I guess fear of something different not working properly? But I've been hearing that AMD has been dominating and I'm open to the idea of pushing this.

We were HPE exclusive then for 5 years migrated to Nutanix running on Supermicro, we have since moved back to HPE and NetApp that we were running prior to Nutanix only due to cost. Both solutions work well.

 

But yes basically everyone is or has been moving to AMD during hardware refreshes but that doesn't mean Intel is not still a good choice. Intel has improved their pricing so in the lower core counts both option are competitive, it just wasn't really possible for Intel to compete previously when the best they could do is 56 cores per server and AMD was offering 128 (it actually had to make sense for your environment to go with 128 per server though).

 

1 hour ago, Bear_99 said:

Haha yea I only got the option in the template to pick read intensive drives. I assume i can go back and ask them to change to mixed use 

They will be able to change it over to MU, just be aware it'll be a fair decent cost increase.

 

Anyway I'll have a look now if any of the 1U HPE options would actually be viable for you, you could get away with it if you did go iSCSI and put 25Gb NICs in the server and brought 25Gb switches but I just don't think overall that would actually be a good idea. Do have a think about it because it would reduce the number of cable connections and PCIe/OCP cards down and may also make 25Gb switching an option with the money saved on server hardware. It would be perfectly safe to share dual 25Gb connectivity with both general network and iSCSI storage network.

[leadeater]

5 hours ago, Bear_99 said:

I will be most likely buying new switches as well. I saw these two in my build template that was given to me. So I figured since i need ilo I can get one that is 10GB switch and 1 larger 1GB switch to support ilo+ some redundancy between two nics.

• 1x Broadcom BCM57412 Ethernet 10Gb 2-port SFP+ OCP3 Adapter for HPE
• 1x Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter for HPE

Something to be aware of when it comes to NIC teaming and multiple paths for general network connectivity is that end to end you can only utilize a single path, there are caveats and it depends to this however it is best to think of it this way. That means if you add 2x or 4x 1Gb connections in to the mix you must make sure in your vSwitch configuration that only the single 10/25 Gb connection is under active and the 1Gb connections under backup otherwise you'll end up fighting random performance problems when traffic flows across the 1Gb paths when you don't actually want it to.

 

Think of your 1Gb connections only as a worst case redundancy option to avoid outages but while they are actively being used due to 10/25Gb being down you will have degraded performance. The benefit here is mainly in being able to do maintenance and firmware/software upgrades of the switches without outages which is not insignificant benefit.

 

Overall just caution on how useful you think the 1Gb connections are going to be even if teamed, they will still realistically only offer 1Gb of actual usable bandwidth rather than multiple of the number connected.

 

Pre-comments/Recommendations

 

Have a think about future cluster options and how each choice might come in to play. This cluster could for example be used as a staging ground to replace your existing main cluster and that could effect whether or not you go with SAS or iSCSI/NFS as well as single socket or dual socket server platform. If it makes sense to use this cluster to replace the existing one and will get expanded out you could go with dual socket servers with a single CPU and then later add the second CPU and RAM etc. The SAS option without a SAS switch (yes these did exist but old hat, https://docs.broadcom.com/doc/12348581) will limit you to 4 hosts.

 

I don't know if expanding this cluster at all make sense for your environment but if it does that may better justify the expenditure of dual 25Gb switches to give a better long term foundation.

 

While you are looking at this for your current use case maybe it might be worth considering how this could factor in to a wider longer term picture. It's also good to draw block diagram on a whiteboard of each option to visualize it and whatever one looks like the most sense typically is the best solution, plus it's also easier to see the differences between each.

 

P.S. All part options below are valid possible options for each server and are listed in their spec sheets, caution is advised however in that I have not done as close checking on parts compatibility and the requirements on using them as I would if I were looking at purchasing myself. Check with HPE or your MSP that these selections are able to be valid configurations, I did not check closely for the riser requirements or cabling requirements etc so stuff like that. 

 

Below will be SAS connected options to your desired SAN

 

Configuration 1

 

HPE DL320 Gen11 8 SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004315enw.pdf?jumpid=in_pdp-psnow-qs

 

Benefits:

• Less rack/cabinet RU used
• Single socket with more cores best satisfies VM 3 requirements
• Not inter-socket data communication

Cons:

• No NIC or HBA redundancy (minor)

 

With this server configuration you would connect to the MSA SAN using 12Gb SAS dual path with 1 port of the HBA going to 1 port on each of the MSA controller cards.

 

Configuration 2

 

HPE DL325 Gen 11 8 SFF

• 1x EPYC 9354P (Select High Per Heatsink/Fan kit)
• 12x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe or MR416i-o OCP (different cable kit required for each option)
• 1x MCX631102AS OCP or BCM57414 OCP, or 2x MCX631102AS OCP or BCM57414
• 1x Potential Half Height, Half Length PCIe slot free (dependent on MR416i-o option being used)

https://www.hpe.com/psnow/doc/a50004297enw

 

Benefits:

• Less rack/cabinet RU used
• Single socket with more cores best satisfies VM 3 requirements
• Not inter-socket data communication
• Platform supports an additional OCP compared to Intel

Cons:

• No HBA redundancy (minor)
• Second PCIe slot when using NS204i-u restricted to Half Height

 

For the above you can go with two OCP network cards for card failure redundancy however to do that requires you use the PCI MR416i-p variant, unimportant other than configuration matrix requirements. Or you could choose one OCP NIC and the OCP MR416i-o and then choose two E208e-p SAS HBA cards for redundancy there, utilizing a single port per SAS card

 

I will not list DL360/365 Gen 11 options as there is no effective difference to the DL320/325 other than the DL360 supporting two OCP slots (2nd CPU required) and the same restrictions around those as the DL325 Gen11 above. The only benefit with either of these platforms is dual socket and going with two 16 core or 24 core CPUs, 24 core CPUs being more ideal for VM 3 but consuming more VMware licenses. For 24 core choices either Intel Xeon 6542Y or AMD EPYC 9254 (maybe 9274F but $$).

 

Configuration 3

 

HPE DL380 Gen 11 8SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004307enw.pdf?jumpid=in_pdp-psnow-qs

 

Benefits:

• Can be expanded to an additional second CPU with supporting riser and OCP slot expansions
• More PCIe slots and expansion capabilities, same as point 1 but more about total PCIe slots given 2U chassis
• Quieter operation, 2U

Cons:

• Requires more rack/cabinet RU space
• Can be more expensive platform being 2U and dual socket, HPE discount heavily dependent

 

 

Configuration 4

 

HPE DL385 Gen 11 8SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004300enw.pdf

 

Benefits:

• Can be expanded to an additional second CPU with supporting riser and OCP slot expansions
• More PCIe slots and expansion capabilities, same as point 1 but more about total PCIe slots given 2U chassis
• Quieter operation, 2U

Cons:

• Requires more rack/cabinet RU space
• Can be more expensive platform being 2U and dual socket, HPE discount heavily dependent

 

Configuration 5

 

HPE DL380 Gen 11 8SFF

• 2x Intel Xeon 6526Y or 6542Y(Select High Perf Heatsink/Fan Kit)
• 16x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004307enw.pdf?jumpid=in_pdp-psnow-qs

 

Benefits:

• Can be expanded to an additional second CPU with supporting riser and OCP slot expansions
• More PCIe slots and expansion capabilities, same as point 1 but more about total PCIe slots given 2U chassis
• Quieter operation, 2U

Cons:

• Requires more rack/cabinet RU space
• Can be more expensive platform being 2U and dual socket, HPE discount heavily dependent

 

 

Configuration 6

 

HPE DL385 Gen 11 8SFF

• 2x AMD EPYC 9124(9174F) or 9254(9274F) (Select High Perf Heatsink/Fan Kit)
• 24x 16GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004300enw.pdf

 

Benefits:

• Can be expanded to an additional second CPU with supporting riser and OCP slot expansions
• More PCIe slots and expansion capabilities, same as point 1 but more about total PCIe slots given 2U chassis
• Quieter operation, 2U

Cons:

• Requires more rack/cabinet RU space
• Can be more expensive platform being 2U and dual socket, HPE discount heavily dependent

 

Decided against showing you Gen 10 Plus v2 options since the CPUs themselves are past the 3 year mark and while totally fine may not be the best choice, performance wise they would be fine. Everything for Gen 10 v2 is basically identical to the above with the exception of the NS204 boot device being a PCIe card that consumes a slot. For CPU options of these if you want to consider them then Intel Xeon 6346 (16 cores) or 6342 )24 cores), AMD EPYC 7343/73F3 (16 cores) or 7443/74F3 (24 cores) and all comments about 24 cores and VM 3 configuration requirement applicable.

 

 

Below will be iSCSI or NFS connected options to your desired SAN (I don't think the MSA supports NFS but this is an option for a different storage solution, we use NFS with NetApp)

 

Benefits/Cons for each will be isolated to iSCSI/NFS vs SAS, see above for more general platform comments. Overall going with iSCSI/NFS may allow you to have more cluster expandability and flexibility and the reduction in PCIe hardware for some option may make 25Gb switch within budget. They difference here is the possibility to grow past 4 servers more easily and to be able to justify dual 25Gb switch and potentially share general network and storage network across the same physical connections, or can be separated out. Offering it as an alternative however not necessarily any better or worse than going SAS.

 

If you stick will 10Gb switches you could still safely achieve that with 4 connections per server, 2 for general network and 2 for storage.

 

Configuration 1

 

HPE DL320 Gen11 8 SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004315enw.pdf?jumpid=in_pdp-psnow-qs

 

Benefits:

• Minor cost reduction removing SAS card
• Reduced cabling and potentially switching requirements

Cons:

• No NIC redundancy (minor)
• Sharing storage and general network bandwidth across same physical connections

 

Configuration 2

 

HPE DL320 Gen11 8 SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x MCX631432AS or BCM57414 PCIe (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004315enw.pdf?jumpid=in_pdp-psnow-qs

 

Benefits:

• NIC redundancy
• 10Gb networking more viable with 4 connections active per server

Cons:

• Slight cost increase over configuration 1, extra NIC

 

Configuration 3

 

HPE DL325 Gen 11 8 SFF

• 1x EPYC 9354P (Select High Per Heatsink/Fan kit)
• 12x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe or MR416i-o OCP (different cable kit required for each option)
• 1x MCX631102AS OCP or BCM57414 OCP, or 2x MCX631102AS OCP or BCM57414

https://www.hpe.com/psnow/doc/a50004297enw

 

Benefits:

• Minor cost reduction removing SAS card
• Reduced cabling and potentially switching requirements

Cons:

• No NIC redundancy (minor)
• Sharing storage and general network bandwidth across same physical connections

 

Configuration 4

 

HPE DL325 Gen 11 8 SFF

• 1x EPYC 9354P (Select High Per Heatsink/Fan kit)
• 12x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe or MR416i-o OCP (different cable kit required for each option)
• 2x MCX631102AS OCP or BCM57414 OCP

https://www.hpe.com/psnow/doc/a50004297enw

 

Benefits:

• NIC redundancy
• 10Gb networking more viable with 4 connections active per server

Cons:

• Slight cost increase over configuration 4, extra NIC

 

Configuration 5

 

HPE DL380 Gen 11 8SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004307enw.pdf?jumpid=in_pdp-psnow-qs

 

Going to save repeating myself here Benefits/Cons wise, I'm sure you can assess that

 

Configuration 6

 

HPE DL380 Gen 11 8SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x MCX631432AS or BCM57414 PCIe (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004307enw.pdf?jumpid=in_pdp-psnow-qs

 

Going to save repeating myself here Benefits/Cons wise, I'm sure you can assess that

 

Configuration 7

 

HPE DL385 Gen 11 8SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x MCX631432AS or BCM57414 PCIe (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004300enw.pdf

 

Going to save repeating myself here Benefits/Cons wise, I'm sure you can asses that

 

Configuration 8

 

HPE DL380 Gen 11 8SFF

• 2x Intel Xeon 6526Y or 6542Y(Select High Perf Heatsink/Fan Kit)
• 16x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 2x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
• 1x PCIe slot usable and free

https://www.hpe.com/psnow/doc/a50004307enw.pdf?jumpid=in_pdp-psnow-qs

 

Going to save repeating myself here Benefits/Cons wise, I'm sure you can asses that

 

Configuration 9

 

HPE DL385 Gen 11 8SFF

• 2x AMD EPYC 9124(9174F) or 9254(9274F) (Select High Perf Heatsink/Fan Kit)
• 24x 16GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x MR416i-p PCIe
• 2x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)

https://www.hpe.com/psnow/doc/a50004300enw.pdf

 

Going to save repeating myself here Benefits/Cons wise, I'm sure you can asses that

[Bear_99]

Wow thanks! I really appreciate the effort and time you put into this. Is there a way for me to buy u coffee or lunch haha.

 

Currently our goal is to get things up and running as simple and fast since we have the vmware licenses. Then 3-5yrs from now we're looking to do a complete refresh of our entire VMStack. So the syseng recommends me going SAS to reduce cost before a major refresh to get things up.

 

8 hours ago, leadeater said:

May I ask why? You have the NS204i-u which is a dual M.2 controller card used for OS so I'm not sure what that additional 480GB SSD in the front bays is for?

Oh I had the extra 480gb drive for incase the NS204i-u fails and for incase they want to expand out these servers for the refresh.

 

I'm going to get pricing for these two configurations and hope it fits within the budget as this definitely went up haha have to buy 3x hosts+msa. I may have to reduce drives in the MSA. 

I will speak to our networking team if were going with 10/25gb cards SFP28 instead of 10gb SFP+. I would have to ask for another $10k to increase it to this model for the cisco switch C9500-24Y4C

 

Regarding these two configurations. I have a quick question.

I know you spoke of using the 4x1gb for redundancy Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter. I see it was left out and curious if I should add that back in? but not the OCP version but the PCIe one.

 

The vendor for the 24core one said recommend is 24 but min 16. Alot to think about once i get the cost back and its over my budget. But this is great.

 

Configuration 3

 

HPE DL380 Gen 11 8SFF

1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
8x 32GB RAM
1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
1x MR416i-p PCIe
1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)
1x PCIe slot usable and free

 

Configuration 1

 

HPE DL320 Gen11 8 SFF

• 1x Intel Xeon 6530 (Select High Perf Heatsink/Fan Kit)
• 8x 32GB RAM
• 1x NS204i-u (has it's down dedicated special slot in this server, does not consume a PCIe slot)
• 1x Smart Array E208e-p SR Gen10 Controller (supported, see link below)
• 1x MR416i-p PCIe
• 1x MCX631432AS or BCM57414 OCP (these are 10/25 Gb and I recommend that over only 10Gb)

[leadeater]

10 hours ago, Bear_99 said:

Currently our goal is to get things up and running as simple and fast since we have the vmware licenses. Then 3-5yrs from now we're looking to do a complete refresh of our entire VMStack. So the syseng recommends me going SAS to reduce cost before a major refresh to get things up.

That makes a lot of sense, yea if the timing isn't right and a larger project is on the cards later then yea lowest possible TCO is best in this situation.

 

10 hours ago, Bear_99 said:

Oh I had the extra 480gb drive for incase the NS204i-u fails and for incase they want to expand out these servers for the refresh.

Personally I would just forgo that and only get it when/if you actually need it. The NS204-u is individually hot swap for each of the 2 M.2 devices so you can do maintenance on them without any outage.

 

 

The only risk is the RAID SoC failing but you have redundancy in the ESXi hosts anyway and ESXi run in memory after boot so the boot device failing is non disruptive in the immediate sense.

 

Also long as you buy the MR416i RAID card you can add storage later that is compatible with ESXi, Intel VROC directly off the CPU from my understanding is not ESXi compatible.

 

10 hours ago, Bear_99 said:

I may have to reduce drives in the MSA. 

22x 1.92TB did seem like a lot for your stated storage requirements on each VM. One thing to note which might be important for you is that ESXi and individually VMs have a limited capability to push I/O through to the underlying storage which is a mistake I made a few years ago that I can only fix next hardware replacement. We have a dedicated ESXi host with direct attached SSDs ~20x internal and another 24x in an DAS shelf each being their own RAID array and ESXi datastore. However this host runs Fortinet FortiAnalyzer and I'm getting high datastore latency alerts even though all SSD etc etc and its simply due to not being able to extract all the performance out of all 24 SSDs in a single array through a single ESXi datastore.

 

Consider splitting your arrays in to groups of 12 (2x 12 for 24) or minimum 6, then creating an ESXi datastore for each. Then distribute VMs across them or for very large VMs like my FortiAnalyzer that uses multiple VMDKs distribute them across datastores which is what I will be doing once the hardware is replaced (also put each virtual disk on is own VM virtual SCSI controller).

 

There's actually quite a bit to optimizing ESXi storage throughput and latency. It is recommended for SQL VMs to do the above, SCSI controller for OS disk, SCSI controller for data disks, SCSI controller for log disks and SCSI controller for TempDB disks aka in the VM config properties 4 SCSI controllers all Paravirtual type.

 

10 hours ago, Bear_99 said:

I know you spoke of using the 4x1gb for redundancy Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T OCP3 Adapter. I see it was left out and curious if I should add that back in? but not the OCP version but the PCIe one.

I think some of the servers come with it regardless so I just didn't mention it. Basically up to you on what you want to do there with the info I gave, it's not wrong to use or not use it, simply a cost equation on going dual 10Gb or 25Gb switching which sounds like it's not really a go here so at least a team of 2x 1Gb as a backup is going to be required. I can't ever support having only a single network connection, something is better than nothing.

[leadeater]

9 hours ago, Bear_99 said:

I will speak to our networking team if were going with 10/25gb cards SFP28 instead of 10gb SFP+. I would have to ask for another $10k to increase it to this model for the cisco switch C9500-24Y4C

FYI those NICs support both SFP+ and SFP28 so that is why I recommend them, you are free to use 10Gb switching and SFP+ transceivers with them and if in the future the opportunity to connect them at SFP28 25Gb comes up it's just a transceiver/DAC swap.

[Bear_99]

57 minutes ago, leadeater said:

Personally I would just forgo that and only get it when/if you actually need it. The NS204-u is individually hot swap for each of the 2 M.2 devices so you can do maintenance on them without any outage.

oh wow then yea I will drop the 480gb ssd to save cost. but keep the MR416i RAID for later in case.

58 minutes ago, leadeater said:

think some of the servers come with it regardless so I just didn't mention it. Basically up to you on what you want to do there with the info I gave, it's not wrong to use or not use it, simply a cost equation on going dual 10Gb or 25Gb switching which sounds like it's not really a go here so at least a team of 2x 1Gb as a backup is going to be required. I can't ever support having only a single network connection, something is better than nothing.

Thanks! I will check with our MSP regarding if the servers will come with default 4x 1gb broadcom card. I was just worried because it was mentioned with single socket best to have 1 nic card + 1 sas. For this option I will get the dual 10/25gb cards which one is better MCX631432AS or BCM57414 OCP?

 

If the MSP does come back and say I need to add the 1gb myself. I assume I would select Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T as an PCIe card vs OCP.

1 hour ago, leadeater said:

There's actually quite a bit to optimizing ESXi storage throughput and latency. It is recommended for SQL VMs to do the above, SCSI controller for OS disk, SCSI controller for data disks, SCSI controller for log disks and SCSI controller for TempDB disks aka in the VM config properties 4 SCSI controllers all Paravirtual type.

ah wow.  there's definitely so much to think about lol. not sure how you learn all of this.

 

1 hour ago, leadeater said:

22x 1.92TB did seem like a lot for your stated storage requirements on each VM. One thing to note which might be important for you is that ESXi and individually VMs have a limited capability to push I/O through to the underlying storage which is a mistake I made a few years ago that I can only fix next hardware replacement.

hmm seems like you

 

1 hour ago, leadeater said:

Consider splitting your arrays in to groups of 12 (2x 12 for 24) or minimum 6, then creating an ESXi datastore for each.

based on my MSA SAS seems like can only support 22 drives. 
22x HPE MSA 1.92TB SAS 12G Read Intensive SFF (2.5in) M2 3yr Wty SSD = ~42tb.

 

I only really need 6-10TB. What drives would you recommend for me to get and how many. I plan to switch to MU.

 

Once again I really appreciate your time and knowledge as I'm still trying understand/learn warp my head around this.

[leadeater]

39 minutes ago, Bear_99 said:

Thanks! I will check with our MSP regarding if the servers will come with default 4x 1gb broadcom card. I was just worried because it was mentioned with single socket best to have 1 nic card + 1 sas.

Multiple NICs connected to a CPU is fine, is just least optimal to have no NICs connected to a CPU. However to actually make use of the NIC per CPU you have to make sure the OS and applications use the correct NIC for the CPU cores/physical CPU they are running on. This is not always true, I'm just OCD and like to stick to this guideline so it's at all possible and also so if a card fails the host isn't offline.

 

41 minutes ago, Bear_99 said:

For this option I will get the dual 10/25gb cards which one is better MCX631432AS or BCM57414 OCP?

Both are good, we use BCM57414 however we did find out that Citrix doesn't official support that NIC for their Netscaler VM product however they do for Mellanox (MCX).

 

43 minutes ago, Bear_99 said:

based on my MSA SAS seems like can only support 22 drives. 
22x HPE MSA 1.92TB SAS 12G Read Intensive SFF (2.5in) M2 3yr Wty SSD = ~42tb.

 

I only really need 6-10TB. What drives would you recommend for me to get and how many. I plan to switch to MU.

 

Once again I really appreciate your time and knowledge as I'm still trying understand/learn warp my head around this.

I would imagine that is just a pre-configured bundle HPE offers, the controller chassis and expansion chasses all support 24x SFF bays in that SFF formfactor (LFF option also exist but you don't want that).

 

Also having a look around at the tech specs for the MSA I noticed only RI SSDs were listed so I did some looking and found this:

Quote

MSA started using SSDs with the MSA Gen4 products and were first introduced with what would now be WRITE INTENSIVE drives.  Gen5 introduced the Mix-Use capacities and RI.  MSA Gen6 only uses RI.  The cost of RI are as expected much less than MU or WI.  The experiences we have with the previous generations have led to the endurance of RI only available.  There is a good explanation of why in the MSA Virtual Storage Technical Guide: https://www.hpe.com/psnow/doc/a00103247enw
See the SSD Endurance section and lifetime of the SSDs that can be achieved.

https://community.hpe.com/t5/msa-storage/msa-2062-write-intensive-ssd-s/td-p/7180970

 

I wasn't aware of that change, interesting so my recommendation of using MU SSDs is moot since it's not "supported".

 

If you want maximum performance then I would go with 16x 960GB SSDs in two RAID5 of 7 SSDs with that last 2 SSDs are global hot-spare. Since the SSD failure rate is so low RAID 5 is quite safe compared to HDDs. You other option is looking at MSA-DP+. Have read about it here, also performance table of different RAID options at the bottom of the PDF: https://www.hpe.com/psnow/doc/a00094630enw

 

Otherwise if you just want the capacity at lowest cost then 8x 1.92TB RAID 5 of 7 SSDs with 1 hot-spare or MSA-DP+

 

I have never used MSA-DP+ btw, ask your MSP/HPE about it.

[leadeater]

1 hour ago, Bear_99 said:

there's definitely so much to think about lol. not sure how you learn all of this.

I have been around the block a few times 

 

Mainly I'm actually interested in this crap along with having to look at performance issues of things like SQL VMs and digging in to the why.

[Bear_99]

7 hours ago, leadeater said:

I wasn't aware of that change, interesting so my recommendation of using MU SSDs is moot since it's not "supported".

 

If you want maximum performance then I would go with 16x 960GB SSDs in two RAID5 of 7 SSDs with that last 2 SSDs are global hot-spare. Since the SSD failure rate is so low RAID 5 is quite safe compared to HDDs. You other option is looking at MSA-DP+. Have read about it here, also performance table of different RAID options at the bottom of the PDF: https://www.hpe.com/psnow/doc/a00094630enw

 

Otherwise if you just want the capacity at lowest cost then 8x 1.92TB RAID 5 of 7 SSDs with 1 hot-spare or MSA-DP+

 

I have never used MSA-DP+ btw, ask your MSP/HPE about it.

 

I also see that for SSD's HPE recommend chart does not show Raid 6 thats very interesting lol. 

https://www.hpe.com/psnow/doc/a00105260enw

 

I assume going 16x960 with two Raid5 + 2 ssds as hot spares is the way to go. if one drive fails we can activate the hot spares.

 

I think the MSA also comes with 2x 1.92TB by default that's why I could only max at 22 drives. I assume I can have these as hotspares or it should be used for something else like SSD ReadCache?

 

Since this offers two Raid 5's one disk group for controller a and one for controller b. I believe this will increase performance to 100%? 

 

Ah Actually was reading this maybe my idea to use both controllers is a bad idea. Since one of the controllers can take over incase for updates etc.

"One pool with 100% headroom— Provisioning a single pool and placing all workloads in it reduces the impact to performance when a controller becomes unavailable and greatly simplifies configuration management. Because typical requirements for small and mediumsized businesses are far below the capacity and performance limits of a single pool, Hewlett Packard Enterprise recommends evaluating a single-pool strategy before considering dual-pools. System configuration limits are documented in the HPE MSA1060/2060/2062 Storage Management Guide. "

 

7 hours ago, leadeater said:

Otherwise if you just want the capacity at lowest cost then 8x 1.92TB RAID 5 of 7 SSDs with 1 hot-spare or MSA-DP+

 

I guess i will go for this if budget is a concern. and will lose out controller b i assume. 

 

or.. the cheapest is 10k drives haha.

 

Thanks!

 

[leadeater]

49 minutes ago, Bear_99 said:

I think the MSA also comes with 2x 1.92TB by default that's why I could only max at 22 drives. I assume I can have these as hotspares or it should be used for something else like SSD ReadCache?

 

That should just be more a quirk of the option(s) being presented to you, you won't be forced in to buying 2x 1.92TB SSDs you don't need if you are buying 960GB SSDs. HPE will pretty much sell you anything you want within reason and support matrix.

 

49 minutes ago, Bear_99 said:

Since this offers two Raid 5's one disk group for controller a and one for controller b. I believe this will increase performance to 100%?

Sort of not really since the controller cards can handle a lot so that few SSDs should max out it's capabilities too easily. That said you can make Controller A primary owners of Array A and Controller B primary owner of Array B and then in your ESXi/HBA pathing make the primary datastore paths match that so you can benefit from the cache in each controller and slightly more useable bandwidth. It's just not going to be double the performance, just technically "better".

 

49 minutes ago, Bear_99 said:

I guess i will go for this if budget is a concern. and will lose out controller b i assume. 

 

or.. the cheapest is 10k drives haha.

So actually this generation of MSA supports data tiering, it's automatic. So if you know how much hot data percentage or TB amount you actually need you can buy as an example 5x 960GB SSDs (4x RAID 5 + 1 HS) and then attach on a 12 LFF shelf and put in larger Near/Midline-SAS HDDs for a 30/70 ratio of fast tier to slow tier.

 

Quote

Performance Tiering and Archive Tiering
Disk tiers are comprised of aggregating 1 or more Disk Groups of similar physical disks. The MSA 2062 supports 3 distinct tiers:
• A Performance tier with SSDs
• A Standard SAS tier with Enterprise SAS HDDs
• An Archive tier utilizing Midline SAS HDDs.
The MSA 2062 supports sub-LUN tiering and automated data movement between tiers. The MSA 2062 automated tiering
engine moves data between available tiers based on the access characteristics of that data. Frequently accessed “pages” will
migrate to the highest available tier delivering maximum I/O´s to the application. 

 

Not saying this is the right way to go but if you must cost down your solution this is a method of doing that and if the performance is not good enough you just add more SSDs. Tiering works very well until it doesn't, then it works exceptionally badly and that is 100% down to knowing your I/O workload and hot data capacity requirements. If in doubt don't do it.

 

49 minutes ago, Bear_99 said:

I guess i will go for this if budget is a concern. and will lose out controller b i assume. 

It's always worth having the second controller to protect from controller failure. If your one and only fails then entire cluster is down, it's a pretty significant single point of failure.

[Bear_99]

16 minutes ago, leadeater said:

ort of not really since the controller cards can handle a lot so that few SSDs should max out it's capabilities too easily. That said you can make Controller A primary owners of Array A and Controller B primary owner of Array B and then in your ESXi/HBA pathing make the primary datastore paths match that so you can benefit from the cache in each controller and slightly more useable bandwidth. It's just not going to be double the performance, just technically "better".

Thanks yea according to our MSP person they said the 2x1.92tb comes with every MSA 2062 and its something we remove after very werid. I will try to ask to confirm again.

 

I will go with this setup. Would i set Pool 1 7x960gb to controller a and Pool 2 7x960gb to controller b then 2 hotspares.

 

or do I set all the pools to Controller A? This way controller b can take over A during firmware upgrades or outages.

[leadeater]

7 minutes ago, Bear_99 said:

I will go with this setup. Would i set Pool 1 7x960gb to controller a and Pool 2 7x960gb to controller b then 2 hotspares.

Yes

 

7 minutes ago, Bear_99 said:

or do I set all the pools to Controller A? This way controller b can take over A during firmware upgrades or outages.

The owner is just the "Primary", you can talk to any array through any controller it's just called indirect access or many other names for it. The only down side is the non-primary controller has to talk to the the primary controller over the controller backplane interconnect, basically very minor latency difference between direct and indirect access.

 

There is no downside in regards to maintenance, dual controller means no loss in access during a failure or controller firmware upgrade.

 

P.S. If you are forced to have those two 1.92GB SSDs then those are your two hot spares

[Bear_99]

1 hour ago, leadeater said:

The owner is just the "Primary", you can talk to any array through any controller it's just called indirect access or many other names for it. The only down side is the non-primary controller has to talk to the the primary controller over the controller backplane interconnect, basically very minor latency difference between direct and indirect access.

 

There is no downside in regards to maintenance, dual controller means no loss in access during a failure or controller firmware upgrade.

Ah nice! 

 

I thikn im going to go with MCX631432AS vs Broadcom since its slightly more expensive and seem more reliable? 

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=54081&vcl=true

 

my final build am i over my PCIe slot limit lol

Servers:

3x DL380 GEN 11

• 8 x 32GB Ram = 256GB
• 2 x 1000w power supply
• 1x HPE NS204i-u Gen11 NVMe Hot Plug Boot Optimized Storage Device
• 1x HPE Smart Array E208e-p SR Gen10 (8 External Lanes/No Cache) 12G SAS PCIe Plug-in Controller
• 1x MR416i-p PCIe
• 1x MCX631432AS 10/25Gb 2-port SFP28 OCP
• 1x Broadcom BCM5719 Ethernet 1Gb 4-port BASE-T PCIE

1x MSA 2062 SAS

• 16x 960gb drives.
• 6x SAS connectors to the server. (1 for controller a 1 for controller b. to each server?)

[leadeater]

34 minutes ago, Bear_99 said:

my final build am i over my PCIe slot limit lol

If you are going with the DL380 Gen 11 then you aren't actually over the PCIe amount usable for a single CPU. That server in that configuration supports 3 single slot PCIe cards and your configuration is only using 3.

 

 

Also looking at the spec sheet again it looks like it is possible to use OCP slot 2 with 1 CPU with this part/add-on

 

OCP 3.0 Enablement HPE ProLiant DL360 Gen11 CPU1 to OCP2 x8 Enablement Kit P51911-B21

 

So both your 10/25Gb NIC option and 4 port 1Gb option could be OCP, just get that doubled checked.

[Bear_99]

1 hour ago, leadeater said:

Also looking at the spec sheet again it looks like it is possible to use OCP slot 2 with 1 CPU with this part/add-on

 

OCP 3.0 Enablement HPE ProLiant DL360 Gen11 CPU1 to OCP2 x8 Enablement Kit P51911-B21

 

So both your 10/25Gb NIC option and 4 port 1Gb option could be OCP, just get that doubled checked.

Thanks! will be asking this later today. 

Raid 5
7x960gb
7x960gb

2spares

question if i was running MSA-DP+ my amount of disks would have to change right? I believe minimum is 12 drives.

 

MSA-DP+

12x960GB

4 Spares

[leadeater]

@Bear_99 Yea the MSA 2062 is actually just a preconfigured bundle

 

 

It's actually just an MSA 2060 so you could get that quoted with SSD and the Advanced Data Services license if you need that.

[Bear_99]

20 hours ago, leadeater said:

@Bear_99 Yea the MSA 2062 is actually just a preconfigured bundle

 

 

It's actually just an MSA 2060 so you could get that quoted with SSD and the Advanced Data Services license if you need that.

The MSP got back to me and recommended that I go MSA-DP+ 

So I assume I would have to up my drives a bit and take a performance hit.

I think Raid 5 7x960gb with two controllers i get ~ 10,000mb/s

 

With MSA-DP+ (min 12 drives) 

16x960GB in Controller A, Pool A 

I get a performance hit and goes down to 6,429 MB/s because I'm only using 1 controller.

I will get 2 for Global Hot spares.

 

Wait I assume the 2x1.92tb drives can be used as hotspares since it a power of 2?

 

Whats your thoughts?

 

[leadeater]

16 minutes ago, Bear_99 said:

Whats your thoughts?

Seems good, I'd still ask what the price difference would be to get the second controller.

 

17 minutes ago, Bear_99 said:

Wait I assume the 2x1.92tb drives can be used as hotspares since it a power of 2?

They can yes, in general your hot spares just need to be the same size or large than the failed storage device. MSA-DP+ works a little differently but this general rule still applies.

[Bear_99]

7 minutes ago, leadeater said:

Seems good, I'd still ask what the price difference would be to get the second controller.

 

I think since we have a deal with HPE to get the 2062 version this is what it comes with
MSA 2062

(2) AC power supplies

(1) black 1.5m C13/C14 PDU style jumper cord each

(2) MSA 2060 Storage Controllers

(2) 1.92TB Read Intensive SSDs

(1) Advanced Data Services LTU

 

I assume I will have to take the performance hit of the MSA-DP+ 

16x960GB in Controller A. While Controller B would just be there for when shit goes wrong haha.

 

or what I can do is load up 22 drives + the 2 in there = Max 24. I will have both controllers going with two pools haha but no Global hot spares. but can survive 2+2 drives failing + my controller b wont just be sitting around