HPE DL380p Gen8 Build

[Windows7ge]

Today I'm embarking on a build with the first of hopefully future servers I would like to play with starting with the HPE DL380p Gen8 an early 2010's system built around the LGA2011-0 server platform featuring the C602 chipset.

 

 

Getting a quick look around the system starting at the front left to right we have

• VGA
• a slim optical drive
• Eight 2.5" drive bays. Unfortunately this server did not come with any usable sleds.
• LED indicator panel for debugging
• USB
• System information tab
• Power button
• USB
• Front UID which pairs with the rear UID

 

 

Right behind that we have a wall of six hot swappable fans.

 

 

These things are no slouches. Each fan can put out up to 39.6W for a total of 237.6 watts of just fans.

 

 

Unreal. Moving farther back gives us a view of the CPU/RAM arrangement.

 

 

Two sockets, 12 DIMM slots per socket, 3 DIMMs per channel. The plastic guard acts as a air flow guide to direct the air more accurately to where it needs to go.

 

In the back we have two three slot PCI_e risers. One per socket.

 

 

We'll take a closer look at the slotted cards on the right in a little while.

 

In the very rear we can see all of our I/O

 

 

• A small allen key for system servicing
• Quad Gigabit NIC
• Serial port
• Our iLO Management Port. This will become very important later
• VGA
• Quad USB
• Rear UID which pairs with the front UID
• Two mini-redundant 750W PSU's. I will be looking at those closer in a bit.

Right now the CPU's have arrived so I can start by installing those but we are waiting on the RAM so I think I'd rather do the two at once. The mechanisms designed which hold everything together are a marvel in themselves so I can't wait to dig into this farther.

[Windows7ge]

The memory has arrived.

 

 

8x8GB(64GB) 2Rx4 Micron RDIMM ECC 1600MHz.

CPU's of choice, Intel Xeon E5-2690v1's.

 

I really like HP's design for CPU and memory installation. First you start by removing the plastic air baffle by squeezing the blue spring loaded clips:

 

 

Exposing the CPU socket is as easy as lifting the blue handle retaining the CPU heatsink. The heatsink itself uses a ZIF design so it just lifts off of it's alignment pegs.

 

 

Then like any other computer that uses the LGA design install the CPU:

 

 

Apply your TIM:

 

 

Rest the heatsink on-top aligning it with the alignment pins making sure it's in the right orientation:

 

 

Lower the retention arm and push down on the blue handle:

 

 

Congratulations you just installed a CPU in a HPE DL380p G8

For the second socket it's the exact same process so I'm not going to go over it.

 

As far as memory placement goes they make it pretty hard to screw up. To make the memory a little easier to access you can remove the fan wall as one assembly using it's blue handle.

 

 

With no memory installed it's actually marked on the air flow baffle which slots to populate first. It tells you what the channels are and everything.

 

 

So if we rest it on top we can see exactly where each module should go.

 

 

As it just so happens HPE idiot proofed the process and actually color coded the first slot of each channel so it's a no-brainer. With eight sticks are we have to do is populate every beige slot.

 

 

Next up I want to look over some of the tech this has in the rear before I turn it on. I also need to install something special so we can test boot.

[Windows7ge]

So, moving past the CPU sockets and memory I think I've identified what some of the controllers are on the motherboard.

 

One is just outright labeled though so that one's a no-brainer.

 

 

It looks to me like the C602 chipset isn't in charge of much. There's an unpopulated SFF-8087 connector next to it. There's what looks to be USB control behind the left riser card for a SD card but not a whole heck of a lot else.

 

The iLO itself and what I believe is a single RAM chip so probably 512MB or 256MB of RAM allocated to that.

 

Then unfortunately it looks like the P420i is actually integrated into the motherboard itself. I was hoping to see it in a slot or socket. Based on the servers documentation the P420i comes with every model of this server but with varying amounts of RAM cache which would be the card slotted to the right so it itself is up-gradable.

 

 

The highest capacity cache available is 2GB and that not what we have here. This is 2nd best which is 1GB. The wires run off to what looks to be a pair of capacitors with act as a battery to preserve the cached data in the event of power failure. The prevents data corruption on the drive array.

 

Taking a closer look at one of the PSUs.

 

 

750W. Appears to claim 94% efficiency but I wouldn't doubt if that's only when it's running on 240V under a 50% or higher load.

 

The Quad NIC in the rear is what the x8 expansion card is.

 

 

Besides the slot this has proprietary form factor written all over it. Based on the documentation you can have either Quad Gigabit or a 10Gig NIC installed here. Possibly fiberoptic too.

 

Now I don't have any sleds for this server and I don't want to test the server running over USB. Instead I'm going to boot off the network using iPXE which is what this NIC will be for.

 

 

The Mellanox ConnectX-3 CX311A.

 

Cool to see that the riser just uses two standard x16 slots.

 

 

I assume one is bifurcated to x8x8 to give us three usable slots.

 

Installing the riser is pretty easy starting with twisting the little blue tabs.

 

 

Like it says on the riser. Push down, twist, fold the tab over, done.

 

From here we're ready to shut the lid, plug everything in the rear, and start testing. See if the server even POSTs and looking into software like the iLO itself which I may get into later today.

[Windows7ge]

We have POST!

 

 

Both CPU's and all RAM detected.

 

 

Now unfortunately the previous owners of this server did me sloppy and didn't reset the iLO. It had a Static IP well in the public range outside the A/B/C/D classes. I have no idea what they were doing so I had to [F8] the thing, figure out what is apparently iLO 4 and setup DHCP.

 

 

I also had to reset the User as the existing user and administrator accounts were messed up. Absolute PITA.

 

Finally connecting to the iLO. Very fancy login page.

 

 

After logging in.

 

 

Now the iLO is far far too complex for me to go over here but for the interest of curiosity lets look at our CPU's and Memory:

 

 

 

Everything reporting exactly how it should. I'm happy. Might use the iLO to do some temperature testing later. Who knows but the level of system control it gives you is amazing. For example Remote Console which is how I'm getting all of these screen shots.

 

Continuing with testing if she'll boot let's get started with FlexBoot.

 

 

The whole process of setting up a server that other servers can boot off of is a process well outside the scope of this build log. If you want more information on that checkout the tutorial I wrote on the process here. It is a VERY involved process. Above iPXE (or FlexBoot) just got an IP from DHCP, discovered the TFTP server, and downloaded the updated version of iPXE which has a custom embedded script.

 

From here the updated version of iPXE loads the system.

 

 

And runs the custom embedded script:

 

 

In this process the script:

1. Queried DHCP
2. Set the iSCSI Initiator
3. Requested to boot from said iSCSI Initiator

From here GRUB takes over which was also custom modified to allow booting as a SAN device.

This then loads Linux which was also custom modified to allow booting as a SAN device.

 

But after it's all said and done. We're in!

 

 

Booting locally is overrated anyways.

 

I feel it's only appropriate to end the build with a traditional neofetch.

 

 

Memory looks a little wonky...not sure what's going on there. Should be 65536MiB but I guess something is reserving it. No matter.

 

I wish I could have made this longer but part of corporate level rack mount servers is you're not building it from scratch. Hopefully we will be able to do this again with both Dell's and IBM's close equivalents those being the Dell PowerEdge R720 and IBM X3650 M4 as I'd like to play with those as well. I'll look forward to it and hope to see you all there! 

 

[Anthony_95]

35 minutes ago, Windows7ge said:

hope to see you all there!

For sure. I love this kind of builds