I've Come Feeling Defeated (Supermicro server build)

[AnAngryMob69]

I'm definitely struggling with this seemingly eternal server project.

 

I started with a Super Micro SuperChassis 933T-R760B with dual AMD Opteron 280s. This was an issue for me because running FreeNAS through a hardware RAID is definitely nothing anyone should ever need to try. EVER.

 

• I soon bought a used X7DWE motherboard because of its compatibility with the CSE-933(T?) chassis in both size and power connector requirements. (This is why the X8 and newer boards were out immediately)
  http://www.supermicro.com/products/motherboard/Xeon1333/5400/X7DWE.cfm
• This was then paired with a set of X5472 processors (obviously I overlooked the maximum TDP of the motherboard at 100W) in order to max out the capability of the FSB and maximum processor speed.
  http://ark.intel.com/products/34447/Intel-Xeon-Processor-X5472-12M-Cache-3_00-GHz-1600-MHz-FSB
• Adding a set of 4x8 GB (32GB Total) ECC sticks of DRAM proved to be no serious issue
• Finding and actually waiting for a LGA771 compatible passive heat sink proved to be a battle of its own.
• Cables were a pain. I had to manually find the generic equivalent of a 20-pin front panel connector because the original was too short for the new MoBo. There is still one connector that I left disconnected because the new MoBo doesn't have an equivalent connector. Super Micro calls it the JAR connector: Power supply alarm reset header. I also have nothing plugged into the J18: System Management Bus header on the X7.

 

After attempting to boot this severely overpowered system, I was met with the lovely temperature LED on the front panel of the chassis. I soon realized my error in recognizing the TDP threshold and the (Very) low TCase set by Intel, and ordered a more reasonable 80W TDP pair of E5462s. This is where I need help...

 

After installing the same LGA 771 passive coolers to the (theoretically) less heated processors, I was met with the same abysmal temperature warning light on the chassis. No boot, nothing but fans.

 

If any of you kind souls would be willing to help, I would be forever in your "virtual debt equivalent."

 

 

Sincerely,

 

About to give up on this heavy piece of $#*^

X7-DWE Manual.pdf

SC933.pdf

[Electronics Wizardy]

You want a heater, you bought a heater. Also you will be soon paying about about 33 dollars a month in power, depends on where you are.

 

 

Id highly suggest you dump that server, and get something like this http://www.ebay.com/itm/Dell-PowerEdge-FS12-TY-C2100-8-Core-2-13GHz-L5630-16GB-12x-HDD-TRAYS-/381744926594?hash=item58e1c54f82:g:ankAAOSw5cNYZCO7

 

Half the power, more drives, faster cpus(you can put 6 cores in it and 288gb of ram), and it just works out of the box.

[AnAngryMob69]

Just now, Electronics Wizardy said:

You want a heater, you bought a heater. Also you will be soon paying about about 33 dollars a month in power, depends on where you are.

 

 

Id highly suggest you dump that server, and get something like this http://www.ebay.com/itm/Dell-PowerEdge-FS12-TY-C2100-8-Core-2-13GHz-L5630-16GB-12x-HDD-TRAYS-/381744926594?hash=item58e1c54f82:g:ankAAOSw5cNYZCO7

 

Half the power, more drives, faster cpus(you can put 6 cores in it and 288gb of ram), and it just works out of the box.

Thank you for the suggestion, But then what to do with the parts that I already have.... I doubt there's still a market for these things and I've already dumped a solid $300 on components.

[leadeater]

First boot the system only using one CPU, if it boots it's the CPU TDP problem if not it's something else.

[Electronics Wizardy]

Just now, AnAngryMob69 said:

Thank you for the suggestion, But then what to do with the parts that I already have.... I doubt there's still a market for these things and I've already dumped a solid $300 on components.

dump them?

 

There not worth 300+, sorry about that, id say thats worth about 150 max, its a heater now.

[leadeater]

3 minutes ago, Electronics Wizardy said:

You want a heater, you bought a heater. Also you will be soon paying about about 33 dollars a month in power, depends on where you are.

 

 

Id highly suggest you dump that server, and get something like this http://www.ebay.com/itm/Dell-PowerEdge-FS12-TY-C2100-8-Core-2-13GHz-L5630-16GB-12x-HDD-TRAYS-/381744926594?hash=item58e1c54f82:g:ankAAOSw5cNYZCO7

 

Half the power, more drives, faster cpus(you can put 6 cores in it and 288gb of ram), and it just works out of the box.

Holy crap, um I might just buy that... lol. Been needing a good rackmount case.

[AnAngryMob69]

Just now, leadeater said:

First boot the system only using one CPU, if it boots it's the CPU TDP problem if not it's something else.

I did try that. I noticed that the CPU 2's heat sink was much warmer than the other's. Once removed, the same indicator persisted.

[AnAngryMob69]

1 minute ago, Electronics Wizardy said:

dump them?

 

There not worth 300+, sorry about that, id say thats worth about 150 max, its a heater now.

I did buy the original chassis for about $120 with the AMDs already configured and working (but everything running through two Hardware RAID cards). The other things are what added up; the new motherboard, new RAM, cable extensions... etc

[leadeater]

4 minutes ago, AnAngryMob69 said:

I did try that. I noticed that the CPU 2's heat sink was much warmer than the other's. Once removed, the same indicator persisted.

If there is only 1 CPU installed and you wait for it to cool down and it won't boot at all it's not a TDP problem, that would be too quick to cause an over temp alarm. It's likely a sensor on the motherboard or chassis that needs to be plugged in or bypassed to get the system to turn on.

 

I'd suggest taking the system out of the chassis and bench running it using a standard PSU and the single CPU. Once you know it will actually boot up that way you can work on getting it running in the chassis with the dual PSU backplane and it's sensors.

[AnAngryMob69]

Just now, leadeater said:

If there is only 1 CPU installed and you wait for it to cool down and it won't boot at all it's not a TDP problem, that would be too quick to cause an over temp alarm. It's likely a sensor on the motherboard or chassis that needs to be plugged in to bypassed to get the system to turn on.

 

I'd suggest taking the system out of the chassis and bench running it using a standard PSU and the single CPU. ONce you know it will actually boot up that way you can work on getting it running in the chassis with the dual PSU backplane and it's sensors.

Thank you, I'll give that a try soon. I'm pretty peeved at the moment, so I'm worried that I'd do much more harm than good.

[Electronics Wizardy]

3 minutes ago, AnAngryMob69 said:

I did buy the original chassis for about $120 with the AMDs already configured and working (but everything running through two Hardware RAID cards). The other things are what added up; the new motherboard, new RAM, cable extensions... etc

you could just use the hardware raid cards, you don't need freenas and could use anther other os just fine.

[leadeater]

Just now, AnAngryMob69 said:

Thank you, I'll give that a try soon. I'm pretty peeved at the moment, so I'm worried that I'd do much more harm than good.

Yea been there before. Typically it something simple and makes it even more annoying when you find it.

[leadeater]

What RAID card is it btw? You could just flash it to an HBA firmware.

[AnAngryMob69]

Just now, leadeater said:

What RAID card is it btw? You could just flash it to an HBA firmware.

Unfortunately not able... Too old.

LSI MegaRAID SATA 300-8XLP

[Electronics Wizardy]

Just now, AnAngryMob69 said:

Unfortunately not able... Too old.

LSI MegaRAID SATA 300-8XLP

why not use hardware raid?

[AnAngryMob69]

Just now, Electronics Wizardy said:

why not use hardware raid?

This card doesn't allow the cache to be disabled, and FreeNAS freaks out when cache is on. I'd definitely like to stick with FreeNAS (despite the murder of Corral).

[Electronics Wizardy]

Just now, AnAngryMob69 said:

This card doesn't allow the cache to be disabled, and FreeNAS freaks out when cache is on. I'd definitely like to stick with FreeNAS (despite the murder of Corral).

How about ufs in freenas?

 

Id personally just  use linux, but people like there own stuff.

[AnAngryMob69]

26 minutes ago, Electronics Wizardy said:

How about ufs in freenas?

 

Id personally just  use linux, but people like there own stuff.

To Be honest, I'm pretty new to the server world, so I have no idea what UFS is. ZFS seemed to check all of the boxes for me in terms of speed and redundancy. I mainly wanted the ability to SSH if necessary and create file shares for media and backups..

[AnAngryMob69]

@leadeater Confirmed, the same overheat LED persisted when powered by a separate PS. I'm not certain at this point what could be causing the issue. I've maintained the thought that it is not a TDP issue, as the system turns on and remains running (which was not the case with the X5472s where it would flash the OH LED and power down). I've already cleared CMOS to attempt resetting any previous fan profiles preventing max voltage to them. Honestly the passive coolers may need to go, but this would confuse me as the AMDs had a higher TDP of 90W and still used the exact same style. I just don't want to sink more $$ into active coolers if they won't solve the problem. ( I don't currently have any spare ones)

[Electronics Wizardy]

9 minutes ago, AnAngryMob69 said:

@leadeater Confirmed, the same overheat LED persisted when powered by a separate PS. I'm not certain at this point what could be causing the issue. I've maintained the thought that it is not a TDP issue, as the system turns on and remains running (which was not the case with the X5472s where it would flash the OH LED and power down). I've already cleared CMOS to attempt resetting any previous fan profiles preventing max voltage to them. Honestly the passive coolers may need to go, but this would confuse me as the AMDs had a higher TDP of 90W and still used the exact same style. I just don't want to sink more $$ into active coolers if they won't solve the problem. ( I don't currently have any spare ones)

could easily be a bad board.

[AnAngryMob69]

Just now, Electronics Wizardy said:

could easily be a bad board.

I would probably die....

[Electronics Wizardy]

17 minutes ago, AnAngryMob69 said:

To Be honest, I'm pretty new to the server world, so I have no idea what UFS is. ZFS seemed to check all of the boxes for me in terms of speed and redundancy. I mainly wanted the ability to SSH if necessary and create file shares for media and backups..

UFS is anouther filesystem in freenas.

 

For filesystems and OS, id personally just use hardware raid on the old hardware and then use a os like nas4free(the old but still maintained version of freenas with good ufs support. Or something like open media vault.

[AnAngryMob69]

Just now, Electronics Wizardy said:

UFS is anouther filesystem in freenas.

 

For filesystems and OS, id personally just use hardware raid on the old hardware and then use a os like nas4free(the old but still maintained version of freenas with good ufs support. Or something like open media vault.

Gotcha. I really still want to avoid hardware RAID like the plague. These particular LSI cards have their own boot sequences that easily clear 5 minutes each. There is the option to stop this in the BIOS but then you get no drives.

[leadeater]

33 minutes ago, AnAngryMob69 said:

@leadeater Confirmed, the same overheat LED persisted when powered by a separate PS. I'm not certain at this point what could be causing the issue. I've maintained the thought that it is not a TDP issue, as the system turns on and remains running (which was not the case with the X5472s where it would flash the OH LED and power down). I've already cleared CMOS to attempt resetting any previous fan profiles preventing max voltage to them. Honestly the passive coolers may need to go, but this would confuse me as the AMDs had a higher TDP of 90W and still used the exact same style. I just don't want to sink more $$ into active coolers if they won't solve the problem. ( I don't currently have any spare ones)

So the system boots and you can get in to the bios? The overheat LED is also for failed fan, I'm thinking you just don't have fans connected to headers it expects to be populated e.g. CPU fan headers.

 

Normally you can go in to the bios and set the RPM indicator for the CPU fans to ignore which should clear the alarm.

 

Edit:

Is the LED flashing or solid on?

[AnAngryMob69]

4 minutes ago, leadeater said:

So the system boots and you can get in to the bios? The overheat LED is also for failed fan, I'm thinking you just don't have fans connected to headers it expects to be populated e.g. CPU fan headers.

 

Normally you can go in to the bios and set the RPM indicator for the CPU fans to ignore which should clear the alarm.

 

Edit:

Is the LED flashing or solid on?

The LED is Solid. I was also not able to get into BIOS, but went as far as shorting the jumper and removing the CMOS battery. The MoBo does not have any dedicated CPU fan headers either.