ULtraMega LANBOX

FedsAgainstGuns · February 17, 2016

I'd like to thank LMG for their videos they are what inspired me to attempt this build, and just now sign up for the LTT forums, its great to be here and i hope to be a helpful contributor to this community.
TL:DR dual socket LGA1366 12 xeon cores 24 thread at 3.3Ghz water cooled(better than i7 5960x even when overclocked in most work loads), up to 192GB DDR3 1333 ECC on a board thats smaller at the rear than a mini ITX, problem is the board is proprietary, all the way from power connection, to mounting holes(the 2 main problems) but i plan to make 2 different case designs, one thats basically like a HAF stacker 915R with the GPU rizzen above the motherboard, the other like a 3U server thats been shrunk down and an 18-24 inch LED TV/monitor hinged over the viewing window to expose motherboard tripple liquid cooling, and the GPU, laying flat, i'm leaning towards a GTX 970 or 980 with hybrid cooling(exhaust+liquid) Both cases are planned to be made out of chrome plated tubular and plate stainless steel
I'm not very far into the build but everything is going WAY better than expected, but there is a long, and maybe hard road ahead before i reach the goal of my dream PC, in portable form factor

This will be my first custom PC, and my third hand assembled PC(my first with intel/server parts) I started my search for a motherboard to match the power of my dell precision T7500 with dual xeon X5660 6x2 core 12x2 thread 2.8-3.1Ghz with a 3Ghz flat turbo (ebay $400 less ram and GPU). But the set in stone rule was that when completed it had to be portable, the problem, and why this is such a unique and difficult (for a first attempt) is that there are no LGA2011 motherboards in the mini ITX form factor, even a single 1366 never seems to be as small as even micro ATX in the consumer market, but then i discovered the high density server market, i soon found many boards with rear board dimensions smaller than mini ITX and they usually had DUAL sockets, i soon settled on a supermicro x8dtt without the superchannel, or whatever it was on the rear IO, it has 2xLGA1366 sockets supporting up to the 6 core xeon 5600 series, and 12 dimm slots for up to 192GB of DDR3 1333mhz fully buffered ECC memory, but it only has 2 rear USB and 2 internal usb doubleheads and one PCIe 16x 2.0, but oh my god the whole board was less than $50, i couldnt believe it, all that potential in such a small cheap package. So before buying it i knew there would be problems, and i was willing and eager to tackle them, and so far its working well, the first hurdle i have partially tackled is power delivery. These boards have 2x20 pin power connectors(meant to be used as redundancy and not parity) and then one molex connector, NO CPU pin connectors. i thought 'OK i can just get a PSU with a really high 12v rail connected to the 20+4 pin or find a way to use 2 PSUs to power the board, and the dual 130 watt processors.' No dice apparently they CANT be use concurrently, and even then PSUs arent designed to supply that much power through the 20+4 pin, and even if THEY WERE designed to do that this motherboard has proprietary wiring. I couldnt tell it from the pictures everywhere but the entire board is actually powered by a single 8pin from the PSU and then rewired to 24 pin, and theres a weird heat generating circuit half way down the line, more on that later. So instead of waiting months on the phone with reps to get a replacement harness only to find they have been discontinued, or cost $150, i went and searched for them on ebay. Unfortunately the only way to get them seems to be buying partial systems, the cheapest one i could find was a listing for a 1U chassis with 2 of everything, PSUs were 256 watts, the case was missing it's top, and the motherboards were listed as not working, and for less than $100 that was fine i'd have a template or two to use for building the case instead of worrying about saw dust metal shavings and other contaminants getting in the working board, and i'd have 2 tiny SFF PSUs to use elsewhere, and i'd have a cable to test with and a cable to make another one from scratch. The best part was that i decided to test both boards that were said to be bad, both posted without issue and booted into linux, though i was only using one processor and 1 stick of ram, and a usb drive for linux, its possible that the other socket or some of the ram slots are bad, so IDK, either way i'm REALLY happy with the purchase. So after sittin on the project for a week because i was sick, i immidiately tried to get the board to post with the used PSU and a single CPU, instant success, browsed the bios, shut down and put in another CPU, didnt have 2 water blocks so i just let a spare cooler rest on the CPU to keep it cool, and it posted, checkd the temps in bios and all were "low". so i tried to boot an OS, unfortunately the PSU had no molex/sata cables, and the CPU 4pin to SAS data+power cable sets in the case were obnoxiously long for me to use with testing, i was able to boot from a HDD with windows already on it using a USB adapter, but i didnt remeber that person's password, so i had to wait a day and get Lubuntu while at work, booted up and ran a stress test across all 8 cores of the dual xeon E5504(80W) and the temps of the H55 liquid cooled CPU never rose above 33C, and the CPU with the loosely mounted SFF cooler never reached over 50C, the interesting thing is that it didnt lock up because of low voltage, i am 99% sure i was over the PSU's rated power with 2x80 watt(heat) CPUs a board that has to at least 10 watts, and a Nvidia GT430, which if i remember correctly, uses 55 watts, that should be right at the rated 256 watts, but no the tiny ~20mm fans werent spinning anywhere near max, and werent much louder than at idle. At this point i'm pulling out my hair because it shouldnt be going this well, i half expected the board to explode when i first powered it on.
The second hurdle i have yet to tackle is motherboard mounting and cases. This board is designed to be in blade, OR multiboard server chassis(very important because most blade only boards cannot be powered without expensive backplane), and like most manufacturers Supermicro decided it would be best if they made this board not work with anything other than what it was designed to work with, so the mounting holes are not standard. I will have to at least make a motherboard tray(not hard) but the problem is that all cases other than the HAF stacker 915R are either too shallow to fit this board(in the case of mini ITX through ATX, or are expecting you to put in a board with 7 PCIe slots and 4 PCI slots eATX through WTX, so my choises of factory made cases are limited to the HAF stacker 915R(not exactly portable without modding) or cases that are entirely too big and heavy to be portable any way you look at it. So either buy an unattractive used HAF 915 for an ungodly high ammount and then chopping it up, or make my own case, i chose the latter. I think it will be easier to converti it to a lanbox. So i've got to take the measurements of my motherboard, and everything else, decide where i want them placed, make a mount for the mobo, a mount for the GPU and GPU riser, then finding dimentions and materials for the case that will make it serviceable, sturdy, portable, and beautiful. Easy peasy but also cheapy(sarcasm) weapy. that means i'm going to have to get better at welding, or spend alot of money to get someone to cut, form, weld, and seal this case, and then electroplate it with chrome, probably going to be either the most time consuming, or most expensive part of the build

More to come as the project progresses, many unknows are left, will a modular PSU be able to power 2 high output CPUs from just a single 8 pin plug? even if the psu is capable of 2000w over a single 12v rail, will that 8 pin socket prevent all that power from going out of just one socket. Will i run into issues with PCIE ribbon, will the board actual power the 130 watt processors, documentation says it supports the xeon 5600 series, does that mean all of them, or just the ones that come in under the undisclosed board power delivery maximum. Will i be able to design a case that works, and will i be able to convey that idea to someone else so that it does work the way i mentally design it, and can i repeat it in case others want me to build them one of these monsters
Note that my dell T7500 has dual x5660 stock 2.8, but i was able to OC the reference clock, never went beyond validation because it had to be done in windows using a PLL modifier, once i verified that the base clock was at 150 instead of 133, i immediately powered off and reset, i HAVE left a C state mod on so they are turboing to 3.1 instead of 3.0 across all 24 threads, but this "success" may mean i will be able to either overclock the planned x5680 from 3.3 base to 4.1base, or do the same thing with the much cheaper x5650, 5660, 5670, and 5675 for future builds. I might try my hand in BIOS modding, i do already have 2 boards to work with, and stand alone working boards are always cheap, also the bios chip appears to be removable, so there's that

Edited February 17, 2016 by FedsAgainstGuns
Forgot to add basic layout of component placement for slim case

BrinkGG · February 17, 2016

6 minutes ago, FedsAgainstGuns said:

I'd like to thank LMG for their videos they are what inspired me to attempt this build, and just now sign up for the LTT forums, its great to be here and i hope to be a helpful contributor to this community.
TL:DR

What do you mean? TL:DR? I did read it.

Followed!! I love builds like this.

FedsAgainstGuns · February 18, 2016

So the cinebench score for the 2Ghz 4 core processors(450) is 50 points above the EXACT same processors running in the Dell Precision T7500(405), my best guess for this difference is that the Dell uses a daughter card for the second CPU and 6xDIMM and it looks like it connects through the blade server like PCIe 2.0 connector either 16 or 32x, on the dell the E5504s were also passively cooled with small heat sinks, relying on case fans for cooling, on the supermicro board I have liquid cooling on one, and a server HAF-HSF with 8 heat pipes and suck type fan

One issue that i didnt expect is boot order, this bios is set up so that it automatically boots to a USB drive when attached because normally to get to USB ports you'd have to pull the rack for service, and i dont know how it causes this but half the time i cant boot to the drive i want until i unplug the one i dont, and the order is entirely random, direct example i was trying to install windows 10pro on a used HDD with an existing install of windows 8.1, but it would always boot to the SAS drive that would fail to boot because there was too much of a hard ware difference, and the UEFI security prevented boot for security reason. I had originally though i would just wipe the drive and it would switch to USB without an OS or MBR on the sata drive, but no, it said no boot media found with no options to choose another. so what i had to do was remove the SATA drive, boot to USB, then plug in the SATA drive while the system was hot, I never want to do that again, sacrilege.

And the weird thing is that now that windows is installed on the 320GB 7200RPM 2.5" drive, the bios now decides it only wants to boot to the USB drive, my best guess is the bios chooses the last successful boot device, no matter what order you set priority in bios and there is no listed option/hotkey to select temporary boot device, and none of the normal hotkeys, like F12 on this lenovo X220T, and F9 on my old toshiba P500D. this may be an issue in the future with dual boot on multiple drives and make it hard to service. Boot drive will almost have to be a SAS SSD

MOAR PICTURES

this is an ebay 454 watt PSU from ebay for $9(for 2 of them) the computer is wired up to them note the always on 20+4 pin mod, and the board being powered by the 8 pin, the single 4 pin on the end is powering a string of 6, yes 6 SAS data+power connectors on a single tangle of cables

This is the 8 pin that's converted to the 20pin board power

This is the strange heat producing IC tied into the 8 pin, with the 256 watt PSU the thing got so hot that i flipped its heatsink over and mated it to to an old Pentium3 heatsink and it would still be hot, but on this PSU the thing barely gets warm

This heatsink pictured here is the opposite of the mysterious heater on the 256 PSU this heat sink was barely warm, but on this PSU i needed to flip around the CPU2 fan to have it blow on it, though it was no longer too hot to touch i decided to prop the other fan up to blow on it too, now it (feels) about 100-120F I may need to make a water-cooling loop and add this too the loop, also note the desperate need for USB ports which leads to our next picture