Building a router out of used/e-waste server parts.

[Windows7ge]

Pulled this guy out of the trash at work. Figured I'd turn it into a router.

 

 

This is the Barracuda Network's Backup 490. It runs on an MSI motherboard with some custom firmware and their own Linux based operating system.

 

 

To turn this into a router I wanted to gut the existing hardware. Turns out the enclosure is a gently tweaked Supermicro CSE-813M which accepts standard ATX motherboards.

 

I also salvaged all of these bits used, for free from work.

 

 

4x MX500 SSDs off a shelf

4x8GB 2Rx8 DDR4 RDIMM ECC from a Dell PowerEdge

HP quad port network card from our old work server

Intel Xeon E5-2637 quad core LGA2011-v3 processor from a Dell PowerEdge

1U Narrow-ILM CPU cooler from a E5-2670 box where it was never used.

 

Parts I need to order

1. 40mm fans
2. PCI_e riser card + bracket
3. A motherboard
4. server rails
5. CSE-813M compatible I/O shield (doesn't take normal ones)

All parts will be sourced from second hand market vendors.

[RollinLower]

noice! what OS do you plan to run here, just OPNSense?

[Windows7ge]

2 hours ago, RollinLower said:

noice! what OS do you plan to run here, just OPNSense?

I was arguing with myself between VyOS and pfSense but forgot OPNSense was a thing. Is it Debian based?

 

VyOS has no WebUI. CLI only.

 

pfSense being FreeBSD based may not support the quad-nic. I have to check the controller it uses. Also all the controversy surrounding the developers makes supporting the project a turn off.

 

I will read into OPNSense otherwise I'm tempted to give VyOS a shot. It's Debian based and should support the NIC. I'll have to brush up on the commands though.

[RollinLower]

27 minutes ago, Windows7ge said:

I was arguing with myself between VyOS and pfSense but forgot OPNSense was a thing. Is it Debian based?

 

VyOS has no WebUI. CLI only.

 

pfSense being FreeBSD based may not support the quad-nic. I have to check the controller it uses. Also all the controversy surrounding the developers makes supporting the project a turn off.

 

I will read into OPNSense otherwise I'm tempted to give VyOS a shot. It's Debian based and should support the NIC. I'll have to brush up on the commands though.

I cant really vouch for vyos tbh, but we only have 1 machine running vyos in deployment and no one knows how it was setup exactly cause the person that did it left the company and didn't write docs. So I might be slightly biased

[Windows7ge]

2 hours ago, RollinLower said:

I cant really vouch for vyos tbh, but we only have 1 machine running vyos in deployment and no one knows how it was setup exactly cause the person that did it left the company and didn't write docs. So I might be slightly biased

Something with a WebUI would be nice just because I'm not going to remember commands and will probably neglect to write them down.

 

I see OPNSense is actually a pfSense fork so it's also FreeBSD based. I'll try OPNSense first, see how it plays with the hardware. If I have issues I might try again but with VyOS.

[OhioYJ]

7 hours ago, Windows7ge said:

pfSense being FreeBSD based may not support the quad-nic. I have to check the controller it uses. Also all the controversy surrounding the developers makes supporting the project a turn off.

The pfSense has plenty of controversy. I switched to OPNsense and haven't looked back. 

 

As for drivers, I will say in either you can add drivers into either. So just because a NIC isn't supported, doesn't mean it is a deal breaker.  That being said your life will be much easier if you just buy a NIC that is supported.

[Windows7ge]

1 hour ago, OhioYJ said:

The pfSense has plenty of controversy. I switched to OPNsense and haven't looked back. 

 

As for drivers, I will say in either you can add drivers into either. So just because a NIC isn't supported, doesn't mean it is a deal breaker.  That being said your life will be much easier if you just buy a NIC that is supported.

I've never been one to do things the easy way.

 

It's a HPE 331T. Don't know if HP spun their own silicon or if it's an Intel controller but we'll find out if support is there out of the box. Have never built a driver from source if that's even how you port a driver into an unsupported distro. Will find out. The goal of the project is to do this as inexpensively as possible. This is what I got for free so I want to avoid buying one unless I have to.

[Windows7ge]

Got the old motherboard removed.

 

 

I also moved some stand offs and removed some standoffs for the "new" board.

 

This is the motherboard I'll be replacing it with. The Supermicro X10SRL-F

 

 

I needed a board that would support the E5-2637v3 which it should. The CPU mount needed to support Narrow-ILM. I wanted this socket orientation so it can get airflow from the front of the chassis. And I needed enough SAS/SATA ports for the drive bays.

 

Wiring up all the pins for the front panel indicators was annoying.

 

 

Looking at the silkscreen the front panel I/O is pretty strait-forward.

 

 

As it turns out the cable supplied with the server was an adapter plugged into an adapter and neither are needed plugging a supermicro board into a supermicro chassis as every pair are properly in line. I can just drop the ribbon directly onto the board.

 

 

According to the manual all polarity dictates GND is ontop. and pin 1 is lower left. This also matches the pin 1 indicator on the ribbon itself.

 

Until the rear I/O shield arrives I can't screw in the board so I'll stop here for tonight.

[Windows7ge]

40mm fans arrived. Server only came with 3.

 

 

I was able to source the same model of fan the server came with very cheaply so they all match and I don't have to worry about airflow balance.

 

I also acquired the x8 riser card and it's bracket. 

 

 

Once I'm ready to screw everything together this is about what it'll look like.

 

 

Still waiting on the I/O shield so I can't secure the motherboard in place just yet.

[Windows7ge]

I/O shield arrived.

 

 

That looks a lot better.

 

 

I was just buttoning everything up and plugging everything in when I discovered an oversight.

 

 

I didn't check if the sleds supported 2.5" drives.

 

 

So I have three options here.

1. I can look for adapters but I know those get expensive.
2. I can see if 2.5" compatible sleds are available for this server.
3. I can just use my fingers and push the drives into the SAS backplane knowing they're not connected to the sleds.

I'm gonna explore all three probably. Depending on prices and availability option 3 is probably gonna be my goto. Once I install these they're only coming out of a drive dies.

[RollinLower]

I have literal mountains of 2,5" drive sleds for these servers just laying around. I could send you 4 for free if you're willing to cover shipping. Im from the netherlands tho so shipping could be expensive.

[Windows7ge]

15 hours ago, RollinLower said:

I have literal mountains of 2,5" drive sleds for these servers just laying around. I could send you 4 for free if you're willing to cover shipping. Im from the netherlands tho so shipping could be expensive.

Expensive and I'd have to wait quite a while...what I need is 3.5" with 2.5" support. The chassis doesn't support 2.5" sleds.

 

For the moment I've just pushed the SSDs onto the SAS backplane and inserted the sleds. I have a few ideas how I might 3D print my own brackets.

[Windows7ge]

No air flow channel? No problem.

Box of shredded wheat and some masking tape.

 

[RollinLower]

9 hours ago, Windows7ge said:

For the moment I've just pushed the SSDs onto the SAS backplane and inserted the sleds. I have a few ideas how I might 3D print my own brackets.

even on the official supermicro adapters you will find it still screws into half of the 3,5" bracket. so you could still use those and just 3d print some spacers to fit them into the other side. 

9 hours ago, Windows7ge said:

what I need is 3.5" with 2.5" support.

i'm fully aware, i have exactly those.

[Windows7ge]

On 5/11/2024 at 3:02 AM, RollinLower said:

even on the official supermicro adapters you will find it still screws into half of the 3,5" bracket. so you could still use those and just 3d print some spacers to fit them into the other side. 

i'm fully aware, i have exactly those.

Photo is a little weird. SSD is supposed to be on the left. Mirrored? Also I don't know how this bracket works. It would put the SSD out of alignment unless it comes with an adapter.

 

I am currently contemplating how I might 3D print brackets. I think I can make it work if I can nail the measurments.

[Windows7ge]

Working on printing a prototype 2.5" to 3.5" sled compatible bracket. I downloaded this reference from the web and modified it:

 

 

I tweaked it a bit cause the design blocks most of the airflow and I added the screw holes which will hopefully lineup.

 

 

I had the print started but experienced a printer  hardware failure. Good news is I had backup parts to fix the issue so now we're attempting this print again. I don't expect iteration 1 to be perfect first try but I hope I can fix all the issues in iteration 2. Then I'll just print 3 more of them and we're good to go.

[Windows7ge]

These worked out surprisingly well. They're not perfect but not bad and get the job done.

 

 

The drive only mounts in with two screws. I had a surplus of 5mm socket head screws. They worked well.

 

 

Then I used the old HDD screws to mount these in the sleds.

 

 

I reduced the height profile of the original design because it reduced airflow for no reason.

 

The accuracy of the tolerances are pleasantly good. Slot right in.

 

 

And they all still show up.

 

 

I will go over the installation but that requires mounting it in my server rack but I don't have the energy for that right now. We'll see what free time I have Saturday.

[OhioYJ]

On 5/9/2024 at 6:38 PM, Windows7ge said:

I/O shield arrived.

 

 

Was that an OE piece or something you had to have made?

[Windows7ge]

9 hours ago, OhioYJ said:

Was that an OE piece or something you had to have made?

This is a SuperMicro manufactured I/O shield that fits the CSE-813M though I imagine they make this for other models.

 

With a SuperMicro motherboard you just have to pay attention to if you're using an X9 or X10 series board so the layout of the rear ports match and buy the corresponding I/O shield. Then you can punch out the port covers necessary for you exact model.

 

For me I had to punch out all of them except the two top Ethernet ports. I only have a dual port built-in NIC not quad port.