Posted April 9, 2022 The Dell S4810. A 48 port SFP+ 10GbE, 4 port QSFP+ 40GbE top of rack network switch. I picked this guy up on eBay for cheap due to it taking some pretty severe damage from mishandling. I plan to fix it up the best I can and install it in my homelab setup. I'll be taking a closer look at the damages later. These switches were made for high availability with up to 640Gbit switching capacity. Just for reference your current modern motherboards only come with 1Gbit, sometimes 2.5Gbit, and the high-end on occasion can come with 10Gbit. Most of the current top of the line NVMe M.2 Gen4 SSD's? They cap around 7GB/s (unless you resort to RAID). This network switch can push 80GB/s... In the world of enterprise grade networking though a switch like this is actually pretty entry level but for a tech enthusiast to have one in their home? Oh how much fun this is going to be. This is what I have planned: A full tear-down Cleaning Possibly swapping thermal paste if I can, where I can. Repairing whatever damage is reasonably repairable within my power. The seller stated 10 out of 48 SFP+ ports are damaged. I hope to recover most of them. Possibly doing a paint-job if I'm feeling cocky? It's so badly scratched up I really want to and I feel anything I do would make it look better than it does right now. Figure out how to interface with it over RS232 or Ethernet and a Terminal. See if can get system stats such as temperature and look into making the fans run quieter if temps permit. Configure some VLANs, and get it installed as my new primary 10Gig home network switch. Stay tuned for more updates. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 10, 2022 2 hours ago, Windows7ge said: In the world of enterprise grade networking though a switch like this is actually pretty entry level but for a tech enthusiast to have one in their home? Oh how much fun this is going to be. hehe one 2 hours ago, Windows7ge said: The seller stated 10 out of 48 SFP+ ports are damaged. I hope to recover most of them. Should be able to, I doubt the actual port connector is damaged and it's just the port cages because someone had no clue how to get SFP+ modules out and just wrenched it out (looks like the case to me). At worst port 38 is dead, best worse case anyway haha. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 10, 2022 Author 59 minutes ago, leadeater said: Should be able to, I doubt the actual port connector is damaged and it's just the port cages because someone had no clue how to get SFP+ modules out and just wrenched it out (looks like the case to me). At worst port 38 is dead, best worse case anyway haha. I don't see anything too alarming yet but I do see the cage on port 38 the inside(rear) was pressed backwards which gives me the impression there was a transceiver installed when the switched slammed into something. I'll have to perform a closer investigation but yeah worst off I may 3D print a sort of blank insert to remind me not to use that port. I'll be posting pictures about the repairs later. Moving forward starting at the back of the unit I got the modular redundant PSU's removed and the modular fans: Yes you weren't lying Leadeater when you said it was loud. It sounds like a hairdryer when it's running. Getting a closer look at the rear components it uses 350W PSU's and what are perfectly standard 40mm 12V 3-pin fans on little simple PCBs. If I can monitor component temperatures this'll make fan swapping/modding a breeze. I like the design. Lifting the skirt on this beauty revels a holy motherboard of heatsinks. There's actually not very much visible dust in here. That's nice. Less work for me. Taking apart the upside down PCB in the rear looks to be the brains of the operation. 8GB SD card for the OS 2GB DDR3 1333MHz memory What looks to be a BIOS battery of sorts (yellow). Power delivery breaking up the incoming power from a high density interconnect. VRM? And finally the CPU...? Appears to be a P2020NSE2MHC. Let me tell you that heatsink did not want to come off easily. It uses a thermal pad 2.0mm thick. Although it was probably fine to reuse I opted to replace it substituting some leftover supplies from another project. What I had on hand was only 1.0mm thick though so I had to double it up. To my own defense if it's using thermal pads and a 2mm thick one at that it can't be THAT hot of a chip. Should be fine it's only part of a switch I have no way of replacing if it breaks. After this I just reattached the heatsink. This will have to be all for tonight. Tomorrow we'll investigate what's under some of the other heatsinks on the mainboard and get a closer look at the 40Gig section. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 10, 2022 Author Continuing where I left off I got both of the heatsinks removed for the 10Gbit ports. Again all of the thermal pads appear to be 2mm in thickness. The IC's appear to be Broadcom BCM84754A1KFSBG's. Looking at the 40Gig section shows it uses a sandwiched daughter-board configuration: Look at that high density connector. Wow. Taking the heatsinks off the 40Gig NICs they appear to use Broadcom BCM84740A1KFSBG's. If I want to remove the primary circuit-board this little guy had to come out. Not sure why Dell opted to make this unit seemingly over-complex. All this standalone board does is power the fans and show some LED's. I don't see a reason for this independent boards existence. Removing the last heatsink exposes what looks to be the actual CPU. What I was looking at before must have been a I/O controller for things like USB, PCI_e, etc. A PCH or South Bridge of sorts. That's a chonky boy. Interesting construction putting everything but the CPU itself on a daughter-board. Unfortunately Dell committed a unforgivable sin here. They used the most unholy thermal compound I've had the displeasure of dealing with. It has the viscosity and stickiness of pre-chewed gum. It's not the first time I've seen it either. Absolutely miserable to clean. After way more time scrubbing, applying alcohol, and elbow grease then I'm willing to admit this will just have to do. It's "clean enough". The Broadcom BCM56845A4KFRBG. I'm gonna stop here and get some things done today. Like feed myself. If I have time left over I'll dive deeper into this project likely starting the repair phase. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 11, 2022 Author With all of the heatsinks removed I was able to remove the entire board from the chassis. Now we can get a closer look at the port damage. On the left side were a couple of port with really easy to fix damage. Looks like the chassis itself was bent causing the cage to bend a bit. Fixed up pretty nicely. Now for the severely damaged part...I believe you @leadeaterhad an interest in how this would go: After: Out of the 10 ports the seller said were damaged 2 were physically unrecoverable. It's clear transceiver modules were installed when this hit or was hit by something. This is the rear of the two most damaged ports: The column was pressed backwards about 2mm. This caused the ports to no longer accept transceivers even if the cage doesn't prevent them from being inserted. Two of the adjacent ports are resistant to accepting transceivers but will with a little finagling. I will be doing a full test of all usable ports when I get this put back together so we'll see if there's any non-visible damage to the port connectors near the 2 unrecoverable ports. So far things are looking pretty promising though. 44~46 SFP+ ports on a fully managed switch for $250. Working on ordering QSFP+ hardware so I can test those interfaces. Something I noticed while cutting new thermal pads and this is something @Schnozmight like is I don't know why Dell opted for 2mm thick pads given I noticed there's a reasonable gap between the heatsink standoffs and the circuit board with a 1mm thick pad installed: Assuming the mounting pressure is still adequate I should be able to improve the cooling by just installing a 1mm thick pad. Now's a good stopping point for tonight. Tomorrow I'll get the switch mostly reassembled and I'll be investigating how I might possibly interface with it assuming I didn't break it. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 12, 2022 Author Re-assembly time! As it turns out the switch has four light pipes for system diagnostics. Apparently they were bumped out of the holes in the front panel somehow but I was able to put them back in. Re-assembling the 40Gig section remembering to put the bottom heatsink on, followed by airflow guide, then daughter-board: I noticed 3 or four of these around the board. They appear to be VRM's but I don't see why they felt the need to put them on daughter-boards except maybe space constraints: Regardless the heatsinks are held on with thermal epoxy and I don't want to break anything trying to get them off. Reattached the heatsinks for all of the switch port IC's. For the CPU itself I'm giving it some of the good stuff. And then we just plop the heatsink back on. And after a few other odds and ends we have everything back together and no extra screws... Please ignore the tray full of screws. Plugging all of the modular components back in the rear and giving it power the switch runs though it's start-up process and gives me all greens on the information LEDs. Now I went ahead and connected an Ethernet cable hoping that Dells enterprise solutions didn't work the same as CISCO's where the switch doesn't automatically collect an IP until you manually configure it over RS232...well that appears to be the case. I searched my router for the MAC address of the switch and nothing...damn. I'm going to have to source what's known as a Rollover Cable. They're cheap and ubiquitous but I don't have any. You can plug them into the RS232 port found on a lot of servers or older desktops & laptops but alternatively you can buy a Serial to USB adapter and run it via USB. I'll investigate both but until I can get one in hand the project is going to be on hold. I may do some basic hardware testing. See what works see what doesn't but we won't be doing any installing until I can configure it. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 13, 2022 By default none of those ports won't be set up into a VLAN. You will need to configure a VLAN and assign an IP to manage the switch before you can add it to your network. I know these switches very well and at one point my site had the highest operational units in one place in the whole of Europe, not sure if this is still true now though. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 13, 2022 Author 14 minutes ago, abit-sean said: By default none of those ports won't be set up into a VLAN. You will need to configure a VLAN and assign an IP to manage the switch before you can add it to your network. I know these switches very well and at one point my site had the highest operational units in one place in the whole of Europe, not sure if this is still true now though. Not according to the quick start guide. Quote The Default VLAN as VLAN 1 is part of the system startup configuration and does not require configuration. Same as CISCO switches. All 48+4 ports should be plug'n'play right now if I didn't care about VLANs. I plan on using the dedicated management port which you are correct about requires me to manually configure before it can be used. Though I will ask wouldn't the IP in step 2 be identical to the IP in the Management Route command? Or is it it just asking for the network? Ex. 192.160.0.0 Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 13, 2022 I've always used the console port to configure new switches. Then once I configured it I add a management VLAN which are use to manage all the switches via the trunk link. I assume that the management ethernet port is by default set to either DHCP or a static IP listed in the manual and that in step 2 you can set to the one you require. By default all ports are shutdown hence the no shutdown command in step 3. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 13, 2022 Author Good news everyone. I was able to get a rollover cable on loan from a old professor of mine. Still have to buy my own but if this works then I know I won't be wasting my time and money buying one for myself. That or I have to build an adapter cable that changes the pin-out. Hopefully I don't have to do that. 2 hours ago, abit-sean said: I've always used the console port to configure new switches. Then once I configured it I add a management VLAN which are use to manage all the switches via the trunk link. I assume that the management ethernet port is by default set to either DHCP or a static IP listed in the manual and that in step 2 you can set to the one you require. By default all ports are shutdown hence the no shutdown command in step 3. For my arrangement a trunk link isn't necessary (yet) and the default state of the Management Ethernet is definitely Static which is unfortunate. I can use the dedicated management port if I understand what it wants me to input. Given how it's written in the user guide I think this is what it's asking for. management route <ip-address/mask> <gateway> management route 192.168.0.0/24 192.168.0.1 If I'm going to manage it from over the internet it'll be via a VPN so I wonder if I can actually omit the gateway address or if it will not function or allow me otherwise. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 14, 2022 Author We're in baby! ThinkPad-A485:~$ telnet 192.168.0.248 Trying 192.168.0.248... Connected to 192.168.0.248. Escape character is '^]'. Login: admin Password: Dell>enable Password: The SupportAssist EULA acceptance option has not been selected. SupportAssist can be enabled once the SupportAssist EULA has been accepted. Use the: 'support-assist activate' command to accept EULA and enable SupportAssist. Dell#config terminal % Warning: The following users are currently configuring the system: User "" on line console0 Dell(conf)# Things just did not want to work on two different Serial communication programs (Minicom & HyperTerminal) but at the end of the day (literally) PuTTY came to the rescue and "just worked". Next I have some availability we'll be doing some hardware installing & configuring! Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 17, 2022 Author Just a short update today. From here things are going to get really command-liney. After successfully establishing a serial connection with the switch over RS232 the first thing I wanted was to establish a means to manage it over the LAN. To do that was outlined in the start guide. First get to configuration mode: Dell>enable Dell#configure terminal Dell(conf)# Then to configure remote access over the management interface: Dell(conf)#interface managementethernet 0/0 Dell(conf-if-ma-0/0)# ip address 192.168.0.248/24 Dell(conf-if-ma-0/0)# no shutdown So what we've done here is: We said we want to something with the Ma0/0 physical interface. We assigned it an IP & Subnet Mask. We enabled the interface. Now to create a user and give him enable permissions: Dell(conf-if-ma-0/0)#exit Dell(conf)#username <username> password <password> Dell(conf)#enable secret level 15 <password> At this point you can now Telnet into the switch remotely and do virtually anything, congrats!...Unfortunately if you shut the switch off or lose power at this point in time you will have to do all of this over again...why? Because we've edited the running-config. Which only lives in RAM. If we want to edit the startup-config which resides in persistent flash we have to tell it to do that: Dell(conf)#exit Dell#copy running-config startup-config Proceed to copy the file [confirm yes/no]: yes ! 4604 bytes successfully copied Dell# This step was not written in the User Guide which I find weird. I assume it's a step they expect you to already know. While we're at it we can assign a new hostname by going back into config mode and using the hostname command. Dell(conf)#hostname Switch-S4810P Switch-S4810P(conf)# Then switching back to enable mode and saving the running-config again. That'll be it for today. Hopefully I have more free time soon and more of the hardware I need to actually build this out shows up. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 21, 2022 Author The cables have finally arrived meaning this week-end I should be able to progress in the project given work permits. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 22, 2022 On 4/14/2022 at 3:24 AM, Windows7ge said: We're in baby! ThinkPad-A485:~$ telnet 192.168.0.248 Trying 192.168.0.248... Connected to 192.168.0.248. Escape character is '^]'. Login: admin Password: Dell>enable Password: The SupportAssist EULA acceptance option has not been selected. SupportAssist can be enabled once the SupportAssist EULA has been accepted. Use the: 'support-assist activate' command to accept EULA and enable SupportAssist. Dell#config terminal % Warning: The following users are currently configuring the system: User "" on line console0 Dell(conf)# Things just did not want to work on two different Serial communication programs (Minicom & HyperTerminal) but at the end of the day (literally) PuTTY came to the rescue and "just worked". Next I have some availability we'll be doing some hardware installing & configuring! bit of a late reaction here, but on our dell routers in the datacenter i connect with minicom with the following settings: 115200 8N1 no flow control works everytime! Thats on S5232F-ON routers though. -Phoenix- -VouwFabriek- -SolidNAS- BlackMac Woody Clack Virtuality Lil'Lad Plot-Device Boerderij all your proteins are belong to us. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 22, 2022 Author 2 hours ago, RollinLower said: bit of a late reaction here, but on our dell routers in the datacenter i connect with minicom with the following settings: 115200 8N1 no flow control works everytime! Thats on S5232F-ON routers though. I have to order a rollover cable for myself in the event I run into trouble with Telnet and/or SSH if I can configure it. I can try those settings then. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 22, 2022 Author I can't start on this today but I wanted to go over what my current game plan is. Now I've not shared this with anyone but my network closet looks atrocious and this addition isn't going to make things any better. The only thing I ask is don't tell Leadeater. So this is what I have to work with: You can see where I've already ran one of two QSFP+ cables. (lower-right). Due to the length of the switch installing it strait out like the current two isn't an option. Rotating the switch 90° wouldn't be an option either as the distance from the wall to the ventilation duct on the left is too short. I had the idea to lay the switch on it's side and screw the ear to the side of the shelf but I don't like the long term stability of that solution. So the current running solution because it has to go in this closet is as follows, beautifully illustrated in what is basically MS Paint. So to explain what you're looking at I'm thinking of starting by re-enforcing the shelf to the 2x4 in the wall with a couple of metal brackets. One for the left and right. This shelf hasn't had any trouble supporting the UPS for a few years but we're adding a chonky boi of a switch and I'd rather over-build it than under-build it. After re-enforcing the shelf I can fabricate some brackets that will attach to the switch ears by hanging over the edge. The top brackets will get holes drilled to pass machine screws through to the bottom where a smaller backplate will act to re-enforce the shelf against the weight of the switch. These will then be held in with basic nuts. From here I can just hang the switch like you would install it in a server rack except facing up and attach its nuts & bolts. Thoughts?...Criticisms? Better ideas? Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 24, 2022 Author Today to start things out I want to fabricate the mounting brackets I need for the plan I laid out above. For materials I went to the hardware store and picked up 3' of both 1" x 1/8" flat and angled Plain Steel. I don't know steel grades very well. Best I can guess would be it's probably 300 series. It's definitely not 400 series given it's a soft steel with not a high enough carbon content for Tool Steel. Measuring the lower shelf with the additional length of the height of the switch brackets I'm going to need two cuts of the angle steel 8 & 7/8th" (or 225.425mm) long. Measuring the lower shelf alone I'm going to need two lengths of the flat bar 7 & 3/16th" (or 182.5625mm). From here I can mark, center punch, drill, & de-burr all of the holes. Moving over to the shelf I've gone ahead and added the reinforcements. 3" screws directly into the 2x4. Stubby screws into the shelf. It probably doesn't need the additional support but it'll help me sleep better at night. Getting the switch hung for the first time. Everything is still loose and I need to put some rust prevention paint on the new brackets but everything went together smoother than I could have hoped. A look at the bottom half of the bracket. I was worried it was going to interfere with the switch but I realized looking at this that any overhang is under the ear not the switch itself. As I predicted due to a lack of forwards & back support it won't handle being bumped into very well but this is an extremely low traffic area so I'm not too worried about that. I might add some shelf brackets behind it if I have any issues though. I'm going to stop for now. I've been at this for the better part of the day. Due to this closet not having a whole heck of a lot of air flow I want to test how loud the switch will get with the door closed. Right now it's not any worse than my server rack but I'll let it warm up and see what happens. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 26, 2022 Author Popping the top off the server so I can install the new NIC everyone can see what I'm working with here. As I stated this NIC will replace three separate NICs totaling 50Gbe which you can see better from the back. So I got the new NIC installed and connected the one 40Gbe cable I already ran to both ends. Started up the server. Saw the new NIC was detected and at 40Gig. Good. Link Staus...down... Little bit of investigation showed I needed to enable the interface on the switch side and that was pretty painless to do. Switch-S4810P>en Switch-S4810P#config t Switch-S4810P(conf)#interface fortyGigE 0/60 Switch-S4810P(conf-if-fo-0/60)#no shut Suddenly I got a little pop-up on the server console: mlx4_en: enp24s0d1: Link Up Now I took the liberty of connecting some 10Gig clients and re-configuring them to compensate for the network changes. Initially no clients wanted to talk to the server or each other for that matter but that was easily rectified by putting the interfaces into switchport mode. Switch-S4810P(conf)#interface range tengigabitethernet 0/0-47 Switch-S4810P(conf-if-range-te-0/0-47)#switchport Switch-S4810P(conf-if-range-te-0/0-47)#exit Switch-S4810P(conf)#interface fortyGigE 0/60 Switch-S4810P(conf-if-fo-0/60)#switchport What this does is it places the specified interfaces into the Default VLAN (VLAN1) which we can verify with a show command: Switch-S4810P#show vlan NUM Status Description Q Ports * 1 Active U Te 0/0-47 U Fo 0/60 And with that we've successfully linked these interfaces together for Layer 2 communication. Now we're not done yet but I'm out of time for today. We'll actually be doing some port/vlan re-arranging and connecting the other 40Gbe cable I bought. I look forward to it. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 26, 2022 Nice work!. Surprising to see the commands are similar to Cisco's How loud is the switch btw? CPU: AMD Ryzen 5 5600X | CPU Cooler: Stock AMD Cooler | Motherboard: Asus ROG STRIX B550-F GAMING (WI-FI) | RAM: Corsair Vengeance LPX 16 GB (2 x 8 GB) DDR4-3000 CL16 | GPU: Nvidia GTX 1060 6GB Zotac Mini | Case: K280 Case | PSU: Cooler Master B600 Power supply | SSD: 1TB | HDDs: 1x 250GB & 1x 1TB WD Blue | Monitors: 24" Acer S240HLBID + 24" Samsung | OS: Win 10 Pro Audio: Behringer Q802USB Xenyx 8 Input Mixer | U-PHORIA UMC204HD | Behringer XM8500 Dynamic Cardioid Vocal Microphone | Sound Blaster Audigy Fx PCI-E card. Home Lab: Lenovo ThinkCenter M82 ESXi 6.7 | Lenovo M93 Tiny Exchange 2019 | TP-LINK TL-SG1024D 24-Port Gigabit | Cisco ASA 5506 firewall | Cisco Catalyst 3750 Gigabit Switch | Cisco 2960C-LL | HP MicroServer G8 NAS | Custom built SCCM Server. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 26, 2022 Author 7 hours ago, Sir Asvald said: Nice work!. Surprising to see the commands are similar to Cisco's How loud is the switch btw? According to what I've been told by an old professor of mine the command similarity is more innate to the OS/platform. Think of the different switch/router operating systems more like different distributions of Linux or UNIX. Each version has it's key features but they all share common factors. How loud is it? If the fans whirl up to 100% like during startup it's somewhere in-between a hair dryer on high & a blowematron. When the fans calm down (after the OS starts) it's like a hair dryer on low. Definitely not something you want to sleep or even sit next to regularly but quiet enough to where if you stuff it in a closet it's not that noticeable with the door shut. It's a lot of power in a 1U enclosure though so I didn't expect it to be silent. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 27, 2022 Author I'm not home at the moment but I wanted to get some configuration out of the way and I'm glad I did because it gave me issues I didn't expect forcing me to change how I setup the VLANs. This is the current arrangement which I hope can stay this way: Switch-S4810P#show vlan NUM Status Description Q Ports * 1 Inactive Default VLAN U Fo 0/56 2 Inactive Primary VLAN T Te 0/12-47 T Fo 0/52 3 Active iPXE Node VLAN T Te 0/0-11 T Fo 0/60 Few things I discovered. I'm honestly not sure if it makes a difference in my configuration but in addition to assigning ports to VLANs the ports must also be tagged to the VLAN (denoted by the letter T next to the ports. Ports cannot be tagged to the Default VLAN (VLAN1) So to start how I did what's shown above I created VLAN's 2 & 3 using the interface vlan # command in configuration mode: Switch-S4810P(conf)#interface vlan 2 Switch-S4810P(conf-if-vl-2)# If the VLAN does not exist this creates it while simultaneously taking you into it's configurator. From here I can tag/assign ports and give it a little description: Switch-S4810P(conf)#interface vlan 2 Switch-S4810P(conf-if-vl-2)#tagged tengigabitethernet 0/12-47 Switch-S4810P(conf-if-vl-2)#tagged fortyGigE 0/52 Switch-S4810P(conf-if-vl-2)#description Primary VLAN Switch-S4810P(conf)#interface vlan 3 Switch-S4810P(conf-if-vl-3)#tagged tengigabitethernet 0/0-11 Switch-S4810P(conf-if-vl-2)#tagged fortyGigE 0/60 Switch-S4810P(conf-if-vl-2)#description iPXE Node VLAN There is a little bit more to the setup than this but I don't expect anyone to be repeating what I did here so some details have been left out. For example all applicable ports have to be in switchport mode before you can do...pretty much any of the things I mentioned above. And last but not least never forget: Switch-S4810P#copy running-config startup-config File with same name already exist. Proceed to copy the file [confirm yes/no]: yes ! 5692 bytes successfully copied It's the equivalent of Ctrl+S or File -> Save. If you forget to do this every time you make a change you'll lose everything you did since the last save if the power goes out. When I get home I'll test and make sure tagging all the ports didn't break anything and connect the second 40GbE cable for the other VLAN. I can make sure the switch is doing it's job of segregating frames based on VLAN given one of the servers running on one VLAN is hosting DHCP. If clients on a different VLAN get an IP then I'll know the switch isn't configured how I need it. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted April 27, 2022 Author OK, I got everything physically plugged in. For my setup, VLAN tagging, bad bad bad, very bad. Everything stopped working. Leaving them untagged but in their default VLAN configuration it's working exactly how I want it to which I can prove with the DHCP server. Here is a 10Gig client attempting iPXE Network Boot connected to a untagged switch port associated with VLAN2: FlexBoot v3.4.752 Features: DNS HTTP iSCSI TFTP VLAN ELF MBOOT PXE bzImage COMBOOT Menu PXEXT net0: e4:1d:2d:7e:98:e0 Using ConnectX-3 on PCI01:00.0 (open) [Link:down, TX:0 TXE:0 RX:0 RXE:0] Waiting for link-up on net0... ok Configuring (net0 e4:1d:2d:7e:98:e0)................................................................. Error 0.040ee119 (http://ipxe.org/040ee119) No more ports. Exiting FlexBoot... Here is the same 10Gig client attempting iPXE Network Boot but connected to a untagged switch port associated with VLAN3: FlexBoot v3.4.752 Features: DNS HTTP iSCSI TFTP VLAN ELF MBOOT PXE bzImage COMBOOT Menu PXEXT net0: e4:1d:2d:7e:98:e0 Using ConnectX-3 on PCI01:00.0 (open) [Link:down, TX:0 TXE:0 RX:0 RXE:0] Waiting for link-up on net0... ok Configuring (net0 e4:1d:2d:7e:98:e0).............................. ok net0: 10.1.0.13/255.255.255.0 net0: fe80::e61d:2dff:fe7e:98c0/64 Now I have a couple file servers that need to synchronize at night. For that I like to have jumbo packets enabled to take some of the load off the CPU. Enabling Jumbo Frames on the switch is pretty easy: Switch-S4810P#config t Switch-S4810P(conf)#int tengigabitethernet <0/#> Switch-S4810P(conf-if-te-0/#)#mtu 9216 Then I just do the same for the 40Gig interface. So this is what the current setup now looks like physically after cleaning up some cables and hardware that are no longer in use. Couldn't fit it all in one image. It turned out really well. Power draw from the UPS claims even with the addition of this monster it should still sustain up to 30mins of run time. That's nice. Power outages here are typically very brief. While I'm at it I'd like to get my other server upgraded to 40Gig but before I can do that I'm going to have to order more hardware. Cables cleaned up pretty well behind the server too. Personally I preferred all those fiber optic cables but this saved me 2 PCI_e slots and is technically 37.5% faster. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted September 26, 2023 Hi @Windows7ge Digging up this old thread after stumbling onto it when I rescued a Dell Force10 S4810 from the recycle pile. Taking a chance you are still active here on LTT... Couple questions; - Are you still using the switch - What SFP+ transceiver modules are you using in the S4810? Can you post a picture of the module label? - There are quite a few inexpensive transceiver modules available out there with support for SFP+, 10GbE, SR, 850nm, 300m using MMF LR cables. I wonder if the switch will tolerate using non-Dell branded transceiver modules, or if they MUST be Dell brand? Have you tried other vendor's modules? Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted September 26, 2023 Author 6 minutes ago, patfonltt said: lcHi @Windows7ge Digging up this old thread after stumbling onto it when I rescued a Dell Force10 S4810 from the recycle pile. Taking a chance you are still active here on LTT... Couple questions; - Are you still using the switch - What SFP+ transceiver modules are you using in the S4810? Can you post a picture of the module label? I am still using this switch. I hope and plan to for long into the future. SFP+ 10Gig Short Range 850nm 300m w/ CISCO firmware from fiberstore. Guides & Tutorials: PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure Mass Deploying Customized Windows 10/11 Installs Building a GNU/Linux Based Windows Deployment Server GNU/Linux Installer Server: Installation & Configuration How to: Use (i)PXE to Install Windows from a Network Why Memorize IP's When You Can Self-Host DNS Instead? Ventoy - The USB Multi-Boot Utility! Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list. *NOTE: I'll only add it to the list if the request is something I know I can do. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Posted September 26, 2023 That is excellent news. Thank you for your response. Not to be overly pedantic, but I believe the FiberStore modules with the "CO" designation are IBM BNT 46C3447 Compatible whereas the Cisco Compatible modules have no designator. And the Dell Compatible modules have a "DE" designation. The good news is, it seems (based on your experience) the S4810 will tolerate use of a non-Dell branded module. Link to comment Share on other sites More sharing options... Link to post Share on other sites More sharing options...
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now