Preparing for 10 GIGABIT Internet! What Could Go Wrong?

[Emily Young]

We're getting 10 gigabit INTERNET installed at the office! But we need to do some housekeeping first in order to use it, and it turns out we suck at networking...

 

 

Buy SuperMicro Servers: https://lmg.gg/8KV6Q

[KostWarCZE]

If you ware using linux for servers you woudn't be at the work on the weekend dumb dumb

[LukeSavenije]

so @Slick can download his cookies at 10 gigabit now? I DEMAND MY DARK THEME SIDE COOKIES LUKE. NOW.

[christoffer15]

Wait are they going to use pfsense for there 10gbps line. Last i remmeberd seeing on the pfsense/netgate forums pfsense cant handle 10gbps troughput with firewalling on. 

 

Correct me if im wrong

[Guest]

26 minutes ago, KostWarCZE said:

If you ware using linux for servers you woudn't be at the work on the weekend dumb dumb

They were using PFsense(FreeBSD)  on the firewall 

[Kevin Soutar]

Now they can download more ram at 1gigabyte per second per second, 256 gigs of ram on every machine? hahaha

[FeedbackGuy]

Good to see that the normal aspect ratio is back!

[Solkre]

I'd love to work on a project like that.  Interested in a followup how close to 10Gb you can get on that hardware.  

[mynameisjuan]

As a network engineer, your network videos make me cringe. Dont get me wrong, you get the basics but then spread misinformation elsewhere to the viewers. Not understanding VLANs, like do they have to be configured or why a PC connected to a trunk and not an access port is not working, at their basics proved that this task should have been handed off. 

 

First, that pfsense box will not be able to push 10gig. On top end hardware with pure routing, no ACLs, firewall just permit any any you might be able to push 8-9gig before it gets crippled. Forget trying to run any other services on the box as well. You do have plenty of headroom for 5gig though. 

 

There were much better options to go and with much more features that can truly handle 10gig with 40gig upgradability. Cisco is a given but even more so should be Fortinet for a 10gig firewall router combo for your use. Juniper also is a very solid option. Anything but PFsense.

 

While I applaud you enthusiasm  and still enjoy you videos, your lack of network engineer and purchasing now resulted:

 

- resolving an issue will be a nightmare without understanding the concepts of why the configuration was set or what is needed to fix it which can lead you to hours of downtime.

- PFsense is behind greatly on updates security and stability

- No upgradability

 

Yes this is an elitist comment but I dont want you to get screwed over in the long run.

[Solkre]

12 minutes ago, mynameisjuan said:

As a network engineer, your network videos make me cringe. Dont get me wrong, you get the basics but then spread misinformation elsewhere to the viewers. Not understanding VLANs, like do they have to be configured or why a PC connected to a trunk and not an access port is not working, at their basics proved that this task should have been handed off. 

 

First, that pfsense box will not be able to push 10gig. On top end hardware with pure routing, no ACLs, firewall just permit any any you might be able to push 8-9gig before it gets crippled. Forget trying to run any other services on the box as well. You do have plenty of headroom for 5gig though. 

 

There were much better options to go and with much more features that can truly handle 10gig with 40gig upgradability. Cisco is a given but even more so should be Fortinet for a 10gig firewall router combo for your use. Juniper also is a very solid option. Anything but PFsense.

 

While I applaud you enthusiasm  and still enjoy you videos, your lack of network engineer and purchasing now resulted:

 

- resolving an issue will be a nightmare without understanding the concepts of why the configuration was set or what is needed to fix it which can lead you to hours of downtime.

- PFsense is behind greatly on updates security and stability

- No upgradability

 

Yes this is an elitist comment but I dont want you to get screwed over in the long run.

Not elitist, 100% correct.  They need to hire a network engineer; and let him make some videos on this stuff.

[GamerXP]

This would be hell without hardware offload and try to optimized OS-level, kernel and stuff.

[GodAtum]

What supermicro router is he using??

[YellowJersey]

Linus always seems so frustrated with Jake.

[GamerXP]

Just now, GodAtum said:

What supermicro router is he using??

My best guess would be motherboard with many NIC other that is off-the-shelf (without case, psu) component.

[Cyberspirit]

Man, that looked kinda fun but, veery frustrating at the same time.

[iris7]

53 minutes ago, mynameisjuan said:

As a network engineer, your network videos make me cringe. Dont get me wrong, you get the basics but then spread misinformation elsewhere to the viewers. Not understanding VLANs, like do they have to be configured or why a PC connected to a trunk and not an access port is not working, at their basics proved that this task should have been handed off. 

 

First, that pfsense box will not be able to push 10gig. On top end hardware with pure routing, no ACLs, firewall just permit any any you might be able to push 8-9gig before it gets crippled. Forget trying to run any other services on the box as well. You do have plenty of headroom for 5gig though. 

 

There were much better options to go and with much more features that can truly handle 10gig with 40gig upgradability. Cisco is a given but even more so should be Fortinet for a 10gig firewall router combo for your use. Juniper also is a very solid option. Anything but PFsense. 

 

While I applaud you enthusiasm  and still enjoy you videos, your lack of network engineer and purchasing now resulted:

 

- resolving an issue will be a nightmare without understanding the concepts of why the configuration was set or what is needed to fix it which can lead you to hours of downtime.

- PFsense is behind greatly on updates security and stability

- No upgradability

 

Yes this is an elitist comment but I dont want you to get screwed over in the long run.

Not elitist.  It was cringy when they didn't understand some of the network fundamentals.  In this situation I would have probably used a router with some modules. 
It probably would have been a good idea for them to contract out the work.

[orbitalbuzzsaw]

Aren't they doing 10gig to the home in Sweden now?

[Trevorb64]

so if your just plugging into the new switch with no vlans just to test if it works, did you make sure to then also remove the sub-interfaces on the router? or are you trying to connect the new switch with no vlans to an already configured switch with vlans? If so the configured switch would be putting the new one into a black hole vlan.

[_bolek_]

Looking on this video make me cry an goosebumps. Like @mynameisjuan say, you are done this horrible this time and wrong - almost horror for network admin and engineer. Horror movie

 

I must point few things:

 

As i know, you don't need do screw server rack door, you can remove them (read manual),

 

Star using server KVM like this site  - cheep, less space and more comfortable to use

 

Stop using different network devices, and believe me when i say if you need then use CISCO for networking (like NEXUS 3000 for 10GbX - YES you can buy refurbished too, for less then 1200$). Reason is simple - VLAN's and other network issue will disapere. About VLAN, on DELL not working that same as on CISCO or other devices compatible with IEEE 802.1q, that same issue you will find in HP, Mikrotik, D-Link and few other especially when using TRUNK.

 

When you cleaning server you should remove most of the parts like, PCIe card, fans, disk and power supply. As IT technician you should know this

 

Ok. Some other advice:

First if you upgraded to PFSense and custom hardware then i presume you also considered to run some sort of IDS or IPS in network. I don't think that on this configuration will work smoothly on only 32GB RAM (you should upgrade to 64 if not 128GB especially snort - only IDS you should run on PFSense - will take almost all resource it that configuration). I advice to make some changes in sysctl for kernel like its pointed on this site, you not be disappointed in result (i do this on any FreeBSD/OpenBSD server i administrate). Of corse better choice will be Fortinet (FortiGate) for the price but its your choice. Other thing is if you want to get full 10GbX on WAN and push this to LAN then you should consider replace 10GbX port to 40GbX - only then you will get full 10GbX on WAN.

As for CARP ... you can do this using BGP, so don't need to have two identical device (i presume you know that if you want take full advantage of fail over the you shoul have two identical devices even if you use CARP - CARP is *BSD [created by OpenBSD] alternative to CISCO HSRP and IETF VRRP  and it working almost identical but using encryption).

 

Configuration on UBIQUITY EdgeRouter Pro is not compatible with PFSense so you need someone who understand both to migrate configuration

Is i see your notwork didn't work, and because i we don't know if PC got IP (and right one) then i will try point few things to check:

- did you configure NAT? dynamic or static?

- did you configure VLAN on TRUNK properly? TRUNK on LAN interface in router - not WAN?

- did you configure routing like entering proper GW for WAN?

- did you configure DHCP properly for each VLAN?

- did you configure TRUNK on proper port in DELL and VLAN on ports when PC is connected? - remember PC port should not be TRUNK

This is all i can help with knowledge i get form video

 

As for servers ...:

- in modern RAID card, the order of disk don't matter. When you create RAID disk all information about RAID ID and RAID type is stored on DISK - that's why when you move disk to other server with that same card model or that same manufacturer RAID card detect RAID configuration and import it.

- you should confider to change consumer grade SSD like Kingston to something more reliable like SAMSUNG Enterprise SSD, INTEL DC SSD, Micron or other enterprise and server grade one.

 

If you need any hep you can msg me

 

[iris7]

25 minutes ago, _bolek_ said:

Looking on this video make me cry an goosebumps. Like @mynameisjuan say, you are done this horrible this time and wrong - almost horror for network admin and engineer. Horror movie

I have personally seen worse.  This seems pretty mild.

 

16 minutes ago, _bolek_ said:

 

Ok. Some other advice:

First if you upgraded to PFSense and custom hardware then i presume you also considered to run some sort of IDS or IPS in network. I don't think that on this configuration will work smoothly on only 32GB RAM (you should upgrade to 64 if not 128GB especially snort - only IDS you should run on PFSense - will take almost all resource it that configuration). I advice to make some changes in sysctl for kernel like its pointed on this site, you not be disappointed in result (i do this on any FreeBSD/OpenBSD server i administrate). Of corse better choice will be Fortinet (FortiGate) for the price but its your choice. Other thing is if you want to get full 10GbX on WAN and push this to LAN then you should consider replace 10GbX port to 40GbX - only then you will get full 10GbX on WAN.

As for CARP ... you can do this using BGP, so don't need to have two identical device (i presume you know that if you want take full advantage of fail over the you shoul have two identical devices even if you use CARP - CARP is *BSD [created by OpenBSD] alternative to CISCO HSRP and IETF VRRP  and it working almost identical but using encryption)

Interesting.  That seems like some pretty extreme system requirements.  Do you have any sort of sources for where I can read more about this.
 

 

[mynameisjuan]

22 minutes ago, _bolek_ said:

Stop using different network devices, and believe me when i say if you need then use CISCO for networking (like NEXUS 3000 for 10GbX - YES you can buy refurbished too, for less then 1200$). Reason is simple - VLAN's and other network issue will disapere. About VLAN, on DELL not working that same as on CISCO or other devices compatible with IEEE 802.1q, that same issue you will find in HP, Mikrotik, D-Link and few other especially when using TRUNK.

This is huge. Once you begin dabbling in a bit of everything terminology and configuration will make troubleshooting and proper setup a nightmare. Configuring VLANs on a Nexus 9k is simple, configuring VLANs and trunking on a Mikrotik requires a week or two of practice to fully understand why its configure in the way they developed RouterOS. 

 

This only gets worse as you go down the rabbit hole of what is an is not supported on each device.

 

26 minutes ago, _bolek_ said:

As for CARP ... you can do this using BGP, so don't need to have two identical device (i presume you know that if you want take full advantage of fail over the you shoul have two identical devices even if you use CARP - CARP is *BSD [created by OpenBSD] alternative to CISCO HSRP and IETF VRRP  and it working almost identical but using encryption).

Another big point that I dont think Linus is prepared for when he gets his second router (which in the video he didnt even leave another 1u slot to mount it) he then needs to properly setup redundancy which for someone not familiar with networking will be a headache on its own but add on that this is ANOTHER service on PFsense, his 10gig link will take a huge hit once again. 

 

Also I wasnt familiar with CARP as I never used PFsense or Open BSD for routing. Thank you for the input!

32 minutes ago, _bolek_ said:

Of corse better choice will be Fortinet (FortiGate) for the price but its your choice. Other thing is if you want to get full 10GbX on WAN and push this to LAN then you should consider replace 10GbX port to 40GbX - only then you will get full 10GbX on WAN.

As said in my post, this is the route I recommend most small enterprises go. 

 

Features - check

Routing 10gig - check

Robust interface - check

Entry to Professional level configuration - Check

Support - check

[mynameisjuan]

1 minute ago, iris7 said:

Interesting.  That seems like some pretty extreme system requirements.  Do you have any sort of sources for where I can read more about this.

Check out r/homelabs or r/networking for pfsense configs. There really are not any guidelines for pfsense and its recommended hardware. 

 

For routing ONLY with a single default route you can handle a gig off a single i3 with like 16gb of ram. This is mainly what people use for home networks with only have one route to go (your ISP)

 

Take that same hardware and say you want to add a few ACLs for security or some routing protocols, you might go from 1gig to 50mbps instantly. Layer 3 is very CPU intensive and this is why dedicated hardware with ASIC chips are necessary. 

[iris7]

@mynameisjuan
Thanks for the explanation.  I really only have experience using Cisco hardware when creating and configuring "large" networks with multiple routers and switches.
I have used pfSense as a router/firewall, but I haven't used it to create networks that use VLANs or other routing protocols (like OSPF or VRRP as an example).

[_bolek_]

4 minutes ago, iris7 said:

Interesting.  That seems like some pretty extreme system requirements.  Do you have any sort of sources for where I can read more about this

I have tested this like Lawrence System did and have that same result (but on hardware not VM). When you have big traffic and at that same time you want to filter it, then you need raw power and i mean RAW. Especially on 10GbX. About my point on 40GbX over 10GbX, i got my first problem about 4 yeas ago when client got 1Gbps symmetric. We couldn't get past 700Mbps. When we ask our ISP the said that they provided us fully 1,2Gbps. We got perfect 1Gbps after some twerks on router (i mostly use Mikrotik, Ubiquity, CISCO or PFSense because - and that may be fun CISCO - its more affordable) like changing deviate to one with 10GbX interface. About the source, just read about OSI/ISO MODEL, Ethernet and TCP/UDP protocol design, its explain almost everything in link i provide to tune FreeBSD are little explanation too. Other thing is that you must remember that maybe expect NetBSD and OpenBSD with is mainly developed for network and server purpose and optimalization (and OBSD with security by default), FreeBDS is mainly for general purpose.

 

15 minutes ago, mynameisjuan said:

This is huge. Once you begin dabbling in a bit of everything terminology and configuration will make troubleshooting and proper setup a nightmare. Configuring VLANs on a Nexus 9k is simple, configuring VLANs and trunking on a Mikrotik requires a week or two of practice to fully understand why its configure in the way they developed RouterOS

Configuring VLAN on Mikrotik is simple (2h with doc) but making it work and compatible with other device that take weeks I think problem is in terminology and implementation. To this day i couldn't make work native VLAN on Mikrotik with CISCO, and TRUNK between DELL,HP and CISCO properly but on CISCO to CISCO its working like charm. I don't think they need 9k series but based on what they need NEXUS series and minimum 3000 or above.

 

23 minutes ago, mynameisjuan said:

As said in my post, this is the route I recommend most small enterprises go

Small enterprise don't need more then 1Gbps these days, most don't need more then 200mbps or even less.

[_bolek_]

23 minutes ago, mynameisjuan said:

Check out r/homelabs or r/networking for pfsense configs. There really are not any guidelines for pfsense and its recommended hardware.

You are wrong, there is dedicated hardware, NetGare offering one.