Jump to content

PFSense with a 10GB Switch

Sentein

I have a PFsense Box, it's been working for years. I am on the fence about buying a bigger 10GB or 40GB switch. I guess the simple question here is, Will PFsense have to deal with any of the internal traffic while using the 10GB switch? I am asking this because i have a bottle neck and i am trying to find it. If PFsense will have to deal with throughput from the internal network that is most likely my bottleneck. I am hoping PFsense just hands out IP addresses and monitors the wan side of things while the switch is supposed to take care of the rest. Any input would be welcome. I am trying to stay away from needing TNSR.

Link to comment
Share on other sites

Link to post
Share on other sites

No, none of your internal traffic will go through the pfSense box other than DHCP requests and such.  Real simple stuff.  My network mostly uses static IPs and I can outright shut down the pfSense box and most of the network continues to functions.  (Though the clients that use DHCP go dark obviously)

Desktop: Ryzen 9 3950X, Asus TUF Gaming X570-Plus, 64GB DDR4, MSI RTX 3080 Gaming X Trio, Creative Sound Blaster AE-7

Gaming PC #2: Ryzen 7 5800X3D, Asus TUF Gaming B550M-Plus, 32GB DDR4, Gigabyte Windforce GTX 1080

Gaming PC #3: Intel i7 4790, Asus B85M-G, 16B DDR3, XFX Radeon R9 390X 8GB

WFH PC: Intel i7 4790, Asus B85M-F, 16GB DDR3, Gigabyte Radeon RX 6400 4GB

UnRAID #1: AMD Ryzen 9 3900X, Asus TUF Gaming B450M-Plus, 64GB DDR4, Radeon HD 5450

UnRAID #2: Intel E5-2603v2, Asus P9X79 LE, 24GB DDR3, Radeon HD 5450

MiniPC: BeeLink SER6 6600H w/ Ryzen 5 6600H, 16GB DDR5 
Windows XP Retro PC: Intel i3 3250, Asus P8B75-M LX, 8GB DDR3, Sapphire Radeon HD 6850, Creative Sound Blaster Audigy

Windows 9X Retro PC: Intel E5800, ASRock 775i65G r2.0, 1GB DDR1, AGP Sapphire Radeon X800 Pro, Creative Sound Blaster Live!

Steam Deck w/ 2TB SSD Upgrade

Link to comment
Share on other sites

Link to post
Share on other sites

Okay thank you for the information. pfSense has a overall upper limit to data transfer and as long as my fiber stuff does not need to go through that box it means i have another issue to track down. I have everything important as static IP as well. 

Link to comment
Share on other sites

Link to post
Share on other sites

It depends on how the network is set up and where things are connected, but out of the box, two computers connected to the same switch will not have to go through your PFSense box. 

Link to comment
Share on other sites

Link to post
Share on other sites

10 hours ago, LAwLz said:

It depends on how the network is set up and where things are connected, but out of the box, two computers connected to the same switch will not have to go through your PFSense box. 

This is what I was going to warn about.  If your PFSense is running as a router on a stick, then yes it can and will bottleneck.  If routing between VLANs occurs on the switch, then there's no bottleneck.

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, jec6613 said:

This is what I was going to warn about.  If your PFSense is running as a router on a stick, then yes it can and will bottleneck.  If routing between VLANs occurs on the switch, then there's no bottleneck.

Or if both devices are on the same VLAN (which they are out of the box) then it won't have to go to the Pfsense box either, even if you're running router-on-a-stick. 

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, LAwLz said:

Or if both devices are on the same VLAN (which they are out of the box) then it won't have to go to the Pfsense box either, even if you're running router-on-a-stick. 

While true OOB, it's not necessarily true.  I can absolutely send all traffic between two devices on the same VLAN through another on a stick device, and do do it on my guest VLAN to prevent any device from talking to any other without hitting the firewall first.

Link to comment
Share on other sites

Link to post
Share on other sites

27 minutes ago, jec6613 said:

While true OOB

Yes, I know you can configure port isolation/private VLANs/<whatever your vendor calls it>, but that's specifically why I said "out of the box" earlier in the thread.

 

In a standard router-on-a-stick config, devices on the same VLAN will not have to go through the default gateway either. What you are saying is only true when you enable port port isolation, and I assume anyone who configures that understand that two devices on the same network can talk to each other without going through a router first.

 

You made it sound like if you have a router-on-a-stick, traffic from the same VLAN will go to the router and not be forwarded in a switch. This is false EXCEPT if you have port isolation enabled. The default behaviour is that, even if you have a router-on-a-stick, traffic between the same VLANs, between hosts in the same network, will not have to be routed. It will be handled by the switch.

 

 

12 hours ago, jec6613 said:

If your PFSense is running as a router on a stick, then yes it can and will bottleneck.

This statement is only true if you have port isolation enabled. You make it sound like router-on-a-stick forces traffic to the firewall. It doesn't.

It's not "router-on-a-stick" that causes intra-VLAN traffic to go to the firewall. Port isolation does.

 

 

12 hours ago, jec6613 said:

If routing between VLANs occurs on the switch, then there's no bottleneck.

This is a nonsense statement because we are not talking about routing here. We are talking about VLANs and switches. No routing needs to happen to send traffic from one host to another on the same VLAN and network UNLESS you have enabled port isolation. 

 

 

I don't think you fully understand what you are talking about and as a result you are bringing up irrelevant details and use the incorrect terminology that will just confuse others reading this.

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, LAwLz said:

Yes, I know you can configure port isolation/private VLANs/<whatever your vendor calls it>, but that's specifically why I said "out of the box" earlier in the thread.

 

In a standard router-on-a-stick config, devices on the same VLAN will not have to go through the default gateway either. What you are saying is only true when you enable port port isolation, and I assume anyone who configures that understand that two devices on the same network can talk to each other without going through a router first.

 

You made it sound like if you have a router-on-a-stick, traffic from the same VLAN will go to the router and not be forwarded in a switch. This is false EXCEPT if you have port isolation enabled. The default behaviour is that, even if you have a router-on-a-stick, traffic between the same VLANs, between hosts in the same network, will not have to be routed. It will be handled by the switch.

 

 

This statement is only true if you have port isolation enabled. You make it sound like router-on-a-stick forces traffic to the firewall. It doesn't.

It's not "router-on-a-stick" that causes intra-VLAN traffic to go to the firewall. Port isolation does.

 

 

This is a nonsense statement because we are not talking about routing here. We are talking about VLANs and switches. No routing needs to happen to send traffic from one host to another on the same VLAN and network UNLESS you have enabled port isolation. 

You're very much confusing intra-VLAN and inter-VLAN (what I called routing between VLANs here).  Port based isolation is not required to enable isolation intra-VLAN, TrendNet, TP-Link, EnGenius, and UBNT switches enable MAC-level isolation by default if you enable L3 routing features on the switches and they are not centrally managed, even when the switch has no interface on the VLAN.

 

And yes, this behavior is a, "Feature," not a, "Bug."

 

Quote

I don't think you fully understand what you are talking about and as a result you are bringing up irrelevant details and use the incorrect terminology that will just confuse others reading this.

I'm bringing up this because I've watched it bite people before more times than I can count on both hands.  This isn't irrelevant, it's a well known, "Watch out for," trait of many of these switches.

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, jec6613 said:

You're very much confusing intra-VLAN and inter-VLAN

No I am not. 

Intra-VLAN = traffic between the same VLAN.

Inter-VLAN = Routing between two different VLANs.

 

If you have a configuration that forces traffic within the same VLAN to go up to your default gateway, then that's some type of port isolation/private VLAN. It is not a characteristic of router-on-a-stick. Router-on-a-stick has absolutely nothing to do with port isolation. The two are completely unrelated technologies.

 

Same thing as I said here:

8 hours ago, LAwLz said:

This statement is only true if you have port isolation enabled. You make it sound like router-on-a-stick forces traffic to the firewall. It doesn't.

It's not "router-on-a-stick" that causes intra-VLAN traffic to go to the firewall. Port isolation does.

 

 

6 hours ago, jec6613 said:

Port based isolation is not required to enable isolation intra-VLAN

Can you please define "isolation intra-VLAN" because that's as far as I am aware a made up term. I want to know what you mean before I reply.

Not sure what "port based isolation" is either. The official terminology for private VLANs and similar features is "port isolation". Port based isolation sounds like it might just be referring to VLANs.

 

6 hours ago, jec6613 said:

TrendNet, TP-Link, EnGenius, and UBNT switches enable MAC-level isolation by default if you enable L3 routing features on the switches and they are not centrally managed, even when the switch has no interface on the VLAN.

Who said anything about turning on L3 routing features on the switch? You don't have to do that. Nobody has even remotely suggested that was going to be done.

I am not even sure the devices you listed actually acts the way you say they do but I will just take your word for it. I think it sounds absolutely stupid to enable port isolation when you enable routing on a switch and I couldn't find anything suggesting it does on Ubiquiti's website but whatever.

 

6 hours ago, jec6613 said:

I'm bringing up this because I've watched it bite people before more times than I can count on both hands.  This isn't irrelevant, it's a well known, "Watch out for," trait of many of these switches.

But then why did you bring up router-on-a-stick if you were talking about enabling L3 features on the switch? That's not what router-on-a-stick means.

This is how the conversation went.

OP: Hey, if I add a switch will traffic go to my PFSense router?

Me: No it won't, out of the box.

You: It will if he runs router-on-a-stick.

Me: No it won't unless you enable port isolation.

You: It does if he has a certain brand of switch and enable certain features and use the switch as his router instead of running his PFSense box as a router-on-a-stick.

 

 

I am honestly at a loss for words because I don't understand how this played out in your head. OP was asking about if traffic will go to this PFSense box and you started talking about running routing on the switch. Even if everything you said was true, it still doesn't make sense because if he does routing on the switch then the traffic still won't go to his PFSense box. It will be routed through his switch.

I don't understand what you are on about and I don't think you know either.

Link to comment
Share on other sites

Link to post
Share on other sites

Not to mention that by default, pfSense wont pass traffic between VLANs or allow any routing between different private subnets.  You'd have to explicitly configure that and at that point I'd expect the person to know the caveats to doing so.

I'd personally argue that it even goes against the whole point of having different VLANs and subnets, to isolate them, if you're then going to route that traffic breaking the isolation.

I've seen people do it to isolate their IoT player from the LAN but want it to access their Plex server.  In that situation IMO you simply want the Plex box on both VLANs or physically wire it to both, rather than bottlenecking on a router.  Then let the firewall on the Plex box deal with blocking access to specific ports.

Router:  Intel N100 (pfSense) WiFi6: Zyxel NWA210AX (1.7Gbit peak at 160Mhz)
WiFi5: Ubiquiti NanoHD OpenWRT (~500Mbit at 80Mhz) Switches: Netgear MS510TXUP, MS510TXPP, GS110EMX
ISPs: Zen Full Fibre 900 (~930Mbit down, 115Mbit up) + Three 5G (~800Mbit down, 115Mbit up)
Upgrading Laptop/Desktop CNVIo WiFi 5 cards to PCIe WiFi6e/7

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×