Network Segmentation and Vlans

[Bdavis]

Over the past few weeks I have been slowly organizing and segmenting my network layout. I feel I have a pretty good layout but what was wondering best practices for separating devices using Vlans.

 

Here's a list of my devices. I use PfSense and ubiquiti managed switches and access points just FYI.

 

-trueNAS server 1 with plex, ubiquiti controller, and home assistant.

-trueNAS server2 (backup in another building via ptp)

- personal desktop

- smart light switches

- ipmi for both servers

- fire TV's

- echo dots

- security cameras

- smart phones

- kids tablets

- printers

- smart thermostats

 

How would you separate all of this? My main issue is the TrueNAS Scale server. I want it to be secure, but it needs to interact with all my smart home stuff for plex and home assistant.

[manikyath]

you dont.

 

segmentation essentially means things cant talk to each other. you can set up pfsense to let things talk to each other trough the firewall, but that beats the purpose of segmenting in the first place.

[heimdali]

1 hour ago, manikyath said:

you dont.

 

segmentation essentially means things cant talk to each other. you can set up pfsense to let things talk to each other trough the firewall, but that beats the purpose of segmenting in the first place.

That's not true.  Using vlans to keep things apart means you can have control over the communication between devices, letting them communicate only along the paths you allow, or not at all.  Not every device needs internet access.

 

@Bdavis

 

So where's the plan of your network you made?  You have a list of possible VLANs so far.

[Needfuldoer]

2 hours ago, heimdali said:

Not every device needs internet access.

You can prevent those devices from accessing the Internet by not setting a default gateway when you assign them a DHCP reservation or static address.

 

Home networking is very easy to overthink and needlessly complicate.

[manikyath]

3 minutes ago, Needfuldoer said:

Home networking is very easy to overthink and needlessly complicate.

as the owner of a home network with 4 vlans.. this.

[Bdavis]

1 hour ago, Needfuldoer said:

You can prevent those devices from accessing the Internet by not setting a default gateway when you assign them a DHCP reservation or static address.

 

Home networking is very easy to overthink and needlessly complicate.

Agreed, maybe I'm making things too complicated. So far I have the lan and 3 vlans. 

- LAN

-Guest VLAN

-IOT VLAN

- Management VLAN

I guess the main issue is my TrueNAS Scale Server. I know with TrueNAS core you could assign a network interface to a jail to keep things separate, but I haven't figured out how to do this in Scale.

[itsparks]

54 minutes ago, Bdavis said:

Agreed, maybe I'm making things too complicated. So far I have the lan and 3 vlans. 

- LAN

-Guest VLAN

-IOT VLAN

- Management VLAN

I guess the main issue is my TrueNAS Scale Server. I know with TrueNAS core you could assign a network interface to a jail to keep things separate, but I haven't figured out how to do this in Scale.

This is similar to what i have, I also have a Security vlan for cameras & the BlueIris Box.  I run OPNsense though, not Pfsense. 

[wseaton]

6 hours ago, heimdali said:

That's not true.  Using vlans to keep things apart means you can have control over the communication between devices, letting them communicate only along the paths you allow, or not at all.  Not every device needs internet access.

 

@Bdavis

 

So where's the plan of your network you made?  You have a list of possible VLANs so far.

Dude, vertical network segmentation isn't security, and pretty much every mid size company I've gone into to scrub Ransomware off of has VLANs all over the place.

 

Managing traffic for the sake of managing traffic doesn't improve your network. It won't make things run better. Ethernet is rolling traffic accident anyways, and aside from some nasty video protocols (Crestron) there's no need to filter things. If you still have NetBEUI running might be time to shut down those NT 4 SP1 boxes.

 

I agree about internet access. If the end point device doesn't need to talk to the internet it doesn't need access. However, that is much easier achieved via tossing those devices in a group on your firewall.  Turning your switches into spaghetti just makes a mess. 

[itsparks]

5 minutes ago, wseaton said:

Dude, vertical network segmentation isn't security, and pretty much every mid size company I've gone into to scrub Ransomware off of has VLANs all over the place.

 

Managing traffic for the sake of managing traffic doesn't improve your network. It won't make things run better. Ethernet is rolling traffic accident anyways, and aside from some nasty video protocols (Crestron) there's no need to filter things. If you still have NetBEUI running might be time to shut down those NT 4 SP1 boxes.

 

I agree about internet access. If the end point device doesn't need to talk to the internet it doesn't need access. However, that is much easier achieved via tossing those devices in a group on your firewall.  Turning your switches into spaghetti just makes a mess. 

So throwing all the video cameras and phones and printers & iot devices on one single /24 networks is perfectly fine ? 

[LIGISTX]

4 minutes ago, wseaton said:

Dude, vertical network segmentation isn't security, and pretty much every mid size company I've gone into to scrub Ransomware off of has VLANs all over the place

How is it not security?

 

I run a simplistic homelab with multiple vlans, 1 for guests with no access to my infrastructure, NAS, etc, a IoT vlan with no access to private subnets with the exception of a few holes punched for specific ports for specific devices for things like Plex streaming or ubiquity queries from home assistant (which lives in IoT land), then I have my privilege vlans for my Pc, laptop, phone etc which can freely talk to my homelab vlan which has my VM’s and such. Control surfaces are all on a higher level management vlan (which, I have not actually segmented out effectively at this point, convince>security sometimes.). My private subnet can hit the management subnet… but IoT, homelab, and guest can not.

 

Lets assume for a second I actually took security a little more seriously and did restrict the management interface to either a VPN connection (so that would require a user interaction and authentication), or a physical different port on my router to then control the management surface. Your going to have trouble explaining to me how this “is not security”. 
 

This would more or less fully protect me from random ransomeware seeing as if they start encrypting my truenas box, I can just revert to a previous snapshot… and there wouldn’t have been a way for said ransomeware to gain access to truenas and either delete the snapshots, or change the password to the box, etc. SSH and webUI are only bound to the management subnet... SMB and NFS are relegated to the lower level subnets. 
 

Would this protect against a nation state level bad actor… no. Of course not. But if I end up with physical human beings trying to gain entry into my home network, something very bad happened and that is likely the least of my concerns. 

[LIGISTX]

9 minutes ago, itsparks said:

So throwing all the video cameras and phones and printers & iot devices on one single /24 networks is perfectly fine ? 

Haha, sounds like it’s a great idea . 
 

I think @wseatonis neglecting to point out… everything helps, but no one thing is the silver bullet. If this is all you do, no, you have failed at opsec. But if you don’t include vlans as part of your tools, you also failed at opsec. 

[heimdali]

4 hours ago, Needfuldoer said:

You can prevent those devices from accessing the Internet by not setting a default gateway when you assign them a DHCP reservation or static address.

That is insecure.  And why would I want all devices to be able to communicate which other?  That isn't secure, either.

4 hours ago, Needfuldoer said:

Home networking is very easy to overthink and needlessly complicate.

Either do it right or don't do it at all.

 

[heimdali]

28 minutes ago, wseaton said:

Dude, vertical network segmentation isn't security, and pretty much every mid size company I've gone into to scrub Ransomware off of has VLANs all over the place.

So what do you suggest?  Run separate cabling everywhere?

[itsparks]

23 minutes ago, heimdali said:

So what do you suggest?  Run separate cabling everywhere?

My iphone 15 has usb c with gigabit ethernet, but im tied to my couch, if i need to go to the kitchen i just use a longer cable.

[Lurick]

1 hour ago, wseaton said:

I agree about internet access. If the end point device doesn't need to talk to the internet it doesn't need access. However, that is much easier achieved via tossing those devices in a group on your firewall.  Turning your switches into spaghetti just makes a mess. 

Ah yes they old dynamic device grouping available on no consumer firewall ever because everyone is running Foritnet or Palo with XDR microsegmentation on their network.....

 

If only there was something that was which would allow you to group devices, it might not be as dynamic but hmmm perhaps they'll call it Virtual Local Area Network or something, maybe VLANs!

[LIGISTX]

1 hour ago, Lurick said:

Ah yes they old dynamic device grouping available on no consumer firewall ever because everyone is running Foritnet or Palo with XDR microsegmentation on their network.....

 

If only there was something that was which would allow you to group devices, it might not be as dynamic but hmmm perhaps they'll call it Virtual Local Area Network or something, maybe VLANs!

I loled at this   

 

Vlans, such a novel idea!!! Someone get this man an award.

 

Oh, wait, what’s that? This has already existed and been a solved problem for decades?! Shit. Sorry, no trophy for @Lurick

 

Im still waiting for @wseatonto explain to me (us) why vlans are a bad idea and don’t improve security. 

[LIGISTX]

11 hours ago, Bdavis said:

Over the past few weeks I have been slowly organizing and segmenting my network layout. I feel I have a pretty good layout but what was wondering best practices for separating devices using Vlans.

 

Here's a list of my devices. I use PfSense and ubiquiti managed switches and access points just FYI.

 

-trueNAS server 1 with plex, ubiquiti controller, and home assistant.

-trueNAS server2 (backup in another building via ptp)

- personal desktop

- smart light switches

- ipmi for both servers

- fire TV's

- echo dots

- security cameras

- smart phones

- kids tablets

- printers

- smart thermostats

 

How would you separate all of this? My main issue is the TrueNAS Scale server. I want it to be secure, but it needs to interact with all my smart home stuff for plex and home assistant.

But back to actually helping OP…

 

You definitely want a VLAN for IoT devices. Stick them all on a subnet with no access to any other subnet. If you need to punch holes for things like Plex, or use avahi for mDNS translation across the subnets, go ahead and do that. I have my network set up this way, and I can chromecast from my private subnet just fine, and Plex clients and talk to my Plex server via firewall rules for specific ports and specific IP’s. 
 

I would then have a subnet just for management surfaces (pfsense webui, NAS webui (assuming your NAS itself isn’t a garbage tier IoT device itself and is actually secure), networking management like UniFi controller, and hypervisor for virtualization environments. Don’t let anything talk to this subnet except for what is required to do management. Ideally, it would be only accessible via enabling a VPN or via plugging in to a specific port on your firewall, but this makes it such a pain to use, so, that’s a hard one. But definitely don’t allow IoT devices touch the management subnet. 
 

This would be a good way to get you going in the right direction. I also have a homelab subnet just for all my VM’s and docker containers etc.

[Bdavis]

8 hours ago, LIGISTX said:

But back to actually helping OP…

 

You definitely want a VLAN for IoT devices. Stick them all on a subnet with no access to any other subnet. If you need to punch holes for things like Plex, or use avahi for mDNS translation across the subnets, go ahead and do that. I have my network set up this way, and I can chromecast from my private subnet just fine, and Plex clients and talk to my Plex server via firewall rules for specific ports and specific IP’s. 
 

I would then have a subnet just for management surfaces (pfsense webui, NAS webui (assuming your NAS itself isn’t a garbage tier IoT device itself and is actually secure), networking management like UniFi controller, and hypervisor for virtualization environments. Don’t let anything talk to this subnet except for what is required to do management. Ideally, it would be only accessible via enabling a VPN or via plugging in to a specific port on your firewall, but this makes it such a pain to use, so, that’s a hard one. But definitely don’t allow IoT devices touch the management subnet. 
 

This would be a good way to get you going in the right direction. I also have a homelab subnet just for all my VM’s and docker containers etc.

Thanks for the info. Now it's time to research avahi and firewall rules. My NAS is home built running TrueNAS Scale with a supermicro server board with IPMI and ECC ram.

[LIGISTX]

1 hour ago, Bdavis said:

Thanks for the info. Now it's time to research avahi and firewall rules. My NAS is home built running TrueNAS Scale with a supermicro server board with IPMI and ECC ram.

Avahi is just a pfsense plugin, download it, tell it which subnets to work on, and boom, chromecast and stuff will work perfectly.

 

IPMI should be on the management subnet as well. Everything that can change security features or settings, can change how things boot, can access BIOS, etc should be in the management subnet. Truenas WebUI (and if enabled SSH access) should be there as well, and only there. Bind those only to the management subnet, and bind SMB and such to whatever other subnet. I have truenas running virtual under proxmox, so I give truenas 2 virtual NIC’s, one gets no vlan tag (so it gets the full on trunk which is the management subnet) and the other gets my homelab vlan tag, managed in proxmox obviously.… which is also “obvious” on the trunk which again is the management subnet. 
 

Also, make sure you have ZFS snapshots on… this is your best protection against accidental user error but also ransomeware. Snapshots are read only except via truenas itself, so no SMB or NFS user can alter snapshots. So if something starts encrypting your data… you can just go into truenas webui after you fix whatever ransomeware issue you have, and revert the data back. It’ll be as if nothing bad every happened

[heimdali]

2 hours ago, LIGISTX said:

IPMI should be on the management subnet as well.

Do you even need IPMI on a server that's in your home and not somewhere remote?  I have two having IPMI, but the functions it offers aren't useful at all --- maybe because these boards are pretty old.  If access to IPMI is needed, you can always plug it into the management vlan.

 

Make sure you don't get locked out of your devices when adjusting them to the management vlan ...

 

2 hours ago, LIGISTX said:

Also, make sure you have ZFS snapshots on… this is your best protection against accidental user error but also ransomeware. Snapshots are read only except via truenas itself, so no SMB or NFS user can alter snapshots. So if something starts encrypting your data… you can just go into truenas webui after you fix whatever ransomeware issue you have, and revert the data back. It’ll be as if nothing bad every happened

Does it automatically take a new snapshot every 5 minutes or so?  Shouldn't you rather make a copy from the snapshot so you still have it in case your data gets encrypted again?

 

[LIGISTX]

7 hours ago, heimdali said:

Does it automatically take a new snapshot every 5 minutes or so?  Shouldn't you rather make a copy from the snapshot so you still have it in case your data gets encrypted again?

 

You can set them up on any schedule you want, and as many schedules as you want, on a directory by directory or all up dataset basis. 
 

ZFS snapshots are just block level deltas,  so if no data ever changes, they are effectively free and take up no space at all. As the directory changes, only the delta is used. Snapshots are forever read only, and can only be deleted via webUI or SSH - they are a function of ZFS itself, not special to truenas at all. 
 

So I have multiple snapshots set up and different setups based on data. For my personal data, I take a snapshot every 10 minutes and store that for 12 hours, I take one every hour and store that for a day, take one every 6 hours and store it for a week etc all the way up to 6 months of retention. So I can revert to any of those places on time, or I can create a new dataset FROM one of those past times to just grab a single file that was deleted for example, restore it, then delete that newly created dataset (the ZFS snapshot will remain, the dataset is just a SMB accessible way to go and retrieve said file). 
 

This is the power of ZFS… you can also use snapshots for ZFS replication tasks. So if you have a remote server that you backup to, you can backup snapshots via ZFS’s built in replication mechanism as well: https://www.google.com/amp/s/klarasystems.com/articles/introduction-to-zfs-replication/%3Famp

 

I am telling you…. ZFS really is the best file system out there, which is why you should grab yourself a LSI HBA and switch to it

[Bdavis]

12 hours ago, LIGISTX said:

Avahi is just a pfsense plugin, download it, tell it which subnets to work on, and boom, chromecast and stuff will work perfectly.

 

IPMI should be on the management subnet as well. Everything that can change security features or settings, can change how things boot, can access BIOS, etc should be in the management subnet. Truenas WebUI (and if enabled SSH access) should be there as well, and only there. Bind those only to the management subnet, and bind SMB and such to whatever other subnet. I have truenas running virtual under proxmox, so I give truenas 2 virtual NIC’s, one gets no vlan tag (so it gets the full on trunk which is the management subnet) and the other gets my homelab vlan tag, managed in proxmox obviously.… which is also “obvious” on the trunk which again is the management subnet. 
 

Also, make sure you have ZFS snapshots on… this is your best protection against accidental user error but also ransomeware. Snapshots are read only except via truenas itself, so no SMB or NFS user can alter snapshots. So if something starts encrypting your data… you can just go into truenas webui after you fix whatever ransomeware issue you have, and revert the data back. It’ll be as if nothing bad every happened

I have nightly snapshots that back up to another TrueNAS server at my brother's house across the street via a ptp link. I'll install avahi as soon as I'm back from my work trip.

[Bdavis]

9 hours ago, heimdali said:

Do you even need IPMI on a server that's in your home and not somewhere remote?  I have two having IPMI, but the functions it offers aren't useful at all --- maybe because these boards are pretty old.  If access to IPMI is needed, you can always plug it into the management vlan.

 

Make sure you don't get locked out of your devices when adjusting them to the management vlan ...

 

Does it automatically take a new snapshot every 5 minutes or so?  Shouldn't you rather make a copy from the snapshot so you still have it in case your data gets encrypted again?

 

IPMI isn't needed, but it's nice when the server has an issue and I suddenly can't access the GUI. I plan to move the TrueNAS box to a shelf above my stairwell so my toddler will stop turning it off a couple times a week. After having IPMI. On my NAS I don't think I'll ever build one without it.

[Bdavis]

On 11/18/2022 at 2:06 AM, manikyath said:

you dont.

 

segmentation essentially means things cant talk to each other. you can set up pfsense to let things talk to each other trough the firewall, but that beats the purpose of segmenting in the first place.

That is exactly what I'm trying to achieve. I only want devices to be able to talk to the devices and resources on the network that the need to. My main concern is my IOT devices. My brother told me the other day, "The "S" in IOT stands for security"

[LIGISTX]

50 minutes ago, Bdavis said:

I have nightly snapshots that back up to another TrueNAS server at my brother's house across the street via a ptp link. I'll install avahi as soon as I'm back from my work trip.

I’d take snapshots more often, they don’t cost anything so might as well, and what do you mean by a ptp link exactly? If you both have homelabs (sounds like you do), set up a point to point VPN tunnel and get both truenas boxes on it as an additional vlan, and do ZFS replication across that link. 

 

38 minutes ago, Bdavis said:

"The "S" in IOT stands for security"

Haha, true story. Get them on their own vlan, both wired and wireless. My IoT stuff has its own SSID, and that SSID is obviously part of the IoT vlan subnet. I have 3 different SSID’s using a single UniFi Wifi 6 AP.