Compicated powerline situation

[ElectroSquid]

Hello dear ltt community. You have helped me in the past, so i am confident that i can get help now to.

 

i have a pretty wired/complicated problem. i live in 2 floor home. on the second floor, i am planning to create a server room. the servers will handle storage, routing and compute.

 

and it is the routing part I'm having a problem with. i need to have a server (repurposed dell Optiplex 330) as my main router. it is equipped with an onboard NIC and and a PCIe NIC. one will handle output and one will handle input, so far so good. but this router needs to be placed on the second floor with the rest of the servers, but my modem and my proprietary modem connector supplied by my ISP Comhem is on the first floor. I need to get an Ethernet connection from the first floor to the second floor (where my Optiplex router will be) and than i need to get an Ethernet wire back down to the first floor, where the living room and most of the Ethernet connected devices are. i thought of using power line adapters, but i realized i needed two of them. and i am on a fairly tight budget (70 dollars) for the Power-line adapters, so i can only afford one. so i wonder if i can run the input and output of the router through one powerline adapter cable (the OS is pf sense or Debian, i have not decided yet)

 

thanks on beforhand

[Sir Asvald]

What type of ISP connection do you have? VSDL, ASDL+ or Cable/fiber? 

 

With a phone line, you can use an Ethernet cable. If you can't do that, why not run an Ethernet cable? I wouldn't recommend a powerline adapter for your router. It's not ideal.

[Ralphred]

What you are looking to achieve is basically called router on a stick. The correct way to do it would be with vlans, but would require a managed switch at the modem to help split the traffic.

It can be done by just using different network addresses for the modem<>router traffic and router<>lan traffic (which you'd have to do using vlans anyway), but you need to be careful about exposing the modem<>router network segment in a way it can be abused, especially if your router is expected to have your actual WAN address.

 

You also talk about servers on the 2nd floor, so effectively you will have to have a different gateway address for the second and first floor clients, though this is easily manageable, and could work out for the best as your DMZ can be a separate network entirely.

 

So, lets assume you have the option to have your router<>modem connection set as a network you decide (as this is the simplest case) and go from there:

Onboard card is for clients on the 2nd floor, we'll call it eth0 and give it 192.168.2.254/24

PCI card is for modem and 1st floor clients, we'll call eth1 and eth1:1 and give it addresses 192.168.1.254/24 and 192.168.3.253/30 respectively

Modem gets 192.168.3.254/30 > this means only eth1:1 and the modem get to talk to each other (layer 2 stuff not withstanding)

2nd floor clients have an address in 192.168.2.0/24 with a gateway of .254

1st floor clients have an address in 192.168.1.0/24 with a gateway of .254

 

If you go with debian, you can now treat eth0, eth1 and eth1:1 within iptables (your firewall/nat/routing software) as three separate interfaces and set it up the same way you would if you had 3 physical interfaces (any normal iptables natting tutorial*** will work, just do it twice so you are connecting both lans to the wan, and add simple rules to allow eth0<>eth1 comms so you can access your servers from the 1st floor) 

You will need a set-up similar to his even if you do have the ability to vlan off 192.168.1.0 and 192.168.3.252 networks (or even just the 192.168.3.252 if the modem can support it) 

 

The only complications I can see are if you need to use pppd* or similar on your router, I don't know how the traffic to initialise/maintain the connection will behave if it's being switched**, everything else mentioned above, I have done at some point, and works.

 

*If you do need to use this, still keep the 192.168.3.252 network in place so you can reach the management interfaces of your modem.

**Technically, if it was connected via a hub at each end of the mains link, it will work, but I've not had time to think about all the security implications...

***Don't make a script, iptables-save is a thing, no need to reinvent the wheel.

[Alex Atkin UK]

6 hours ago, Ralphred said:

What you are looking to achieve is basically called router on a stick. The correct way to do it would be with vlans, but would require a managed switch at the modem to help split the traffic.

It can be done by just using different network addresses for the modem<>router traffic and router<>lan traffic (which you'd have to do using vlans anyway), but you need to be careful about exposing the modem<>router network segment in a way it can be abused, especially if your router is expected to have your actual WAN address.

 

You also talk about servers on the 2nd floor, so effectively you will have to have a different gateway address for the second and first floor clients, though this is easily manageable, and could work out for the best as your DMZ can be a separate network entirely.

 

So, lets assume you have the option to have your router<>modem connection set as a network you decide (as this is the simplest case) and go from there:

Onboard card is for clients on the 2nd floor, we'll call it eth0 and give it 192.168.2.254/24

PCI card is for modem and 1st floor clients, we'll call eth1 and eth1:1 and give it addresses 192.168.1.254/24 and 192.168.3.253/30 respectively

Modem gets 192.168.3.254/30 > this means only eth1:1 and the modem get to talk to each other (layer 2 stuff not withstanding)

2nd floor clients have an address in 192.168.2.0/24 with a gateway of .254

1st floor clients have an address in 192.168.1.0/24 with a gateway of .254

 

If you go with debian, you can now treat eth0, eth1 and eth1:1 within iptables (your firewall/nat/routing software) as three separate interfaces and set it up the same way you would if you had 3 physical interfaces (any normal iptables natting tutorial*** will work, just do it twice so you are connecting both lans to the wan, and add simple rules to allow eth0<>eth1 comms so you can access your servers from the 1st floor) 

You will need a set-up similar to his even if you do have the ability to vlan off 192.168.1.0 and 192.168.3.252 networks (or even just the 192.168.3.252 if the modem can support it) 

 

The only complications I can see are if you need to use pppd* or similar on your router, I don't know how the traffic to initialise/maintain the connection will behave if it's being switched**, everything else mentioned above, I have done at some point, and works.

 

*If you do need to use this, still keep the 192.168.3.252 network in place so you can reach the management interfaces of your modem.

**Technically, if it was connected via a hub at each end of the mains link, it will work, but I've not had time to think about all the security implications...

***Don't make a script, iptables-save is a thing, no need to reinvent the wheel.

A smart/managed switch could probably solve this.

 

Basically you'd just use a single NIC in the router and the switch to send traffic from the port the modem is plugged into down one VLAN to the router, then have the LAN traffic on a different VLAN with the router configured to send that traffic untagged to the rest of the ports.

 

I believe you could then use the second NIC bridged to the LAN VLAN to hook up the rest of the equipment that's in the same room as the router.   I'm not really familiar with VLANs so don't know the specifics.

 

Be aware that if you did it that way then the router would be software switching between the upstairs and downstairs equipment, so you might hit CPU bottlenecks if you are doing large transfers between the two, depending on how powerful the router is.  The only solution to what would be a second managed switch, which would likely be pushing the cost up to similar to having two different powerline kits.  But then I don't think its advised to have more than one powerline network running on the same wiring so this may be the only practical option anyway.

[Ralphred]

23 hours ago, Alex Atkin UK said:

Be aware that if you did it that way then the router would be software switching between the upstairs and downstairs equipment, so you might hit CPU bottlenecks if you are doing large transfers between the two.

Even the dankest 330 has enough power to not even break a sweat routing 2 1gig cards. I have a (now retired) k4-2 that used to do the same, when the cpu fan got too loud I unplugged it, thing just kept humping packets.

[Alex Atkin UK]

On 10/21/2018 at 4:13 PM, Ralphred said:

Even the dankest 330 has enough power to not even break a sweat routing 2 1gig cards. I have a (now retired) k4-2 that used to do the same, when the cpu fan got too loud I unplugged it, thing just kept humping packets.

Good to know, I just hear a lot of people saying you shouldn't use CPU bridging yet I have never seen any negative impact myself either.  Even latency wise, there's literally about 0.1ms difference between it and a hardware Switch.

[Ralphred]

13 hours ago, Alex Atkin UK said:

I just hear a lot of people saying you shouldn't use CPU bridging

Yeah, it's a nuanced subject that people hold too definitive views on. The way I've learned to see it, if you can use an ASIC to hump your packets they way you want, go for it, but if you need a specific version of firmware to do what you need then you're probably doing it in software anyway.

A better way to put it would probably be "you shouldn't use CPU bridging for things you don't have to", even when we are dealing with a massive surplus of CPU power like we are here, that would still be best practice.

 

In this case, the guy is limited by physical connections, if he had a managed switch at each end that can do inter VLAN routing in hardware, then that would offset any performance concerns between upstairs and downstairs networks, or allow for consolidation of those networks, but that's not £s I'd spend until I'd proved my "router" was a bottleneck, or that the hardware I could justify buying would do it better.

 

FYI, I get better performance out of appliances I have built myself, out of old machines, than any "commercial router", where "commercial router" ranges from free rubbish from ISP's to Cisco SME kit, because I do some very odd things.