Building a home pfSense router

[KrYpToCiD]

Hello there. I want to build a router for my new 1Gb internet plan provided not long ago by my ISP. The problem is that they provided a crappy router is weak in every possible way. From time to time, I need to restart it and a friend of mine actually got passed it's firewall in minutes (he is working as technician at a local firm and knows a bit of cyber security and yeah, we alerted the manufacture but in response we found out the fact that my ISP is writing the Firmware on the units. I've called them and after 4 weeks no firmware update, nothing showed up). And to make the things worst the performance of the router is more than poor.

 

Now, I want to build a pfSense router due the fact that I can play with it  and anyway i need to spend those money on a third-party wireless router due the fact that the WAN line is fiber and I've got my hands on SFP+ NIC so right now i need only the system. I've watched Linus's router build video for too many times but I still have a question. He said that it was overkill and from my understanding, all what it has overkill there is the PSU and Mobo+CPU combo. Also, by doing some research I've found out that pfSense doesn't scale that much over many cores and it is worth to go with higher frequency instead. Now here is my problem: I want to use an VPN while I'm away due the fact that I live in a pretty low-free internet paradise and I want to use the 1Gb down-speed (I'm kinda greedy  this service is really cheap, less than $8 USD tax included ) but that would require more power from my CPU and also I want the system to be as low power and smaller as it can. So, long story short, I can't make my mind on the CPU I should use. 

 

Anyone has any ideea or advice?

 

I should mention that I'll have on the network 1 workstation, 1 NAS, 1 AIO pc for web browsing, and maybe 3 other machines. It is kinda stupid but I'll keep my archer wifi router with the DHCP on due the fact that I want to keep phones and guests isolated from my main network.

 

Thank you in advance for your response.

Best regards, Kryptocid.

[AngryBeaver]

14 minutes ago, KrYpToCiD said:

Hello there. I want to build a router for my new 1Gb internet plan provided not long ago by my ISP. The problem is that they provided a crappy router is weak in every possible way. From time to time, I need to restart it and a friend of mine actually got passed it's firewall in minutes (he is working as technician at a local firm and knows a bit of cyber security and yeah, we alerted the manufacture but in response we found out the fact that my ISP is writing the Firmware on the units. I've called them and after 4 weeks no firmware update, nothing showed up). And to make the things worst the performance of the router is more than poor.

 

Now, I want to build a pfSense router due the fact that I can play with it  and anyway i need to spend those money on a third-party wireless router due the fact that the WAN line is fiber and I've got my hands on SFP+ NIC so right now i need only the system. I've watched Linus's router build video for too many times but I still have a question. He said that it was overkill and from my understanding, all what it has overkill there is the PSU and Mobo+CPU combo. Also, by doing some research I've found out that pfSense doesn't scale that much over many cores and it is worth to go with higher frequency instead. Now here is my problem: I want to use an VPN while I'm away due the fact that I live in a pretty low-free internet paradise and I want to use the 1Gb down-speed (I'm kinda greedy  this service is really cheap, less than $8 USD tax included ) but that would require more power from my CPU and also I want the system to be as low power and smaller as it can. So, long story short, I can't make my mind on the CPU I should use. 

 

Anyone has any ideea or advice?

 

I should mention that I'll have on the network 1 workstation, 1 NAS, 1 AIO pc for web browsing, and maybe 3 other machines. It is kinda stupid but I'll keep my archer wifi router with the DHCP on due the fact that I want to keep phones and guests isolated from my main network.

 

Thank you in advance for your response.

Best regards, Kryptocid.

Just purchased yourself a nice router instead of going through all this hassle. Call your ISP and tell them you want the Modem, Router, or gateway to be placed in passthrough mode. That will insure it only acts as a modem and you can pass all of that straight to a good router/security solution

[Windows7ge]

If I'm not mistaken you need a modem you can't just plug the fiber line into the pfSense box. You can buy your own modem and disable functions such as DHCP and setup bridged mode then plug that into the pfSense box. From there you can setup interfaces and everything else.

 

You should also know that a SFP+ card is overkill. SFP would allow full utilization of your 1Gbit internet connection. However needing a modem you'd likely be plugging a Ethernet cable into your router anyways unless you got a modem that supports SFP ports.

[KrYpToCiD]

1 minute ago, AngryBeaver said:

Just purchased yourself a nice router instead of going through all this hassle. Call your ISP and tell them you want the Modem, Router, or gateway to be placed in passthrough mode. That will insure it only acts as a modem and you can pass all of that straight to a good router/security solution

Yeah, I forgot to mention..... here I have the low price internet but I pay a lot more on network devices. As an example, you can buy at $600 an router launched in 2011, here, by getting a dedicated router or switch for home use is more than rare....... most of the people are going to buy first wifi router that they can find and fit their needs and change it every year or so.

[Just.Oblivious]

It all depends on how your ISP delivers its service. Does the fiber line go directly from the wall into the router, or does it go through a media converter with ethernet output first?

 

VPN, as in connecting to a public VPN provider? 99% of the public VPN providers use OpenVPN, which uses a hell of a lot of CPU resources. Pushing 200+ Mbps over OpenVPN is a challenge (think Core i3/i5 server with AES-NI support and no other services running). If you want VPN just for making remote connections to your home network you can use a less resource-intensive protocol like L2TP/IPSEC.

 

If you already have a server running (or if you plan to get a home server) you should consider building a virtual router, that way you can share resources and save power.

[KrYpToCiD]

35 minutes ago, Windows7ge said:

If I'm not mistaken you need a modem you can't just plug the fiber line into the pfSense box. You can buy your own modem and disable functions such as DHCP and setup bridged mode then plug that into the pfSense box. From there you can setup interfaces and everything else.

 

You should also know that a SFP+ card is overkill. SFP would allow full utilization of your 1Gbit internet connection. However needing a modem you'd likely be plugging a Ethernet cable into your router anyways unless you got a modem that supports SFP ports.

 If I'm right there is something like an adapter for fiber to SFP. Also, I don't know really how or why, but all the network here is running on fiber and/ or CAT7.... so you can get away without any modem. This is why I want SFP as WAN port  

 

EDIT: I've seen one of those fiber to SFP adapters on LTT channel in one of the episodes from moving blog .

[KrYpToCiD]

2 minutes ago, Just.Oblivious said:

It all depends on how your ISP delivers its service. Does the fiber line go directly from the wall into the router, or does it go through a media converter with ethernet output first?

 

VPN, as in connecting to a public VPN provider? 99% of the public VPN providers use OpenVPN, which uses a hell of a lot of CPU resources. Pushing 200+ Mbps over OpenVPN is a challenge (think Core i3/i5 server with AES-NI support and no other services running). If you want VPN just for making remote connections to your home network you can use a less resource-intensive protocol like L2TP/IPSEC.

 

If you already have a server running (or if you plan to get a home server) you should consider building a virtual router, that way you can share resources and save power.

There is no modem require, you get a cable long enough to reach to the furthest point of your house or flat. 

 

I mean personal use VPN, and for using it only for hollydays abroad, and most of the time there won't be speeds like that. The higher speed that I've get was around 60-70 Mbps so there will be no need for speeds higher than 100Mbps. All the idea with this is to get my ass covered against getting a ticket or being taken into court for my "good" habit to download torrents or accessing sites available only for my country.

 

I've taught about your idea with running pfSense as VM but the security is weaker than a crappy wifi router due the fact that is easy to go arount the firewall due the fact that you are tunneling the port/ports and this meant only to divide two machines, to make them not interfere, not to secure them apart.

[Windows7ge]

10 minutes ago, KrYpToCiD said:

 If I'm right there is something like an adapter for fiber to SFP. Also, I don't know really how or why, but all the network here is running on fiber and/ or CAT7.... so you can get away without any modem. This is why I want SFP as WAN port  

I considered the possibility public networking works differently in different areas. In my area with coax cable you need a modem before your pfSense box.

 

The connector type is called a transceiver and varies according to the connector you're working with. What I see most commonly for home installations is single strand fiber cable. I don't know if a transceiver exists for this. Now if you have a LC cable coming into your home these are very common on transceivers.

[AngryBeaver]

OP, Do you not have the option to purchase a cheap router that is compatible with OpenWRT? I mean a dedicated router is going to be your best option at this point. Yes you can build a cheap PC to do it, but then user and implementation errors become much more prevalent and will leave your network completely vulnerable.

[Just.Oblivious]

8 minutes ago, KrYpToCiD said:

There is no modem require, you get a cable long enough to reach to the furthest point of your house or flat. 

What type of cable is it that you get from them? There are a few dozen flavors of optical fiber cable, the SFP module you select has to be compatible with the technology used by your ISP.

 

3 minutes ago, KrYpToCiD said:

I've taught about your idea with running pfSense as VM but the security is weaker than a crappy wifi router due the fact that is easy to go arount the firewall due the fact that you are tunneling the port/ports and this meant only to divide two machines, to make them not interfere, not to secure them apart.

Not if you do it properly... Virtualizing routers, firewalls and security appliances is common practice in big enterprises. Breaking out of a virtual machine is a few hundred times more difficult than pwning a cheap-ass home router, believe me.

 

Of course you should use a proper hypervisor and segment the interfaces, but that's a given with virtualization.

 

Another solution is to get a hardware firewall with your virtual routing appliance behind it.

 

As far as VPN performance goes, 100 Mbps should be doable with something like a Celeron-based Intel NUC.

[KrYpToCiD]

5 minutes ago, Windows7ge said:

I considered the possibility public networking works differently in different areas. In my area with coax cable you need a modem before your pfSense box.

 

The connector type is called a transceiver and varies according to the connector you're working with. What I see most commonly for home installations is single strand fiber cable. I don't know if a transceiver exists for this. Now if you have a LC cable coming into your home these are very common on transceivers.

They can be used with a single fiber cable. I've asked a person that I know from Cisco and he have some spares that can be used like that. But we called adapters, witch are a fairly loose term for my language.

[Windows7ge]

2 minutes ago, KrYpToCiD said:

They can be used with a single fiber cable. I've asked a person that I know from Cisco and he have some spares that can be used like that. But we called adapters, witch are a fairly loose term for my language.

Then you're all set. Are you just looking to where you can get one of these transceivers?

 

Are you also asking what hardware you should put in the box?

[KrYpToCiD]

5 minutes ago, Just.Oblivious said:

What type of cable is it that you get from them? There are a few dozen flavors of optical fiber cable, the SFP module you select has to be compatible with the technology used by your ISP.

 

Not if you do it properly... Virtualizing routers, firewalls and security appliances is common practice in big enterprises. Breaking out of a virtual machine is a few hundred times more difficult than pwning a cheap-ass home router, believe me.

 

Of course you should use a proper hypervisor and segment the interfaces, but that's a given with virtualization.

 

Another solution is to get a hardware firewall with your virtual routing appliance behind it.

 

As far as VPN performance goes, 100 Mbps should be doable with something like a Celeron-based Intel NUC.

My ISP use a GPON fiber cable  

 

And with the enterprises running routers as vm, you made me turn around and reconsider this idea ( i might do it if I'll have enough PCIE lanes )

About hardware firewall..... i might run a cheap ass machine as bridge to my router and i might  keep my money for further upgrades  

[KrYpToCiD]

7 minutes ago, Windows7ge said:

Then you're all set. Are you just looking to where you can get one of these transceivers?

 

Are you also asking what hardware you should put in the box?

Well. The question was what CPU should I use for this project  @Just.Oblivious gave me a pretty good idea but any other input is welcomed :))

[Windows7ge]

20 minutes ago, KrYpToCiD said:

Well. The question was what CPU should I use for this project  @Just.Oblivious gave me a pretty good idea but any other input is welcomed :))

I have future plans to build a pfSense box. I'm planning to use the ASRock C2750D4I. It's not the cheapest but it has an 8 core BGA CPU and 4 RAM slots in a ITX form factor. The system draws VERY little power. There's a 4 core version C2550D4I.

 

Both only have 1 PCI_ex8 slot so expansion is limited.

[KrYpToCiD]

13 minutes ago, Windows7ge said:

I have future plans to build a pfSense box. I'm planning to use the ASRock C2750D4I. It's not the cheapest but it has an 8 core BGA CPU and 4 RAM slots in a ITX form factor. The system draws VERY little power. There's a 4 core version C2550D4I.

 

Both only have 1 PCI_ex8 slot so expansion is limited.

Well... I've already explored that option but in my country those are very rare and to buy from other place, i would be forced to pay another 25% of the price tax only to be able to get it thru customs.

[Windows7ge]

4 minutes ago, KrYpToCiD said:

Well... I've already explored that option but in my country those are very rare and to buy from other place, i would be forced to pay another 25% of the price tax only to be able to get it thru customs.

It's only a suggestion. If this is going to manage your whole network it's a good idea to use server grade hardware. You can look into hardware from Supermicro. Pick yourself up some server ECC memory & a xeon. Don't use desktop parts. If the router goes offline because of half assed hardware your whole network is down indefinitely.

[KrYpToCiD]

1 minute ago, Windows7ge said:

It's only a suggestion. If this is going to manage your whole network it's a good idea to use server grade hardware. You can look into hardware from Supermicro. Pick yourself up some server ECC memory & a xeon. Don't use desktop parts. If the router goes offline because of half assed hardware your whole network is down indefinitely.

Yeah, I'm aware of that. To be ore on point: It will work properly with a 4c/8t xeon at 2.4GHz or i need to go bigger? That was my question. I've got a good deal on one with mobo combo :)))

[Windows7ge]

1 minute ago, KrYpToCiD said:

Yeah, I'm aware of that. To be ore on point: It will work properly with a 4c/8t xeon at 2.4GHz or i need to go bigger? That was my question. I've got a good deal on one with mobo combo :)))

For a router that's plenty of power. Most consumer routers aren't that strong. 

[Just.Oblivious]

Another option is to ditch the idea of a x86 pc-based router. If you separate the VPN load from the routing/firewall tasks you can get by with a good low-power ethernet+SFP router (like the Ubiquiti EdgeRouter 4). Just run the VPN client on the desktop (or in the future on a VM) and you're good.

 

BTW you absolutely don't need server-grade hardware for running a home-router, just keep the ISP router as a cold spare in case something fails...

 

The only case where I would consider server hardware (and ECC memory) at home is for a storage server. If your stuff is mission critical you should always setup two boxes with automatic failover, don't forget a UPS-system with a generator out back and a second internet line, you know, just in case your cat blog goes offline  

[KrYpToCiD]

21 hours ago, Just.Oblivious said:

BTW you absolutely don't need server-grade hardware for running a home-router, just keep the ISP router as a cold spare in case something fails...

 

The only case where I would consider server hardware (and ECC memory) at home is for a storage server. If your stuff is mission critical you should always setup two boxes with automatic failover, don't forget a UPS-system with a generator out back and a second internet line, you know, just in case your cat blog goes offline  

My things are not that mission critical :))) are important but not that important  Besides I had a NAS box that died on me due the fact that one of my friends fried my board and damaged my Xeon CPU and still now I don't get how that was even possible. I'm have 2 SSD in Raid 0 for the sole purpose of cashing data so i should be in very unlucky time to have power outage and I usually keep a save of project or file until I can be sure that it was saved in the main pool  . 

 

I will keep the ISP router as a spare and, even if it hurts me, I use it to connect ,without wifi enabled, my workstation because my 4G over USB from phone is a bit to slow for the project uploading. 

21 hours ago, Just.Oblivious said:

Another option is to ditch the idea of a x86 pc-based router. If you separate the VPN load from the routing/firewall tasks you can get by with a good low-power ethernet+SFP router (like the Ubiquiti EdgeRouter 4). Just run the VPN client on the desktop (or in the future on a VM) and you're good.

 

That is quite a good idea. Finally, I've got from asking what CPU should I use for router to getting another solutions for my problem  

[Alex Atkin UK]

I'd still go for x86 personally (I have) as the latency and flexibility will be much better than most if not all consumer routers.  They tend to bottleneck on the WiFi alone, which is why I still use one just to handle the WiFi (pfSense is not great for WiFi as Linux has far better support than FreeBSD).

More importantly, you can put extensive blocklists into the router (see pfBlockerNG) to help prevent malware and network intrusion attempts.  I have my firewall specifically set to only allow traffic to my NAS, web server, chat server and VPN from countries I know I might use it from, reducing the potential for compromise.  You can also direct certain clients over different VPNs, useful for torrents or accessing region locked services.  DNS can be setup to use Cloudflare SSL so your ISP cannot snoop/alter DNS.

 

Many of these things can be done on OpenWRT but consumer hardware wouldn't be powerful enough to do it on a fast connection.

 

I recently upgraded my pfSense box from an Atom DN2800 to an i5 3470T as I already had the case, motherboard, RAM, etc going spare from a previous upgrade.  Its overkill for my workload, but it means if I upgrade to faster broadband later on it will easily handle it, plus I needed AES-NI support for compatibility with the next major pfSense release.
 

Also remember that if the OpenVPN server is on your router then clients will be fully integrated into the LAN as the router is already your default route for the whole LAN.  This is a huge advantage over having to fiddle with your VPN server and a consumer router that might not have great support for adding manual routes.

Yes power consumption its higher than a consumer router, but if you factor in flexibility, ease of use, the potential for having to upgrade that router again later if you find its reached capacity, and of course the greater security options, it pays for itself.

 

The only place OpenWRT wins over pfSense (other than power consumption on consumer routers) is it currently has a better method to prevent bufferbloat, but I'm sure pfSense will catch up eventually.  Its also likely to be a none-issue if you have gigabit broadband.

[KrYpToCiD]

18 hours ago, Alex Atkin UK said:

I'd still go for x86 personally (I have) as the latency and flexibility will be much better than most if not all consumer routers.  They tend to bottleneck on the WiFi alone, which is why I still use one just to handle the WiFi (pfSense is not great for WiFi as Linux has far better support than FreeBSD).

More importantly, you can put extensive blocklists into the router (see pfBlockerNG) to help prevent malware and network intrusion attempts.  I have my firewall specifically set to only allow traffic to my NAS, web server, chat server and VPN from countries I know I might use it from, reducing the potential for compromise.  You can also direct certain clients over different VPNs, useful for torrents or accessing region locked services.  DNS can be setup to use Cloudflare SSL so your ISP cannot snoop/alter DNS.

 

Many of these things can be done on OpenWRT but consumer hardware wouldn't be powerful enough to do it on a fast connection.

 

I recently upgraded my pfSense box from an Atom DN2800 to an i5 3470T as I already had the case, motherboard, RAM, etc going spare from a previous upgrade.  Its overkill for my workload, but it means if I upgrade to faster broadband later on it will easily handle it, plus I needed AES-NI support for compatibility with the next major pfSense release.
 

Also remember that if the OpenVPN server is on your router then clients will be fully integrated into the LAN as the router is already your default route for the whole LAN.  This is a huge advantage over having to fiddle with your VPN server and a consumer router that might not have great support for adding manual routes.

Yes power consumption its higher than a consumer router, but if you factor in flexibility, ease of use, the potential for having to upgrade that router again later if you find its reached capacity, and of course the greater security options, it pays for itself.

 

The only place OpenWRT wins over pfSense (other than power consumption on consumer routers) is it currently has a better method to prevent bufferbloat, but I'm sure pfSense will catch up eventually.  Its also likely to be a none-issue if you have gigabit broadband.

Yeah, that's what I've had in my mind from the beginning  right now I was looking for some i5 chips, but I haven't found yet, one that is low power. Those are pretty rare. I've found on Amazon a motherboard that looks really nice with 6 gigabit LAN ports but they require skylake or kabylake processors...... there were some nITX motherboards that were low power but they had some J1900 chipset without AES-NI so that was a pass from me. Right now, I'm back at searching for low power i5  

[Falconevo]

I have put similar in to an office recently, their budget was minimal but needed  close to 1G WAN<>LAN throughput and high performance VPN capability as their previous Cisco FW was trash for VPN speeds.  Fortunately they had rack space available already so this may or may not be suitable but it will point you in the right direction as their are desktop variants of the Dell PowerEdge series.

 

You want a CPU with AES-NI present if you are looking to do anything via VPN, if you are going Intel Xeon this will be anything after the Westmere architecture, check Intel's ARK however as some odd low power chips may not have it.

 

Here's what I used for them;

Dell R610 Chassis

2x Intel E5640 (2.67Ghz 6 core CPUs)

64GB Ram

2x 60GB Intel 520 SSDs [RAID1] (these were just lying around doing nothing so were chosen, pfSense config set to ramdisk)

1x Intel x520 Single port card but ideal in case they get anything more than 1G in the future, using 1G SFP currently

1x Intel x540-DA2 (RJ45) RJ45 internally was chosen due to all their switching being on 1G with 2x 10G RJ45 uplinks which were utilised as normal ethernet interfaces for 10G switching.  Switches are Cisco 3560Xs POE model with the 2x 10G RJ45 module in each of them.

 

Was configured as;

 

WAN > pfSense  > 10G RJ45 > Switch #1 10G port > Users
                           > 10G RJ45 > Switch #2 10G port > Infrastructure / Servers

Users were vlan'd and zoned off from access to infrastructure/servers only allowed via specific ports, the main file server was configured on Switch #2 on a single 10G RJ45 using another Intel x540, would of preferred redundancy here but not many 10G ports going around and they didn't wanna spend any more cash .  Still managed to pull 6.7Gb/s from the file server via pfSense from one of the clients, I didn't tinker with pfSense much as that was near 6x the speed from the file server previously and the disks in the file server can't really exceed that unless the IO workload is just sequential.

During testing pfSense was able to pull a full 1G minus over heads up and down with minimal system interrupts or anything crazy going on.  CPU on the pfSense box sits somewhere around 8-10% throughout the busiest part of the day with the file server getting a battering mainly.  VPN performance was limited by their other office Cisco firewall being a shitbox rather than the pfSense box, so around 300Mbit/s was seen over IPsec VPN.

The box is massive overkill, next is to sort out with the ISP to have it on active/passive fail over via CARP. 

[Falconevo]

Haha, just realised u said low power  the above isn't going to work if u want a low power solution.

Low power doesn't generally equate to performance in x86 land as it's mostly done in software especially with pfSense not being amazing for hardware acceleration.

 

I would probably look for a Micro ATX Xeon motherboard like the Gigabyte GA-6LASL which has 2x Intel i210 network interfaces on board for LAN connectivity.  Throw in a E3-1220 v3 Xeon which is a quad core clocked at 3.1Ghz base and 3.5Ghz boost and a bit of ram.  Job done.

 

You will also need a SFP capable Intel card in the PCI-E slot, x520 if you want to have 10G future proofing or stick with 1G and get an Intel 82576EB with SFP ports on it.