Routing cluster

[Mornincupofhate]

I've been contemplating lately about what would happen if I put together a bunch of routers and connected them all to one switch while using CARP. For some reason I just can't wrap my head around how this would work. Would I theoretically be able to achieve 30gbps / 44.64 Million packets per second of simultaneous throughput on the same machine? (The machine would have to be capable of handling it, of course, but that's not what I'm asking.)

 

Here's a diagram of what I'm attempting.

 

[Nikolithebear]

I'm not sure what the connection could be on the intake of the routers (outside the firewall for example)  However I think this could be done through a load balancing configuration between the routers.  Once that is achieved you could either keep each 10gb link to the layer 3 switch as independent or set as teams.  (Depending on the load balance configuration).  I'm not an expert but that would be my suggestion.  

 

 

Edit: One question could be what Router OS or platform would you use?  (Cisco, Alcatel-Lucent, PfSense etc) 

[Lurick]

You could, in theory, do a transfer from 1 box on a 40Gbps link to three servers, each with their own 10Gbps link, but going from you to a single destination will not give you 30Gbps.

[Mornincupofhate]

2 hours ago, Lurick said:

You could, in theory, do a transfer from 1 box on a 40Gbps link to three servers, each with their own 10Gbps link, but going from you to a single destination will not give you 30Gbps.

What if I did a round robin sort of thing? Sending data through all 3 pipes at the same time. Would it then be able to transfer at the same time at 30gbps?

 

If you're wondering why I'm doing this sketch looking setup, it's because I'd rather not buy a router that costs the same as a house.

[Mornincupofhate]

2 hours ago, Nikolithebear said:

I'm not sure what the connection could be on the intake of the routers (outside the firewall for example)  However I think this could be done through a load balancing configuration between the routers.  Once that is achieved you could either keep each 10gb link to the layer 3 switch as independent or set as teams.  (Depending on the load balance configuration).  I'm not an expert but that would be my suggestion.  

 

 

Edit: One question could be what Router OS or platform would you use?  (Cisco, Alcatel-Lucent, PfSense etc) 

FreeBSD.

[Lurick]

11 hours ago, Mornincupofhate said:

What if I did a round robin sort of thing? Sending data through all 3 pipes at the same time. Would it then be able to transfer at the same time at 30gbps?

 

If you're wondering why I'm doing this sketch looking setup, it's because I'd rather not buy a router that costs the same as a house.

Unless you buy a load balancer with 8x 10Gbps ports or 2x 40Gbps ports, it's not going to happen. Even then you'll have to really tweak the balancer to properly hash a single stream and find one that offers that capability.

[Mornincupofhate]

12 hours ago, Lurick said:

Unless you buy a load balancer with 8x 10Gbps ports or 2x 40Gbps ports, it's not going to happen. Even then you'll have to really tweak the balancer to properly hash a single stream and find one that offers that capability.

Couldn't I just set my server/switch to use LACP and load balance it from there, and then just have the ISP round robin or load balance the traffic coming in?

[LAwLz]

I don't know about CARP, but this would be possible with VRRP. But there might be quite a bit of configuring that needs to be done before it works fully (like with VRRP/HSRP where you need to create multiple instances for it to work).

[Erkel]

Simple answer is 40 or 100Gb ethernet,

[Mornincupofhate]

Just now, Erkel said:

Simple answer is 40 or 100Gb ethernet,

As I said, the problem is packets per second and the fact I don't want to buy a cisco machine that costs as much as my house.

[Erkel]

Refering to one of your previous posts.

"Let's just get this straight, I'm no network engineer, nor am I old enough to even have a job."

 

So this an academy question rather than a reality related question.    The problem does not realistically exist in reality, in the context that you are asking it,  You can load balance across the links using routing policy etc,  But you are talking millions of TCP connections potentially, not just one.

 

Any one wanting to do this in reality would be running what would affectively amount to an active/passive cluster of big iron that costs the same as a house.

[Lurick]

10 hours ago, Mornincupofhate said:

Couldn't I just set my server/switch to use LACP and load balance it from there, and then just have the ISP round robin or load balance the traffic coming in?

LACP does not work like that. You get failover protection, it doesn't load balance across all available links for a single stream. You need multiple streams to multiple destinations to see the load balancing take effect.

[LAwLz]

3 minutes ago, Lurick said:

LACP does not work like that. You get failover protection, it doesn't load balance across all available links for a single stream. You need multiple streams to multiple destinations to see the load balancing take effect.

That's not really true. Depending on what equipment you use, it is possible to use load balancing over single streams with LACP.

One of the common load balancing schemes is to do it on a port basis as well, which (preferably combined with IP source) would allow a single host to use both connections, assuming each data stream uses a different port number. It is also possible to do it over the same port by using vendor specific solutions such as Juniper's adaptive load balancing or per-packet random spray.

 

So no, LACP is not just for failover protection. It is for load balancing too, even with a single destination.

[Lurick]

13 minutes ago, LAwLz said:

That's not really true. Depending on what equipment you use, it is possible to use load balancing over single streams with LACP.

One of the common load balancing schemes is to do it on a port basis as well, which (preferably combined with IP source) would allow a single host to use both connections, assuming each data stream uses a different port number. It is also possible to do it over the same port by using vendor specific solutions such as Juniper's adaptive load balancing or per-packet random spray.

 

So no, LACP is not just for failover protection. It is for load balancing too, even with a single destination.

Everything OP talked about so far led me to believe they wanted to achieve 30Gbps of throughput using a single data stream, which wouldn't, by default, hash over multiple links. If there is a second connection with a different port or anything, and the proper hashing algorithm is used, then you are correct. I was going off what I had interpreted them to be wanting to achieve but looking at the diagram again I realize that I was wrong since there are three servers so there would have to be three different connections, at minimum.

[leadeater]

You need to be very careful how you load balance traffic internally and externally. You also need to keep in mind protocols like VRRP, HSRP, CARP etc are all primarily designed for high-availability/resiliency and not for load-balancing or increasing bandwidth. While you can achieve increased bandwidth this is network wide and not for a single session from a single device.

 

Anything that supports a round-robin type mode must be used with extreme caution and usually comes with tons of warnings on their own manual pages. If you round-robin you violate packet/frame ordering which can be very bad and have a negative effect not just on achievable bandwidth but on network device load and buffers having to re-order packets.

 

When it comes to internet connections while you'll be able to do some kind of outbound round-robin you are not going to be able to get inbound round-robin, I specifically mean round-robin, so don't ask for it or you'll get some rather ... abrupt no's .

 

FreeBSD does support round-robin for NIC bonding on outbound traffic but your problem will be getting the switch to round-robin traffic from a single host/session to the FreeBSD server.

 

When it comes to load-balancing the proper way to do it and generally the only supported way is to keep session or connection persistence unless there is a link or device failure. Moving an existing and open connection to another network path can break stuff unless it is done application aware.

 

Very generally speaking the only safe way to increase bandwidth is if it is done natively at the protocol or application layer i.e SMB3 Multi-Channel.