Weird Unifi switch issue

[Phibsta]

Hoping the LTT brains trust can help me on this, unifi support certainly couldn't.

 

I replaced an old unifi POE switch with a new USW Enterprise 48 PoE.  I have my router (DAC), 2 unraid servers (SFP to RJ45) and my NVR (DAC) connected into the 10gb SFP ports and the rest of my network is flat with no additional switches anywhere.

 

It seems that whenever I get a bunch of high volume traffic, usually to one of my unraid servers, that ports just start dropping traffic causing my cameras to go offline for a few seconds.  This traffic can be anything from a large SMB file transfer to a bunch of linux ISO torrent files being downloaded.

 

When I first had this issue support went through some testing and then eventually we RMA'd the switch.  The new one does the exact same and they have again suggested an RMA, however I don't have another switch to keep my network going while I wait for that (the first time I actually purchased a new one and then got a refund for the original).  Given that it has happened on 2 there is obviously something with either a faulty product line or something in my network.

 

Does anyone have any ideas on further troubleshooting I can do to try rule out anything in my network?

 

Devices connected:

2 x U7 Pro APs

a few Windows PC's

9 Unifi PoE cameras

A few NVIDIA shield

2 solar inverters

 

TIA

[MrSimplicity]

This sounds like 1 of 2 things. 1. definitely something faulty with their product QC. 2. Honestly, this seems more likely in my experience, you're stuck with dirty power. Are these machines strictly running off of the solar inverters? kind of a dumb question as that would be A LOT of solar panels. Only way you could potentially test this is to disconnect from WAN and start running stress tests on machines simultaneously and see if that does it as well. There's a chance that along side all of your systems the new switch uses too much power or your power is too dirty to stay stable. My only recommended remedy would be setting up some UPS' (battery backups) to help with clean power transmission. Dirty power has been a problem for me constantly even in the US and ultimately made me buy a UPS for my main PC and one for my server which also has my router and modem plugged into it as well.

 

If you don't think it's a power issue then maybe you might just want to look into a different PoE switch

[brwainer]

On the old switch, were things also connected at 10Gb?

[Falcon1986]

@Phibsta 

 

Try toggling on (or off if already on) flow control for the switch through UniFi Network.

 

Make sure UniFi OS, Network and switch firmware are all up to date.

[Phibsta]

20 hours ago, MrSimplicity said:

This sounds like 1 of 2 things. 1. definitely something faulty with their product QC. 2. Honestly, this seems more likely in my experience, you're stuck with dirty power. Are these machines strictly running off of the solar inverters? kind of a dumb question as that would be A LOT of solar panels. Only way you could potentially test this is to disconnect from WAN and start running stress tests on machines simultaneously and see if that does it as well. There's a chance that along side all of your systems the new switch uses too much power or your power is too dirty to stay stable. My only recommended remedy would be setting up some UPS' (battery backups) to help with clean power transmission. Dirty power has been a problem for me constantly even in the US and ultimately made me buy a UPS for my main PC and one for my server which also has my router and modem plugged into it as well.

 

If you don't think it's a power issue then maybe you might just want to look into a different PoE switch

Hi MrSimplicity,

 

Confirming it isn't dirty power.  I am connected to grid power, the inverters are just connected for data and the switch is behind a UPS also.  I wish I had enough panels to fully power the house!

 

I will try the disconnect from WAN and stress test between machines to see if it is related to WAN but I suspect it is not as it has previously shown for data transfers internally where no external source is related.

11 hours ago, brwainer said:

On the old switch, were things also connected at 10Gb?

The old switch was 48x1gb and 2 x 10gb SFP which were connected to the 2 unraid servers, so almost identical just limiting the client to server connection speed to 1gb which might have been masking the problem.

6 hours ago, Falcon1986 said:

@Phibsta 

 

Try toggling on (or off if already on) flow control for the switch through UniFi Network.

 

Make sure UniFi OS, Network and switch firmware are all up to date.

Firmware is all up to date including ubiquiti providing me a RC firmware which didn't fix it.  I will check the flow control and see if that helps.

 

Edit: Flow control was OFF so I have now turned that on.  Now to do some testing.

Edited June 6, 2025 by Phibsta
Update to info

[Phibsta]

On 6/6/2025 at 6:37 AM, Falcon1986 said:

@Phibsta 

 

Try toggling on (or off if already on) flow control for the switch through UniFi Network.

 

Make sure UniFi OS, Network and switch firmware are all up to date.

So turning flow control on seems to have fixed it.  I have had no issues in the week since I have changed that setting so touch wood it was the problem.  Thank you.

[brwainer]

On 6/5/2025 at 10:39 PM, Phibsta said:

so almost identical just limiting the client to server connection speed to 1gb which might have been masking the problem.

 

5 hours ago, Phibsta said:

So turning flow control on seems to have fixed it. 

And here’s your answer. With the client able to transfer faster, it is overloading something somewhere with trying to send too much data at once. Flow Control pauses things when buffers are full, at any place. Its a bandaid for whatever is overloaded, but identifying and fixing the root problem would be better.

[Falcon1986]

5 hours ago, Phibsta said:

So turning flow control on seems to have fixed it.  I have had no issues in the week since I have changed that setting so touch wood it was the problem.  Thank you.

I'm surprised Ubiquiti's support didn't tell you this from the get-go. Might have saved you the hassle of the RMA.

 

Am sure you're not the only person who pushes these switches to the max. However, most prosumers might not reach this level of use to ever notice and never activate flow control.