Whistleblower: Ubiquiti Breach “Catastrophic”

[Sir Asvald]

9 minutes ago, SlidewaysZ said:

I agree any management should be done through a VPN

That's correct! as for Ubiquiti devices, you to use a separate device in order to manage your already expensive switches and cameras is annoying. I dont want to spend and extra $200 for  stupid "cloud key". Allow me to manage everything as I want...

[jagdtigger]

54 minutes ago, Sir Asvald said:

That's correct! as for Ubiquiti devices, you to use a separate device in order to manage your already expensive switches and cameras is annoying. I dont want to spend and extra $200 for  stupid "cloud key". Allow me to manage everything as I want...

You dont have to buy anything, the controller sw is available to download for anyone. Its just one more VM....

[Sir Asvald]

2 minutes ago, jagdtigger said:

You dont have to buy anything, the controller sw is available to download for anyone. Its just one more VM....

Then why sell the cloud key?

[leadeater]

9 minutes ago, Sir Asvald said:

Then why sell the cloud key?

For those that don't want to run run a VM or a PC for it, it's for those that want a nice tidy all in one (vendor) package. See my example on the other page.

[StDragon]

3 hours ago, jagdtigger said:

And contrary to some whining here its actually rock stable. No signal loss, no sudden reboots, no freezing.

When it comes to WiFi hardware performance, range, and stability of connectivity, I've got nothing but praise for Ubiquiti. The issue of software bugs (namely API) and features, it's always been an issue. Both support and initial rollout is extremely beta-ish. General consensus is to just wait a few months on any new major update or feature.

 

 

[Tieox]

@LinusTechTagging you out of concern for LTT since I am fairly sure your access points are Ubiquiti in the offices, and your home too?

 

 

[LAwLz]

9 hours ago, leadeater said:

Their UAP access points are actually good for the price, not much better in the target market. I would personally keep deployments to around 6-10 max before going with a better brand. The config and management is just at the right level for homes and smaller businesses to be able to self manage after being deployed by someone more expert.

 

Friends parents place has 3 UAP-AC Pros in it with the Cloud Key (the older one) and their small PoE switch. The house is used as a rental/borders accommodation so there's a private SSID and a guest client isolated SSID, generating password with timed expiry is simple and not something you could really do or easily (without extra crap) on anything else consumer oriented.

That's why I called it a "hobby vendor". 

It's perfect for the scenarios you described. For example some home network or some non-business critical stuff (like wifi in a rental house). 

 

For other installations though I think it's best to go with a proper enterprise vendor. Both Cisco and HP have fairly competitive offerings. Especially if you get it through a partnered reseller thst can get some decent rebates. 

 

I've heard good things about the Aruba instant on line, as well as the Meraki Go line. I haven't had any small enough customers to deploy it though. 

 

7 hours ago, RadientPapaya said:

I really hope this is not true but if UBNT is now a company more interested in the bottom line then in quality/goodwill then these are more or less the only two paths to for them.

It's a company. It has always been more interested in the bottom line. If you have gotten the impression that their main focus is anything but to protect their bottom line and make money then you have had the wrong impression. 

[LAwLz]

4 hours ago, SlidewaysZ said:

Can anyone confirm if your AP was just connected to a controller hosted locally running their controller software without remote connection access are you safe? That's what he seems like to me. Obviously update account passwords but does the password to my AP need to be updated or do I need to wipe everything?

We don't know. You're probably safe but honestly, we have no idea what might have happened. Since they had access to things like their severs, they could have pushed out a compromised update to your devices. If you have updated your devices sometime after the hacker got in then you might be compromised (although the risk is fairly low) even if you don't use the cloud management. 

 

 

2 hours ago, Sir Asvald said:

And that why kids, I don't recommend Cloud "networking gear" because shit like this happens. Why does "cloud management" have to be a thing? I get it is a nice "feature" that you can use to manage your gear from anywhere but come on... 

Nothing wrong with cloud management. I don't think it's necessary for consumer stuff but for enterprise it's great. 

 

 

2 hours ago, SlidewaysZ said:

I agree any management should be done through a VPN

So you don't like "login exposed to the internet and once you're in you can manage your device" (cloud management) but you do recommend "login exposed to the internet and once you're in you can manage your device, in addition to being inside the network" (management over VPN). 

 

It's easy to fall into black and white thinking where something is either bad or good, but the reality is often various shades of gray. 

[Sir Asvald]

13 minutes ago, LAwLz said:

Nothing wrong with cloud management. I don't think it's necessary for consumer stuff but for enterprise it's great. 

Large Enterprises will never have cloud management for their devices too much of a secure risk.

13 minutes ago, LAwLz said:

So you don't like "login exposed to the internet and once you're in you can manage your device" (cloud management) but you do recommend "login exposed to the internet and once you're in you can manage your device, in addition to being inside the network" (management over VPN). 

 

It's easy to fall into black and white thinking where something is either bad or good, but the reality is often various shades of gray.

There is a difference between the two. For the cloud management all I need is username/email and password. As for managing network devices over VPN is much more secure as you will need not only the company device username/password, but you also need to factor in 2FA, such as a smart card or even passcode that you'll receive via text message in order to use the VPN.

[leadeater]

1 hour ago, Sir Asvald said:

There is a difference between the two. For the cloud management all I need is username/email and password. As for managing network devices over VPN is much more secure as you will need not only the company device username/password, but you also need to factor in 2FA, such as a smart card or even passcode that you'll receive via text message in order to use the VPN.

So PaaS/Cloud Management providers also have 2FA support, you just need to enable it. Not sure about Ubnt though, but 2FA is not exclusive to your VPN example.

[leadeater]

49 minutes ago, comander said:

VPN is not necessarily the ideal way forward since it's too close to an all or nothing approach

Yep, I feel a lot of people treat VPN as silver bullet to anything and everything were as I largely consider VPN more as a potential security risk. You have to ensure that VPN configuration as whole, system design, is actually secure otherwise you're introducing a critical external risk and achieving the reverse purpose of what people think a VPN does.

[LAwLz]

11 hours ago, Sir Asvald said:

Large Enterprises will never have cloud management for their devices too much of a secure risk.

Not sure what you classify as "large enterprise" but I got several large customers (large by Swedish standards at least) that uses cloud managed products like Meraki. Meraki isn't something I would typically recommend to large enterprises but it's not because of security risks. It's because of pricing and features.

 

Not to mention things like Azure which are arguably "cloud managed" as well if we aren't talking about networking specifically. You'd agree that those are used by large enterprises, right?

 

 

11 hours ago, Sir Asvald said:

There is a difference between the two. For the cloud management all I need is username/email and password. As for managing network devices over VPN is much more secure as you will need not only the company device username/password, but you also need to factor in 2FA, such as a smart card or even passcode that you'll receive via text message in order to use the VPN.

You can set up MFA for cloud managed things as well... It's not something exclusive for VPNs. You can even enable it on your Ubiquiti account.

[LAwLz]

10 hours ago, leadeater said:

Yep, I feel a lot of people treat VPN as silver bullet to anything and everything were as I largely consider VPN more as a potential security risk. You have to ensure that VPN configuration as whole, system design, is actually secure otherwise you're introducing a critical external risk and achieving the reverse purpose of what people think a VPN does.

Which is funny because I am in this thread arguing against someone proposing VPNs for management, but just a couple of weeks ago I was making arguments for why VPNs for management aren't bad (with @comander).

On 2/23/2021 at 9:36 AM, LAwLz said:

Client VPN doesn't mean you give everyone that logs on the same permissions or unrestricted access. If that's how you think it works then I understand that you are against it. You can apply firewall policies to VPNs if you want (and you probably should). 

 

 

It seems like people in general these days operate on absolute black and white thinking for everything. Something has to be either the greatest thing ever and used all the time, or it's the worst and should never be used and is always bad. I am probably guilty of it too, despite trying not to.

[Overtaxed]

15 hours ago, Sir Asvald said:

And that why kids, I don't recommend Cloud "networking gear" because shit like this happens. Why does "cloud management" have to be a thing? I get it is a nice "feature" that you can use to manage your gear from anywhere but come on... 

While I tend to agree with you, there is one application where UBNT's "cloud management" is really nice.  If you're a VAR setting up a lot of their gear for customers and managing it long term, having a cloud connection to all the gear (rather than 100's of VPNs to deal with) is very nice.  Allows you to see all your customers at once and manage a fleet of lots of devices easily.  

 

That said, it's a big security hole in any network by allowing "cloud" management like this.  It's a very "nice to have" feature for a provider, but for a customer with a single site install, it's, IMHO, way more risk than value.  If you want to manage your gear remotely, setup a VPN!  That also gives you unified access to the entire network, not just the Ubiquiti gear.  

 

That said, I do question "why" a single site user would want to manage their network remotely.  I'm sure there are use cases, but I've had a pretty complex network, lots of cameras, switches, APs, and dozens of endpoints at home for years now and exactly "never" felt the need to manage it.  If I'm not home, I'm not much concerned with what my network is doing.  And given that, by far, the most likely failure in my network is the Internet connection itself which is also the management link, I'm just not seeing much (any, honestly) use for that functionality in my life.

[leadeater]

1 hour ago, LAwLz said:

It seems like people in general these days operate on absolute black and white thinking for everything. Something has to be either the greatest thing ever and used all the time, or it's the worst and should never be used and is always bad. I am probably guilty of it too, despite trying not to.

We get that a bit at work too. Security team wanted to stop the usage of RDS Gateway and use SSLVPN instead, but that would actually be dumb. RDS Gateway is an HTTPS/SSL Application gateway/reverse proxy that has access and authorization policies so you can define who can connect and what they can connect to, and this is secured behind a Citrix ADC appliance with traffic going to it going through the datacenter firewall and then traffic from it going back through the datacenter firewall then to the destination server.

 

This was going to be replaced with an SSLVPN hosted on the datacenter firewall (for ITS Engineers only etc, regular one is on campus firewall), however since everyone in my team needs access to all servers on multiple different ports and services across a huge range blah blah having our home PCs/laptops and whatever else device connecting in and have that sort of network access is simply worse than RDS Gateway we already have in place.

 

Sometimes the wheel is already round enough, you can't make it rounder but you can square off the edges lol aka leave well enough alone.

[b1k3rdude]

Isn't this the same system Linus used for his home security setup..?

[Tieox]

5 hours ago, b1k3rdude said:

Isn't this the same system Linus used for his home security setup..?

I believe so.

[PudgyBuns]

I just watched the WAN show and Linus mentioned this thread so I came over to read it all. I made an account just now too.

 

I use Ubiquiti equipment at my business as a WISP provider exclusively, for over 600 customers just using this point to point network we built in our county. We monitor it all on a VM because we considered the cloud hosted UNMS/UISP an unnecessary cost for whatever they are charging to back up ~200mb, which is the backup file size that gets sent to the NAS daily for a little peace of mind. I'm very glad that call was made now. VMs really are delightful. 

 

The extensive access granted to a malicious attacker would be astounding. You can access a CLI and SSH into peoples dishes on their roofs or their access points in their hallways. 

One example of issues these devices face include the following:

Ubiquiti Networks EdgeSwitch version 1.7.3 and prior suffer from an improperly neutralized element in an OS command due to lack of protection on the admin CLI, leading to code execution and privilege escalation greater than administrators themselves are allowed. An attacker with access to an admin account could escape the restricted CLI and execute arbitrary shell instructions.
- https://www.cvedetails.com/vulnerability-list/vendor_id-12765/Ubnt.html

Now I'm not very good with command line, as I just put down my anti static bracelet a few months ago, but I assume this is a problem due to the fact you have some pretty high level permission on a device inside a persons network. My coworker was recently working on writing a "rubber ducky shellcode" was the words he kept using without explaining it, but if Rubber Ducky's website has a clip from Mr. Robot you know bad stuff is going to happen to your credentials. 

 

All of this said. I didn't know that cve website existed, and I know why people don't want multifactor authentication at a business. Sharing password manager accounts at business isn't too uncommon. I saw it several times when working with businesses who would have dozens of desktops, laptops, tablets, and phones they need to keep track of and they have potentially 10 people who need access to those passwords to help manage them due to the high volume of staff they have. 

 

Its just sort of.... hard to keep employees obeying the company policies about passwords and managers, and sticky notes, and emailing passwords without verification of recipient or request being approved by a high enough authority when you don't write anything in stone about it. And when you do, you will have those lazy boners still using their PW manager over the internal business PW server hosted in a local only VM CUS ITS GOT AN EXTENSION THO. Hard argument to fight until you don't have a job cus your small business burned to the ground.... cus you left your pc open at Starbucks.... cus you had to take a wicked piss. 

 

Oh also their equipment lasts in fires and storms no problem. We have about a dozen aircubes out in the field that we're slowly replacing whenever a customer complains about anything if they have one. On the other hand we have dozens of NanoStation Loco M5s that are performing perfectly, and I'm unsure of their age but it seems to be about 5 years. 

Some very specific products like some of their access points get a lot of DFS hits on one firmware and not on another. Its kind of annoying because it's too complicated to troubleshoot without wasting a reasonable amount time and at that point you just want to downgrade it to the firmware that was working fine before this. Bam security flaw and you didn't think of it. 

 

 

 

[Bombastinator]

On 3/30/2021 at 7:30 PM, TempestCatto said:

Well this sucks. I had recommended Ubiquiti to a number of people over last several months. Most of those people have indeed purchased Ubiquiti products. Now I'll look like a giant ass for that.

 

Also, this is irresponsible as hell. They should have said how bad it was from the start, rather than a whistleblower having to break the news. People could have at least taken some action, like removing those devices entirely from their network or home/business. This does not bode well at all. Kinda reminds me of what happened with NordVPN.

Maybe maybe not.  If it was done AFTER things got fixed things should be relatively simple.  Before the breach was discovered though I got nuttin’

[BlackManINC]

 Well this is the last fucking time I'll ever recommend another Ubiquiti product to anyone ever again. Jesus, can't really trust anyone these days. 

[MrDevanWright]

Any suggestions for a Ubiquiti competitor that has a quality aesthetic and cover similar avenues? (ie: networking, cameras, access, etc)