(WanShow) Air Gap Network

[Thiago Cabral]

You can setup a realy small air-gap solution using solid state relays and arduino/esp32 to put inline with the servers.
It can be FULL offline and management free.

[wanderingfool2]

Quite frankly, unless a company is really a target even a simpler non-air gap solution would likely work. (like a cheaper/easier to manage solution that offers still protection)

 

e.g.  Create a server (Lets call it BackServ), that utilizes a different OS than the server hosting the data (Lets call it HostServ).  If they host their primary storage on Linux, then use Windows or vise versa.

 

BackServ is plugged into it's own dedicated firewall (software side, like Windows Firewall and blocking all inbound ports and also hardware side...again blocking all inbound ports).  The firewall is configured to block all outbound connections as well, except to the IP range of HostServ.

You then have BackServ initiate a connection with HostServ and copy over the data folders from HostServ (Could even use a quickly made custom program to do the "file IO" stuff thus eliminating the SMB exploit possibility).

 

And there you have it, a backup system that should be pretty resilient to malware.  The only way malware could get onto the system would be primary host being infected with something that can transverse different OS's (less likely) and one that uses active zero-day exploits against the IP protocol itself (from my knowledge pretty rare).  Since HostServ would never be able to connect to BackServ...only BackServ can initiate the connection and from there only the communication being dictated by program *albeit a simple one, but in this case security through obscurity would work*.  It would I *think* be one that would be difficult to break.

[GhostRoadieBL]

Totally agree with @wanderingfool2 OS hopping is probably the best defence outside overly complex solutions.

Even just running a scripted backup on a linux middleman pointed to a rasbian OS NAS (or vice versa) would be more than enough hops for the vast majority of companies who could fall victim to this.

 

[XWAUForceflow]

If only there would be a high capacity automated storage solution that has an air-gap built in. Like something that automatically plugs in and removes the storage media, is highly reliable and fairly cost effective when talking dollar per GB...

Oh wait, there is. A tape library does exactly that. I mean LTT has already worked with IBM before and they do have some really incredible high performance tape solutions available.

@LinusTechwhy not approach them and ask if they'd be willing to sponsor a video supplying one if their smaller libraries, a few drives and tapes?

It might not be the 'coolest' solution, but if you really are afraid of either a targeted attack or some drive-by ransomware infection that is a pretty darn good and reliable solution. I mean it's not a zero effort solution and someone does need to actively manage and monitor it, but that's hopefully a lesson you learned with your recent storage issues.

[kumicota]

They don't need to create something like this, they would need to setup a Cold Storage System, where it would turn off the HDD when they aren't being used

[wanderingfool2]

3 hours ago, XWAUForceflow said:

Oh wait, there is. A tape library does exactly that. I mean LTT has already worked with IBM before and they do have some really incredible high performance tape solutions available.

Tape solutions are okay, but also really expensive to initially setup...given that at the scale LTT would have to do would probably require one that can autoswitch between tapes given the amount of data...and then it takes a long time to verify the data (because that would be something that should be done periodically checking the backups).  It also leaves it open to attack during the time that it's "connected" and some of the malware out there specifically targets things like tape-backups as well.

[Needfuldoer]

Theoretically you could shut its ports off in a managed switch. Someone would have to know which shut-off port your backup server is connected to and how to get into the switch and turn the port back on in order to hit the machine. (And then we're back to Layer 8 vulnerabilities, since at that point there's a good chance you're dealing with a targeted social engineering attack or an inside job.)

[wanderingfool2]

5 hours ago, Needfuldoer said:

Theoretically you could shut its ports off in a managed switch. Someone would have to know which shut-off port your backup server is connected to and how to get into the switch and turn the port back on in order to hit the machine

I mean realistically this kind of approach (and even the one that was spitballed during WAN show) ultimately has the problem that during backup the system itself would be connected and vulnerable (and could be spread).  Although technically not really air-gap (as truly air gap would mean no communication...but sort of self defeating at that stage, and air-gap I think used in a general term is slightly okay in this case...even if it's not).  i.e. In cases of malware it could be actively looking for connections so it doesn't matter if it's disconnected for 23 hours of the day...the 1 hour is all that matters

 

That's why I mentioned an alternative thought to the problem, because while not air-gap it provides a solid level of protection while being minimal work/effort to implement. (Mainly you would need to do physical access or govt level intrusion to do anything)  Chances of a malware attacking 2 different OS's are small.  Lower when communication is limited to only the connection/port that was opened during the backup via a firewall, and lower still if using a less standard way of  transferring the files over the network (i.e. not via smb)

[NovaNightmare]

On 4/2/2022 at 1:39 AM, Larbaco said:

You can setup a realy small air-gap solution using solid state relays and arduino/esp32 to put inline with the servers.
It can be FULL offline and management free.

Clone your environment, restore a backup to it daily. The cloned environment can be completely isolated in scif like room. 

 

Trying to automate an air gapped environment adds risk that the gap can be breached. Turning off ports? Those ports could get turned on. Powering off equipment on a schedule or remotely and then doing the same to bring it back online, schedule or remote control can be compromised. 

 

A real air gapped network is going to involve a human being interacting with the system in order to move data. 

 

So in my scenario, you have a cloned environment and between your cloned environment and your production environment you have a data integrity station. 

 

Physically remove a drive, place it on into the data integrity station and use system that scans the files and ensures the data is not encrypted. The system copies scanned data from drive 1 production to drive 1 isolated, drive 1 is then inserted into the air gapped system and automatically data is transferred to overwrite old data. Using de-duplication the only data copied is changed data, to increase the speed of the process.

[RandomDane]

I have just watched the video that LMG Clips put up, about the fraud, and air gapping is actually "easy" there is a lot of work to set it up, but then when it's up it's very easy to maintain 

 

The air gapping part, it's actually "pretty easy", have 2 types of net works, 1 where there is internet on, and one for only data transfer, we had that at the factory i worked at, every one had 2 computers, and a couple thunderbolt SSD's to transfer files, and every time the transfer happend first it's plugged in to a thunderbolt hub that is connected to and "malware scanner" it was just a pc with 2 16 core xeons that ran ransomware detectors and anti malware and then after scanning the drives it goes on to our "data network" where there is no connection to the internet just a data like MP4 files and word docs

[kumicota]

Which is the video about airgap?

[emosun]

well airgapping is easy if the backup times are extended and online backups are redundant

Basically if you backup once a month or longer to the airgapped machines then the risks are reduced greatly , and if you increase the redundancy of the online backups then the data has a lower chance of being fowled before the monthly backup.

[RandomDane]

2 minutes ago, kumicota said:

Which is the video about airgap?

 

[RandomDane]

2 minutes ago, emosun said:

well airgapping is easy if the backup times are extended and online backups are redundant

Basically if you backup once a month or longer to the airgapped machines then the risks are reduced greatly , and if you increase the redundancy of the online backups then the data has a lower chance of being fowled before the monthly backup.

True, the server that had got the files after the transfer put them on some 8 or 10 tb drives can remember which one, and does drives were hotswapable at the time i was working there, and the drives was put in another server that was in the bank vault at the factory so there was 2 backups the first offline server, and then another offline server

[kumicota]

While airgap is probably is the best solution for security, there's a few things that linux could also do:

• Create a server/filesystem where the person can create files but it can't modify, run or delete(chattr and chmod can do that) and save daily ZFS snapshots there
• Block root access from SSH
• Linus team copies file through Samba, they can edit that the samba group/user can't access anything else besides those folder and aren't able to run anything
• Using a hardware token like ubikey as the way to log-in as root

This things can be bypassed by a zero-day bug but besides that, it's extremely safe

Edited April 6, 2022 by kumicota
clarifications

[ThePasswordIsPassword]

I got curious about airgapping too, and some cursory Googling led me to this: https://www.l-com.com/secure-data-l-com-category-6-a-b-physical-layer-air-gap-network-switches-rj45-ports

 

I'm sure there are other companies / products that do the same thing, but this was the first example I came across. It's essentially a physical-layer switch (as in a light switch, not a network switch per se) that can be controlled by a physical button, a serial connection, or a network connection. I'm pretty sure that's the same functionality as Linus's physical-disconnect idea, but with less robot arm. Whether or not the robot arm is a hard requirement is, of course, anybody's guess given the way LTT usually likes to do things...

[LogicalDrm]

2 hours ago, RandomDane said:

I have just watched the video that LMG Clips put up, about the fraud, and air gapping is actually "easy" there is a lot of work to set it up, but then when it's up it's very easy to maintain 

 

The air gapping part, it's actually "pretty easy", have 2 types of net works, 1 where there is internet on, and one for only data transfer, we had that at the factory i worked at, every one had 2 computers, and a couple thunderbolt SSD's to transfer files, and every time the transfer happend first it's plugged in to a thunderbolt hub that is connected to and "malware scanner" it was just a pc with 2 16 core xeons that ran ransomware detectors and anti malware and then after scanning the drives it goes on to our "data network" where there is no connection to the internet just a data like MP4 files and word docs

Merged to earlier thread about same subject.

[NovaNightmare]

9 hours ago, kumicota said:

While airgap is probably is the best solution for security, there's a few things that linux could also do:

• Create a server/filesystem where the person can create files but it can't modify, run or delete(chattr and chmod can do that) and save daily ZFS snapshots there
• Block root access from SSH
• Linus team copies file through Samba, they can edit that the samba group/user can't access anything else besides those folder and aren't able to run anything
• Using a hardware token like ubikey as the way to log-in as root

This things can be bypassed by a zero-day bug but besides that, it's extremely safe

Permissions are not effective protection for ransomware. Yes, they can stop some things, but often the attack will exploit a vulnerablity that gives it greater access than expected. 

 

As you said, a zero day or any unpatched exploit that any vendor may fail to deal with properly. 

 

A real air gapped (isolated) network must be completely and physically separated and use none of the equipment from the production network. Anything else is a misconfiguration or a multi step breach away from being breached. The SolarWinds Orion hack was a multi step state level breach that went further and further and then began targeting specific entities.