brute force attack Securing server suggestions that don't cost a fortune!

[Lasky]

Hello, all members of LTT.

 

Linus, I wanted to first start with saying how amazing your videos are on youtube.  I have been watching your videos a lot, and I must admit that some actually made me chuckle!

 

I am from UK, and I am new to this board, so please bare with me as I attempt to explain my issue in as much detail as possible.

 

I have been hosting game servers for around 2 years on a dedicated server that is located in Pennsylvania, USA.  We were running Windows Server 2008 R2 until around 2 weeks ago when somehow, someone managed to get in and install ransomware, that demanded we attach 2 files to get some sort of decryption key to get our files back.  I am guessing that after they get the files, they will demand some sort of ransom.  Suffice to say that we did not respond to their demands and we just had the entire server rebooted, after I had our hosts completely erase all the drives and reboot Windows from scratch.  We had backups of our most important files, I had copies of them here at home.  We are now running Server 2016.

 

I am not sure how these hackers got in to our server, as we had a very secure password, and the system was fully up to date at the time.  I have been working constantly to create a more secure server and have done what I can thus far, but I am wondering if there is anything that experts here can suggest to be able to make it almost impossible to get in to, in any way.  To be able to better explain my situation, I have included a few screenshots that I took from the server before we had it rebooted for reference purposes, and wondered if anyone knows of a possible way that they got in to install this ransomware, something called "Mr.Dec".  I have no idea what this is, but here are some screenshots for the experts to try and assist us to make the server as secure as possible to prevent similar attacks of this nature.

 

NOTE: That this is in no way intended to be spam, I am trying to get some useful advice here, as so far, we have had a PSU failure that fried the entire rack we were located in (not hosts fault) and this issue that has become a real headache.

 

Can anyone assist me please?  Thanks in advance.

 

[givingtnt]

possible infiltrations may have happened if
someone plugged an unknown usb stick in the server or a pc that has acess to the servers (unlikely)
a user (say you or whoever has acess to the server might have some virus on their computer that they got somehow, that through the network gave them acess to it.
a user has compromised accounts (email or whatever) and someone found a way to access the server via that. (employee's password and the sorts)

 

or perhaps some vulnerable software that you are using to access the server.

[Lasky]

Yes, I am creating a regular backup as we speak, and server logs and custom code/scripts are planned to be saved on Google Drive, which was remarkably, previously unaffected by this attack.  We are also saving the shortcuts (that contain command lines) to Google Drive as well.  This way, we know that they are backed up after losing them before when our entire box fried as mentioned above.

 

@givingtnt, just saw your updated post...

 

We don't have any employees, we are a small gaming organisation, nothing more.  As far as I am aware, only the staff at the data centre have access to the box as it were and the likely hood of them doing anything to cause this is not really possible, I don't think they would do this.

 

We are using RDP over SSH to access the box, and we are very careful about what we download to it.  Generally, I download any applications or programs to my home computer to scan it for any viruses or malware and if it is safe, then I send it to the server via Google Drive.

[Electronics Wizardy]

34 minutes ago, Lasky said:

am not sure how these hackers got in to our server, as we had a very secure password, and the system was fully up to date at the time. 

There have been many windows bugs that allow remote access with no password. Take a look at the cve list. There is also stuff from websites that can do this, do you use a web browser on this system?

 

 

Really you just have to plan for and expect this to happen. Its impossible to have a perfetly secure system. So just split the tasks into vms to reduce the chance and scope of any problems and keep image backups for a easy restore.

 

[Lasky]

There is only one major issue with this...

I live several thousand miles away from the server, I live in UK, server is located in USA, there is quite a big distance between us.  I am creating archives of our stuff and sending it to myself via Google Drive (files get deleted afterwards).  We use Google Chrome on it to look up mods and such, we don't leave it running, ever.

[LinuxTek]

Been reading up on this hosting Steam Servers. Interesting.

 

More interesting is Valve is paying hackers to ...hack Steam.

 

 

[Lasky]

We don't host Steam servers, we host games from Steam, if that makes sense?

[Electronics Wizardy]

54 minutes ago, Lasky said:

There is only one major issue with this...

I live several thousand miles away from the server, I live in UK, server is located in USA, there is quite a big distance between us.  I am creating archives of our stuff and sending it to myself via Google Drive (files get deleted afterwards).  We use Google Chrome on it to look up mods and such, we don't leave it running, ever.

Bad backup plan. 

 

What service are you using? Id look at something like aws here. That way you can make snapshots easily, and everyhting runs in a vm, makeing it much easier to restore.

[Lasky]

But what if these hackers break in to the main server, the one that it actually hosting it all...

 

From my stand point, I am not going to win this war at the moment, it's almost as though I am being targeted and I don't know why.

 

Checked the Event Viewer logs last night, and sure enough, once again, multiple IP's again trying to brute force my server...

[AngryBeaver]

Looks like a wannacry variant. Honestly, a few things you can do would be to close off any port not in use, upgrade to 2012r2 or 2016. I wouldn't say to go as far as getting and IDS/IPS solution for just one server, but you will need to have some type of endpoint protection.

 

Since it is just a game server I would make sure the save folders are being backed up at least weekly. Learn how to use Nessus and lookup anything over medium risk. See if there is an active exploit for it in metasploit, Chances are that is how you got popped.

[beersykins]

37 minutes ago, Lasky said:

But what if

What happens when your server gets nuked by North Korea?  We need a mitigation strategy for that too.  It's easy to sit there and claim what if for every scenario, negating any suggestion in the process due to lack of understanding.

37 minutes ago, Lasky said:

 

Checked the Event Viewer logs last night, and sure enough, once again, multiple IP's again trying to brute force my server..

This happens to every host on the internet.  If it's directly public facing you're going to see this always.

 

I would bet it was cve/exploit related

7 hours ago, Lasky said:

We use Google Chrome on it

I'd shy away from doing that, that was probably the attack vector..  You can transfer whatever files you need over rdp instead.

 

[lacion]

I would say that right off the bat stop using password no matter how secure you think they are. switch to cert only auth (I'm sure windows has a policy for it) I like 99% sure winRM supports that, you can also go for smart cards.

 

 

only open used ports (i know this is hard as game servers usually use huge ranges for ports) also any access to remote manage the server I recommend you block and control that as much as possible, maybe limit to certain IP's.

 

I'm not sure if win servers have this but there is an util in Linux world called fail2ban that will simply ban TCP/UDP access of any kind to someone that tried to auth unsuccessfully x number of times for y amount of time (super useful to frustrate and slow down attempts at brute-forcing.)

 

sorry, I can't be of much help I have not used win servers for ages. but I'm sure there are guides around of basic security settings for windows servers. also, remember to keep security updates going. and do them ASAP

[Lasky]

We are now using Server 2016.  That was the first thing I asked our hosts to install.

 

I am nowhere near the server, it is thousands of miles away!

[Electronics Wizardy]

28 minutes ago, Lasky said:

We are now using Server 2016.  That was the first thing I asked our hosts to install.

 

I am nowhere near the server, it is thousands of miles away!

Is this a physical system or a vm?

 

Id run everything in a vm. You have hyper-v in windows, but Id look at anouther hypervisor like esxi or proxmox.

 

If this is a vm already, then you can make easy snapshots. Look at aws ec2 for this.

[Lasky]

No, sadly, it is a full dedicated server.

 

Would you like the specs?  I tell you, they are "not to shabby"!!

[Electronics Wizardy]

7 hours ago, Lasky said:

No, sadly, it is a full dedicated server.

 

Would you like the specs?  I tell you, they are "not to shabby"!!

Sure I don't mind specs.

 

Why not run all the different servers in their own vm? Then use the host as a hypervisor. 

[Lasky]

Hypervisor?

 

A pair of Intel Xeon X5650's (yes I do mean that I have two of them),

72GB DDR3 1600MHz RAM,

240GB SSD, (this drive also holds a 56GB page file, we cannot add anymore RAM without replacing all the modules, currently all 4GB DIMM's) Total of 128GB of virtual RAM.

1TB WD Black (7200RPM 32MB Cache)  This drive hosts our servers,

1TB Seagate HDD, for storing installers and maps for the game servers, and;

a pair of 1TB HP hard drives. which I plan to use for VM's perhaps.

 

I still have 3 drive bays remaining, so I can add another 3 drives, which I might do in due course, might get three 2TB WD Blacks installed.

 

The WD Black is as far as I am aware a high performance hard drive, and is perfect for our game servers to run on them, if anyone has a better drive, please do let me know.

[Electronics Wizardy]

On 9/19/2018 at 5:03 AM, Lasky said:

Hypervisor?

 

A pair of Intel Xeon X5650's (yes I do mean that I have two of them),

72GB DDR3 1600MHz RAM,

240GB SSD, (this drive also holds a 56GB page file, we cannot add anymore RAM without replacing all the modules, currently all 4GB DIMM's) Total of 128GB of virtual RAM.

1TB WD Black (7200RPM 32MB Cache)  This drive hosts our servers,

1TB Seagate HDD, for storing installers and maps for the game servers, and;

a pair of 1TB HP hard drives. which I plan to use for VM's perhaps.

 

I still have 3 drive bays remaining, so I can add another 3 drives, which I might do in due course, might get three 2TB WD Blacks installed.

 

The WD Black is as far as I am aware a high performance hard drive, and is perfect for our game servers to run on them, if anyone has a better drive, please do let me know.

Id make a vm for every different server. that makes it easy to administer and more secure.

 

Why wd blacks? Get ssds if you can, otherwise get reds here. Id user storage spaces with a tiering if you can't afford all ssds.

[Lasky]

I could not afford an all SSD system, I would need several terabytes of SSD storage to accommodate all of our game servers!

 

So I did a compromise, and had an SSD for Windows (so it boots faster when restarted) and a high performance drive (WD Black) for the game servers.  I have other drives, but they are being used for other things.  I wanted to host my websites on our server, but the problem is that if I did, I would have a major issue if we have another ransomware attack similar to the one on the first post on this thread.  I would again lose everything.

[Electronics Wizardy]

10 minutes ago, Lasky said:

I could not afford an all SSD system, I would need several terabytes of SSD storage to accommodate all of our game servers!

 

So I did a compromise, and had an SSD for Windows (so it boots faster when restarted) and a high performance drive (WD Black) for the game servers.  I have other drives, but they are being used for other things.  I wanted to host my websites on our server, but the problem is that if I did, I would have a major issue if we have another ransomware attack similar to the one on the first post on this thread.  I would again lose everything.

Setup up vms. That will solve a good amount of the security problem

 

Next time get wd reds or red pros or golds or seagate ironwolf or exos. Don't use desktop drives in a server, that often leads to issues.

 

You can cache the hdd with a ssd,  should be a good amount faster, and much cheaper than ssd only.

[Lasky]

Well, we do have logging enabled, the files themselves are only a max of around 2MB, which we plan to start storing on Google Drive which will always sync as a backup.

 

WD Red's to my understanding are for NAS servers and not for heavy workloads like running game servers.  Please correct m if I am wrong here.

 

I am aware that WD Blacks are desktop drives, but they are the only ones that I know of that provide the performance that I require for running our game servers.  We did start with a 5400rpm drive and loading was very slow and needed to be improved.  I do agree that I COULD use an SSD as a caching drive, but SSD's with our hosts cost a fortune where as hard drives are only a fraction of the cost.  currently, we have a 240GB SSD (which runs Windows and programs) and four 1TB hard drives, one the WD Black I have described above.

 

I have 72GB of RAM, which helps a lot, but I also have a 56GB page file on the SSD for a virtual 128GB of RAM.

[Electronics Wizardy]

1 minute ago, Lasky said:

Well, we do have logging enabled, the files themselves are only a max of around 2MB, which we plan to start storing on Google Drive which will always sync as a backup.

 

WD Red's to my understanding are for NAS servers and not for heavy workloads like running game servers.  Please correct m if I am wrong here.

 

I am aware that WD Blacks are desktop drives, but they are the only ones that I know of that provide the performance that I require for running our game servers.  We did start with a 5400rpm drive and loading was very slow and needed to be improved.  I do agree that I COULD use an SSD as a caching drive, but SSD's with our hosts cost a fortune where as hard drives are only a fraction of the cost.  currently, we have a 240GB SSD (which runs Windows and programs) and four 1TB hard drives, one the WD Black I have described above.

 

I have 72GB of RAM, which helps a lot, but I also have a 56GB page file on the SSD for a virtual 128GB of RAM.

 

WD reds have things like tler and vibration restance that makes them much better in server uses.

 

A caching ssd really isn't that expensive, adding a 250gb ssd will be less than 100usd

 

Why not run vms? That would make this much easier to secure. Virtual machines are nice and just make everything better. You have hyper-v in server 2016 or use a hypervisor like proxmox or esxi.

[Lasky]

I only have a single 1Gbps port at the moment, and I have tried to set up VM's and at this stage, the port goes from 1Gbps to 10Gbps, but the through put on the main server is only 100Mbps despite it being a 1Gbps port.

[Electronics Wizardy]

Just now, Lasky said:

I only have a single 1Gbps port at the moment, and I have tried to set up VM's and at this stage, the port goes from 1Gbps to 10Gbps, but the through put on the main server is only 100Mbps despite it being a 1Gbps port.

huh?

 

You can setup vms with a single network port. Network speed won't affect vms at all.

 

It will make a virtual switch to connect the vms to the network. That switch is 10gbe+, but the rest of the network is normal

[Lasky]

When VM's are set up, the port on the main server drops to 100Mbps.