Adding performance to an old office server.

[Comrade Suhov]

Good day,
I need some advice in bringing some more juice to a 2012 'server' PC tasked with running a legacy accounting program and printing tasks. It's configured as a terminal server for a small office with 3 RDP clients.

 

The current specs are:

• Sandy Bridge i5-2400 4C/4T @ 3.1 GHz,
• Gigabyte H61M-DS2V motherboard,
• one stick of 2 Gb DDR3 1333 MHz
• 320 Gb 7200 Rpm Seagate HDD
• Windows Server 2003

 

The server uptime is about 140 hours a week, and it's pretty much idling most of the time. However, the nature of the load is when the terminal client performs some specific action in the accounting program, every client could face a small 5-30 second hang in said program. After closely monitoring the performance, I saw that that client consumes 100% of their allocated CPU (aka 1 core) during that time. Ram is rarely even at 40% of load, and, surprisingly, disk I/O load is sitting at 10-20% at most heavy tasks.

 

Unfortunately, there is nearly no budget to upgrade components, let alone to get a new platform. About 200 USD at best.

 

My initial plan was to upgrade the disk performance with some 256 GB SATA SSD with DRAM cache and reliable 5+ year warranty brand (aka Samsung 870 EVO or Kingston KC600) and leave the current HDD as a mirror image backup and backup for the accounting program database.

 

Next is the CPU. The type of a current OS (Server 2003) and the program itself (the last stable release was in 2003 for 32-bit Win XP and 2000) do not scream "multithreading" exactly, but I believe something faster will help. I'm thinking of moving from Sandy Bridge to Ivy Bride i7-3770 (K) for raw performance boost and HT. And then, I found that there are some used Intel Xeon E3 are for sale for about the same price: E3-1275 v2, E3-1245 v2 and 1265Lv2. No idea if Xeon's are any better, but at least it says 'server', not a gaming rig from 2012.

 

And lastly, while the H61 chipset does support Ivy Bridge CPU, I was surprised to find out that it has only SATA2 ports. It barely supports anything, truth to be told, even RAID. However, used Q65-Z68 motherboards cost way more than they should, let alone Q-/H-/Z77 ones. Paying nearly a hundred bucks for a ten-year-old used mainboard doesn't sound reasonable. Is there a need to replace this motherboard? Just a month before, I was pleasantly surprised how was 2006 rig with W10 and a single SATA 2 got 'Brrr' when I plugged an SSD there.

 

Is this a good upgrade or am I missing something?

[leadeater]

44 minutes ago, Comrade Suhov said:

The type of a current OS (Server 2003) and the program itself (the last stable release was in 2003 for 32-bit Win XP and 2000) do not scream "multithreading" exactly, but I believe something faster will help

Seriously that OS is a huge risk to be running. Run up a VM on a desktop or something and see if you can get it working in Server 2008 32bit. If that works then try Windows Server 2016, you can run 32bit Apps on 64bit OS usually perfectly fine however there are rare cases where it doesn't work, but those are very rare.

 

44 minutes ago, Comrade Suhov said:

Unfortunately, there is nearly no budget to upgrade components, let alone to get a new platform. About 200 USD at best.

Well that isn't going to go very far now days sadly.

 

44 minutes ago, Comrade Suhov said:

My initial plan was to upgrade the disk performance with some 256 GB SATA SSD with DRAM cache and reliable 5+ year warranty brand (aka Samsung 870 EVO or Kingston KC600) and leave the current HDD as a mirror image backup and backup for the accounting program database.

Sounds good to me.

 

44 minutes ago, Comrade Suhov said:

Next is the CPU. The type of a current OS (Server 2003) and the program itself (the last stable release was in 2003 for 32-bit Win XP and 2000) do not scream "multithreading" exactly, but I believe something faster will help. I'm thinking of moving from Sandy Bridge to Ivy Bride i7-3770 (K) for raw performance boost and HT. And then, I found that there are some used Intel Xeon E3 are for sale for about the same price: E3-1275 v2, E3-1245 v2 and 1265Lv2. No idea if Xeon's are any better, but at least it says 'server', not a gaming rig from 2012.

For Xeon in this situation not at all worth it to be honest. Consumer desktop CPUs are "faster" core for core, Xeons is about ECC support and on the larger platforms getting a lot more cores but neither is of much benefit here. Money for no improvement so I'd not do this. If you have money available after anything else and you can find something cheap enough, should easily be able to, then it might be worth doing 3770K or Xeon.

 

44 minutes ago, Comrade Suhov said:

Is there a need to replace this motherboard?

No, SSD even on a SATA2 port will give a vast improvement to system performance and responsiveness. Of all the improvements you can make it goes

1. Minimum viable RAM
2. An SSD
3. Increase RAM
4. CPU

Main thing for me however is this entire situation is a huge risk and the business should be prioritizing it much more. If not having this server online, aka it's dead forever, and that significantly impacts the business operations then sufficient funds should be allocated to it, simple as that. Business has various different forms of insurance, well this is literally the same thing.

[wseaton]

Long story short is you have a legacy app on a legacy server and *have* to maintain it. Sigh....

 

I try to run from scenarios like this unless its a cash strapped non profit with no other recourse.

 

Internet sites have warehouses full of refurbished higher end i5 and i7 boxes for a couple hundred bucks. Load it up on an SSD and migrate the old PC to virtualbox on whatever host OS you want.

 

The application hangs bug me more. My suspicion is its a database issue with the old app. Probably table extent rot in a table, but its only a guess. Any tools in the app to due data maintenance? Check windows logs for any kind of hardware or disc errors. 

[Sir Asvald]

20 hours ago, Comrade Suhov said:

 

 

SERVER 2003?!!?1?1?1?1?/1?!

 

Spoiler

 

 

 

Best thing you can do is this:

 

Spoiler

 

Trust me, you're going to need a new system, 200 USD is the ONLY BUGDET?! seriously how can these people think that you can only say 200 is enough, you need do speak to management and tell them you're going to need more than 200 USD.

[Comrade Suhov]

On 1/15/2022 at 6:27 PM, leadeater said:

Seriously that OS is a huge risk to be running. Run up a VM on a desktop or something and see if you can get it working in Server 2008 32bit. If that works then try Windows Server 2016, you can run 32bit Apps on 64bit OS usually perfectly fine however there are rare cases where it doesn't work, but those are very rare.

When this system was built around 2012 and probably I was tasked to get "just a PC" for it, the Tech guy said it's safer to go with 2003 instead of 2008 for whatever reasons. Now I'm much more tech-savvy and see the flaws of a build and a system. Since this summer, I was step-by-step upgrading this organization clients to Windows 10 (usually from XP SP3), adding RAM and SSD's here and there. Even replaced a 16-year-old PSU with a power-on button that pulsed... Menacingly.

 

Good thing you mentioned 32-bit nature. After some research, I discovered that this program cannot be installed in any modern OS, but it can be installed in a 32bit environment and then simply copied to a new system as an unpacked folder.

Quote

For Xeon in this situation not at all worth it to be honest.

The thing with Xeons - they are 10-15 bucks cheaper on an aftermarket compared to 3770k, even a K-less one. Damn this chip shortage. Also, isn't Intel moved from solder to thermal paste at this time? Do Xeons have a solder thermal interface compared to the Core series?

Quote

Of all the improvements you can make it goes

1. Minimum viable RAM
2. An SSD
3. Increase RAM
4. CPU

Main thing for me however is this entire situation is a huge risk and the business should be prioritizing it much more. If not having this server online, aka it's dead forever, and that significantly impacts the business operations then sufficient funds should be allocated to it, simple as that. Business has various different forms of insurance, well this is literally the same thing.

With 2003 server it's not a RAM bound, I was quite surprised to find it only got one stick. Probably 2012 or 2016 server uses much more than 2003, though.

In that case, I t won't rush with CPU upgrade then. I'll remove the dust, replace thermal paste and plug in a Samsung drive and try to set up a new OS there.

 

I thought the server could use some more horsepower in case this year this organization will switch to a more modern version of an accounting program, which is quite heavy on CPU and disk load even in single-user scenarios. I had one installed locally in the office as a test playground. In short, it was ugly. It instantly ate all the 2 GB of RAM and pagefile and most button presses were greeted with 15-60 second hangs until I moved it to an SSD and get some more RAM.

 

16 hours ago, wseaton said:

Internet sites have warehouses full of refurbished higher end i5 and i7 boxes for a couple hundred bucks. Load it up on an SSD and migrate the old PC to virtualbox on whatever host OS you want.

Maybe it's a COVID and a chip shortage but I never even seen an old CPU's could cost as much. I'm seeing some Haswell i5's bundled with entry level motherboards for about 200 USD, and some Ryzen 2600 for about the same money.

Quote

The application hangs bug me more. My suspicion is its a database issue with the old app. Probably table extent rot in a table, but its only a guess. Any tools in the app to due data maintenance? Check windows logs for any kind of hardware or disc errors. 

Before I thought it was purely caused by HDD, it's got 68k hours Power-On and about 18k of Hardware ECC Recovered - SMART is marketed as unsafe. But when I used HWMonitor I saw that at worst case hangs HDD is barely 15% loaded. The old version of the accounting base got some tools to re-organise its tables and stuff, but again, they're all from 2003. I searched online for the same problems and most IT guys from the past wrote that this program only can be sped up by raw GHz per user and RAID 1 configs.

 

[wseaton]

I know everybody is jumping on you about 2003, but so what. I've worked with just as many crappy 64bit apps with loads of security holes and I want to remind the 2003 haters that we have log4j problems on state of the art platforms. I have clients running proprietary software on NT 4. Its been migrated to a VM and can run forever. 

As long as the box isn't surfing the internet or directly exposed to the internet get over it folks. Often the software maker puts a gun to your head and forces you to a subscription or cloud model with an excessive price.

 

An advantage to virtualizing this machine to a higher performance desktop is you can test if that helps performance while still leaving the production server online. Basically you are copying it. 

 

RAID 1 won't improve database performance.

[leadeater]

14 hours ago, Comrade Suhov said:

The thing with Xeons - they are 10-15 bucks cheaper on an aftermarket compared to 3770k, even a K-less one. Damn this chip shortage. Also, isn't Intel moved from solder to thermal paste at this time? Do Xeons have a solder thermal interface compared to the Core series?

Not surprising the Xeon is cheaper to be honest, way more used Xeons hit the market than desktop CPUs. I believe the Xeons were soldered yes but it's been a long time since I looked in to that. For this generation it really doesn't matter either way as the peak power is quite low.

 

14 hours ago, Comrade Suhov said:

Probably 2012 or 2016 server uses much more than 2003, though.

Correct, it's a rather bad time to run these with anything less than 4GB. XP/2003 was pretty sweet in that regard. Oh how times have changed.

 

14 hours ago, Comrade Suhov said:

I thought the server could use some more horsepower in case this year this organization will switch to a more modern version of an accounting program, which is quite heavy on CPU and disk load even in single-user scenarios. I had one installed locally in the office as a test playground. In short, it was ugly. It instantly ate all the 2 GB of RAM and pagefile and most button presses were greeted with 15-60 second hangs until I moved it to an SSD and get some more RAM.

Modern day software for you, basically most things are like this now. But yea with how little performance uplift you'll realistically get sticking with the same platform generation and CPUs supported it's just really not worth upgrading for such a minor improvement. Even going to something like Intel 7th or 8th Gen would lead to a massive upgrade at what I would assume reasonable cost, I have not checked but surely there are used Xeons in the E3 line for low cost, it'll be the motherboard that's the cost killer.

 

But you can get off-lease/ex-lease OEM (HP, Dell etc) business product line desktops/workstations ready to run with typically 1-3 month warranty, sometimes with 1 year warranty purchase upgrade possible, for extremely reasonable cost.

 

6 hours ago, wseaton said:

As long as the box isn't surfing the internet or directly exposed to the internet get over it folks

Diligence still dictates advising migrating away from it if possible.

[jec6613]

First, I'd skip the SSD, provided you're not seeing any queue depth spikes.  With a $200 budget and a limit of a 32 bit OS, there isn't going to be much you can do, and you really need to get this thing fully overhauled with a proper budget.  You're on an unsupported OS that's waiting to get owned - if anything makes it past the firewall to any system, this server is hosed.  Second, you're on a 32 bit OS version, which for the era this was installed indicates to me that there are 16 bit components to the application - definitely no bueno.

 

Long term: new system on a modern OS with a new accounting system.  This will cost more than $200 though (probably five digits), and you need to levelset to somebody with the checkbook that this needs to be done SOON.  Like, before the end of 2022 soon, budget it now and plan now.

 

To kick the can down the road until then though, you're single thread bound it appears, so I'd do a few things:

1. Add another memory module.  Preferably an exact match to your existing one (should be pretty cheap).  This gives your 32 bit OS more room to memory manage into and dedicated kernel space, but more importantly doubles your memory bandwidth to avoid stalls bringing data to the CPU.  You should expect a noticeable performance bump from just this.
2. Upgrade the CPU to an i7 or equivalent Xeon E3.  The E3 series Xeons are basically i5's and i7's with ECC support, and you want the one with the highest out of the box boost clocks you can get your hands on.  Performance won't be any different between the i7 and a Xeon E3, since the delta in this generation is ECC support vs overclocking support, but you need more single threaded performance.  Also verify that the motherboard will support the CPU in question - many won't.
3. See if you can enable it to use two cores.  With HT and a fully patched Server 2003 scheduler, this should help a lot of things responsiveness-wise, but you'll want that sweet quad core CPU before doing so.
4. Make sure backups are working and test a restore.  Like, seriously, this is no parts cost and for an accounting system, extremely critical for something of this age.  You could blow a capacitor tomorrow and be up a creek scrambling to find parts, this system is already on borrowed time as it is.
5. Go and grab all of the 2003 out of band patches issued after OS support ended and make sure they're applied.  There are a few, some fairly recently, for highly wormable vulnerabilities.
6. Make sure things like password, account lockout, and cryptographic policies are set to something sane and secure.  Everything else you have is newer, so there's no need for LM and NTLM, just NTLMv2.  Ditto old cryptographic protocols.  You can at minimum kill off a lot of known attack vectors this way, just look at the Server 2019 security baseline configuration (not so much 2022 for a variety of reasons), many of those things can be applied all the way back to Server 2003 and should be provided nothing breaks when you enable them (nothing should, but test first).  Oh, and disable the print spooler unless you really need to be using it.
   Edit: #6 will also improve performance by shutting off a lot of unneeded services.

[Comrade Suhov]

On 1/17/2022 at 11:43 AM, leadeater said:

Even going to something like Intel 7th or 8th Gen would lead to a massive upgrade at what I would assume reasonable cost

So, I was sitting there, watching the prices on used i5's of 7th and 8th gen on a market. And then I was, "hold on, these aren't right". Checked the local store prices and indeed - these used i5 and i7 from 2018 cost 0-20 USD less compared to bright new i5-10400 on a shelf.

 

I quickly wrote down something like:

• i5 10600 with Box cooler
• Asus/MSI microATX with H470 chipset
• Some Crucial DDR 4 2x8Gb 3200 Hz 22-22-22
• Intel 760p 256 Gb

And the net total was like 430 USD. Asked the guys around, and most said that the new version of said accounting program is fine even with 8Gb for a time being, given the low number of concurrent users, it mostly needs crazy IOPS performance and CPU power.

 And then I looked down on that config again and thought "hey, if I add a generic case and some 80+ Bronze PSU, it's an entirely new rig", and I could probably sell the current 'server' for about $100 as an office machine and save extra. On the other hand, I have my own 7700k z270 build, which is just a few weeks from becoming 5 years old. If only I could find a good use for it...

[jec6613]

4 hours ago, Comrade Suhov said:

So, I was sitting there, watching the prices on used i5's of 7th and 8th gen on a market. And then I was, "hold on, these aren't right". Checked the local store prices and indeed - these used i5 and i7 from 2018 cost 0-20 USD less compared to bright new i5-10400 on a shelf.

 

I quickly wrote down something like:

• i5 10600 with Box cooler
• Asus/MSI microATX with H470 chipset
• Some Crucial DDR 4 2x8Gb 3200 Hz 22-22-22
• Intel 760p 256 Gb

And the net total was like 430 USD. Asked the guys around, and most said that the new version of said accounting program is fine even with 8Gb for a time being, given the low number of concurrent users, it mostly needs crazy IOPS performance and CPU power.

 And then I looked down on that config again and thought "hey, if I add a generic case and some 80+ Bronze PSU, it's an entirely new rig", and I could probably sell the current 'server' for about $100 as an office machine and save extra. On the other hand, I have my own 7700k z270 build, which is just a few weeks from becoming 5 years old. If only I could find a good use for it...

On a going forward basis, do yourself a favor and get 11th Gen.  Waste of sand on the gaming side, but for productivity, the VBS feature in Windows nets less than 10% of the performance penalty as it does on 10th gen ... and the cost is effectively free.