Medical CT Computer BSOD City (Windows XP)

spl3001 · July 31, 2019

This story begins Monday morning. I'm informed that there is a problem with one of our CT computers and it has a BSOD. I had just sat down for the day, and I got the feeling this was going to foreshadow the rest of the week.

Patient's vitals:

Windows XP (unsure of service pack version)
Dell Precision T5500 (6 core Intel Xeon)
12 GB DDR3 ECC RAM (don't have spares I can test at the moment)
300 GB Fujitsu SAS Hard Drive connected to a...
Dell USC-61 (made by LSI) RAID controller card
Two DVD writer drives

Patient's initial symptoms included:

STOP: 0x0000008E (0xC0000005,0xE0D02E33,0xF26889B8,0x00000000)

With a memory dump that I thought I'd be able to get to in short order, but that would not be the case. A Google search indicated that this stop code is related to PAGE_FAULT_IN_NON_PAGED_AREA. Or some general posts I found talked about the memory modules having gone bad. I took photo evidence of the error and rebooted.

Windows XP booted up to the login screen. I attempt to log in with the correct credentials. The computer thought about this for a minute and decided it was not interested in logging in today:

CONFIG_INITIALIZATION_FAILED

STOP: 0x00000067 (0x00000001,0x0000000F,0xC000009A,0x00000000)

Ok, new error. This seemed to be pointing to either more memory trouble OR some issue with the registry being corrupted (or both, who knows).

Let's try safe mode first:

Windows could not start because the following file is missing or corrupt:

system32\DRIVERS\isapnp.syk



You can attempted to repair this file by starting Windows Setup using the original Setup CD-ROM.

Select 'r' at the first screen to start repair

That is NOT a type. It said isapnp.syk and NOT isapnp.sys. I tried googling for that exact phrase but could not find ANYTHING closely relevant. This is where things started to get...complicated.

So I thankfully found an ISO for Windows XP SP2 that I had laying around from a decade or so ago. I used the Rufus utility to make a bootable USB flash drive and walked back to the computer and used Dells F12 menu to select USB Device as the boot option. To my surprise (but later I learned at least why this happened), I got another kind of BSOD:

STOP: 0x0000007B (etc.)

If I remember correctly, this had to do with boot media not being present. This is really a side note because it had to do with the BIOS being in RAID (and NEEDING to be in RAID because of the system drive being on a RAID controller), and not something like compatibility/IDE mode (this also seems to happen when AHCI is selected on older systems).

With that brushed aside mentally, I started to hunt for a burnable CD, thinking I might be out of luck because CDs are soooooo 2000s! Deep in an office cabinet I found an unmistakeable 100 DVD-R (and turns out a mix of CD-Rs) plastic cylinder. I whisk away the whole thing and make a Windows XP SP2 CD for the first time in more than 10 years.

Disc in drive. Boot up. Press any key. Press F6 for third party drivers prompt. Don't need em. Wait. Wait. Wait. No drive found. Uh oh...I need third party drivers. Google for the Dell UCS-6 Windows XP drivers for nearly an hour because who would have THAT any more? I actually found them but learned a terrible bit of information in the process.

Windows XP Setup will ONLY load third party drivers with the F6 prompt from the A: drive. This computer:

1.) Doesn't have a floppy drive

2.) Who on the entire planet has a floppy?

Because our entire lab operates on a shoestring budget and equipment from 1995, a coworker not only had a USB floppy drive, but a perfectly functional 3.25" floppy I could borrow because he still transfers files from a lab machine to his computer with it.

I didn't have much hope, but cutting to the chase, I booted ALL the way into the Recovery Console.

chkdsk found errors and repaired (why doesn't it give more detail than that?). I ran it again. No errors the second time. Wish I could could have done a system file checker command (can't remember what the command is at the moment), but that's not apart of Windows XP Recovery Console. I closed the system and ran MemTest86+ for 18 hours overnight. All tests passed.

I eventually find out that problems have cropped up before when "the drive fills up". It has two partitions, a 20 GB for C:\, and approximately 260 GB for what appears to be a weird RAID setup (Linux showed an LBA over a NTFS partition?). I booted into Ubuntu from a USB (with no issue) to examine the layout, but I didn't take a picture. I moved some files onto a USB hard drive, but couldn't clear out too much space on C (it currently has 12 GB free) as most of the folders are special/weird and I don't want to screw this up more than it already is. Could the page file not have room to be created/used any more?

I learned about Hiren Boot CD near the end of the day today and tried the latest version, but realized it is a Windows 10/PE environment and may not be suitable for repairing Windows XP. I started downloading version 15.2 from their archive which appears to be around circa 2012 and may work based on a forum post I found using it to fix a similar issue to the one this computer has. It has some registry back up recovery abilities apparently.

I did try a system restore early in the process but that gave me either one of the above BSODs or the isapnp.syk error (I can't keep track any more). I did copy isapnp.sys to the drivers folder (keeping the old one), but it had no affect.

Sorry to drag this out, but I wanted to detail all the attempts made so far and to provide a relatively complete picture of problem.

Is there anything I should try next? Hiren "Registry Repair Wizard"? Some other utility to try to restore the registry? Manually restore it? Are there even backups of the registry on Windows XP? I've considered imaging the disk with an offline utility and seeing if it can boot from that, but I'm going to wait until tomorrow.

Any help or suggestions are greatly appreciated.

Arttu89 · July 31, 2019

If all the system repair options fail then your best bet would probably be setting it all up from scratch, as in reinstalling the OS and setting up all the drivers for the CT scanner, that's optimistically assuming that none of the hardware actually died.

I've had the luck of not having to work with any older digital medical equipment, but I know it can be a pain to get it back up and running on any kind of modern day PC, depending on what old standards it relied on to interface with the computer correctly.

spl3001 · August 1, 2019

I'm afraid the vendor most likely has all the software cards up its sleeve. The hope was it could be repaired without resorting to a $1350 300 GB hard drive and a $1350 Dell UCS-61 RAID controller card (actual quoted prices from the CT vendor, labor NOT included [another $3,000]).

In any other sane situation, I'd whole sale replace the system with modern computer hardare, failing that, rebuilding the system on a new drive/controller (combined price of ~$70). Kinda got my hands tied. And this was just a last ditch effort to avoid those costs, so we may have to go that route anyway. My extremely optimistic hope would be to get the system to some kind of working order and then make a dozen clones of the working drive.