Random BSODs

[porina]

It is Friday morning where I am. Since Thursday evening I've had 4 BSODs, with three different reasons so they're not consistent.

 

System is the top one in my sig but key parts are:

7980XE

Asus TUF X299 Mark 2 latest bios

Corsair something 8x 8GB 3000 happened with both XMP on and off

 

The CPU was installed on Tuesday evening and had been working fine until Thursday evening. It replaced my 7920X which rarely gave problems on boot.

 

I did update GPU driver yesterday, but rolled back to older version and still happening.

 

What I don't get is that if the CPU is unstable, why did it work fine for 2 days then BSOD 4 times in the last half day? If it isn't the CPU, I'm at a lost to what it could be.

 

Other things I've tried:

Verified Windows files, no errors.

Read scan of the OS SSD, no errors.

Increase ram voltage slightly, no change. At defaults with XMP off, it was slightly below nominal.

 

I'm about to try resetting the bios in case any settings should differ between the two CPUs, which seems unlikely. If nothing seems to help, I'll put the old CPU back in to see if it is a CPU problem or rest of system problem.

 

Edit: another BSOD just after posting the above! This is getting really annoying.

[Selestial]

16 minutes ago, porina said:

It is Friday morning where I am. Since Thursday evening I've had 4 BSODs, with three different reasons so they're not consistent.

 

System is the top one in my sig but key parts are:

7980XE

Asus TUF X299 Mark 2 latest bios

Corsair something 8x 8GB 3000 happened with both XMP on and off

 

The CPU was installed on Tuesday evening and had been working fine until Thursday evening. It replaced my 7920X which rarely gave problems on boot.

 

I did update GPU driver yesterday, but rolled back to older version and still happening.

 

What I don't get is that if the CPU is unstable, why did it work fine for 2 days then BSOD 4 times in the last half day? If it isn't the CPU, I'm at a lost to what it could be.

 

Other things I've tried:

Verified Windows files, no errors.

Read scan of the OS SSD, no errors.

Increase ram voltage slightly, no change. At defaults with XMP off, it was slightly below nominal.

 

I'm about to try resetting the bios in case any settings should differ between the two CPUs, which seems unlikely. If nothing seems to help, I'll put the old CPU back in to see if it is a CPU problem or rest of system problem.

 

Edit: another BSOD just after posting the above! This is getting really annoying.

If you can, check event viewer and see if it posted any errors that might lead to a bsod

[Levent]

Do you have any control over vcore? if so raise it slightly and see if anything changes.

[porina]

1 minute ago, Meowth LVL255 said:

If you can, check event viewer and see if it posted any errors that might lead to a bsod

The ones I looked at did not point to any particular common source.

 

Forgot to mention in OP, I have tried short stability tests and other checks when I first got the CPU. I'm about to do a memtest on it since I haven't done one in quite a while.

 

Just now, Levent said:

Do you have any control over vcore? if so raise it slightly and see if anything changes.

Actually now you mention it, I did try a small offset of +0.05V, and I think the last BSOD happened while in that state.

[WereCat]

RAM test? If it's unstable it may be corrupting your system. 

[RONOTHAN##]

24 minutes ago, porina said:

three different reasons so they're not consistent

What are those reasons? Different BSOD codes can come up for the same specific hardware issues (pretty sure there's 5+ ones for RAM alone), so it still might point in the general direction of what's the issue. 

[Levent]

8 minutes ago, porina said:

The ones I looked at did not point to any particular common source.

 

Forgot to mention in OP, I have tried short stability tests and other checks when I first got the CPU. I'm about to do a memtest on it since I haven't done one in quite a while.

 

Actually now you mention it, I did try a small offset of +0.05V, and I think the last BSOD happened while in that state.

You can try higher voltages on the ram and see if that changes anything, 0.05v seems a little low but I never used offset mode so I cant really be sure.

[Selestial]

28 minutes ago, porina said:

The ones I looked at did not point to any particular common source.

 

Forgot to mention in OP, I have tried short stability tests and other checks when I first got the CPU. I'm about to do a memtest on it since I haven't done one in quite a while.

 

Actually now you mention it, I did try a small offset of +0.05V, and I think the last BSOD happened while in that state.

Try removing all overclocks and see what happens. 

[porina]

38 minutes ago, WereCat said:

RAM test? If it's unstable it may be corrupting your system. 

One is running now.

 

Is there any preference on using memtest86 or memtest86+ these days? I'm using the first, latest version available today.

 

31 minutes ago, RONOTHAN## said:

What are those reasons? Different BSOD codes can come up for the same specific hardware issues (pretty sure there's 5+ ones for RAM alone), so it still might point in the general direction of what's the issue. 

system thread exception not handled

IRQ not less or equal

 

There was another one but I can't remember it, and can't check it right now. I'll have another look after memtest is done.

 

29 minutes ago, Levent said:

You can try higher voltages on the ram and see if that changes anything, 0.05v seems a little low but I never used offset mode so I cant really be sure.

Since the CPU voltage varies with load, offset just raises it all round.

 

9 minutes ago, Meowth LVL255 said:

Try removing all overclocks and see what happens. 

It is not overclocked and XMP is off.

[porina]

One pass cleared without error.

[porina]

1 hour ago, RONOTHAN## said:

What are those reasons? Different BSOD codes can come up for the same specific hardware issues (pretty sure there's 5+ ones for RAM alone), so it still might point in the general direction of what's the issue. 

SYSTEM_THREAD_EXCEPTION_NOT_HANDLED 0x0000007e (three times)

SYSTEM_SERVICE_EXCEPTION 0x0000003b 

IRQL_NOT_LESS_OR_EQUAL 0x0000000a 

 

[Bjoolz]

Random crash errors and process blamed is usually RAM because it's random which process/driver has data in the bad region. 

 

Go to C:\Windows\Minidump and check if you have any minidump files. If you do, go back to the Windows folder and copy the Minidump folder itself to the Downloads folder (You can use the desktop if you don't have OneDrive syncing files). Zip the copied folder and attach it to a post. Please follow the instructions to the letter as Windows doesn't like you messing with files in this location.

[porina]

On 1/19/2024 at 11:26 AM, Bjoolz said:

Random crash errors and process blamed is usually RAM because it's random which process/driver has data in the bad region. 

On the flip side, I have no evidence to show ram is unstable, even running XMP off and increasing voltage. I know that doesn't prove it is good.

 

On 1/19/2024 at 11:26 AM, Bjoolz said:

Go to C:\Windows\Minidump and check if you have any minidump files. If you do, go back to the Windows folder and copy the Minidump folder itself to the Downloads folder (You can use the desktop if you don't have OneDrive syncing files). Zip the copied folder and attach it to a post. Please follow the instructions to the letter as Windows doesn't like you messing with files in this location.

I found that out earlier when I tried to look at them myself. I have no clue what to look for in them though.

 

https://drive.google.com/file/d/1Gmjac-D2sK60cKm8HKFbzDVpawyzXtQD/view?usp=sharing

Forum wont let me attach it, so I've put it at above link instead. 2.4MB zip. 

[Bjoolz]

10 hours ago, porina said:

On the flip side, I have no evidence to show ram is unstable, even running XMP off and increasing voltage. I know that doesn't prove it is good.

 

I found that out earlier when I tried to look at them myself. I have no clue what to look for in them though.

 

https://drive.google.com/file/d/1Gmjac-D2sK60cKm8HKFbzDVpawyzXtQD/view?usp=sharing

Forum wont let me attach it, so I've put it at above link instead. 2.4MB zip. 

It looks like RAM. There are other issues that can look like RAM like storage, the memory controller or the motherboard, but testing RAM is much easier. Storage and RAM can look a lot like each other, but you usually have a few crashes that quite strongly point to storage which wasn't the case here. 

 

You can use memory testers like Memtest86, but in my experience they miss bad RAM too often to be trusted with clean scans. No false positives that I'm aware of so if it finds errors there are issues with the RAM (Not sure if an issue with the memory controller would show as errors). I much prefer just using the machine normally with half the RAM at a time and seeing if only one half has crashes. If only one half has crashes, that half probably has a faulty stick. With 8 sticks this could be annoying. Almost all RAM vendors offer lifetime warranty, but they want the entire kit back because replacing one stick can give compatibility issues. On that same note, just replacing one of the sticks could cause crashes due to compatibility with the other 7 sticks. I see that all your sticks are the same model number, but I can't tell if they are from the same kit.

 

RAM vendors don't make the chips so every batch could have completely different components. They buy the cheapest parts that can do the specs they want. That makes replacing a stick later possibly annoying and not necessarily as easy as just buying a stick with the same model number. 

[porina]

2 minutes ago, Bjoolz said:

I much prefer just using the machine normally with half the RAM at a time and seeing if only one half has crashes.

The problem with this approach is the randomness of the crashes. Since the last crash, I've not had any more. I did tinkering as discussed but I didn't have high hopes any of it would really make a difference.

 

I do have a sledgehammer approach to this. I have a separate 4x 16GB 3200 kit. I could drop that in and see what happens. If the problem continues, it is unlikely to be the ram. That or I have two similarly bad kits. The thing is I have had the crashes with ram XMP on and off, so 3000 and 2133. If it is relatively speed independent, that could point to a hard(er) fault, which should show up in memtest like tests.

 

2 minutes ago, Bjoolz said:

I see that all your sticks are the same model number, but I can't tell if they are from the same kit.

It was sold as a kit of 8 modules. I am curious what info you can pull out of those dumps.

 

2 minutes ago, Bjoolz said:

RAM vendors don't make the chips so every batch could have completely different components.

Yup, well known by people in the PC building game for a while.

[Mark Kaine]

so you have a new cpu... did you update chipset drivers and bios? 

 

sorry if i missed it.

 

 

[porina]

7 hours ago, Mark Kaine said:

so you have a new cpu... did you update chipset drivers and bios? 

The CPU is same generation with support from the launch of the mobo, similarly with chipset drivers. BIOS has previously been updated to latest and I tried resetting it.

 

 

Anyway, after the bunch of crashes through yesterday morning, the system was behaving itself through the afternoon through overnight. Only this morning did I get another crash. KMODE something something. I guess I'll make the first change - replacing 8x 8GB with 4x 16GB and see if that makes a difference. If crashes keep happening, then I'll swap the old CPU back in but that is a bigger job.

[porina]

System uptime has been over 48 hours without a single crash since the ram change. I guess the problem is gone and it was ram related after all. It leaves open the door, was it the ram itself, the motherboard, the CPU? All the above? Still, I'm happy to have an apparently stable system right now.

 

The ram I took out, some day, I'll try it in other systems. Probably wont be any time soon.

[porina]

 

 

Ok, I'm NOT done with this yet. Yes, it has taken me 4 days to realise that I'm missing a channel - I haven't rebooted the system since I swapped the ram. I'm in the middle of a BOINC challenge elsewhere so I don't want to touch the hardware for another 2 days when it is over. But I guess that kinda shows the problem I had.

 

I think this might be beyond just reseating ram. Gonna get the alcohol out and give the CPU contacts a deeper clean. I did think the pads on the CPU placed in the system days before the crashes were less shiny than the one it replaced.

 

Edit: I can see the LEDs on all 4 modules are lit up. CPU-Z can read the SPD on all 4 modules.

[porina]

I took the CPU out and gave it a good clean on the contacts using alcohol. The swab was visibly dirty. Put everything back and... no change.

 

After some more testing, it seems to be the ram, not the mobo.

Slot 1 -4, ram sticks A-D

1A, 2B, 3C, 4D - 48GB detected
1A, 2B - 16GB detected - either the ram or the slots are the problem. Ram C+D assumed good
1C, 2D - 32GB detected - with ram assumed good, this shows these slots are also good. Ram A or B are suspect.
1C, 2D, 3A, 4B - 48GB detected - A or B still suspect.
1C, 2D, 3A - 48GB detected - stick B suspected bad
1C, 2D, 4A - 48GB detected - confirms all slots seem ok
Clean the contacts on module B
1C, 2D, 3B, 4A - 48GB detected - returning B did not add capacity. I'll pull it and write it off for now. Might look at it again another time.

 

I was sure all 4 sticks were working when I took them out of another system. I'm not in the mood to diagnose further right now and will just run with 3 sticks.

[porina]

To follow up on my ram woes, new information.

 

In my previous post with one stick not working out of a 4x 16GB kit, I took that sus stick and put it in my 11700k system by itself. It booted to Windows. I didn't test beyond that yet.

 

 

Back to the 8x 8GB kit I originally had problems with. 4 of those sticks are in my other X299 system. It has worked problem free since I did that. I'll assume it is ok, but without in depth testing. The other 4 sticks I put in my 11700k system. A couple of games CTD without error. Very sus. I ran Aida64 memory test. In about a minute it came back with an error! There it is!

 

Or is it? So those 4 sticks are now sus. I left 2 in, and ran Aida64 memory test. 5 minutes clear. Maybe it was the other two sticks? Swapped it over. Another 5 minutes clear. I know 5 minutes is short, but the first time it errored in a minute. Can I repeat that? All 4 sticks back in. Error in about 12s this time, and I repeated it twice. Ram might not have been in same slots which might account difference in speed of error.

 

I know, 4 sticks in a dual channel system is a "worse" load than 2 sticks. Forgot to add, this testing was at JEDEC 2133 speed. I didn't even turn on XMP. It's about as safe as it gets.

 

In the past I had suspected the IMC of my first 6700k to weaken over time. 3200 2R ram which worked great for years suddenly started erroring unless I backed the speed down to 2133. I still use those sticks in my 11700k system normally as they're higher performance than 1R. Now I have too many variables. For the 8x 8GB kit, could the ram have weakened somehow over time that running 2DPC is no longer stable? That it happened on two different systems doesn't point so much to the IMC.

 

I still need to do further testing on the 16GB module from the other kit. I might try it in the X299 system again and see if it gets picked up now.

 

I think this is some sign I should buy a DDR5 era system.