Bug Check 0x50, WINDOWS 2008 X86 Server

Good day All,

Welcome back, Today will share with all some issue we encountered on a Windows 2008 x86 Version of a Server,

For some client requirement we had to build a Windows 2008 x86 Server, it was stable for a while till recently every other week it would do a unexpected reboot..As the Server was having 36 GB of memory we only had mini dump enabled.. and Mini dump was pointing to Bug Check 50.. When we check in Windbg about bug check this is what it showed..

Resolution

Resolving a faulty hardware problem: If hardware has been added to the system recently, remove it to see if the error recurs. If existing hardware has failed, remove or replace the faulty component. You should run hardware diagnostics supplied by the system manufacturer. For details on these procedures, see the owner's manual for your computer.
Resolving a faulty system service problem: Disable the service and confirm that this resolves the error. If so, contact the manufacturer of the system service about a possible update. If the error occurs during system startup, restart your computer, and press F8 at the character-mode menu that displays the operating system choices. At the resulting Windows Advanced Options menu, choose the Last Known Good Configuration option. This option is most effective when only one driver or service is added at a time.
Resolving an antivirus software problem: Disable the program and confirm that this resolves the error. If it does, contact the manufacturer of the program about a possible update.
Resolving a corrupted NTFS volume problem: Run Chkdsk /f /r to detect and repair disk errors. You must restart the system before the disk scan begins on a system partition. If the hard disk is SCSI, check for problems between the SCSI controller and the disk.

Finally, check the System Log in Event Viewer for additional error messages that might help pinpoint the device or driver that is causing the error. Disabling memory caching of the BIOS might also resolve it.

Well the first thing we did was updating the Server with Latest HP Support Pack 2014, that didn't help and Server had unexpected reboot again in weeks time..So next thing we check was any antivirus problem with antivirus team and it was clean .. later even checked chkdsk for any disk errors but nothing we could find.. the server was  rebooting every week..and the issue was getting heated up..

So we circled back raised a ticket with hardware vendor .. the only thing they found was memory modules was not in a order so they asked us to try putting in a order .. well we requested downtime and then tried it too.. well this time after 10 days we had the same issue..

Now the issue was getting more attention even though it was internal server we had this issue close to 2 months now..
After all the options the only options we had left was enable Full memory dump , so move files around to accommodate at-least 50 GB  of dump because as i said Server is stacked with 36 GB Memory. We started to wait for a couple of days and then we could capture Memory dump and it was close to 46 GB...

So we started to analyze using Windbg and this is what the analyze -v showed for stack

STACK_TEXT: 
9f9f7964 81c67de4 00000000 e3d64e18 00000000 nt!MmAccessFault+0x10b
9f9f7964 81d96782 00000000 e3d64e18 00000000 nt!KiTrap0E+0xdc
9f9f7a40 81d96258 f307da20 00000000 e3d64024 nt!CmpCheckValueList+0x83
9f9f7a8c 81d9c81a 01000001 009c4020 009c3f70 nt!CmpCheckKey+0x5b4
9f9f7abc 81d9ce48 f307da20 01000001 00000006 nt!CmpCheckRegistry2+0x8c
9f9f7b04 81d9786e 01000001 9f9f7c60 80005a74 nt!CmCheckRegistry+0xf5
9f9f7b60 81d99fdd 9f9f7bb4 00000005 00000000 nt!CmpInitializeHive+0x4c1
9f9f7bd8 81d9c27d 9f9f7c60 00000000 9f9f7c4c nt!CmpInitHiveFromFile+0x19e
9f9f7c18 81d924c5 9f9f7c60 00000000 9f9f7c7b nt!CmpCmdHiveOpen+0x36
9f9f7d14 81d926fa 00000002 81d125a0 00000002 nt!CmpFlushBackupHive+0x2fd
9f9f7d38 81e71cbd 81d1c13c 967612d8 81cbfd4a nt!CmpSyncBackupHives+0x90
9f9f7d44 81cbfd4a 00000000 00000000 967612d8 nt!CmpPeriodicBackupFlushWorker+0x32
9f9f7d7c 81df001c 00000000 c8084d5c 00000000 nt!ExpWorkerThread+0xfd
9f9f7dc0 81c58eee 81cbfc4d 00000001 00000000 nt!PspSystemThreadStartup+0x9d
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

We still seeing the Registry Hive error so we started to search in the Microsoft site for any Registry Hive and Bug check 50, so we came across this article ..
http://support.microsoft.com/kb/2709236/en-us

The article resembled close to our issues, keeping fingers crossed we applied the hotfix and guess what that turned out to be the fix...

Hope this helps someone out there....