Trap in WSAD j9vm21.dll in debug mode

For a long time now, a few of my co-developers have been trying to figure out what caused this trap in WSAD:

Type=GPF vmState=0xffffffff
Target=20030915 (Windows XP 5.1 build 2600 Service Pack 2 x86)
ExceptionCode=0xc0000005 ExceptionAddress=0x1000ac84 ContextFlags=0x0001003f
Handler1=0x111016c0 Handler2=0x10019730
Module=c:\WSAD512\runtimes\base_v51\java\jre\bin\j9vm21.dll
Module_base_address=0x10000000
Offset_in_DLL=0x0000ac84
EDI=0x0011d9c8 ESI=0x008e5e60 EAX=0x74753f3e
EBX=0x0011d9c8 ECX=0x03b5cf14 EDX=0x7463656a
EBP=0x000be300 ESP=0x04381644 EIP=0x1000ac84
Generating crash dump: C:\Documents and Settings\c09974\j9.dmp

I had a quick look a while back, and couldn’t see anything obvious, and my WSAD was working OK, so I didn’t spend too much time on it Anyway, yesterday I found that I was in the same position, and the only thing that had changed on my machine was that I had installed Mercury Interactive QuickTestPro.

Not being able to have my server running in debug model with hot-swap turned on was completely unacceptable, so I spent a bit of time trying to track down what the problem was.

Luckily, I was only temporarily installing QuickTestPro to help another colleague, however, the problem remained even after I had uninstalled it! Yikes!

So, I used a copy of ListDLLs from SysInternals to see what DLLs were in use at the time of the dump. I then compared that list to the list of DLLs in use on a colleague’s machine that wasn’t dumping. The only difference between was a file called c:\windows\system32\micsupp.dll. Doing a hexdump of that DLL gives me a few clues such as the following strings:

micsupp.dll
ClassLoadHook
JVM_OnLoad
SOFTWARE\Mercury Interactive\Java Add-in\Modules


Lovely! It looks like Mercury is mucking around with my default Java installation. I tried to renamed the dll to something else, but then I can’t even start WSAD! So, what to do? Perhaps it didn’t uninstall properly, so I had a look at the c:\Program Files\Mercury Interactive\QuickTest Professional\dat directory. In there, I found a bunch of uninstaller files that I tried to run by using the following command:

\windows\miuninst6.exe miuninst.ini
\windows\miuninst6.exe QTP65P13_NS702_SUPPORT.ini
\windows\miuninst6.exe QTP65P16.ini
\windows\miuninst6.exe QTP65P63.ini
\windows\miuninst6.exe QTP65P84.ini
\windows\miuninst6.exe QTPJAVA65P08.ini


However, that still didn’t solve the problem! However, I did find a file called install.log in that directory that contained (amongst other stuff):

Installing Standalone VMs (JDK/JRE/Oracle1.3.X/Netscape 6)
[
  Copying micsupp.dll from C:\Program Files\Common Files\Mercury Interactive\SharedFiles\JavaAddin\bin to C:\WINNT\syste
m32\
  ...succeeded (return value is 0 )
  Adding _classload_hook environemtn variable
  Adding _JAVA_OPTIONS environemtn variable
]

Based on this, I thought that perhaps I should check my environment variables. Looking at those, I found that Mercury Interactive had created the following environment variables:

_classload_hook
_JAVA_OPTIONS
CLASSPATH
IBM_JAVA_OPTIONS
LSERVRC
MIC_CLASSES


So, I just removed them all, and believe it or not, WSAD started up with no problems, and I could now resume debugging with hot-swap. Thanks so much Mercury Interactive for such a quality product – not!!

Now, fortunately for me, I don’t actually need the Mercury Interactive QuickTestPro product on my machine, so I am confident that this fix won’t affect my normal operation. However, if you need to use QuickTestPro, I am not sure what affect removing these environment variables will have on your installation.