websphere - WAS 6.1 java.lang.VerifyError: class loading constraint violated

Question

The environment is WAS 6.1 on Linux, deploying a webapp that uses classes from xercesImpl.jar.

Due to company policy restrictions, the app must be deployed with settings:

Class Loader Order
    Classes loaded with parent class loader first
->  Classes loaded with application class loader first

WAR class loader policy
    Class loader for each WAR file in application
->  Single class loader for application

The WAR file contains a copy of xercesImpl.jar, the same one that was in the classpath when the app was compiled.

When launching the webapp, when Spring tries to parse its configs, it throws:

java.lang.VerifyError: class loading constraint violated 
    (class: org/apache/xerces/jaxp/DocumentBuilderImpl 
    method: parse(Lorg/xml/sax/InputSource;)Lorg/w3c/dom/Document;)

ANALYSIS SO FAR

It appears that WAS provides an implementation of org.apache.xerces.jaxp.DocumentBuilderImpl, because we can remove xercesImpl.jar from the WAR file and still get the same error (not ClassNotFoundException). Thus WAS seems to be resolving the references using its own copy that is incompatible with the references in our compiled class files. However, the only other instance of 'xercesImpl.jar' I can find (other than the copy deployed with our app) is in directory deploytool, which seems to be outside the app server.

I scanned all the jars in WAS (all 1300 of them) with

for i in `find . -name *.jar`; do jar tvf $i|grep -qi xerces && echo $i ; done

and found that ./java/jre/lib/xml.jar contains all the classes in org.apache.xerces.*, so this is likely where the classloader is resolving the reference.

HERE'S THE WEIRD PART:

If we change to "parent class loader first" we do not see the exception. This goes counter to the expected behavior. We would expect that with "application classloader first" it would use the xercesImpl.jar that we provided, and use WAS's version only if we set "parent classloader first". This appears to be backwards from what we actually see.

THE QUESTION:

How is the classloader delegation setting interacting with the above information to result in the observed behavior?

Answer 1

Your WAR is also including either org.xml.sax or org.w3c.dom classes, and then you're referencing a class that is outside your application that also references these classes. This sets up a scenario where your application class loader has visibility to two instances of the same class, which is a linkage error.

For example, if your application uses javax.xml.bind.Unmarshaller.unmarshal(InputSource), then Unmarshaller would be loaded from the JDK, and the Unmarshaller class only has visibility to the JDK InputSource. When your application creates its InputSource, it will load the class from the WAR (because "app first" policy), and then your application would attempt to pass an instance of the WAR InputSource to the JDK Unmarshaller, which can only accept an instance of the JDK InputSource.

There are two solutions:

1. Remove all API jars from your application, and use the ones from the JDK. For example, remove the jar containing org.xml.sax or org.w3c.dom.
2. Include all libraries in your WAR that reference the classes you want to reference. For example, include a copy of the JAXB library in your WAR.

In my experience, linkage errors are quite difficult to track down because JVMs give lousy information about what causes linkages to be added. I usually enable class loader trace, reproduce the problem, and then walk backwards until I find a class loaded from outside the application that "sounds like" it might reference a class that is known to exist inside the application.