JNDI lookup from one application to another failing because of classloader

[Former Member]

I have two separate applications - lets say app1.ear and app2.ear. In app2.ear, a ServletContextListener starts a timer thread that tries to lookup a session bean running in app1.ear. FYI, it does this in a timer thread so it can retry at a specific interval to account for the fact that app1.ear may not be accessible yet. I get the following exception:

java.lang.ClassCastException: turtle.registry.SBRegistryRemoteHome

at com.sap.engine.services.cross.PortableRemoteObjectContainer.narrow(PortableRemoteObjectContainer.java:209)

at com.sap.engine.system.PortableRemoteObjectProxy.narrow(PortableRemoteObjectProxy.java:24)

at javax.rmi.PortableRemoteObject.narrow(PortableRemoteObject.java:137)

at turtle.common.server.connect.RemoteFactory.getRemote(RemoteFactory.java:96)

at turtle.common.server.connect.RemoteProxyFactory.getRemote(RemoteProxyFactory.java:99)

at turtle.common.server.connect.RemoteProxyFactory.create(RemoteProxyFactory.java:91)

at turtle.common.server.connect.RemoteProxyFactory.createProxy(RemoteProxyFactory.java:52)

at turtle.server.runtime.registry.RegistryFacade.connectToRegistry(RegistryFacade.java:815)

at turtle.server.runtime.registry.RegistryFacade.getRegistryBI(RegistryFacade.java:870)

at turtle.server.runtime.registry.RegistryFacade.bind(RegistryFacade.java:541)

at turtle.registry.ProjectRegistrar$1$1.run(ProjectRegistrar.java:133)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:337)

at turtle.common.security.auth.AuthenticatorBase.doAs(AuthenticatorBase.java:387)

at turtle.registry.ProjectRegistrar$1.run(ProjectRegistrar.java:168)

at java.util.TimerThread.mainLoop(Timer.java:512)

at java.util.TimerThread.run(Timer.java:462)

I have turned on logging for com.sap.engine.services.cross to get the following output:

#1.5 #00065B3FCBDD000A0000012800001018FFFFFFFFFFFFFFFF#1173631540711#com.sap.engine.services.cross#turtle.com/s2pctra_Test#com.sap.engine.services.cross#usernameTBD#10#####Thread[Timer-6,5,ApplicationThreadGroup]##0#0#Debug##Plain###PortableRemoteObjectContainer.narrow(java.lang.Object, Class) : Casted object classloader = SharedClassLoader[/L_turtle.com\#turtleProject]#

#1.5 #00065B3FCBDD000A0000012900001018FFFFFFFFFFFFFFFF#1173631540711#com.sap.engine.services.cross#turtle.com/s2pctra_Test#com.sap.engine.services.cross#usernameTBD#10#####Thread[Timer-6,5,ApplicationThreadGroup]##0#0#Debug##Plain###PortableRemoteObjectContainer.narrow(java.lang.Object, Class) : Classloader of the interface = SharedClassLoader[/L_turtle.com\#s2pctra_Test]#

Which clearly indicates that the objects have different classloaders. I realize that this problem is caused by having the same classes defined in both applications.

I tried adding a reference from the calling application to the callee application but that causes many other problems. One problem is that now it loads all property files (all of our generated applications have thier own property files with the same names in each application) from the callee application.

How can I work around this issue? Can I make the system ignore the classloader of the class? All of our applications are delivered with the same set of jar files and many of the same supporting classes are delivered directly in each ejb.jar. Creating another application whose sole purpose is to contain the shared classes would be really painful - especially since we don't have to do that for the other application servers that we support. This kind of thing would make it very difficult to port to SAP WAS because we would have to rearrange the entire make up of every ejb.jar file.

I have read the document about Accessing EJB Applications using JNDI but it sounds like the only route is to add application references.

Does anyone have any suggestions?

[suresh_krishnamoorthy]

Hi,

Check this link:

http://help.sap.com/saphelp_nw04/helpdata/en/be/2e2c4142aef623e10000000a155106/frameset.htm

Regards, Suresh KB

[Former Member]

Thanks for the quick response Suresh. I had already visited that link and I don't think it will help me in this case. Unless I am missing something?

I think this link better describes the exact problem I am having:

http://help.sap.com/saphelp_nw04/helpdata/en/4d/993441c73cef23e10000000a155106/frameset.htm

In my case:

app1.ear has registry_ejb.jar with stateless session bean SBRegistry and all related classes in the registry_ejb.jar.

app2.ear has app2_ejb.jar which has a SessionContextListener that starts a timer thread that does jndi lookup for SBRegistryRemoteHome (remote home interface). app2_ejb.jar also has all SBRegistry related classes.

When calling narrow() on the remote home I get a class cast exception because the remote home class returned from the lookup call was loaded with app1 classloader while the second parameter to the narrow() method is from app2 classloader.

I added an application reference from app2 to app1 which essentially makes app1 the parent class loader of app2. Not unexpectedly everything is loaded from app1 (unless it doesn't exist there) including property files loaded as resources from the classloader. The properties files have the same names in each application with different values for the properties so this causes problems. I am wondering what my options are?

[Former Member]

Would it be possible to somehow serialize the EJBHome (getHomeHandle()) and deserialize it forcing it to use app2.ear's classloader? I tried this but it doesn't even seem to work within the same ear? When I try this I get the following error:

turtle.common.exceptions.UnrecoverableException: java.lang.ClassNotFoundException: Loader /L_service:ejb could not load class turtle.registry.SBRegistryRemoteHome

at turtle.common.server.connect.RemoteFactory.getRemoteSAPWAS(RemoteFactory.java:231)

at turtle.common.server.connect.RemoteFactory.getRemote(RemoteFactory.java:47)

at turtle.common.server.connect.RemoteProxyFactory.getRemote(RemoteProxyFactory.java:99)

at turtle.common.server.connect.RemoteProxyFactory.create(RemoteProxyFactory.java:91)

at turtle.common.server.connect.RemoteProxyFactory.createProxy(RemoteProxyFactory.java:52)

at turtle.server.runtime.registry.RegistryFacade.connectToRegistry(RegistryFacade.java:815)

at turtle.server.runtime.registry.RegistryFacade.getRegistryBI(RegistryFacade.java:870)

at turtle.server.runtime.registry.RegistryFacade.bind(RegistryFacade.java:541)

at turtle.registry.ProjectRegistrar$1$1.run(ProjectRegistrar.java:141)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:337)

at turtle.common.security.auth.AuthenticatorBase.doAs(AuthenticatorBase.java:387)

at turtle.registry.ProjectRegistrar$1.run(ProjectRegistrar.java:178)

at java.util.TimerThread.mainLoop(Timer.java:512)

at java.util.TimerThread.run(Timer.java:462)

Caused by: java.lang.ClassNotFoundException: Loader /L_service:ejb could not load class turtle.registry.SBRegistryRemoteHome

at com.sap.vmc.core.impl.sapjvm.sharing.NativeSharedClassLoaderImpl.loadClass(NativeSharedClassLoaderImpl.java:609)

at java.lang.ClassLoader.loadClass(ClassLoader.java:265)

at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:333)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:242)

at java.io.ObjectInputStream.resolveProxyClass(ObjectInputStream.java:656)

at java.io.ObjectInputStream.readProxyDesc(ObjectInputStream.java:1499)

at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1462)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1698)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1304)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at com.sap.engine.services.ejb3.runtime.impl.LocalHandleDelegate.readStub(LocalHandleDelegate.java:177)

at com.sap.engine.services.ejb3.runtime.impl.ServerHandleDelegate.readStub(ServerHandleDelegate.java:64)

at com.sap.engine.services.ejb3.runtime.impl.LocalHandleDelegate.readEJBHome(LocalHandleDelegate.java:101)

at com.sap.engine.services.ejb3.runtime.impl.StatelessHomeHandleImpl.readObject(StatelessHomeHandleImpl.java:73)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:946)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1818)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1718)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1304)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)

at turtle.common.server.connect.RemoteFactory.convertObject(RemoteFactory.java:255)

at turtle.common.server.connect.RemoteFactory.getRemoteSAPWAS(RemoteFactory.java:206)

... 14 more

[Vlado]

Hi Jay,

How do you obtain the SBRegistryRemoteHome object inside app2 in the first place (in order to serialize it)?! Or are you working merely with the javax.ejb.EJBHome interface? Something like:

Context ctx = new InitialContext();
Object obj =  ctx.lookup("name");
EJBHome ejbHome = (EJBHome) PortableRemoteObject.narrow(obj, EJBHome.class);
HomeHandle homeHandle = ejbHome.getHomeHandle();

If so, I'm afraid this will not work either. The reason: you still hope to load class turtle.registry.SBRegistryRemoteHome[app1] (because this is what you get from the lookup) but in app2's classpath you have class turtle.registry.SBRegistryRemoteHome[app2] which, in spite of the name, is a different one.

-Vladimir

[Vlado]

Indeed, you've already done most of the investigations and have perfectly realized and presented the roots of the issue. There's not much that can be added. Actually, you have already outlined the options you have, too.

1) Remove the duplicated classes from all resources of app2 and add reference to app1.

2) Create another app_common to contain all common classes of app1, app2, ..., appN and add the corresponding references to it.

Having a certain class / resource in more than one server-side component and having these components interacting with each other always leads to problems with classloaders. That's why the problem analysis scenario you've linked above also postulates the solution as creating the necessary references.

> Can I make the system ignore the classloader of the class?

This is not possible as it would clearly violate the class loading spec.

> Creating another application whose sole purpose is to contain

> the shared classes would be really painful - especially since

> we don't have to do that for the other application servers that we support.

It would be interesting to know how you solve then this issue on the other application servers? I mean if you don't get a ClassCastException this looks like a violation of the specs.

-Vladimir

[Former Member]

My code is almost identical to what you have.

I am executing the code in app1. It gets the EJBHome for a bean in app2. So now I have ejbHome = SBRegistryRemoteHome[app2] for code executing in app1. This is only okay because the variable ejbHome is of type EJBHome which is loaded in a classloader higher in the hiearchy. I would not be able to cast ejbHome to SBRegistryRemoteHome because that would try to cast to SBRegistryRemoteHome[app1] which is not the same as SBRegistryRemoteHome[app2]. My thought was to get the handle as your code shows. Serialize the home handle to a byte array stream. That would decouple the class from the classloader. Then I would deserialize the byte array in the context of app1 forcing it to be created as SBRegistryRemoteHome[app1].

Unfortunately, this did not work. It did not even work within the same ear. What I mean by that, is that it doesn't seem that you can serialize and then successfully deserialize a HomeHandle. If my code was executing in app2 where the SBRegistryBean resides, then getting the remote home would not cross ears so it would be SBRegistryBean[app2] within the context of the app2 classloader.

I have actually solved the original problem a different way but I would be interested to know if you can serialize and deserialize a HomeHandle. It didn't work with a Handle object (from the EJBObject) either.

Let me know if I can provide any more information. This topic can get confusing.

[Former Member]

I am only familiar with the JBoss implementation, although we do also support WebSphere and WebLogic so they must provide some mechanism to support this.

In JBoss you also have the means of specifying that you want isolated classloaders for each application (ear) and that you want to use call by value for remote interfaces. This avoids the classloader mismatch because remote interfaces are serialized and deserialized when the invocation is across ears.

I basically implemented the same functionality in the SAP app server to get around the problem. The possible options were not really feasible without completely altering our architecture.

This <a href="http://www.jboss.org/wiki/Wiki.jsp?page=ClassLoadingConfiguration">link</a> provides more information if your interested.