Bugs item #516684, was opened at 2002-02-12 14:51
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=376685&aid=516684&group_id=22866

Category: JBossServer
Group: v3.0 Rabbit Hole
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Scott M Stark (starksm)
Assigned to: Scott M Stark (starksm)
Summary: CCE on redeployment of ear with sar/ejb

Initial Comment:
The perf-service.ear in the current main testsuite is 
not usable across redeployments. Steps to reproduce:

1. Deploy the perf-service.ear and then execute the 
jboss.test:service=PerfTest runTests(100) operation. 
Ignore the NPE shown under testTimings.

2. Simply touch the perf-service.ear and then execute 
the runTests(100) operation again on the 
jboss.test:service=PerfTest mbean, you will see:
+++ testTimingsCMT()
testTimingsCMT failed:
java.lang.ClassCastException: $Proxy19
        at 
org.jboss.test.perf.test.PerfTest.testTimingsCMT
(PerfTest.java:134)
        at org.jboss.test.perf.test.PerfTest.runTests
(PerfTest.java:66)
        at java.lang.reflect.Method.invoke(Native 
Method)
        at 
com.sun.management.jmx.MBeanServerImpl.invoke
(MBeanServerImpl.java:1628)
        at 
com.sun.management.jmx.MBeanServerImpl.invoke
(MBeanServerImpl.java:1523)
        at com.sun.jdmk.comm.HtmlInvokePage.buildPage
(HtmlInvokePage.java:240)
        at 
com.sun.jdmk.comm.HtmlRequestHandler.processGetRequest
(HtmlRequestHandler.java:325)
        at 
com.sun.jdmk.comm.HtmlRequestHandler.processRequest
(HtmlRequestHandler.java:152)
        at com.sun.jdmk.comm.HtmlRequestHandler.doRun
(HtmlRequestHandler.java:79)
        at com.sun.jdmk.comm.ClientHandler.run
(ClientHandler.java:84)
        at java.lang.Thread.run(Thread.java:484)


----------------------------------------------------------------------

Comment By: David Jencks (d_jencks)
Date: 2002-02-17 16:36

Message:
Logged In: YES 
user_id=60525

"If the mbean service 
along with any external classes it references is reloaded,
the next execution of the mbean service uses the same
MBeanClassLoader instance to load classes,"

Did you make sure it is the same MBeanClassLoader? There is
code to remove the old MBeanClassLoader when the mbean is
removed from the mbeanserver and then a new one is created
(with the same object name) for the new deployment.  If the
"remove mbeanClassLoader" code is not being executed that
could be the problem, maybe...



----------------------------------------------------------------------

Comment By: Scott M Stark (starksm)
Date: 2002-02-17 16:17

Message:
Logged In: YES 
user_id=175228

The real issue appears to be the new class loader model and 
caching of classes at the vm level. When the PerfTest 
service looks up the ProbeHome interface this is the stack 
trace where the class for the ProbeHome interface class 
gets loaded:

java.lang.Class.forName (Class.java:195)
sun.rmi.server.MarshalInputStream.resolveProxyClass 
(MarshalInputStream.java:183)
java.io.ObjectInputStream.inputProxyClassDescriptor 
(ObjectInputStream.java:982)
java.io.ObjectInputStream.readObject 
(ObjectInputStream.java:370)
java.io.ObjectInputStream.readObject 
(ObjectInputStream.java:236)
java.io.ObjectInputStream.inputObject 
(ObjectInputStream.java:1186)
java.io.ObjectInputStream.readObject 
(ObjectInputStream.java:386)
java.io.ObjectInputStream.readObject 
(ObjectInputStream.java:236)
java.rmi.MarshalledObject.get (MarshalledObject.java:138)
org.jnp.interfaces.NamingContext.lookup 
(NamingContext.java:376)
org.jnp.interfaces.NamingContext.lookup 
(NamingContext.java:356)
javax.naming.InitialContext.lookup (InitialContext.java:350)
org.jboss.test.perf.test.PerfTest.testTimings 
(PerfTest.java:102)
org.jboss.test.perf.test.PerfTest.runTests 
(PerfTest.java:56)
java.lang.reflect.Method.invoke (Native Method)
com.sun.management.jmx.MBeanServerImpl.invoke 
(MBeanServerImpl.java:1628)
com.sun.management.jmx.MBeanServerImpl.invoke 
(MBeanServerImpl.java:1523)
com.sun.jdmk.comm.HtmlInvokePage.buildPage 
(HtmlInvokePage.java:240)
com.sun.jdmk.comm.HtmlRequestHandler.processGetRequest 
(HtmlRequestHandler.java:325)
com.sun.jdmk.comm.HtmlRequestHandler.processRequest 
(HtmlRequestHandler.java:152)
com.sun.jdmk.comm.HtmlRequestHandler.doRun 
(HtmlRequestHandler.java:79)
com.sun.jdmk.comm.ClientHandler.run (ClientHandler.java:84)
java.lang.Thread.run (Thread.java:484)

The public static Class forName(String name, boolean 
initialize, ClassLoader loader)
throws ClassNotFoundException

in turn calls the forName0 native method:

private static native Class forName0(String name, boolean 
initialize, ClassLoader loader)
throws ClassNotFoundException;

This ultimate calls this native function:

static ClassClass *
LookupLoaderCache(char *hashed_name, struct 
Hjava_lang_ClassLoader *loader)
from j2sdk1.3.1-src/src/share/javavm/runtime/classresolver.c

This is simply a hash table of class bytes keyed by the
loader and class name. The issue is that mbean services
run with the same thread context class MBeanClassLoader
and only when a class is first seen is the is this
load asked to load the class which results in a call to
the ServiceLibraries repository. If the mbean service 
along with any external classes it references is reloaded,
the next execution of the mbean service uses the same
MBeanClassLoader instance to load classes, and those that
were originally cached are still returned and these will
conflict if when the class loader that loaded the class 
into the ServiceLibraries repository was not the main 
MBeanClassLoader.


----------------------------------------------------------------------

Comment By: David Jencks (d_jencks)
Date: 2002-02-14 20:25

Message:
Logged In: YES 
user_id=60525

I have found a workaround for this.

Instead of e.g.

ProbeHome ph = (ProbeHome) new
InitialContext().lookup("jndiname");

set the context classloader first to that of the class we
are expecting:

ClassLoader oldClass =
Thread.currentThread().getContextClassLoader();
Thread.currentThread().setContextClassLoader(ProbeHome.class.getClassLoader());
try
{
   ProbeHome ph = (ProbeHome) new
InitialContext().lookup("jndiname");
...
...
}
finally
{
   Thread.currentThread().setContextClassLoader(oldCl);
}


The rmi unmarshalling code, in the absence of location
annotations, ends up using the context classloader of the
caller when constructing a new proxy.  In order to get the
correct class, you apparently have to set this classloader
to the classloader actually used for the class you are
looking for.  The mbeanClassLoader doesn't work.

I'm not sure if this should be considered a problem with 

--sun's marshalling code

--the location annotations (currently none, since jboss
URLClassLoader return no URLs) (Putting them back in doesn't
remove the problem, however)

--our extensible classloader framework.


I believe that returning the original object bound rather
than a serialized-deserialized version would always work.


----------------------------------------------------------------------

Comment By: David Jencks (d_jencks)
Date: 2002-02-14 08:46

Message:
Logged In: YES 
user_id=60525

I think this may have something to do with
ObjectInput/OutputStream and marshalling.

On each deploy, a new proxy is bound into jndi.  It is
stored as a MarshalledObject.  For instance, I see it being
bound as classes $Proxy23, $Proxy42 and $Proxy57 on
successive deployments.

When it is retrieved, the className is different: $Proxy27

Once it is retrieved, the retrieved class is always the
same: $Proxy27 in this example.

I haven't dug into ObjectInput/Output stream enough to see
what is going on.

However....since serialization is the root of all
application lethargy,  is it really necessary to bind the
proxy serialized?  For in vm calls, wouldn't it be
significantly faster to use an ObjectFactory/reference
scheme to return the actual object bound?

I don't know enough about how jndi/rmi works to know if this
would work with remote access.  Would it be possible to bind
using an ObjectFactory for local access and also a
pre-serialized MarshalledObject for remote access?

----------------------------------------------------------------------

Comment By: Scott M Stark (starksm)
Date: 2002-02-13 22:33

Message:
Logged In: YES 
user_id=175228

The most recent PerfTest mbean output shows clearly what 
the problem is. On the initial deployment and execution of 
the runTests operation, both the ProbeHome interface as 
seen by the proxy bound in JNDI and the class reference in 
the PerfTest service mbean are seen to be loaded from the 
same perf.jar location:

ProbeHome Interfaces:
++interface org.jboss.test.perf.interfaces.ProbeHome
++++CodeSource: (file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/62.60.perf.jar )
++interface javax.ejb.Handle
++++CodeSource: (file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/6.jboss-
j2ee.jar )
PerfTest ProbHome CodeSource: 
(file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/62.60.perf.jar )

After a new deployment of the perf-service.ear, the proxy 
bound in JNDI continues to reference the ProxyHome class 
loaded from the original perf.jar, while the PerfTest 
service mbean ProbeHome now refers to the new perf.jar 
deployment:

ProbeHome Interfaces:
++interface org.jboss.test.perf.interfaces.ProbeHome
++++CodeSource: (file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/62.60.perf.jar )
++interface javax.ejb.Handle
++++CodeSource: (file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/6.jboss-
j2ee.jar )
PerfTest ProbHome CodeSource: 
(file:/D:/usr/local/src/cvsroot/Main/jboss-
all/build/output/jboss-3.0.0DR2/tmp/deploy/68.66.perf.jar )

The JNDI bindings of the proxies are not being updated 
correctly on redeployment.


----------------------------------------------------------------------

Comment By: Scott M Stark (starksm)
Date: 2002-02-13 10:17

Message:
Logged In: YES 
user_id=175228

That is correct.

----------------------------------------------------------------------

Comment By: David Jencks (d_jencks)
Date: 2002-02-13 09:12

Message:
Logged In: YES 
user_id=60525

I notice that you can redeploy many times successfully as
long as you don't run the test.  Once you have run the test,
after redeploying, the test fails.

deploy
redeploy
redeploy
run test (succeeds)
redeploy
run test (fails)

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=376685&aid=516684&group_id=22866

_______________________________________________
Jboss-development mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/jboss-development

Reply via email to