Re: [discuss] Release Geronimo 1.2

Sat, 07 Apr 2007 14:25:26 -0700 

I think I forgot to attach it (of course, the one that is probably right).

Hopefully I'll remember now (but just in case here it is inline):
Index: modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java 
===================================================================
--- modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java (revision 526318) +++ modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java (working copy) 
@@ -16,8 +16,8 @@
 */
package org.apache.openejb.util;

-import java.util.LinkedList;
import java.io.Serializable;
+import java.util.Stack;

import org.apache.openejb.cache.InstanceFactory;
import org.apache.openejb.cache.InstancePool;
@@ -30,19 +30,19 @@
public final class SoftLimitedInstancePool implements InstancePool, Serializable { 
    private final InstanceFactory factory;
    private final int maxSize;
-    private transient final LinkedList pool;
+    private transient final Stack pool;
public SoftLimitedInstancePool(final InstanceFactory factory, final int maxSize) { 
        this.factory = factory;
        this.maxSize = maxSize;
-        pool = new LinkedList();
+        pool = new Stack();
    }

    public Object acquire() throws Exception {
        // get the instance from the pool if possible
        synchronized (this) {
            if (!pool.isEmpty()) {
-                return pool.removeFirst();
+                return pool.pop();
            }
        }

@@ -55,7 +55,7 @@
// if we are under the limit put it back in the pool at the head // this encourages reuse of the same instances to improve memory management 
            if (pool.size() < maxSize) {
-                pool.addFirst(instance);
+                pool.push(instance);
                return true;
            }
        }
@@ -84,7 +84,7 @@
        synchronized (this) {
            // add this new instance to the end
            // we prefer other users get older instances first
-            pool.addLast(instance);
+            pool.insertElementAt(instance, 0);
        }
    }



Jay

David Jencks wrote:
I think the mailing list removed your patch... anyway I don't see it. Can you attach it to a jira or include it inline? 

thanks
david jencks

On Apr 6, 2007, at 8:16 PM, Jay D. McHugh wrote:


Whew!

Maybe now (I ran the openejb tests this time)
From what I understand, java.util.Stack is internally sychronized since it is an extension of Vector which is synchronized. 

So, here is a patch that replaces the LinkedList with a Stack.
It does pass the OpenEJB tests and will hopefully stand up under stress with daytrader under load.
Sorry about the previous noise - I'm anxious to get G1.2 out so that everyone can get back to G2 - Plus, this same issue will need to be fixed in OpenEJB 3 if this corrects the problem. 

Anyway, hopefully this will get 1.2 closer to the door,


Jay

David Jencks wrote:
I don't think this is acceptable. There should be only one thread working with the context at a time. Either this exception is caused by modifying the collection in the same thread in which case we can fix it easily or it is caused by more than one thread having access to a context at once. Since by my reading of the code this is a context attached to a stateless session bean instance, that would mean that more than one thread is using a stateless session bean instance at once, which is definitely cause to -1 the release.
I hope there's another possibility I haven't thought of..... but hiding the problem is not acceptable unless we really understand what is going on and are really convinced it's harmless. 

thanks
david jencks

On Apr 6, 2007, at 1:40 PM, Jay D. McHugh wrote:


Chris (do you go by Chris or Christopher?),
Here is a patch that I just wrote that allows the exit routine of ConnectionTrackingCoordinator to finish cleanly after a number (5) of attempts at removing the resource.
If it fails after five tries, then the routine exits and throws a ResourceException (that will hopefully be caught further up the stack). 

Would you like to try it to see if it solves your concurrency problem?

Jay

Christopher Blythe wrote:
Doubtful... everything tested fine under light browser based testing. As the exception suggests, this is a concurrency problem that you would only hit under load.
On 4/6/07, * Jay D. McHugh* <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
If Matt had no problem deploying and testing DT, could it be a Java 
    version or classpath issue?
That could explain the difference in the exception during deployment (and the problems during deployment could possibly explain the run 
    time
    problems).

    Jay

    Christopher Blythe wrote:
> I use a commercial load driving tool... FYI, I'm fairly certain that 
    > G-2.0 has the same issue.
    >
    > On 4/6/07, *David Jencks* < [EMAIL PROTECTED]
    <mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> 
    wrote:
    >
    >     I think we need to figure out why the
    >     concurrentModificationException is happening before we
    release.  I
> think that one possible reason is that we are multithreading 
    >     stateless session bean instances.  I hope this isn't the
    cause....
    >     but IMO we need to find out.
    >
> Chris, how do you run the several clients? manually or with 
    a tool?
    >
    >     thanks
    >     david jencks
    >
    >
    >     On Apr 6, 2007, at 11:09 AM, Christopher Blythe wrote:
    >
>> Gave it a shot... no luck. As soon as I started 2 clients, the 
    >>     same exceptions started to pile up. I have attached the
    >>     geronimo.log. Also, noticed the following exception during
    startup.
    >>
    >>     14:05:00,640 ERROR [TransportConnector] Could not accept
>> connection from /127.0.0.1:28428: java.io.IOException: Wire >> format negociation timeout: peer did not send his wire format. >> java.io.IOException: Wire format negociation timeout: peer did 
    >>     not send his wire format.
    >>         at
    org.apache.activemq.transport.WireFormatNegotiator.oneway
    >>     (WireFormatNegotiator.java :88)
    >>         at
>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:47) >> at org.apache.activemq.broker.TransportConnection.dispatch 
    >>     (TransportConnection.java :1138)
    >>         at
>> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:805) 
    >>         at
    >>     org.apache.activemq.broker.TransportConnection.start
    (TransportConnection.java
    >>     :885)
>> at org.apache.activemq.broker.TransportConnector$1.onAccept 
    >>     (TransportConnector.java:148)
    >>         at
    >>     org.apache.activemq.transport.tcp.TcpTransportServer.run
    (TcpTransportServer.java:167)
    >>         at java.lang.Thread.run (Thread.java:797)
    >>
    >>
    >>
    >>     On 4/6/07, *Matt Hogstrom* < [EMAIL PROTECTED]
    <mailto:[EMAIL PROTECTED]>
>> <mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> wrote: 
    >>
    >>         Only a very light load from a few browsers.  One thing
    to try
    >>         is to increase the number of SLSBs in the pool.
    >>
    >>         Can you add
    >>
    >>                         <session>
    >>                             <ejb-name>TradeJDBC</ejb-name>
>> <jndi-name>ejb/TradeJDBC</jndi-name> 
    >>                             <cache-size>100</cache-size>
    >>                         </session>
    >>
    >>         to your plan and redeploy.  I added some support for
    multiple
>> SLSBs in a pool for 1.2 which we did not have before. This 
    >>         will hopefully make it better and not worse :)
    >>
    >>         On Apr 6, 2007, at 11:32 AM, Christopher Blythe wrote:
    >>
    >>>         Matt...
    >>>
>>> You mentioned that you deployed DayTrader 1.2... did you >>> happen to run it under load? JDBC/Direct mode looks good; 
    >>>         however, I am still seeing
    ConcurrentModificationExceptions
    >>>         while attempting to run more than 1 client in Session
    Direct
    >>>         mode (
    https://issues.apache.org/jira/browse/GERONIMO-2708).
>>> These exceptions are thrown throughout the duration of the >>> run. FYI - I deployed the same ear on Geronimo 1.1.1 and >>> didn't have a problem scaling up the users for Session 
    >>>         Direct mode.
    >>>
    >>>         java.util.ConcurrentModificationException
    >>>             at
    java.util.HashMap$HashIterator.remove(HashMap.java:861)
    >>>             at
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator.exit 
    >>>         ( ConnectionTrackingCoordinator.java :127)
    >>>             at
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator$$FastClassByCGLIB$$5d33aabf.invoke(<generated>) 

    >>>             at net.sf.cglib.reflect.FastMethod.invoke
    >>>         (FastMethod.java :53)
    >>>             at
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
    (FastMethodInvoker.java:38)
    >>>             at
>>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122) 
    >>>             at
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke 
    >>>         (GBeanInstance.java:820)
    >>>             at
>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) 
    >>>             at
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35) 
    >>>             at
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept 
    >>>         ( ProxyMethodInterceptor.java:96)
    >>>             at
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTracker$$EnhancerByCGLIB$$b6b1324a.exit(<generated>) 
    >>>             at
>>> org.apache.openejb.NoConnectionEnlistingInterceptor.invoke 
    >>>         (NoConnectionEnlistingInterceptor.java:70)
    >>>             at
    >>>         org.apache.openejb.SystemExceptionInterceptor.invoke
    (SystemExceptionInterceptor.java:35)
    >>>             at
>>> org.apache.openejb.security.DefaultSubjectInterceptor.invoke(DefaultSubjectInterceptor.java 
    >>>         :49)
    >>>             at
>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer.invoke(DefaultStatelessEjbContainer.java:178) 
    >>>             at
>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer$$FastClassByCGLIB$$7ad7a562.invoke 
    (<generated>)
    >>>
    >>>             at
>>> net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53) 
    >>>             at
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke 
    (FastMethodInvoker.java:38)
    >>>             at
>>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke 
    >>>         (GBeanOperation.java:122)
    >>>             at
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:820) 
    >>>             at
>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java 
    :57)
    >>>             at
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke 
    >>>         (RawOperationInvoker.java:35)
    >>>             at
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96) 
    >>>             at
>>> org.apache.openejb.StatelessEjbContainer$$EnhancerByCGLIB$$5c554f35.invoke 

    >>>         (<generated>)
    >>>             at
>>> org.apache.openejb.AbstractEjbDeployment.invoke(AbstractEjbDeployment.java:195) 
    >>>             at
>>> org.apache.openejb.proxy.EJBMethodInterceptor.intercept(EJBMethodInterceptor.java:145) 
    >>>             at
>>> org.apache.openejb.proxy.SessionEJBObject$$EnhancerByCGLIB$$f5a9c1b2.login 
    >>>         (<generated>)
    >>>             at
>>> org.apache.geronimo.samples.daytrader.TradeAction.login(TradeAction.java:449) 
    >>>             at org.apache.geronimo.samples.daytrader.
    >>>         web.TradeServletAction.doLogin
>>> <http://web.TradeServletAction.doLogin>(TradeServletAction.java:364) 
    >>>             at org.apache.geronimo.samples.daytrader .
    >>>         web.TradeAppServlet.performTask
>>> <http://web.TradeAppServlet.performTask>(TradeAppServlet.java:126) 
    >>>             at org.apache.geronimo.samples.daytrader.
    >>>         web.TradeAppServlet.doPost
>>> <http://web.TradeAppServlet.doPost>(TradeAppServlet.java :91) 
    >>>             at javax.servlet.http.HttpServlet.service
    >>>         (HttpServlet.java:617)
    >>>             at
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java :690) 
    >>>             at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
    >>>         (ApplicationFilterChain.java:252)
    >>>             at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter 
    >>>         (ApplicationFilterChain.java:173)
    >>>             at org.apache.geronimo.samples.daytrader.
    >>>         web.OrdersAlertFilter.doFilter
>>> <http://web.OrdersAlertFilter.doFilter>(OrdersAlertFilter.java:91) 
    >>>             at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
    (ApplicationFilterChain.java
    >>>         :202)
    >>>             at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter 
    >>>         (ApplicationFilterChain.java :173)
    >>>             at
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) 
    >>>             at
    org.apache.catalina.core.StandardContextValve.invoke
    >>>         (StandardContextValve.java:178)
    >>>             at
>>> org.apache.geronimo.tomcat.valve.DefaultSubjectValve.invoke(DefaultSubjectValve.java:56) 
    >>>             at
>>> org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java 
    >>>         :328)
    >>>             at
>>> org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke 
    (GeronimoBeforeAfterValve.java:47)
    >>>             at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) >>> at org.apache.catalina.valves.ErrorReportValve.invoke 
    >>>         (ErrorReportValve.java:105)
    >>>             at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) 
    >>>             at
>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541) >>> at org.apache.catalina.connector.CoyoteAdapter.service 
    >>>         (CoyoteAdapter.java :148)
    >>>             at
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) 
    >>>             at
>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection 
    (Http11BaseProtocol.java
    >>>         :667)
    >>>             at
>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) 
    >>>             at
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) 
    >>>             at
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run 
    >>>         (ThreadPool.java:684)
    >>>             at java.lang.Thread.run(Thread.java:797)
    >>>
    >>>         On 4/5/07, *Jason Dillon* < [EMAIL PROTECTED]
    <mailto:[EMAIL PROTECTED]>
    >>>         <mailto:[EMAIL PROTECTED]
    <mailto:[EMAIL PROTECTED]>>> wrote:
    >>>
>>> Aight, no worries. I still don't fully understand 
    all
    >>>             that plugin stuff... yet ;-)
    >>>
    >>>             --jason
    >>>
    >>>
    >>>             On Apr 5, 2007, at 3:38 PM, Paul McMahan wrote:
    >>>
    >>>>             The change I have cued up replaces "
    1.2-SNAPSHOT" with
    >>>>             " 1.2" for all the catalog entries.  So it would
    break
    >>>>             anyone using the Geronimo plugin repo from a
>>>> 1.2-SNAPSHOT server (maybe not a huge deal). Also, 
    >>>>             I've tested the catalog updates by looping http
    >>>>             requests to repo1.maven.org/maven2
    <http://repo1.maven.org/maven2>
    >>>>             <http://repo1.maven.org/maven2> back to my local
    maven
>>>> repo. So I've made some assumptions about the repo 
    >>>>             layout that should probably be verified.
    >>>>
    >>>>             Best wishes,
    >>>>             Paul
    >>>>
    >>>>             On Apr 5, 2007, at 6:22 PM, Jason Dillon wrote:
    >>>>
>>>>> Will it hurt anything to commit it now? Or will it 
    >>>>>             break things?
    >>>>>
    >>>>>             --jason
    >>>>>
    >>>>>
    >>>>>             On Apr 5, 2007, at 3:14 PM, Paul McMahan wrote:
    >>>>>
    >>>>>>
    >>>>>>             On Apr 5, 2007, at 2:11 PM, Joe Bohn wrote:
    >>>>>>
>>>>>>> I couldn't do much with the framework assembly 
    as it
>>>>>>> requires a plugin repository with 1.2 plugins and >>>>>>> AFAIK there is no such plugin repository available >>>>>>> yet. Will you be making the plugins available for 
    >>>>>>>             1.2 as you make the release available?  If
    not, then
    >>>>>>>             perhaps we shouldn't include the framework
    assembly
    >>>>>>>             in the distribution.
    >>>>>>
    >>>>>>             I updated the plugin catalog stuff in
    >>>>>>             site/trunk/docs/plugins/geronimo- 1.2 locally
    and ran
>>>>>> some quick tests of plugin download & install from >>>>>> maven repo. I'm ready to commit if/when the 1.2 
    >>>>>>             artifacts are published to central.
    >>>>>>
    >>>>>>             Best wishes,
    >>>>>>             Paul
    >>>>>>
    >>>>>
    >>>>
    >>>
    >>>
    >>>
    >>>
    >>>         --
>>> "I say never be complete, I say stop being perfect, I say >>> let... lets evolve, let the chips fall where they may." - 
    >>>         Tyler Durden
    >>
    >>
    >>
    >>
    >>     --
    >>     "I say never be complete, I say stop being perfect, I say
    let...
>> lets evolve, let the chips fall where they may." - Tyler Durden 
    >>     <geronimo.log>
    >
    >
    >
    >
    > --
> "I say never be complete, I say stop being perfect, I say let... 
    lets
    > evolve, let the chips fall where they may." - Tyler Durden
--"I say never be complete, I say stop being perfect, I say let... lets evolve, let the chips fall where they may." - Tyler Durden
Index: /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java 
===================================================================
--- /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java (revision 526213) +++ /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java (working copy) 
@@ -124,7 +124,23 @@
// if no connection remain clear context... we could support automatic commit, rollback or exception here 
                 if (connections.isEmpty()) {
-                    i.remove();
+                    boolean retry = false;
+                    int numberOfRetries = 0;
+                    do {
+                        try {
+                            i.remove();
+ } catch (java.util.ConcurrentModificationException ex) { 
+                            if (numberOfRetries < 5) {
+                                retry = true;
+                            } else {
+                                retry = false;
+                            }
+                            numberOfRetries += 1;
+                        }
+                    } while (retry);
+                    if (numberOfRetries >= 5) {
+ throw new ResourceException("ConcurrentModificationException - Unable to remove resource"); 
+                    }
                 }
             }
         } finally {











Index: 
modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java
===================================================================
--- 
modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java
     (revision 526318)
+++ 
modules/openejb-core/src/main/java/org/apache/openejb/util/SoftLimitedInstancePool.java
     (working copy)
@@ -16,8 +16,8 @@
  */
 package org.apache.openejb.util;
 
-import java.util.LinkedList;
 import java.io.Serializable;
+import java.util.Stack;
 
 import org.apache.openejb.cache.InstanceFactory;
 import org.apache.openejb.cache.InstancePool;
@@ -30,19 +30,19 @@
 public final class SoftLimitedInstancePool implements InstancePool, 
Serializable {
     private final InstanceFactory factory;
     private final int maxSize;
-    private transient final LinkedList pool;
+    private transient final Stack pool;
 
     public SoftLimitedInstancePool(final InstanceFactory factory, final int 
maxSize) {
         this.factory = factory;
         this.maxSize = maxSize;
-        pool = new LinkedList();
+        pool = new Stack();
     }
 
     public Object acquire() throws Exception {
         // get the instance from the pool if possible
         synchronized (this) {
             if (!pool.isEmpty()) {
-                return pool.removeFirst();
+                return pool.pop();
             }
         }
 
@@ -55,7 +55,7 @@
             // if we are under the limit put it back in the pool at the head
             // this encourages reuse of the same instances to improve memory 
management
             if (pool.size() < maxSize) {
-                pool.addFirst(instance);
+                pool.push(instance);
                 return true;
             }
         }
@@ -84,7 +84,7 @@
         synchronized (this) {
             // add this new instance to the end
             // we prefer other users get older instances first
-            pool.addLast(instance);
+            pool.insertElementAt(instance, 0);
         }
     }

Reply via email to