Thanks for the explanation; I think your change should be good for everyone, not just amq. Iirc it used to be common (very bad) practice to get a connection handle and cache it in a non Singleton session bean and expect the connection infrastructure to associate it to whatever managed connection was associated to the current transaction. If that worked this certainly ought to!
Thanks David Jencks Sent from my iPhone > On Sep 3, 2019, at 6:43 PM, Jonathan S. Fisher <exabr...@gmail.com> wrote: > > Ahhh... The CDI spec for JMSContext says it's either scoped to either the > current RequestScope, or in our case TransactionScope. I'll take an > educated guess and bet TransactionScoped beans are destroyed using a > [Transaction] Synchronization's afterCompletion() call. When the > TransactionScope is destroyed is when the JMSContext is destroyed, and it's > at that point it closes the connection. I'm guessing AutoConnectionTracker > was probably written before this was possible, so users were calling > connection.close() ergo handleClosed() before beforeCompletion(). Now the > CDI Extension for JMS2.0 in TomEE is calling close() much later. > > My PR moves what was in beforeCompletion() into afterCompletion(). For > ActiveMQ I'm fairly certain this won't be an issue, as they allow you to > really abuse when you open connections and even how man sessions per > connection you open :) > > On Tue, Sep 3, 2019 at 7:50 PM David Jencks <david.a.jen...@gmail.com> > wrote: > >> Well, beforeCompletion() is called as a result of commit() being called on >> the transaction, presumably by an EJB “interceptor”, and handleClosed() is >> called as a result of the “user level” connection being closed. I’m used to >> the latter being called by user code... perhaps with all the CDI and >> dependency injection I haven’t kept up with this is no longer the case? >> >> Thanks. >> David Jencks >> Sent from my iPhone >> >>> On Sep 3, 2019, at 3:25 PM, Jonathan S. Fisher <exabr...@gmail.com> >> wrote: >>> >>> Honestly I have no idea. And the interface specification is silent >>> unfortunately: >>> >> https://github.com/apache/geronimo-txmanager/blob/trunk/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTracker.java >>> >>> On Tue, Sep 3, 2019 at 3:32 PM David Jencks <david.a.jen...@gmail.com> >>> wrote: >>> >>>> You might have already explained this, but… why is beforeCompletion() >>>> called before handleReleased()? If that’s happening, I’d expect >> something >>>> is wrong. However, I haven’t looked at this code in years. >>>> >>>> thanks! >>>> David Jencks >>>> >>>>> On Sep 3, 2019, at 12:27 PM, Jonathan S. Fisher <exabr...@gmail.com> >>>> wrote: >>>>> >>>>> Two more updates: >>>>> >>>>> For the log message, it looks like beforeCompletion() Is being called >>>>> before handleReleased(), leading to that warning. I ran a couple >> thousand >>>>> messages through and took a heap dump and didn't get any leak suspects, >>>> so >>>>> I think it's working correctly despite the warning. I'll add that to my >>>>> existing PR with your tests. >>>>> >>>>> Second, on my 10K messages test, it looks like it might be Arquillian >>>>> crashing, not TomEE. I'm going to try the same test but without >>>> Arquillian, >>>>> just deploy those classes to a server and see what happens. >>>>> >>>>> On Tue, Sep 3, 2019 at 10:54 AM Jonathan S. Fisher <exabr...@gmail.com >>> >>>>> wrote: >>>>> >>>>>> If I bump the number of messages up to 10k or so I get a VM Crash... I >>>>>> cannot figure out how to get arquillian to give me a heap dump on exit >>>>>> though. >>>>>> >>>>>> On Tue, Sep 3, 2019 at 10:01 AM Jonathan S. Fisher < >> exabr...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> https://github.com/apache/tomee/pull/546/files >>>>>>> >>>>>>> This passes consistently for me with no issues >>>>>>> >>>>>>> On Tue, Sep 3, 2019 at 9:08 AM Jonathan S. Fisher < >> exabr...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> There are actually a few log messages we regularly ignore all the >> time >>>>>>>> from the transaction manager ::wince face:: I'm not sure if we >> should >>>> be >>>>>>>> concerned with that one. >>>>>>>> >>>>>>>> On your test, first, how is the broker xml declared? Often something >>>>>>>> that trips our newbies up to TomEE is having a persistent broker >> that >>>> is >>>>>>>> storing messages between TomEE runs. The tricky thing is that the >>>> broker >>>>>>>> store does always appear in /target, so it might not get cleaned up >>>> with >>>>>>>> mvn clean install. As such, for local development we use this >>>>>>>> URL: >>>> >> broker:(vm://localhost)?persistent=false&deleteAllMessagesOnStartup=true. >>>>>>>> Next, on JMSReceiverBean, you're missing transactionAttribute. >>>> Technically >>>>>>>> it should work without it, but I'd add it just in case. Finally, I'd >>>> add a >>>>>>>> small thread.sleep, or check to the queue length == 0 before doing >>>> your >>>>>>>> assert on messagecounter. It could be you're beating the receiver >>>> bean to >>>>>>>> the finish line. The default messaging mode is auto-ack, so >>>> technically the >>>>>>>> message just has to be on the broker, it doesn't have to be >> processed >>>>>>>> before your sender bean will return. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Sep 2, 2019 at 2:28 PM Jonathan Gallimore < >>>>>>>> jonathan.gallim...@gmail.com> wrote: >>>>>>>> >>>>>>>>> In case it means anything to anyone, the unexpected output I'm >>>> getting >>>>>>>>> is >>>>>>>>> an abandoned connection notification: >>>>>>>>> >>>>>>>>> WARNING: Transaction complete, but connection still has handles >>>>>>>>> associated: >>>>>>>>> ManagedConnectionInfo: >>>>>>>>> >>>> org.apache.geronimo.connector.outbound.ManagedConnectionInfo@5b5a4aed. >>>>>>>>> mc: >>>>>>>>> >>>>>>>>> >>>> >> [org.apache.openejb.resource.activemq.jms2.TomEEManagedConnection@37e69c43 >>>>>>>>> ,ActiveMQConnection >>>>>>>>> >>>>>>>>> >>>> >> {id=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-7:1,clientId=ID:MacBook-Pro.jlpsoftwareltd.net-50025-1567452049768-6:1,started=false}]] >>>>>>>>> Abandoned connection information: >>>>>>>>> Connection handle opened at >>>>>>>>> >>>>>>>>> >>>> >> org.apache.geronimo.connector.outbound.ConnectionInfo.setTrace(ConnectionInfo.java:119), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.geronimo.connector.outbound.ConnectionHandleInterceptor.getConnection(ConnectionHandleInterceptor.java:57), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.geronimo.connector.outbound.ConnectionTrackingInterceptor.getConnection(ConnectionTrackingInterceptor.java:66), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.geronimo.connector.outbound.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:81), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:94), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.activemq.ra.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:67), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.connection(JMSContextImpl.java:83), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.session(JMSContextImpl.java:99), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.resource.activemq.jms2.JMSContextImpl.createQueue(JMSContextImpl.java:309), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.resource.activemq.jms2.cdi.JMS2CDIExtension$InternalJMSContext.createQueue(JMS2CDIExtension.java:371), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:37), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.tests.jms.JMSSenderBean.sendToQueue(JMSSenderBean.java:33), >>>>>>>>> sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:191), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:102), >>>>>>>>> sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:205), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:186), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:85), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.singleton.SingletonContainer._invoke(SingletonContainer.java:272), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.singleton.SingletonContainer.invoke(SingletonContainer.java:221), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.ivm.EjbObjectProxyHandler.synchronizedBusinessMethod(EjbObjectProxyHandler.java:265), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:260), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:89), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:347), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.tests.jms.JMSSenderBean$$LocalBeanProxy.sendToQueue(org/apache/openejb/arquillian/tests/jms/JMSSenderBean.java), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.tests.jms.JMSContextInjectionTest.testShouldSendAndReceiveMessages(JMSContextInjectionTest.java:61), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50), >>>>>>>>> >>>>>>>>> >>>> >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12), >>>>>>>>> >>>>>>>>> >>>> >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$8$1.invoke(Arquillian.java:379), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.execution.LocalTestExecuter.execute(LocalTestExecuter.java:60), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:114), >>>>>>>>> org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.client.protocol.local.LocalContainerMethodExecutor.invoke(LocalContainerMethodExecutor.java:50), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.execution.RemoteTestExecuter.execute(RemoteTestExecuter.java:109), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:114), >>>>>>>>> org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.execution.ClientTestExecuter.execute(ClientTestExecuter.java:57), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.common.TestObserver.switchLoader(TestObserver.java:98), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.common.TestObserver.observesTest(TestObserver.java:75), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.client.ContainerEventController.createTestContext(ContainerEventController.java:129), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:130), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.test(EventTestRunnerAdaptor.java:136), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$8.evaluate(Arquillian.java:372), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$4.evaluate(Arquillian.java:246), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:431), >>>>>>>>> >> org.jboss.arquillian.junit.Arquillian.access$200(Arquillian.java:55), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$5.evaluate(Arquillian.java:260), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:324), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:99), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.on(ClientBeforeAfterLifecycleEventExecuter.java:72), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.common.TestObserver.switchLoader(TestObserver.java:98), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.openejb.arquillian.common.TestObserver.observesTest(TestObserver.java:75), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.container.test.impl.client.ContainerEventController.createBeforeContext(ContainerEventController.java:124), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:130), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73), >>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), >>>>>>>>> >>>>>>>>> >>>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), >>>>>>>>> java.lang.reflect.Method.invoke(Method.java:498), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143), >>>>>>>>> >>>> org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:114), >>>>>>>>> >>>>>>>>> >>>> >> org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.fireCustomLifecycle(EventTestRunnerAdaptor.java:159), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$7.evaluate(Arquillian.java:317), >>>>>>>>> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325), >>>>>>>>> >>>>>>>>> >>>> >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78), >>>>>>>>> >>>>>>>>> >>>> >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57), >>>>>>>>> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290), >>>>>>>>> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71), >>>>>>>>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288), >>>>>>>>> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58), >>>>>>>>> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$2.evaluate(Arquillian.java:205), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:431), >>>>>>>>> >> org.jboss.arquillian.junit.Arquillian.access$200(Arquillian.java:55), >>>>>>>>> >>>> org.jboss.arquillian.junit.Arquillian$3.evaluate(Arquillian.java:219), >>>>>>>>> org.junit.runners.ParentRunner.run(ParentRunner.java:363), >>>>>>>>> org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:167), >>>>>>>>> org.junit.runners.Suite.runChild(Suite.java:128), >>>>>>>>> org.junit.runners.Suite.runChild(Suite.java:27), >>>>>>>>> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290), >>>>>>>>> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71), >>>>>>>>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288), >>>>>>>>> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58), >>>>>>>>> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268), >>>>>>>>> org.junit.runners.ParentRunner.run(ParentRunner.java:363), >>>>>>>>> >> org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:157), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386), >>>>>>>>> >>>>>>>>> >>>> >> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323), >>>>>>>>> org.apache. >>>>>>>>> >>>>>>>>> Also, I see out by one errors - e.g. if I send 200 messages, >>>> sometimes I >>>>>>>>> receive 201 or 199. Not sure if its related to the above warning or >>>> not. >>>>>>>>> >>>>>>>>> Jon >>>>>>>>> >>>>>>>>> On Mon, Sep 2, 2019 at 8:25 PM Jonathan Gallimore < >>>>>>>>> jonathan.gallim...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> I had a play around with this, and got my previously failing >>>>>>>>>> JMSContextInjectionTest to work -- some of the time. ( >>>>>>>>>> >>>>>>>>> >>>> >> https://github.com/apache/tomee/blob/e58ff848a3a80938d4d99fc9bcfeaade5a72d644/arquillian/arquillian-tomee-tests/arquillian-tomee-jms-tests/src/test/java/org/apache/openejb/arquillian/tests/jms/JMSContextInjectionTest.java >>>>>>>>>> ). >>>>>>>>>> >>>>>>>>>> I'm getting some log output that I'm not expecting so I've rolled >>>>>>>>> back my >>>>>>>>>> removal of @Ignore so I can dig into it some more. >>>>>>>>>> >>>>>>>>>> Jon >>>>>>>>>> >>>>>>>>>> On Tue, Aug 27, 2019 at 5:36 PM Jonathan Gallimore < >>>>>>>>>> jonathan.gallim...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Jonathan >>>>>>>>>>> >>>>>>>>>>> Thanks for this. I think I had run into some of the things you >>>> fixed >>>>>>>>> I >>>>>>>>>>> little while back. I'll take a look at your change later today. >>>>>>>>> Happy for >>>>>>>>>>> you to commit and I can review after, too. :) >>>>>>>>>>> >>>>>>>>>>> Cheers >>>>>>>>>>> >>>>>>>>>>> The other Jonathan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Aug 27, 2019 at 2:59 AM Jonathan S. Fisher < >>>>>>>>> exabr...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> https://github.com/exabrial/tomee/commit/08dd4c744818702f3be5edfd8a1c4cf2b69d524d >>>>>>>>>>>> >>>>>>>>>>>> With these patches the JMS2.0 API works pretty well now :) If >>>> anyone >>>>>>>>>>>> wants >>>>>>>>>>>> to review, please go ahead, otherwise I'll commit them tomorrow. >>>>>>>>>>>> >>>>>>>>>>>> cheers, >>>>>>>>>>>> - [The other] Jonathan >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Aug 26, 2019 at 3:45 PM Jonathan S. Fisher < >>>>>>>>> exabr...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I've narrowed down the problem to AutoConnectionTracker. It's >> not >>>>>>>>>>>>> completing, which isn't allowing the connections to be returned >>>>>>>>> to the >>>>>>>>>>>> pool. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> https://github.com/apache/tomee/blob/master/container/openejb-core/src/main/java/org/apache/openejb/resource/AutoConnectionTracker.java#L174 >>>>>>>>>>>>> >>>>>>>>>>>>> getResource() is throwing an IllegalStateException. The >> JavaDoc ( >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> https://docs.oracle.com/javaee/7/api/javax/transaction/TransactionSynchronizationRegistry.html#getResource-java.lang.Object- >>>>>>>>>>>> ) >>>>>>>>>>>>> states it should throw an ISE if a current transaction is not >>>>>>>>> Active. >>>>>>>>>>>> The >>>>>>>>>>>>> transaction is in the state ROLLED_BACK when >>>> AutoConnectionTracker >>>>>>>>>>>> tries to >>>>>>>>>>>>> call getResource(). >>>>>>>>>>>>> >>>>>>>>>>>>> I think the Geronimo implementation ( >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> https://github.com/apache/geronimo-txmanager/blame/trunk/geronimo-transaction/src/main/java/org/apache/geronimo/transaction/manager/TransactionManagerImpl.java#L203 >>>>>>>>>>>> ) >>>>>>>>>>>>> maybe be a little too strict. The JTA Spec pdf doesn't offer >>>> exact >>>>>>>>>>>> hints of >>>>>>>>>>>>> which statuses ( >>>>>>>>>>>>> >>>>>>>>> https://docs.oracle.com/javaee/7/api/javax/transaction/Status.html >> ) >>>>>>>>>>>>> should be have getResource _not_ throw an ISE unfortunately. I >>>> was >>>>>>>>>>>> thinking >>>>>>>>>>>>> of changing Geronimo's implementation to check for anything >>>>>>>>>>>>> but STATUS_UNKNOWN, and STATUS_NO_TRANSACTION. >>>>>>>>>>>>> >>>>>>>>>>>>> The other way is to cast Transaction to the Geronimo >>>>>>>>> implementation and >>>>>>>>>>>>> use Geronimo specific APIs to get call getResource(). Do you >> guys >>>>>>>>> have >>>>>>>>>>>> any >>>>>>>>>>>>> preference which route I should take to fix? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Aug 26, 2019 at 9:15 AM Jonathan S. Fisher < >>>>>>>>> exabr...@gmail.com >>>>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >> https://github.com/exabrial/tomee-jms2-bug/tree/connection-pool-leak >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here's a project that reproduces the bug. This project >>>>>>>>> intentionally >>>>>>>>>>>>>> exceeds the transaction timeout (of 1s). Each invocation, the >>>>>>>>>>>> connection is >>>>>>>>>>>>>> not returned to the pool and eventually you run out, causing >>>> your >>>>>>>>>>>>>> application to freeze. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Aug 23, 2019 at 2:37 PM Jonathan S. Fisher < >>>>>>>>>>>> exabr...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello Apache friends :) I have a question about the JTA and >>>>>>>>> JMS/RA >>>>>>>>>>>> specs: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you borrow something from a RA, like a JMS Connection, and >>>>>>>>> you're >>>>>>>>>>>> in >>>>>>>>>>>>>>> XA Transaction, is it necessary to call connection.close()? >> It >>>>>>>>> would >>>>>>>>>>>> seem >>>>>>>>>>>>>>> JTA should be smart enough to know the connection is enrolled >>>>>>>>> for 2 >>>>>>>>>>>> phase >>>>>>>>>>>>>>> commit and should be smart enough to close it, but I'm not >> sure >>>>>>>>> if >>>>>>>>>>>> that's >>>>>>>>>>>>>>> part of the spec. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In TomEE 7.0.6 we're noticing that if you borrow a JMS >>>>>>>>> Connection >>>>>>>>>>>> with >>>>>>>>>>>>>>> connectionFactory.createConnection(), and your code fails to >>>>>>>>> call >>>>>>>>>>>> close() >>>>>>>>>>>>>>> before the transaction completion, the connection leaks. (And >>>>>>>>>>>>>>> unfortunately, calling close() after the transaction >> completes >>>>>>>>>>>> doesn't >>>>>>>>>>>>>>> mitigate the problem). It took awhile for us to track this >>>> down. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This becomes a huge problem if you're calling external >> services >>>>>>>>> in >>>>>>>>>>>> your >>>>>>>>>>>>>>> transaction. Let's say you have a reasonable transaction >>>>>>>>> timeout of >>>>>>>>>>>> 30s >>>>>>>>>>>>>>> set. You call three services, and they end up taking 15s a >>>>>>>>> piece. >>>>>>>>>>>> Even if >>>>>>>>>>>>>>> you're doing the right thing and you have connection.close() >> in >>>>>>>>> a >>>>>>>>>>>> finally >>>>>>>>>>>>>>> block, because your transaction isn't active when you call >>>>>>>>> close, it >>>>>>>>>>>> leaks >>>>>>>>>>>>>>> and it just gets "stuck" as an active connection, which >>>>>>>>> eventually >>>>>>>>>>>> you hit >>>>>>>>>>>>>>> the pool limit and your app freezes. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On a separate note, we noticed if you open a connection >> outside >>>>>>>>> of >>>>>>>>>>>> the >>>>>>>>>>>>>>> scope of a transaction, then start a transaction, then >> create a >>>>>>>>>>>> session >>>>>>>>>>>>>>> with session_transacted option, the session does not >>>>>>>>> participate in >>>>>>>>>>>> the JTA >>>>>>>>>>>>>>> (which seems out of spec). One most open the connection >> inside >>>>>>>>> the >>>>>>>>>>>>>>> transaction, AND open the session in the transaction, and >> close >>>>>>>>> the >>>>>>>>>>>>>>> connection in the transaction for everything to work. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'll get a reproducing project created, but I was curious if >>>>>>>>> anyone >>>>>>>>>>>> knew >>>>>>>>>>>>>>> offhand what the spec says. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> cheers, and thanks, >>>>>>>>>>>>>>> -[the other] Jonathan >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Jonathan | exabr...@gmail.com >>>>>>>>>>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, >>>>>>>>> see it >>>>>>>>>>>> as >>>>>>>>>>>>>>> half full. >>>>>>>>>>>>>>> Engineers, of course, understand the glass is twice as big as >>>> it >>>>>>>>>>>> needs >>>>>>>>>>>>>>> to be. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Jonathan | exabr...@gmail.com >>>>>>>>>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, >> see >>>>>>>>> it as >>>>>>>>>>>>>> half full. >>>>>>>>>>>>>> Engineers, of course, understand the glass is twice as big as >> it >>>>>>>>>>>> needs to >>>>>>>>>>>>>> be. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Jonathan | exabr...@gmail.com >>>>>>>>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, >> see >>>>>>>>> it as >>>>>>>>>>>>> half full. >>>>>>>>>>>>> Engineers, of course, understand the glass is twice as big as >> it >>>>>>>>> needs >>>>>>>>>>>> to >>>>>>>>>>>>> be. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Jonathan | exabr...@gmail.com >>>>>>>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, see >>>> it >>>>>>>>> as >>>>>>>>>>>> half >>>>>>>>>>>> full. >>>>>>>>>>>> Engineers, of course, understand the glass is twice as big as it >>>>>>>>> needs to >>>>>>>>>>>> be. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Jonathan | exabr...@gmail.com >>>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, see it >> as >>>>>>>> half full. >>>>>>>> Engineers, of course, understand the glass is twice as big as it >> needs >>>>>>>> to be. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Jonathan | exabr...@gmail.com >>>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, see it >> as >>>>>>> half full. >>>>>>> Engineers, of course, understand the glass is twice as big as it >> needs >>>> to >>>>>>> be. >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Jonathan | exabr...@gmail.com >>>>>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >>>>>> half full. >>>>>> Engineers, of course, understand the glass is twice as big as it needs >>>> to >>>>>> be. >>>>>> >>>>> >>>>> >>>>> -- >>>>> Jonathan | exabr...@gmail.com >>>>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >>>> half >>>>> full. >>>>> Engineers, of course, understand the glass is twice as big as it needs >> to >>>>> be. >>>> >>>> >>> >>> -- >>> Jonathan | exabr...@gmail.com >>> Pessimists, see a jar as half empty. Optimists, in contrast, see it as >> half >>> full. >>> Engineers, of course, understand the glass is twice as big as it needs to >>> be. >> >> > > -- > Jonathan | exabr...@gmail.com > Pessimists, see a jar as half empty. Optimists, in contrast, see it as half > full. > Engineers, of course, understand the glass is twice as big as it needs to > be.
