[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-12-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724254#comment-16724254
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on the issue:

https://github.com/apache/activemq-artemis/pull/1761
  
I agree, closing as requested in previous comment.


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>Priority: Major
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Thread[Thread-5 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-12-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724255#comment-16724255
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek closed the pull request at:

https://github.com/apache/activemq-artemis/pull/1761


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>Priority: Major
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Thread[Thread-5 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> jav

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-12-18 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724220#comment-16724220
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user jbertram commented on the issue:

https://github.com/apache/activemq-artemis/pull/1761
  
@JiriOndrusek, I don't think this is going to get merged. Can you go ahead 
and close this PR?


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>Priority: Major
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Thread[Thread-5 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320630#comment-16320630
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1761
  
-1000 on this PR...

it's a brittle change over a test failure. .you could find other way to fix 
the test?


it's a huge change on the maintenance 1.x branch... it's a risky change... 
if this is a go... it needs to be on master. (I wouldn't ever cherry-pick 
something this size into 1.x anyways).


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318499#comment-16318499
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1761#discussion_r160421530
  
--- Diff: 
artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientSessionFactoryImpl.java
 ---
@@ -990,17 +,26 @@ public CloseRunnable(RemotingConnection conn, String 
scaleDownTargetNodeID) {
   // can cause reconnect loop
   @Override
   public void run() {
- try {
-CLOSE_RUNNABLES.add(this);
-if (scaleDownTargetNodeID == null) {
-   
conn.fail(ActiveMQClientMessageBundle.BUNDLE.disconnected());
-} else {
-   
conn.fail(ActiveMQClientMessageBundle.BUNDLE.disconnected(), 
scaleDownTargetNodeID);
-}
- } finally {
-CLOSE_RUNNABLES.remove(this);
+ CLOSE_RUNNABLES.add(this);
+ CountDownLatch latch;
+ if (scaleDownTargetNodeID == null) {
+latch = 
conn.fail(ActiveMQClientMessageBundle.BUNDLE.disconnected());
+ } else {
+latch = 
conn.fail(ActiveMQClientMessageBundle.BUNDLE.disconnected(), 
scaleDownTargetNodeID);
  }
 
+ //TODO(jondruse) is there constant or property usable for this 
kind of wait interval?
--- End diff --

I have too use "constant" for repetitive checking, whether action is 
already finished. May I use some generic timeout used in project (or define new 
attribute, ...)


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryI

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318498#comment-16318498
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on the issue:

https://github.com/apache/activemq-artemis/pull/1761
  
In 2 places I have to use "constant" for repetitive checking, whether 
action is already finished. May I use some generic timeout used in project (or 
define new attribute, ...)
(ClientSessionFactoryImpl line 1123 and RemotingConnectionImpl line 254) 


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Th

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318493#comment-16318493
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1761#discussion_r160420636
  
--- Diff: 
artemis-core-client/src/main/java/org/apache/activemq/artemis/core/protocol/core/impl/RemotingConnectionImpl.java
 ---
@@ -205,14 +211,59 @@ public void fail(final ActiveMQException me, String 
scaleDownTargetNodeID) {
   }
 
   // Then call the listeners
-  callFailureListeners(me, scaleDownTargetNodeID);
+  List failureLatches = callFailureListeners(me, 
scaleDownTargetNodeID);
 
   callClosingListeners();
 
-  internalClose();
+  CountDownLatch latch = new CountDownLatch(1);
 
-  for (Channel channel : channels.values()) {
- channel.returnBlocking(me);
+  new ScheduledInternalClose(failureLatches, me, latch).run();
+
+  return latch;
+   }
+
+   // internalClose has to be called after failureListeners are finished.
+   private class ScheduledInternalClose implements Runnable {
+
+  private final List failureLatches;
+  private final ActiveMQException me;
+  private final CountDownLatch latch;
+
+  public ScheduledInternalClose(List failureLatches, 
final ActiveMQException me, CountDownLatch latch) {
+ this.failureLatches = failureLatches;
+ this.me = me;
+ this.latch = latch;
+  }
+
+  @Override
+  public void run() {
+ List running = new LinkedList<>();
+ failureLatches.forEach((l) -> {
+try {
+   //interval is defined during scheduled execution
+   if (!l.await(1, TimeUnit.MILLISECONDS)) {
+  running.add(l);
+   }
+} catch (InterruptedException e) {
+   ActiveMQClientLogger.LOGGER.warn(e.getMessage(), e);
+}
+ });
+
+ if (!running.isEmpty()) {
+//TODO(jondruse) is there constant or propertyF usable for 
this kind of wait interval?
+scheduledExecutorService.schedule(new 
ScheduledInternalClose(running, me, latch), 500, TimeUnit.MILLISECONDS);
--- End diff --

I have too use "constant" for repetitive checking, whether action is 
already finished. May I use some generic timeout used in project (or define new 
attribute, ...)


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> 

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318490#comment-16318490
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on the issue:

https://github.com/apache/activemq-artemis/pull/1676
  
Replaced with https://github.com/apache/activemq-artemis/pull/1761


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Thread[Thread-5 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.pa

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318488#comment-16318488
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

GitHub user JiriOndrusek opened a pull request:

https://github.com/apache/activemq-artemis/pull/1761

[ARTEMIS-1527] - [Artemis Testsuite] ActiveMQMessageHandlerTest#testS…

…erverShutdownAndReconnect fails

Issue: https://issues.apache.org/jira/browse/ARTEMIS-1527

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JiriOndrusek/activemq-artemis ARTEMIS-1527-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/1761.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1761


commit 367e633846b020bb002e0cc0ff228df6bddefaa3
Author: JiriOndrusek 
Date:   2018-01-09T08:24:03Z

[ARTEMIS-1527] - [Artemis Testsuite] 
ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails




> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemi

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2018-01-09 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16318491#comment-16318491
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek closed the pull request at:

https://github.com/apache/activemq-artemis/pull/1676


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   java.lang.Thread.run(Thread.java:785)
> Thread[Thread-5 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.Abstra

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2017-11-29 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270414#comment-16270414
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user JiriOndrusek commented on the issue:

https://github.com/apache/activemq-artemis/pull/1676
  
Thank you for your responses.

@clebertsuconic you are right, with this change FailoverTest is failing

I'll try to find solution for FailoverTest to succeed.
(it will take me some time to debug and understand it)

@clebertsuconic @mtaylor I'm opened to any suggestions, as there are a lot 
of aspects, about which I don't know/understand. 


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:100)
>   
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
>   
> java.util.con

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2017-11-29 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270370#comment-16270370
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user mtaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/1676
  
@clebertsuconic The problem here is that any thread that blocks 
indefinitely takes up resources in the Thread pool.  If you have several 
threads trying reconnect, the ThreadPool may be exhausted and can not be used 
to perform any other tasks.  My opinion here is that no task we add to the 
ThreadPool should block indefinitely, for retry here we should attempt to 
connect, if it fails schedule another attempt connect call, and give up the 
thread.  Jiri's made a start on that, but we may need to flesh out some of the 
details.  There's a similar problem with LargeMessage receive().


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.ac

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2017-11-28 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270046#comment-16270046
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1676
  
The failover is blocking because client applications would fail and would 
require to block.. it's customer's code.

I reckon we could do asynchronously for MDBs or message listeners. so, how 
do you differentiate the two here?


I suspect Failover (FailoverTest and others on the testsuite) wouldn't work.


Also.. at least you should make this change on master.. it's a significant 
refactoring to make something like this to work properly. Failover itself was a 
pain.. making this level.. even more difficult.


> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.fail(AbstractRemotingConnection.java:206)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1014)
>   
> org.apache.activemq.artemis.utils.OrderedExecutorFa

[jira] [Commented] (ARTEMIS-1527) [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails

2017-11-28 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268624#comment-16268624
 ] 

ASF GitHub Bot commented on ARTEMIS-1527:
-

GitHub user JiriOndrusek opened a pull request:

https://github.com/apache/activemq-artemis/pull/1676

[ARTEMIS-1527] - [Artemis Testsuite] ActiveMQMessageHandlerTest#testS…

…erverShutdownAndReconnect fails

Issue: https://issues.apache.org/jira/browse/ARTEMIS-1527

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JiriOndrusek/activemq-artemis ARTEMIS-1527

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/1676.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1676


commit 354409952ed7cd3c9d7883e21389e75ed9a022b5
Author: JiriOndrusek 
Date:   2017-11-22T14:38:03Z

[ARTEMIS-1527] - [Artemis Testsuite] 
ActiveMQMessageHandlerTest#testServerShutdownAndReconnect fails




> [Artemis Testsuite] ActiveMQMessageHandlerTest#testServerShutdownAndReconnect 
> fails
> ---
>
> Key: ARTEMIS-1527
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1527
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>Affects Versions: 1.5.5
>Reporter: Jiri Ondrusek
>
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.activemq.artemis.tests.integration.ra.ActiveMQMessageHandlerTest.testServerShutdownAndReconnect(ActiveMQMessageHandlerTest.java:245)
> {code}
> Maximum size of clients global thread pool is computed as {{8 * 
> Runtime.getRuntime().availableProcessors()}}. In the case of CPU with one 
> core, the thread pool has maximum size 8.
> The test fails because of following situation. When the server is stopped, 
> all ClientSessionFactoryImpl try to reconnect and they add executor to the 
> thread pool which tries to reconnect ad anfinitum, see \[1\]. These threads 
> employ the whole thread pool and the rest is not able to work in normal way. 
> For example, I can see warnings \[2\].
> This situation is known as Starvation or Livelock \[3\]. I think that the 
> system should work properly even if the maximum size of thread pool will be 2.
> *Steps to reproduce:*
> Modify the test in the following way:
> {code}
> @Test
>public void testServerShutdownAndReconnect() throws Exception {
>   ActiveMQClient.clearThreadPools();
>   ActiveMQClient.setGlobalThreadPoolProperties(2, 1);
>   ActiveMQResourceAdapter qResourceAdapter = newResourceAdapter();
>   ...
> {code}
> \[1\]
> {code}
> Thread[Signal Dispatcher,5,main]
>   com.ibm.misc.SignalDispatcher.waitForSignal(Native Method)
>   com.ibm.misc.SignalDispatcher.run(SignalDispatcher.java:73)
> Thread[Thread-4 
> (ActiveMQ-client-global-threads--90143455),5,ActiveMQ-client-global-threads--90143455]
>   sun.misc.Unsafe.park(Native Method)
>   java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1048)
>   
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1339)
>   java.util.concurrent.CountDownLatch.await(CountDownLatch.java:288)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.waitOnLatch(ActiveMQClientProtocolManager.java:134)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:829)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.reconnectSessions(ClientSessionFactoryImpl.java:753)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:615)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:513)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$400(ClientSessionFactoryImpl.java:70)
>   
> org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1191)
>   
> org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:64)
>   
> org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:210)
>   
> org.apache.activemq.artemis.sp