*My Setup: *
Systems:
>OS: Windows Servers 2016
>ActiveMQ Version: 5.14.5
>Java: 1.8.141
I have setup 2 - 3 node Master/Slave/Slave clusters on AWS
2 on-prem ActiveMQ servers
>1 running 5.14.5
>1 running 5.10.0
Current network configuration
Germany (AWS) <-> USEast (AWS) <-> Cambridge (on-prem)
*Problem:*
I have been seeing my network connector drop and reconnect every 30 sec or
so.
*Network Connector Config:*
*Fmigr (DEXXXq01)*
<networkConnectors>
<networkConnector name="fmiuseast1" duplex="true" networkTTL="2"
staticBridge="true" userName="${bridge.username}"
password="${bridge.password}"
uri="masterslave:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)">
<staticallyIncludedDestinations>
<topic physicalName="trfTrackingEvent" />
</staticallyIncludedDestinations>
</networkConnector>
<networkConnector name="fmiuseast2" duplex="false" networkTTL="2"
staticBridge="true" userName="${bridge.username}"
password="${bridge.password}"
uri="masterslave:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)">
<staticallyIncludedDestinations>
<topic physicalName="medicalreport2" />
<topic physicalName="VirtualTopic.heartbeat1" />
</staticallyIncludedDestinations>
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://0.0.0.0:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="amqp"
uri="amqp://0.0.0.0:5672?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="stomp"
uri="stomp://0.0.0.0:61613?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="ssl"
uri="ssl://0.0.0.0:61617?transport.enabledProtocols=TLSv1,TLSv1.1,TLSv1.2&connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="stomp+ssl"
uri="stomp+ssl://0.0.0.0:61612?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="amqp+ssl"
uri="amqp+ssl://0.0.0.0:5671?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
</transportConnectors>
*USEast Side:*
<networkConnectors>
<networkConnector name="fmigr" duplex="false" networkTTL="2"
staticBridge="true" userName="${bridge.username}"
password="${bridge.password}"
uri="masterslave:(ssl://DEXXXQ01:61617,ssl://DEXXXQ02:61617,ssl://DEXXXQ03:61617)">
<staticallyIncludedDestinations>
<topic physicalName="medicalReportingNotification" />
<topic physicalName="VirtualTopic.heartbeat" />
</staticallyIncludedDestinations>
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://0.0.0.0:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="amqp"
uri="amqp://0.0.0.0:5672?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="stomp"
uri="stomp://0.0.0.0:61613?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="ssl"
uri="ssl://0.0.0.0:61617?transport.enabledProtocols=TLSv1,TLSv1.1,TLSv1.2&connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="stomp+ssl"
uri="stomp+ssl://0.0.0.0:61612?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
<transportConnector name="amqp+ssl"
uri="amqp+ssl://0.0.0.0:5671?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0"/>
</transportConnectors>
*My fmigr activeMQ Log:*
2017-09-19 18:25:02,578 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-34
2017-09-19 18:25:03,213 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 322718397
2017-09-19 18:25:03,321 | INFO | Network connection between
vm://fmigrq01#178 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#178
2017-09-19 18:25:33,433 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:25:33,433 | WARN | Network connection between
vm://fmigrq01#178 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:25:33,439 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-298
2017-09-19 18:25:34,435 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-35
2017-09-19 18:25:35,062 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 663286350
2017-09-19 18:25:35,169 | INFO | Network connection between
vm://fmigrq01#182 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#182
2017-09-19 18:26:05,301 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:26:05,302 | WARN | Network connection between
vm://fmigrq01#182 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:26:05,307 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-307
2017-09-19 18:26:06,304 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-36
2017-09-19 18:26:06,937 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 1852967876
2017-09-19 18:26:07,041 | INFO | Network connection between
vm://fmigrq01#188 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#188
2017-09-19 18:26:37,156 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:26:37,156 | WARN | Network connection between
vm://fmigrq01#188 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:26:37,162 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-314
2017-09-19 18:26:38,158 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-37
2017-09-19 18:26:38,784 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 1838742535
2017-09-19 18:26:38,893 | INFO | Network connection between
vm://fmigrq01#192 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#192
2017-09-19 18:27:09,006 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:27:09,006 | WARN | Network connection between
vm://fmigrq01#192 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:27:09,012 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-323
2017-09-19 18:27:10,008 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-38
2017-09-19 18:27:10,640 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 1244983570
2017-09-19 18:27:10,747 | INFO | Network connection between
vm://fmigrq01#198 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#198
2017-09-19 18:27:40,859 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:27:40,859 | WARN | Network connection between
vm://fmigrq01#198 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:27:40,865 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-330
2017-09-19 18:27:41,861 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-39
2017-09-19 18:27:42,497 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 1096056638
2017-09-19 18:27:42,608 | INFO | Network connection between
vm://fmigrq01#202 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#202
2017-09-19 18:28:12,717 | WARN | Transport
(ssl://USAEXXXQ01.corp.local:61617) failed , not attempting to automatically
reconnect: java.io.EOFException |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:28:12,717 | WARN | Network connection between
vm://fmigrq01#202 and unconnected shutdown due to a remote error:
java.io.EOFException |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: ssl://USAEXXXQ01.corp.local/10.120.237.233:61617
2017-09-19 18:28:12,724 | INFO | fmigrq01 bridge to fmiuseastq01 stopped |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[fmigrq01] Task-339
2017-09-19 18:28:13,719 | INFO | Establishing network connection from
vm://fmigrq01?async=false&create=false to
failover:(ssl://USAEXXXQ01.corp.local:61617,ssl://USAEXXXQ02.corp.local:61617,ssl://USAEXXXQ03.corp.local:61617)?randomize=false&maxReconnectAttempts=0
| org.apache.activemq.network.DiscoveryNetworkConnector | Simple Discovery
Agent-40
2017-09-19 18:28:14,349 | INFO | Successfully connected to
ssl://USAEXXXQ01.corp.local:61617 |
org.apache.activemq.transport.failover.FailoverTransport | ActiveMQ Failover
Worker: 597416418
2017-09-19 18:28:14,455 | INFO | Network connection between
vm://fmigrq01#208 and ssl://USAEXXXQ01.corp.local:61617 (fmiuseastq01) has
been established. |
org.apache.activemq.network.DemandForwardingBridgeSupport |
triggerStartAsyncNetworkBridgeCreation: remoteBroker=unconnected,
localBroker= vm://fmigrq01#208
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
