Jiri Formanek created AMQ-6557: ---------------------------------- Summary: Constant reconnecting after ActiveMQ restart with HTTP Layer Key: AMQ-6557 URL: https://issues.apache.org/jira/browse/AMQ-6557 Project: ActiveMQ Issue Type: Bug Components: Broker, networkbridge Affects Versions: 5.14.1 Environment: Linux openSuse Reporter: Jiri Formanek
The ActiveMQ broker reconnects constantly to other brokers after restart, when HTTP is used as transport layer. Let's consider following simple broker network, where machine A connects to machine B... Machine A config: {code} <networkConnector duplex="true" uri="static:(http://machineB:61617)" name="bridgeToB" /> {code} Machine B config: {code} <transportConnector name="http" uri="http://0.0.0.0:61617?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/> {code} When ActiveMQ on machine A is restarted, then the connection to machine B is successfully re-established after restart, but after 30 seconds, when keep-alive mechanism sends a HTTP GET request on machine B, then this fails with some java.io.IOException. Then the connection breaks up. It's established new connection to machine B and after 30 seconds again fails the keep-alive GET request. And so on over and over till ActiveMQ on machine B is restarted. See logs: {noformat} 2017-01-11 14:32:19,005 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1017 2017-01-11 14:32:20,726 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1017 2017-01-11 14:32:20,923 | INFO | Network connection between vm://localhost#9512 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9512 2017-01-11 14:32:54,912 | WARN | Network connection between vm://localhost#9512 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE T on: http://machineB:61617 Reason: Read timed out | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: HTTP Reader http://machineB:61617 2017-01-11 14:32:54,917 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-4377 2017-01-11 14:32:55,913 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1018 2017-01-11 14:32:57,642 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1018 2017-01-11 14:32:57,841 | INFO | Network connection between vm://localhost#9516 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9516 2017-01-11 14:33:31,429 | WARN | Network connection between vm://localhost#9516 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE T on: http://machineB:61617 Reason: Read timed out | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: HTTP Reader http://machineB:61617 2017-01-11 14:33:31,433 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-4390 2017-01-11 14:33:32,430 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1019 2017-01-11 14:33:34,157 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1019 2017-01-11 14:33:34,355 | INFO | Network connection between vm://localhost#9520 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9520 2017-01-11 14:34:07,169 | WARN | Network connection between vm://localhost#9520 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE T on: http://machineB:61617 Reason: Read timed out | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: HTTP Reader http://machineB:61617 2017-01-11 14:34:07,175 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-4400 2017-01-11 14:34:08,170 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1020 2017-01-11 14:34:09,898 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1020 2017-01-11 14:34:10,095 | INFO | Network connection between vm://localhost#9524 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9524 2017-01-11 14:34:42,505 | WARN | Network connection between vm://localhost#9524 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE T on: http://machineB:61617 Reason: Read timed out | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: HTTP Reader http://machineB:61617 2017-01-11 14:34:42,508 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-4407 2017-01-11 14:34:43,506 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1021 2017-01-11 14:34:45,221 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1021 2017-01-11 14:34:45,419 | INFO | Network connection between vm://localhost#9528 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9528 2017-01-11 14:35:19,002 | WARN | Network connection between vm://localhost#9528 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE T on: http://machineB:61617 Reason: Read timed out | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ Transport: HTTP Reader http://machineB:61617 2017-01-11 14:35:19,006 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-4415 2017-01-11 14:35:20,003 | INFO | Establishing network connection from vm://localhost?async=false&create=false to http://machineB:61617 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1022 2017-01-11 14:35:21,751 | INFO | Broker Servlet supports GZip compression. | org.apache.activemq.transport.http.HttpClientTransport | ActiveMQ Task-1022 2017-01-11 14:35:21,949 | INFO | Network connection between vm://localhost#9532 and HTTP Reader http://machineB:61617 (localhost) has been established. | org.apache.activemq.network.DemandFor wardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=HTTP Reader http://machineB:61617, localBroker= vm://localhost#9532 2017-01-11 14:35:55,973 | WARN | Network connection between vm://localhost#9532 and HTTP Reader http://machineB:61617 shutdown due to a remote error: java.io.IOException: Failed to perform GE {noformat} We have installed currently 5.14.1, but I suppose, that newest version has the same behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)