I'm trying to get an Artemis 2.0 cluster working. I have 3 Nodes Running on AWS Windows 2016 and Artemis 2.0 Java jdk 1.8.0_131
I can't seem to get them to fail over when I take the Live server offline I see no changes to the backup servers. I'm not sure whats happening anyone have any ideas? Thanks Mike Live Server excerpts: 16:48:17,209 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@bae7dc0[owner=ClusterConnectionImpl@547201549[nodeUUID=c20c017e-35c7-11e7-97fc-0e677709a282, connector=TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=10-121-51-121, address=jms, server=ActiveMQServerImpl::serverUUID=c20c017e-35c7-11e7-97fc-0e677709a282]] is sending topology to Remote Proxy on channel 6c3f9531 16:48:17,211 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@bae7dc0[owner=ClusterConnectionImpl@547201549[nodeUUID=c20c017e-35c7-11e7-97fc-0e677709a282, connector=TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=10-121-51-121, address=jms, server=ActiveMQServerImpl::serverUUID=c20c017e-35c7-11e7-97fc-0e677709a282]] sending c20c017e-35c7-11e7-97fc-0e677709a282 / Pair[a=TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=10-121-51-121, b=null] to Remote Proxy on channel 6c3f9531 16:48:17,295 DEBUG [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl] ClientSessionFactoryImpl received backup update for live/backup pair = TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=10-121-51-121 / null but it didn't belong to TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=10-121-51-121 16:48:17,887 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://localhost:8161 16:48:17,899 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://localhost:8161/jolokia 16:48:28,204 DEBUG [org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl] ClusterCommunication::Sending notification for addBinding LocalQueueBinding [address=ActiveMQ.Advisory.TempQueue, queue=QueueImpl[name=6addd362-772b-4675-9c97-fca4eb1fdada, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=c20c017e-35c7-11e7-97fc-0e677709a282], temp=true]@dae557e, filter=FilterImpl [sfilterString=__AMQ_CID<>'ID:b3be4c964f10-37643-1494535648494-1:1'], name=6addd362-772b-4675-9c97-fca4eb1fdada, clusterName=6addd362-772b-4675-9c97-fca4eb1fdadac20c017e-35c7-11e7-97fc-0e677709a282] from server ActiveMQServerImpl::serverUUID=c20c017e-35c7-11e7-97fc-0e677709a282 16:48:28,222 DEBUG [org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl] Couldn't find any bindings for address=activemq.notifications on message=CoreMessage[messageID=6442452924,durable=true,userID=null,priority=0, timestamp=0,expiration=0, durable=true, address=activemq.notifications,properties=TypedProperties[_AMQ_Binding_Type=0,_AMQ_RoutingName=6addd362-772b-4675-9c97-fca4eb1fdada,_AMQ_Distance=0,_AMQ_Address=ActiveMQ.Advisory.TempQueue,_AMQ_NotifType=BINDING_ADDED,_AMQ_Binding_ID=6442452923,_AMQ_FilterString=__AMQ_CID<>'ID:b3be4c964f10-37643-1494535648494-1:1',_AMQ_NotifTimestamp=1494535708220,_AMQ_ClusterName=6addd362-772b-4675-9c97-fca4eb1fdadac20c017e-35c7-11e7-97fc-0e677709a282,foobar=2f6bb043-368b-11e7-a106-0e677709a282]]@2101112334 16:48:28,222 DEBUG [org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl] Message CoreMessage[messageID=6442452924,durable=true,userID=null,priority=0, timestamp=0,expiration=0, durable=true, address=activemq.notifications,properties=TypedProperties[_AMQ_Binding_Type=0,_AMQ_RoutingName=6addd362-772b-4675-9c97-fca4eb1fdada,_AMQ_Distance=0,_AMQ_Address=ActiveMQ.Advisory.TempQueue,_AMQ_NotifType=BINDING_ADDED,_AMQ_Binding_ID=6442452923,_AMQ_FilterString=__AMQ_CID<>'ID:b3be4c964f10-37643-1494535648494-1:1',_AMQ_NotifTimestamp=1494535708220,_AMQ_ClusterName=6addd362-772b-4675-9c97-fca4eb1fdadac20c017e-35c7-11e7-97fc-0e677709a282,foobar=2f6bb043-368b-11e7-a106-0e677709a282]]@2101112334 is not going anywhere as it didn't have a binding on address:activemq.notifications 16:48:28,293 DEBUG [org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl] Couldn't find any bindings for address=activemq.notifications on message=CoreMessage[messageID=6442452927,durable=true,userID=null,priority=0, timestamp=0,expiration=0, durable=true, address=activemq.notifications,properties=TypedProperties[_AMQ_RoutingName=6addd362-772b-4675-9c97-fca4eb1fdada,_AMQ_Distance=0,_AMQ_ConsumerCount=1,_AMQ_User=fmiuser,_AMQ_SessionName=ID:b3be4c964f10-37643-1494535648494-1:1:-1,_AMQ_Address=ActiveMQ.Advisory.TempQueue,_AMQ_RemoteAddress=/10.121.43.230:51534,_AMQ_NotifType=CONSUMER_CREATED,_AMQ_NotifTimestamp=1494535708292,_AMQ_ClusterName=6addd362-772b-4675-9c97-fca4eb1fdadac20c017e-35c7-11e7-97fc-0e677709a282]]@1025865428 16:48:28,293 DEBUG [org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl] Message CoreMessage[messageID=6442452927,durable=true,userID=null,priority=0, timestamp=0,expiration=0, durable=true, address=activemq.notifications,properties=TypedProperties[_AMQ_RoutingName=6addd362-772b-4675-9c97-fca4eb1fdada,_AMQ_Distance=0,_AMQ_ConsumerCount=1,_AMQ_User=fmiuser,_AMQ_SessionName=ID:b3be4c964f10-37643-1494535648494-1:1:-1,_AMQ_Address=ActiveMQ.Advisory.TempQueue,_AMQ_RemoteAddress=/10.121.43.230:51534,_AMQ_NotifType=CONSUMER_CREATED,_AMQ_NotifTimestamp=1494535708292,_AMQ_ClusterName=6addd362-772b-4675-9c97-fca4eb1fdadac20c017e-35c7-11e7-97fc-0e677709a282]]@1025865428 is not going anywhere as it didn't have a binding on address:activemq.notifications Backup Server 1: 16:20:53,563 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server 16:20:53,603 INFO [org.apache.activemq.artemis.core.server] AMQ221000: backup Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=./data/journal,bindingsDirectory=./data/bindings,largeMessagesDirectory=./data/large-messages,pagingDirectory=./data/paging) 16:20:53,629 INFO [org.apache.activemq.artemis.core.server] AMQ221055: There were too many old replicated folders upon startup, removing C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal\oldreplica.10 16:20:53,636 INFO [org.apache.activemq.artemis.core.server] AMQ222162: Moving data directory C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal to C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal\oldreplica.12 16:20:53,840 INFO [org.apache.activemq.artemis.core.server] AMQ221013: Using NIO Journal 16:20:53,885 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.numHeapArenas: 4 16:20:53,887 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.numDirectArenas: 4 16:20:53,894 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.pageSize: 8192 16:20:53,896 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.maxOrder: 11 16:20:53,898 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.chunkSize: 16777216 16:20:53,899 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.tinyCacheSize: 512 16:20:53,900 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.smallCacheSize: 256 16:20:53,901 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.normalCacheSize: 64 16:20:53,906 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.maxCachedBufferCapacity: 32768 16:20:53,906 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.cacheTrimInterval: 8192 16:20:53,953 FINE [io.netty.buffer.AbstractByteBuf] -Dio.netty.buffer.bytebuf.checkAccessible: true 16:20:53,962 FINE [io.netty.util.ResourceLeakDetector] -Dio.netty.leakDetection.level: simple 16:20:53,962 FINE [io.netty.util.ResourceLeakDetector] -Dio.netty.leakDetection.maxRecords: 4 16:20:53,965 FINE [io.netty.util.ResourceLeakDetectorFactory] Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@30e2e2a8 16:20:54,132 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@4d2d5542[owner=ServerLocatorImpl [initialConnectors=[], discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] is sending topology to QuorumManager(server=null) 16:20:54,165 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 16:20:54,169 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP 16:20:54,170 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 16:20:54,171 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolManagerFactory@705239f 16:20:54,171 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT 16:20:54,173 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.openwire.OpenWireProtocolManagerFactory@504ab59c 16:20:54,174 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE 16:20:54,174 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.stomp.StompProtocolManagerFactory@3b825a20 16:20:54,174 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP 16:20:54,280 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@4d2d5542[owner=ServerLocatorImpl [initialConnectors=[], discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] is sending topology to org.apache.activemq.artemis.core.server.impl.AnyLiveNodeLocatorForReplication@7a2aca8e 16:20:54,692 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://localhost:8161 16:20:54,693 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://localhost:8161/jolokia Backup Server 2: 16:21:11,762 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server 16:21:11,825 INFO [org.apache.activemq.artemis.core.server] AMQ221000: backup Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=./data/journal,bindingsDirectory=./data/bindings,largeMessagesDirectory=./data/large-messages,pagingDirectory=./data/paging) 16:21:11,888 INFO [org.apache.activemq.artemis.core.server] AMQ221055: There were too many old replicated folders upon startup, removing C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal\oldreplica.11 16:21:11,892 INFO [org.apache.activemq.artemis.core.server] AMQ222162: Moving data directory C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal to C:\Apache\apache-artemis-2.0.0\bin\brokerAWS1\.\data\journal\oldreplica.13 16:21:11,947 INFO [org.apache.activemq.artemis.core.server] AMQ221013: Using NIO Journal 16:21:11,976 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.numHeapArenas: 4 16:21:11,976 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.numDirectArenas: 4 16:21:11,981 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.pageSize: 8192 16:21:11,983 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.maxOrder: 11 16:21:11,985 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.chunkSize: 16777216 16:21:11,987 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.tinyCacheSize: 512 16:21:11,988 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.smallCacheSize: 256 16:21:12,008 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.normalCacheSize: 64 16:21:12,010 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.maxCachedBufferCapacity: 32768 16:21:12,014 FINE [io.netty.buffer.PooledByteBufAllocator] -Dio.netty.allocator.cacheTrimInterval: 8192 16:21:12,077 FINE [io.netty.buffer.AbstractByteBuf] -Dio.netty.buffer.bytebuf.checkAccessible: true 16:21:12,084 FINE [io.netty.util.ResourceLeakDetector] -Dio.netty.leakDetection.level: simple 16:21:12,092 FINE [io.netty.util.ResourceLeakDetector] -Dio.netty.leakDetection.maxRecords: 4 16:21:12,095 FINE [io.netty.util.ResourceLeakDetectorFactory] Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@5bcba7e2 16:21:12,302 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@3bd801d9[owner=ServerLocatorImpl [initialConnectors=[], discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] is sending topology to QuorumManager(server=null) 16:21:12,341 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 16:21:12,343 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP 16:21:12,345 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 16:21:12,348 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolManagerFactory@60819872 16:21:12,349 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT 16:21:12,350 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.openwire.OpenWireProtocolManagerFactory@13959ee 16:21:12,351 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE 16:21:12,352 WARN [org.apache.activemq.artemis.spi.core.protocol.MessagePersister] Cannot find persister for org.apache.activemq.artemis.core.protocol.stomp.StompProtocolManagerFactory@767fe033 16:21:12,353 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP 16:21:12,453 DEBUG [org.apache.activemq.artemis.core.client.impl.Topology] Topology@3bd801d9[owner=ServerLocatorImpl [initialConnectors=[], discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] is sending topology to org.apache.activemq.artemis.core.server.impl.AnyLiveNodeLocatorForReplication@35cdb75c 16:21:12,915 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://localhost:8161 16:21:12,922 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://localhost:8161/jolokia -- View this message in context: http://activemq.2283324.n4.nabble.com/Artemis-2-0-cluster-fail-over-issue-tp4726002.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
