I'm not really sure how what you are seeing is possible. I looked at your
stack trace and it says that getInFlightUsage() is the culprit but there is
a check for 0 already.
public int getInFlightUsage() {
if (info.getPrefetchSize() > 0) {
return (getInFlightSize() * 100)/info.getPrefetchSize();
}
return Integer.MAX_VALUE;
}
So not really sure how you could be seeing what you are seeing because the
ConsumerInfo is only set in the constructor so it shouldn't be changing
after the fact.
On Wed, Jul 13, 2016 at 8:41 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> This is new, so please submit a bug in JIRA. Please include as much
> information as you can about your configuration and how your broker is
> being used; because you're seeing problems while no one else is and this
> appears to be code everyone should be hitting, it's possible that there's
> something unique about your setup or your usage that triggers the problem,
> and whoever investigates the issue will benefit from having as much detail
> as possible.
>
> Tim
> On Jul 13, 2016 12:26 AM, "Johan Carlquist" <jo...@su.se> wrote:
>
> Hi!
>
> We have two brokers esb-test-mq01 and esb-test-mq02 configured in a
> network of brokers. Each broker has configuration like this:
>
> <networkConnector
> networkTTL="-1"
> conduitSubscriptions="false"
> decreaseNetworkConsumerPriority="true"
> uri="static:(ssl://${activemq.networkPartner}:61616)"
> userName="${activemq.username}"
> password="${activemq.password}"
> />
>
> Both brokers running ActiveMQ 5.13.0, Java 1.8.0_45 and Ubuntu 14.04
> (3.13.0-87-generic x86_64).
>
> Sometimes (we havn't found any patterns for when it occurs) one of the
> connection between the hosts closes with the following message:
>
> "Network connection between vm://esb-test-mq01.it.su.se#18 and ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616 shutdown due to a local error:
> java.lang.ArithmeticException: / by zero"
>
> After this the broker never calls back. Yesterday we finally caught this
> division with TRACE turned on, but we can't make anything out of the
> logs. We now wondering, is this a known bug that we haven't found any
> information on, just missconfiguration or should we report the bug on
> issues.apache.org?
>
> Greatful for any input.
>
> --
> Johan
>
> And the log lines (sorry for the json encryption):
>
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897046,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3082","level":"DEBUG","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"bridging
> (esb-test-mq01.it.su.se -> esb-test-mq02.it.su.se), consumer:
> ActiveMQBytesMessage {commandId = 1218913, responseRequired = true,
> messageId = ID:esb-test-mq02-43870-1468234158631-7:21062:0:0:1,
> originalDestination = null, originalTransactionId = null, producerId =
>
> esb-test-mq01.it.su.se->esb-test-mq02.it.su.se-45448-1467985313001-17:2:1:1,
> destination = queue://aliveness-test, transactionId = null, expiration = 0,
> timestamp = 1468286832000, arrival = 0, brokerInTime = 1468294896175,
> brokerOutTime = 1468294897046, correlationId = null, replyTo = null,
> persistent = true, type = null, priority = 4, groupID = null, groupSequence
> = 0, targetConsumerId = null, compressed = false, userID = null, content =
> org.apache.activemq.util.ByteSequence@4bd15e41, marshalledProperties =
> org.apache.activemq.util.ByteSequence@44e78a09, dataStructure = null,
> redeliveryCounter = 865, size = 1193, properties = {JMS_AMQP_NATIVE=true,
> JMS_AMQP_MESSAGE_FORMAT=0, JMS_AMQP_FirstAcquirer=false},
> readOnlyProperties = false, readOnlyBody = false, droppable = false,
> jmsXGroupFirstForConsumer = false} ActiveMQBytesMessage{ bytesOut = null,
> dataOut = null, dataIn = null }, destination:
>
> esb-test-mq01.it.su.se->esb-test-mq02.it.su.se-45448-1467985313001-17:1:1:36663,
> brokerPath: queue://aliveness-test, message: [esb-test-mq02.it.su.se,
> esb-test-mq01.it.su.se, esb-test-mq02.it.su.se, esb-test-mq01.it.su.se,
> esb-test-mq02.it.su.se, esb-test-mq01.it.su.se, esb-test-mq02.it.su.se,
> esb-test-mq01.it.su.se, esb-test-mq02.it.su.se, esb-test-mq01.it.su.se,
> esb-test-mq02.it.su.se, esb-test-mq01.it.su.se, esb-test-mq02.it.su.se,
> esb-test-mq01.it.su.se, esb-test-mq02.it.su.se, esb-test-mq01.it.su.se,
> esb-test-mq02.it.su.se, esb-test-mq01.it.su.se, esb-test-mq02.it.su.se,
> esb-test-mq01.it.su.se, esb-test-mq02.it.su.se, esb-test-m
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897057,"thread":"ActiveMQ Transport: ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616
> ","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"serviceLocalException:
> disposed false ex","thrown":{"commonElementCount":0,"localizedMessage":"/
> by zero","message":"/ by
>
> zero","name":"java.lang.ArithmeticException","extendedStackTrace":[{"class":"org.apache.activemq.broker.region.AbstractSubscription","method":"getInFlightUsage","file":"AbstractSubscription.java","line":240,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"isOptimizeStorage","file":"Queue.java","line":2322,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"doMessageSend","file":"Queue.java","line":839,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"send","file":"Queue.java","line":731,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.DestinationFilter","method":"send","file":"DestinationFilter.java","line":132,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.AbstractRegion","method":"send","file":"AbstractRegion.java","line":503,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.RegionBroker","method":"send","file":"RegionBroker.java","line":468,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.jmx.ManagedRegionBroker","method":"send","file":"ManagedRegionBroker.java","line":296,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.BrokerFilter","method":"send","file":"BrokerFilter.java","li
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897060,"thread":"ActiveMQ Transport: ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616
> ","level":"INFO","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"Network
> connection between vm://esb-test-mq01.it.su.se#18 and ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616 shutdown due to a local error:
> java.lang.ArithmeticException: / by
> zero","endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897060,"thread":"ActiveMQ Transport: ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616
> ","level":"DEBUG","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"The
> local Exception was: java.lang.ArithmeticException: / by
> zero","thrown":{"commonElementCount":0,"localizedMessage":"/ by
> zero","message":"/ by
>
> zero","name":"java.lang.ArithmeticException","extendedStackTrace":[{"class":"org.apache.activemq.broker.region.AbstractSubscription","method":"getInFlightUsage","file":"AbstractSubscription.java","line":240,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"isOptimizeStorage","file":"Queue.java","line":2322,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"doMessageSend","file":"Queue.java","line":839,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"send","file":"Queue.java","line":731,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.DestinationFilter","method":"send","file":"DestinationFilter.java","line":132,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.AbstractRegion","method":"send","file":"AbstractRegion.java","line":503,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.RegionBroker","method":"send","file":"RegionBroker.java","line":468,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.jmx.ManagedRegionBroker","method":"send","file":"ManagedRegionBroker.java","line":296,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.BrokerFilter","method":"send","file
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897062,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3079","level":"DEBUG","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"
> stopping esb-test-mq01.it.su.se bridge to esb-test-mq02.it.su.se
> ","endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897062,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3079","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"
> esb-test-mq01.it.su.se unregister bridge
> (org.apache.activemq.network.DemandForwardingBridge@199245a1) to
> esb-test-mq02.it.su.se
> ","endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897062,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3084","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"serviceLocalException:
> disposed true ex","thrown":{"commonElementCount":0,"localizedMessage":"peer
> (vm://esb-test-mq01.it.su.se#19) stopped.","message":"peer (vm://
> esb-test-mq01.it.su.se#19)
>
> stopped.","name":"org.apache.activemq.transport.TransportDisposedIOException","extendedStackTrace":[{"class":"org.apache.activemq.transport.vm.VMTransport","method":"stop","file":"VMTransport.java","line":230,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.transport.TransportFilter","method":"stop","file":"TransportFilter.java","line":65,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.transport.TransportFilter","method":"stop","file":"TransportFilter.java","line":65,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.transport.ResponseCorrelator","method":"stop","file":"ResponseCorrelator.java","line":132,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.TransportConnection","method":"doStop","file":"TransportConnection.java","line":1165,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.TransportConnection$4","method":"run","file":"TransportConnection.java","line":1131,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","file":"ThreadPoolExecutor.java","line":1142,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","file":"ThreadPoolExecutor.java","line":617,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.lang.Thread","method":"run","file":"Thread.j
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897062,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3079","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"serviceLocalException:
> disposed true
>
> ex","thrown":{"commonElementCount":0,"localizedMessage":"Stopped.","message":"Stopped.","name":"java.io.IOException","extendedStackTrace":[{"class":"org.apache.activemq.transport.ResponseCorrelator","method":"stop","file":"ResponseCorrelator.java","line":131,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.util.ServiceStopper","method":"stop","file":"ServiceStopper.java","line":41,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.network.DemandForwardingBridgeSupport","method":"stop","file":"DemandForwardingBridgeSupport.java","line":305,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.util.ServiceSupport","method":"dispose","file":"ServiceSupport.java","line":43,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.network.DemandForwardingBridgeSupport$9","method":"run","file":"DemandForwardingBridgeSupport.java","line":878,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","file":"ThreadPoolExecutor.java","line":1142,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","file":"ThreadPoolExecutor.java","line":617,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.lang.Thread","method":"run","file":"Thread.java","line":745,"exact":true,"location":"?","version":"1.8.0_45"}]},"endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897061,"thread":"ActiveMQ Transport: ssl://
> esb-test-mq02.it.su.se/77.238.37.208:61616
> ","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"fire
> bridge failed, listener:
>
> org.apache.activemq.network.DiscoveryNetworkConnector$1DiscoverNetworkBridgeListener@67a83e67
> ","thrown":{"commonElementCount":0,"localizedMessage":"/
> by zero","message":"/ by
>
> zero","name":"java.lang.ArithmeticException","extendedStackTrace":[{"class":"org.apache.activemq.broker.region.AbstractSubscription","method":"getInFlightUsage","file":"AbstractSubscription.java","line":240,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"isOptimizeStorage","file":"Queue.java","line":2322,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"doMessageSend","file":"Queue.java","line":839,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.Queue","method":"send","file":"Queue.java","line":731,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.DestinationFilter","method":"send","file":"DestinationFilter.java","line":132,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.AbstractRegion","method":"send","file":"AbstractRegion.java","line":503,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.region.RegionBroker","method":"send","file":"RegionBroker.java","line":468,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.broker.jmx.ManagedRegionBroker","method":"send","file":"ManagedRegionBroker.java","line":296,"exact":false,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897063,"thread":"ActiveMQ
>
> Task-3","level":"INFO","loggerName":"org.apache.activemq.network.DiscoveryNetworkConnector","message":"Establishing
> network connection from vm://esb-test-mq01.it.su.se?async=false to ssl://
> esb-test-mq02.it.su.se:61616
> ","endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
> Jul 12 05:41:37 esb-test-mq01.it.su.se {"timeMillis":
> 1468294897062,"thread":"ActiveMQ BrokerService[esb-test-mq01.it.su.se]
>
> Task-3079","level":"TRACE","loggerName":"org.apache.activemq.network.DemandForwardingBridgeSupport","message":"serviceLocalException:
> disposed true
>
> ex","thrown":{"commonElementCount":0,"localizedMessage":"Stopped.","message":"Stopped.","name":"java.io.IOException","extendedStackTrace":[{"class":"org.apache.activemq.transport.ResponseCorrelator","method":"stop","file":"ResponseCorrelator.java","line":131,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.util.ServiceStopper","method":"stop","file":"ServiceStopper.java","line":41,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.network.DemandForwardingBridgeSupport","method":"stop","file":"DemandForwardingBridgeSupport.java","line":305,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.util.ServiceSupport","method":"dispose","file":"ServiceSupport.java","line":43,"exact":true,"location":"activemq-client-5.13.0.jar","version":"5.13.0"},{"class":"org.apache.activemq.network.DemandForwardingBridgeSupport$9","method":"run","file":"DemandForwardingBridgeSupport.java","line":878,"exact":true,"location":"activemq-broker-5.13.0.jar","version":"5.13.0"},{"class":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","file":"ThreadPoolExecutor.java","line":1142,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","file":"ThreadPoolExecutor.java","line":617,"exact":true,"location":"?","version":"1.8.0_45"},{"class":"java.lang.Thread","method":"run","file":"Thread.java","line":745,"exact":true,"location":"?","version":"1.8.0_45"}]},"endOfBatch":false,"loggerFqcn":"org.slf4j.impl.Log4jLoggerAdapter"}
>
