Gary,
It does appear that it is just the stats that are wrong in this case. The
messages
are getting delivered as expected (as long as there is an application
consumer on
that broker).
A couple of other questions:
• Since I only seem to see this issue when there are 2 or more brokers, is
there
anything in our networkConnector string that could cause this (or any other
part
of the server configuration that would be advisable to re-check)?
<networkConnector name="ActiveMQ Cluster"
uri="static:(tcp://broker1:61616,tcp://broker2:61616,tcp://broker3:61616)"
conduitSubscriptions="true" dynamicOnly="true"
decreaseNetworkConsumerPriority="false"
suppressDuplicateQueueSubscriptions="true" duplex="false" networkTTL="5"/>
• Also, currently we are running ActiveMQ server *5.4.0*, but our clients
(consumers)
are still running version 5.2.0 - is there any possibility that this could
be causing
the issue (I am in the process of rebuilding the consumers with 5.4.0
libraries to
confirm this).
Thanks,
On Thu, Sep 9, 2010 at 8:49 AM, Gary Tully <gary.tu...@gmail.com> wrote:
> ok, I guess this is going to be very hard to track down without a
> reproducible test case. Can you confirm that it is just the stats that
> are wrong in this case, and that all expected messages are getting
> delivered as expected.
>
> On 9 September 2010 14:28, Colleen Velo <cmv...@gmail.com> wrote:
> > Gary,
> >
> > That doesn't seem to be the case here - namely, the JConsole queue count
> is
> > accurately
> > reflecting what the ActiveMQ web console is showing. I have rechecked
> our
> > application
> > code as well as the ActiveMQ server settings and in both cases we are
> using
> > the default
> > *dispatchAsync* value of false which, according to your blog post from
> Jan
> > 2010, should
> > take care of the "inflight" counts issue.
> >
> > It seems to be more a case where the queue "counters" seem to get "thrown
> > off"
> > and never get back in-sync until the broker is restarted.
> > For example, I can see a number "-10" in the "Pending Messages" on the
> > ActiveMQ
> > web admin console (and in JConsole), but clicking on the queue name and
> > showing
> > the actual contents of the queue might reveal 0 items or 5 items, etc.
> > (Note: in the above scenario, the producers were not putting any new
> items
> > onto the
> > queue, so the # of "Pending Messages" wasn't changing).
> >
> >
> > On Thu, Sep 9, 2010 at 5:44 AM, Gary Tully <gary.tu...@gmail.com> wrote:
> >
> >> Is it by any chance the inflight count in jconsole that is negative?
> >> There is a good explanation for that,
> >>
> http://blog.garytully.com/2010/01/activemq-prefetch-and-asyncdispatch.html
> >>
> >> On 8 September 2010 21:42, Colleen Velo <cmv...@gmail.com> wrote:
> >> > Gary,
> >> > * *
> >> > *Sorry - last message was incomplete - technical difficulties*
> >> >
> >> > Haven't had any luck reproducing this issue with a JMeter test,
> however,
> >> I
> >> > can reproduce
> >> > it on a fairly regular basis using our in-house application script,
> which
> >> > puts tons of application
> >> > messages on the queues.
> >> >
> >> > Also, I don't seem to see this issue when there is only a single
> broker
> >> > running, only when
> >> > there is two or more running in a network of brokers configuration.
> >> >
> >> > When I do see this issue, I •do• see it on JConsole as well as in the
> >> > ActiveMQ WebAdmin console.
> >> >
> >> > *Possible Test Case:*
> >> >
> >> > To reproduce, start up multiple Producers, both of which are throwing
> >> > 80,000+ messages
> >> > to the two brokers (which are configured as a network of brokers ) and
> >> have
> >> > two consumers running on
> >> > separate servers pulling items off of the application queue(s).
> >> >
> >> >
> >> > On Wed, Sep 8, 2010 at 3:39 PM, Colleen Velo <cmv...@gmail.com>
> wrote:
> >> >
> >> >> Gary,
> >> >>
> >> >> Haven't had any luck reproducing this issue with a JMeter test,
> however,
> >> I
> >> >> can reproduce
> >> >> it on a fairly regular basis using our in-house application script,
> >> which
> >> >> puts
> >> >>
> >> >> Also, I don't seem to see this issue when there is only a single
> broker
> >> >> running, only when
> >> >> there is two or more running in a network of brokers configuration.
> >> >>
> >> >> When I do see this issue, I •do• see it on JConsole as well as in the
> >> >> ActiveMQ WebAdmin console.
> >> >>
> >> >>
> >> >
> >> >> To reproduce, start up multiple Producers, both of which are throwing
> >> >> 80,000+ messages
> >> >> at two brokers (which are networked together) and have the two
> consumers
> >> >> running on
> >> >> separate servers pulling items off of the application queue.
> >> >>
> >> >> On Wed, Sep 1, 2010 at 8:52 AM, Gary Tully <gary.tu...@gmail.com>
> >> wrote:
> >> >>
> >> >>> does jconsole show the same negative number? Do you have a test
> case?
> >> >>>
> >> >>> On 1 September 2010 13:52, Colleen Velo <cmv...@gmail.com> wrote:
> >> >>> > After only running my 3 ActiveMQ brokers for short time, I am
> seeing
> >> >>> > negative numbers in
> >> >>> > the "Number of Pending Messages" column. I started seeing this at
> >> 5.3.2
> >> >>> > (after upgrading
> >> >>> > from 5.2.0) and opened and issue on it:
> >> >>> > • https://issues.apache.org/activemq/browse/AMQ-2887
> >> >>> >
> >> >>> > It was recommended that I upgrade to ActiveMQ 5.4.0, which I did,
> >> >>> however, I
> >> >>> > am still
> >> >>> > seeing this issue.
> >> >>> >
> >> >>> > Current Environment:
> >> >>> > • CentOS release 5.2 (both consumers and brokers)
> >> >>> > • 3 ActiveMQ brokers
> >> >>> > • 7 consumers (across different queues)
> >> >>> >
> >> >>> >
> >> >>> > Thanks,
> >> >>> >
> >> >>> > --
> >> >>> > Colleen Velo
> >> >>> > email: cmv...@gmail.com
> >> >>> >
> >> >>> --
> >> >>> http://blog.garytully.com
> >> >>>
> >> >>> Open Source Integration
> >> >>> http://fusesource.com
> >> >>>
> >>
> >> >
> >>
> >>
> >>
> >> --
> >> http://blog.garytully.com
> >>
> >> Open Source Integration
> >> http://fusesource.com
> >>
> >
> >
> >
> > --
> > Colleen Velo
> > email: cmv...@gmail.com
> >
>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
--
Colleen Velo
email: cmv...@gmail.com
