Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-30 Thread Kevin Burton
> > > > I think there's a limit to how many redelivery attempts you're willing to > take before to send the message to the DLQ, which I think would cover most > scenarios when that would happen in the wild. (You could always construct > an arbitrarily bad failure case, but the odds of actually see

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-30 Thread Tim Bain
On Fri, Oct 30, 2015 at 6:29 PM, Kevin Burton wrote: > sorry for the delay in reply. was dealing with a family issue that I > needed to prioritize... > > On Wed, Oct 21, 2015 at 6:52 AM, Tim Bain wrote: > > > Right off the top, can't you use INDIVIDUAL_ACK here, rather than > > committing trans

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-30 Thread Kevin Burton
> buffer filled first, starving out other workers that do nothing... >> >> This leads to massive starvation where some of my boxes are at 100% CPU >> and >> others are at 10-20% starved for work. >> >> So I'm working on a new design where by I use a listener

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-30 Thread Kevin Burton
sorry for the delay in reply. was dealing with a family issue that I needed to prioritize... On Wed, Oct 21, 2015 at 6:52 AM, Tim Bain wrote: > Right off the top, can't you use INDIVIDUAL_ACK here, rather than > committing transactions? That seems like the ideal mode to let you choose > which

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-22 Thread Tim Bain
t;> others are at 10-20% starved for work. >> >> So I'm working on a new design where by I use a listener, then I allow it >> to prefetch and I use a countdown latch from within the message listener >> to >> wait for the thread to process the message. The

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-22 Thread Martin Lichtin
rs are at 10-20% starved for work. So I'm working on a new design where by I use a listener, then I allow it to prefetch and I use a countdown latch from within the message listener to wait for the thread to process the message. Then I commit the message. This solves the over-prefetch problem beca

Re: Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-21 Thread Tim Bain
here some of my boxes are at 100% CPU and > others are at 10-20% starved for work. > > So I'm working on a new design where by I use a listener, then I allow it > to prefetch and I use a countdown latch from within the message listener to > wait for the thread to process the mess

Dealing with the "over-prefetch" problem with large numbers of workers and many queue servers

2015-10-19 Thread Kevin Burton
I allow it to prefetch and I use a countdown latch from within the message listener to wait for the thread to process the message. Then I commit the message. This solves the over-prefetch problem because we don't attempt to pre-fetch until the message is processed. Since I can't commit

Re: Messages on queue, but clients not picking them up (prefetch problem)

2013-10-09 Thread mrich
this message in context: http://activemq.2283324.n4.nabble.com/Messages-on-queue-but-clients-not-picking-them-up-prefetch-problem-tp4672384p4672442.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Messages on queue, but clients not picking them up (prefetch problem)

2013-10-08 Thread Graham Leggett
On 08 Oct 2013, at 6:51 PM, Graham Leggett wrote: > I am using qpid v0.26-SNAPSHOT (AKA trunk) to work around another qpid bug. > > Is it possible to set the prefetch policy on the activemq server, or is this > only possible on the client side? Trying the following URL shows some even more str

Re: Messages on queue, but clients not picking them up (prefetch problem)

2013-10-08 Thread Graham Leggett
On 08 Oct 2013, at 5:48 PM, mrich wrote: > You are using ActiveMQ with AMPQ (1.0). Which client library are you using > for this? > > I recently ran into an issue with the Qpid 1.0 client as it doesnt appear to > handle the same URL parameters as earlier versions of the client (e.g. amqp > 0.10)

Re: Messages on queue, but clients not picking them up (prefetch problem)

2013-10-08 Thread mrich
library for your client connections. -- View this message in context: http://activemq.2283324.n4.nabble.com/Messages-on-queue-but-clients-not-picking-them-up-prefetch-problem-tp4672384p4672386.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Messages on queue, but clients not picking them up (prefetch problem)

2013-10-08 Thread Graham Leggett
Hi all, I am having a further problem that has me stumped. I have 8 nodes, each node makes a short lived connection to poll a central activemq server in the expectation of picking up exactly one message for processing, processing may take many seconds to minutes. The activemq server has a queu

Re: Prefetch problem/misuderstanding.

2012-01-11 Thread John-MQ
Still having this same issue after testing a few more things I found on these forums... If anyone has any suggestions at all it would be greatly appreciated -- View this message in context: http://activemq.2283324.n4.nabble.com/Prefetch-problem-misuderstanding-tp4222551p4285065.html Sent from

Re: Prefetch problem/misuderstanding.

2011-12-21 Thread John-MQ
the connection.createSession("true" in the consumer should have been false, but the problem remains. -- View this message in context: http://activemq.2283324.n4.nabble.com/Prefetch-problem-misuderstanding-tp4222551p4222555.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Prefetch problem/misuderstanding.

2011-12-21 Thread John-MQ
extMessage.getText()); } catch (JMSException e) { e.printStackTrace(); } } }); } catch (JMSException e1) { e1.printStackTrace(); } -- View this message in context: http://activemq.2283324.n4.nabble.com/

Re: Prefetch problem

2010-11-05 Thread MARollins
...even better. Thanks Gary -- View this message in context: http://activemq.2283324.n4.nabble.com/Prefetch-problem-tp3028494p3028879.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Prefetch problem

2010-11-05 Thread Gary Tully
: > > ...looks like I've just solved it!! > > The prefetch should be set on the connection as follows... > > tcp://SERVERNAME:61616?wireFormat.maxInactivityDuration=0&jms.prefetchPolicy.all=1 > > > Thanks!! > -- > View this message in context: > http://ac

Re: Prefetch problem

2010-11-05 Thread MARollins
...looks like I've just solved it!! The prefetch should be set on the connection as follows... tcp://SERVERNAME:61616?wireFormat.maxInactivityDuration=0&jms.prefetchPolicy.all=1 Thanks!! -- View this message in context: http://activemq.2283324.n4.nabble.com/Prefetc

Prefetch problem

2010-11-05 Thread MARollins
uns out of memory. Any help would be appreciated. Thanks -- View this message in context: http://activemq.2283324.n4.nabble.com/Prefetch-problem-tp3028494p3028494.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.