The default behavior for the broker is to block a send pending resources b/c exceeding resource limits could blow the VM with OOM, producer flow control is about blocking producers before the client does a send, rather than blocking on the broker, b/c blocking on the broker will block the connection, not just the producer.
So you have a memory limit, a vm message cursor, which will hold all messages in memory and you have disabled producer flow control... so when the memory limit is reached as the queue depth increased, the send blocks on the broker till some memory is released by a consumer taking some messages. So you see a stop start effect on your producer. The problem is the vmQueueCursor, this keeps all messages in memory and will consume all of your destination memory limit. If you revert to the default store cursor <storeCursor/>, it will cache messages in memory up to some portion of the queue limit and leave the rest in the store. So when you have fast consumers they can be served from the cache and when you have slow consumers and exceed the cache, the messages will get paged in from the store on demand. More detail on cursors at http://activemq.apache.org/message-cursors.html On 5 November 2010 17:30, dcheckoway <dchecko...@gmail.com> wrote: > > 5.3.2, 5.4.1, and 5.5-SNAPSHOT -- doesn't matter which version, but I'm > currently running 5.5-SNAPSHOT. It's running with the stock persistence > enabled and the stock -Xmx memory settings. > > PROBLEM: as consumers slow down, producers slow down -- despite > producerFlowControl explicitly being disabled. > > I'm having trouble understanding what AUTHORITATIVELY enables/disables > producer flow control. This page > (http://activemq.apache.org/producer-flow-control.html) talks about > conditions that would cause flow control to happen. It's extremely vague > about memoryLimit, systemUsage, etc., but it implies that those are factored > in. But it also says you can disable flow control...which I have done. > What's authoritative? Why is ActiveMQ doing flow control when I asked it > not to? > > Here's a snippet from my activemq.xml: > > <policyEntry queue=">" producerFlowControl="false" > memoryLimit="20mb"> > <pendingQueuePolicy> > <vmQueueCursor/> > </pendingQueuePolicy> > </policyEntry> > > I'm doing sync sends to my queue (at least I believe I am, by virtue of NOT > doing explicitly async sends). As consumers slow down, the producers get > blocked. Can somebody please tell me why producer flow control is still > kicking in? If the policyEntry->producerFlowControl setting is not > AUTHORITATIVE, maybe somebody can tell me what is? > > Is the vmQueueCursor conflicting with producerFlowControl="false" in some > way? > > Are the default systemUsage/memoryLimit settings conflicting or causing > producer flow control to kick in? Is the memoryLimit="20mb" on my queue > conflicting? > > The biggest piece that I don't understand is: (a) IF a queue grows in size > beyond some limit (which is very hazy to me) it allows producer flow control > to kick in, and (b) if that's the case, how is that limit derived? The > combination of systemUsage/memoryLimit settings have always been extremely > elusive to me. > > I really need help understanding how I can, WITHOUT QUESTION, turn producer > flow control OFF COMPLETELY. > > Thanks! > > ----- > Dan Checkoway > dcheckoway gmail com > -- > View this message in context: > http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029064.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > -- http://blog.garytully.com http://fusesource.com
