Thanks Gary. I think we switched to vmQueueCursor at one point several weeks ago as an attempt to get better throughput...it was a desperate attempt that ended up working (made things faster), and so we kept vmQueueCursor enabled. I now understand that memory is the limiting factor in this config.
That being said, it might be helpful if activemq logged a warning about this. i.e. "You turned producerFlowControl off but we're overriding your setting in order not to blow up....and here's why..." At least something in activemq log output would be extremely helpful in identifying the fact that my setting was overridden. Anyway, just a suggestion. Thanks again for the reply! Dan On Fri, Nov 5, 2010 at 1:59 PM, Gary Tully [via ActiveMQ] < ml-node+3029106-1912193586-140...@n4.nabble.com<ml-node%2b3029106-1912193586-140...@n4.nabble.com> > wrote: > The default behavior for the broker is to block a send pending > resources b/c exceeding resource limits could blow the VM with OOM, > producer flow control is about blocking producers before the client > does a send, rather than blocking on the broker, b/c blocking on the > broker will block the connection, not just the producer. > > So you have a memory limit, a vm message cursor, which will hold all > messages in memory and you have disabled producer flow control... so > when the memory limit is reached as the queue depth increased, the > send blocks on the broker till some memory is released by a consumer > taking some messages. So you see a stop start effect on your producer. > > The problem is the vmQueueCursor, this keeps all messages in memory > and will consume all of your destination memory limit. > > If you revert to the default store cursor <storeCursor/>, it will > cache messages in memory up to some portion of the queue limit and > leave the rest in the store. > So when you have fast consumers they can be served from the cache and > when you have slow consumers and exceed the cache, the messages will > get paged in from the store on demand. > > More detail on cursors at http://activemq.apache.org/message-cursors.html > > > On 5 November 2010 17:30, dcheckoway <[hidden > email]<http://user/SendEmail.jtp?type=node&node=3029106&i=0>> > wrote: > > > > > 5.3.2, 5.4.1, and 5.5-SNAPSHOT -- doesn't matter which version, but I'm > > currently running 5.5-SNAPSHOT. It's running with the stock persistence > > enabled and the stock -Xmx memory settings. > > > > PROBLEM: as consumers slow down, producers slow down -- despite > > producerFlowControl explicitly being disabled. > > > > I'm having trouble understanding what AUTHORITATIVELY enables/disables > > producer flow control. This page > > (http://activemq.apache.org/producer-flow-control.html) talks about > > conditions that would cause flow control to happen. It's extremely vague > > > about memoryLimit, systemUsage, etc., but it implies that those are > factored > > in. But it also says you can disable flow control...which I have done. > > What's authoritative? Why is ActiveMQ doing flow control when I asked it > > > not to? > > > > Here's a snippet from my activemq.xml: > > > > <policyEntry queue=">" producerFlowControl="false" > > memoryLimit="20mb"> > > <pendingQueuePolicy> > > <vmQueueCursor/> > > </pendingQueuePolicy> > > </policyEntry> > > > > I'm doing sync sends to my queue (at least I believe I am, by virtue of > NOT > > doing explicitly async sends). As consumers slow down, the producers get > > > blocked. Can somebody please tell me why producer flow control is still > > kicking in? If the policyEntry->producerFlowControl setting is not > > AUTHORITATIVE, maybe somebody can tell me what is? > > > > Is the vmQueueCursor conflicting with producerFlowControl="false" in some > > > way? > > > > Are the default systemUsage/memoryLimit settings conflicting or causing > > producer flow control to kick in? Is the memoryLimit="20mb" on my queue > > conflicting? > > > > The biggest piece that I don't understand is: (a) IF a queue grows in > size > > beyond some limit (which is very hazy to me) it allows producer flow > control > > to kick in, and (b) if that's the case, how is that limit derived? The > > combination of systemUsage/memoryLimit settings have always been > extremely > > elusive to me. > > > > I really need help understanding how I can, WITHOUT QUESTION, turn > producer > > flow control OFF COMPLETELY. > > > > Thanks! > > > > ----- > > Dan Checkoway > > dcheckoway gmail com > > -- > > View this message in context: > http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029064.html<http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029064.html?by-user=t> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > > > > > -- > http://blog.garytully.com <http://blog.garytully.com?by-user=t> > http://fusesource.com <http://fusesource.com?by-user=t> > > > ------------------------------ > View message @ > http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029106.html > To unsubscribe from Producer Flow Control - what is AUTHORITATIVE?, click > here<http://activemq.2283324.n4.nabble.com/template/TplServlet.jtp?tpl=unsubscribe_by_code&node=3029064&code=ZGNoZWNrb3dheUBnbWFpbC5jb218MzAyOTA2NHw3MDc4NzEwMTU=>. > > > ----- Dan Checkoway dcheckoway gmail com -- View this message in context: http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029126.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
