Gary, One more question...since I had configured memoryLimit="20mb" in the ">" catch-all for queues, is 20mb the limiting factor for vmQueueCursor size, or does some other systemUsage/memoryLimit setting control the vmQueueCursor size limit?
Thanks, Dan On Fri, Nov 5, 2010 at 2:13 PM, Dan Checkoway <dchecko...@gmail.com> wrote: > Thanks Gary. I think we switched to vmQueueCursor at one point several > weeks ago as an attempt to get better throughput...it was a desperate > attempt that ended up working (made things faster), and so we kept > vmQueueCursor enabled. I now understand that memory is the limiting factor > in this config. > > That being said, it might be helpful if activemq logged a warning about > this. i.e. "You turned producerFlowControl off but we're overriding your > setting in order not to blow up....and here's why..." At least something in > activemq log output would be extremely helpful in identifying the fact that > my setting was overridden. Anyway, just a suggestion. > > Thanks again for the reply! > > Dan > > > > On Fri, Nov 5, 2010 at 1:59 PM, Gary Tully [via ActiveMQ] < > ml-node+3029106-1912193586-140...@n4.nabble.com<ml-node%2b3029106-1912193586-140...@n4.nabble.com> > > wrote: > >> The default behavior for the broker is to block a send pending >> resources b/c exceeding resource limits could blow the VM with OOM, >> producer flow control is about blocking producers before the client >> does a send, rather than blocking on the broker, b/c blocking on the >> broker will block the connection, not just the producer. >> >> So you have a memory limit, a vm message cursor, which will hold all >> messages in memory and you have disabled producer flow control... so >> when the memory limit is reached as the queue depth increased, the >> send blocks on the broker till some memory is released by a consumer >> taking some messages. So you see a stop start effect on your producer. >> >> The problem is the vmQueueCursor, this keeps all messages in memory >> and will consume all of your destination memory limit. >> >> If you revert to the default store cursor <storeCursor/>, it will >> cache messages in memory up to some portion of the queue limit and >> leave the rest in the store. >> So when you have fast consumers they can be served from the cache and >> when you have slow consumers and exceed the cache, the messages will >> get paged in from the store on demand. >> >> More detail on cursors at http://activemq.apache.org/message-cursors.html >> >> >> On 5 November 2010 17:30, dcheckoway <[hidden >> email]<http://user/SendEmail.jtp?type=node&node=3029106&i=0>> >> wrote: >> >> > >> > 5.3.2, 5.4.1, and 5.5-SNAPSHOT -- doesn't matter which version, but I'm >> > currently running 5.5-SNAPSHOT. It's running with the stock persistence >> >> > enabled and the stock -Xmx memory settings. >> > >> > PROBLEM: as consumers slow down, producers slow down -- despite >> > producerFlowControl explicitly being disabled. >> > >> > I'm having trouble understanding what AUTHORITATIVELY enables/disables >> > producer flow control. This page >> > (http://activemq.apache.org/producer-flow-control.html) talks about >> > conditions that would cause flow control to happen. It's extremely >> vague >> > about memoryLimit, systemUsage, etc., but it implies that those are >> factored >> > in. But it also says you can disable flow control...which I have done. >> > What's authoritative? Why is ActiveMQ doing flow control when I asked >> it >> > not to? >> > >> > Here's a snippet from my activemq.xml: >> > >> > <policyEntry queue=">" producerFlowControl="false" >> > memoryLimit="20mb"> >> > <pendingQueuePolicy> >> > <vmQueueCursor/> >> > </pendingQueuePolicy> >> > </policyEntry> >> > >> > I'm doing sync sends to my queue (at least I believe I am, by virtue of >> NOT >> > doing explicitly async sends). As consumers slow down, the producers >> get >> > blocked. Can somebody please tell me why producer flow control is still >> >> > kicking in? If the policyEntry->producerFlowControl setting is not >> > AUTHORITATIVE, maybe somebody can tell me what is? >> > >> > Is the vmQueueCursor conflicting with producerFlowControl="false" in >> some >> > way? >> > >> > Are the default systemUsage/memoryLimit settings conflicting or causing >> > producer flow control to kick in? Is the memoryLimit="20mb" on my queue >> >> > conflicting? >> > >> > The biggest piece that I don't understand is: (a) IF a queue grows in >> size >> > beyond some limit (which is very hazy to me) it allows producer flow >> control >> > to kick in, and (b) if that's the case, how is that limit derived? The >> > combination of systemUsage/memoryLimit settings have always been >> extremely >> > elusive to me. >> > >> > I really need help understanding how I can, WITHOUT QUESTION, turn >> producer >> > flow control OFF COMPLETELY. >> > >> > Thanks! >> > >> > ----- >> > Dan Checkoway >> > dcheckoway gmail com >> > -- >> > View this message in context: >> http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029064.html<http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029064.html?by-user=t> >> > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> > >> >> >> >> -- >> http://blog.garytully.com <http://blog.garytully.com?by-user=t> >> http://fusesource.com <http://fusesource.com?by-user=t> >> >> >> ------------------------------ >> View message @ >> http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029106.html >> To unsubscribe from Producer Flow Control - what is AUTHORITATIVE?, click >> here<http://activemq.2283324.n4.nabble.com/template/TplServlet.jtp?tpl=unsubscribe_by_code&node=3029064&code=ZGNoZWNrb3dheUBnbWFpbC5jb218MzAyOTA2NHw3MDc4NzEwMTU=>. >> >> >> > ----- Dan Checkoway dcheckoway gmail com -- View this message in context: http://activemq.2283324.n4.nabble.com/Producer-Flow-Control-what-is-AUTHORITATIVE-tp3029064p3029134.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
