[ https://issues.apache.org/jira/browse/QPID-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625581#comment-13625581 ]
Fraser Adams commented on QPID-4728: ------------------------------------ If this relates to what I think it does (I've never been able to articulate the problem very well!!) then this sounds *extremely* useful for increasing the reliability of federation links. I've had an issue for ages where there "appeared" to be a memory leak on some of our source brokers, but after a lot of head scratching it turned out to be an issue due to there being "unbounded buffers" on the federation links. In my scenario we run a topology whereby a real time producer writes to a co-located broker and that broker is federated to another broker etc. In most cases this works very well but in one particular set of nodes in our topology the producer is particularly high rate and quite "bursty" and what we'd see is the memory utilisation of the source broker start to increase noticeably and eventually we'd hit swap. What ultimately appeared to be causing the issue was that the network between the brokers was "throttling" the communication (I did say it was a fast bursty producer) so from the perspective of the source broker we appeared to have a fast producer with a slow consumer. As I was trying to get to the bottom of this I ended up trying to reproduce it at the lowest level I could, so wrote simple qpid::messaging and qpid::client producer consumer clients to try and "emulate" the federation bridge code and found that the qpid::client code appeared to "leak" but the "equivalent" qpid::messaging code didn't - then it dawned on me that by default qpid::messaging has consumer capacity of 1 (so I generally set it to something that gives reasonable prefetch) whereas qpid::client defaults to unlimited. Our eventual workaround was to have this particular fast bursty producer deliver directly to the second stage broker rather than to its co-located broker, the theory being that by doing this the rate limiting effect of the network actually becomes our friend, so rather than the fast data source hitting a broker pretty much as fast as it can it now hits it as fast as the network supports, which then generally avoids the unbounded buffer growing in an unmanageable way. It's a pretty subtle issue that took us ages to diagnose and caused more than a few sleepless nights, so I'm pretty much +1 million :-) on adding the ability to make the federation link capacity bounded. > Allow option to configure credit on Federation sessions. > -------------------------------------------------------- > > Key: QPID-4728 > URL: https://issues.apache.org/jira/browse/QPID-4728 > Project: Qpid > Issue Type: Improvement > Components: C++ Broker > Affects Versions: 0.23 > Reporter: Ken Giusti > Assignee: Ken Giusti > Fix For: Future > > > By default, sessions created for Federation links use unlimited credit. This > JIRA requests the ability to configure the credit used by a Federation link. > E.g.: > qpid-route --credit 100 route add .... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org