Re: [ANNOUNCEMENT] Apache Camel 3.0.0-RC1 (Release Candidate 1) Released
Hi Knut-Håvard, thank you for noticing that, it's one of those things when you look at a it and over time it seems more right than wrong so you don't notice the wrong any more. I've pushed a fix for this[1][2], and it's now live. As a reminder to the rest of the community, now that we have the website in git/GitHub it's easy to contribute fixes or features, design changes and so on. This is a really nice and easy way to start contributing to Camel. Have a look at some `help-wanted` issues in the `website` component on JIRA[3], for example CAMEL-13818[4] for creating a design for the news section. zoran [1] https://issues.apache.org/jira/browse/CAMEL-13937 [2] https://github.com/apache/camel-website/commit/b4cf98b785669b8262bbed72cb064b549e653586 [3] https://issues.apache.org/jira/issues/?jql=project%20%3D%20CAMEL%20AND%20status%20%3D%20Open%20AND%20component%20%3D%20website%20AND%20labels%20%3D%20help-wanted [4] https://issues.apache.org/jira/browse/CAMEL-13818 On Tue, Sep 3, 2019 at 10:41 AM Knut-Håvard Aksnes wrote: > > There seems to be a bug in the way the release notes are listed on the > Apache Camel News page, the date is wrong. (In the future) I suspect month > number is shown instead of day of month. > > man. 2. sep. 2019, 19.51 skrev Gregor Zurowski : > > > The Camel community announces the immediate availability of Camel > > 3.0.0-RC1, the first release candidate towards a new 3.0.0 major > > release with 148 improvements and fixes. > > > > The artifacts are published and ready for you to download from the > > Central Maven repository. For more details please take a look at the > > release notes [1]. > > > > Many thanks to all who made this release possible. > > > > On behalf of the Camel PMC, > > Gregor Zurowski > > > > [1] > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345723&projectId=12311211 > > -- Zoran Regvart
Re: [ANNOUNCEMENT] Apache Camel 3.0.0-RC1 (Release Candidate 1) Released
Thanks a lot Zoran Il mer 4 set 2019, 10:24 Zoran Regvart ha scritto: > Hi Knut-Håvard, > thank you for noticing that, it's one of those things when you look at > a it and over time it seems more right than wrong so you don't notice > the wrong any more. > > I've pushed a fix for this[1][2], and it's now live. > > As a reminder to the rest of the community, now that we have the > website in git/GitHub it's easy to contribute fixes or features, > design changes and so on. This is a really nice and easy way to start > contributing to Camel. > > Have a look at some `help-wanted` issues in the `website` component on > JIRA[3], for example CAMEL-13818[4] for creating a design for the news > section. > > zoran > > [1] https://issues.apache.org/jira/browse/CAMEL-13937 > [2] > https://github.com/apache/camel-website/commit/b4cf98b785669b8262bbed72cb064b549e653586 > [3] > https://issues.apache.org/jira/issues/?jql=project%20%3D%20CAMEL%20AND%20status%20%3D%20Open%20AND%20component%20%3D%20website%20AND%20labels%20%3D%20help-wanted > [4] https://issues.apache.org/jira/browse/CAMEL-13818 > > On Tue, Sep 3, 2019 at 10:41 AM Knut-Håvard Aksnes > wrote: > > > > There seems to be a bug in the way the release notes are listed on the > > Apache Camel News page, the date is wrong. (In the future) I suspect > month > > number is shown instead of day of month. > > > > man. 2. sep. 2019, 19.51 skrev Gregor Zurowski >: > > > > > The Camel community announces the immediate availability of Camel > > > 3.0.0-RC1, the first release candidate towards a new 3.0.0 major > > > release with 148 improvements and fixes. > > > > > > The artifacts are published and ready for you to download from the > > > Central Maven repository. For more details please take a look at the > > > release notes [1]. > > > > > > Many thanks to all who made this release possible. > > > > > > On behalf of the Camel PMC, > > > Gregor Zurowski > > > > > > [1] > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345723&projectId=12311211 > > > > > > > -- > Zoran Regvart >
Re: JMS pub/sub (using Camel) performance issue with MQ in XA transactions
camel-jms uses spring jms, so there are many users with this combo also with IBM MQ and XA. Look at the many options spring jms has to tune its polling, and also cache levels you can tweak. There may be some more idle options you can set to make it "sleep" longer when there are no messages. But you can't avoid that spring jms is designed to keep polling for new messages also when the queue is empty. On Tue, Sep 3, 2019 at 7:04 PM Benoit Fortin wrote: > > We have a java application doing JMS subscriptions that is using Camel as > its JMS provider. > > The application is subscribing to a topic using XA. It consumes 1 message > in the queue, and then closes the XA transaction (each message is part of > an XA transaction). Then the application re-attaches itself to the topic to > start over the same process for each message. > > When there are no messages to process, the client waits for 45 seconds > (which is the XA transaction timeout) before closing the request and the XA > transaction and starting a new iteration. > > I have analyzed how this process is actually being done using a tcpdump, > and here is what I found: > > —— > > MQ Client --[XA_START]--> MQ Server > > MQ Client <--[XA_START_REPLY]-- MQ Server > > > MQ Client --[SPI (SUBSCRIBE)]--> MQ Server > > MQ Client <--[SPI REPLY]-- MQ Server > > > MQ Client --[REQUEST_MSGS]--> MQ Server > > MQ Client --[REQUEST_MSGS]--> MQ Server > > MQ Client <--[NOTIFICATION]-- MQ Server > > > MQ Client --[MQCLOSE]--> MQ Server > > MQ Client <--[MQCLOSE_REPLY]-- MQ Server > > > MQ Client --[XA_END]--> MQ Server > > MQ Client <--[XA_END_REPLY]-- MQ Server > > > MQ Client --[XA_COMMIT]--> MQ Server > > MQ Client <--[XA_COMMIT_REPLY]-- MQ Server > > —— > > This is process is on a durable subscription, so each of the iterations > (even when there is no message to consume) ends up generating I/O on the MQ > server (I think, mostly on the SYSTEM.DURABLE.SUBSCRIBER.QUEUE queue, where > MQ keeps trace of its subscribers). > > Having a somewhat high number of subscribers doing this process, the MQ > server ends up I/O bound, with +4000 IOPS on the MQ logs disks (even > outside of business hours, when there are no messages to consume). This > process also consumes ~3 CPU outside of business hours. > > I am a bit puzzled that using XA on pub/sub scales so bad, and I am > wondering if there is any way to implement this solution without doing so > much subscribe operations. -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: JMS pub/sub (using Camel) performance issue with MQ in XA transactions
The cache shouldn’t be enabled, as we are using XA (as per Camel’s recommendation). We have also tuned the polling interval. That being said, what hurts the I/O subsystem on the MQ server is the Subscribe operation itself (not the polling for messages). I am wondering is if there would be any way (inside or outside of Camel) to have the XA transaction “scoped” on the REQUEST_MSGS requests instead of the SUBSCRIBE operation? So that each polling doesn’t do a subscribe operation every time and instead stays attached to an already opened subscription? As a reminder, I am attaching below how the XA transaction and subscription operations are being done (according to a tcpdump I analyzed): MQ Client --[XA_START]--> MQ Server MQ Client <--[XA_START_REPLY]-- MQ Server MQ Client --[SPI (SUBSCRIBE)]--> MQ Server MQ Client <--[SPI REPLY]-- MQ Server MQ Client --[REQUEST_MSGS]--> MQ Server MQ Client --[REQUEST_MSGS]--> MQ Server MQ Client <--[NOTIFICATION]-- MQ Server MQ Client --[MQCLOSE]--> MQ Server MQ Client <--[MQCLOSE_REPLY]-- MQ Server MQ Client --[XA_END]--> MQ Server MQ Client <--[XA_END_REPLY]-- MQ Server MQ Client --[XA_COMMIT]--> MQ Server MQ Client <--[XA_COMMIT_REPLY]-- MQ Server On Wed, Sep 4, 2019 at 05:46 Claus Ibsen wrote: > camel-jms uses spring jms, so there are many users with this combo > also with IBM MQ and XA. > Look at the many options spring jms has to tune its polling, and also > cache levels you can tweak. > There may be some more idle options you can set to make it "sleep" > longer when there are no messages. > But you can't avoid that spring jms is designed to keep polling for > new messages also when the queue is empty. > > > > On Tue, Sep 3, 2019 at 7:04 PM Benoit Fortin > wrote: > > > > We have a java application doing JMS subscriptions that is using Camel as > > its JMS provider. > > > > The application is subscribing to a topic using XA. It consumes 1 message > > in the queue, and then closes the XA transaction (each message is part of > > an XA transaction). Then the application re-attaches itself to the topic > to > > start over the same process for each message. > > > > When there are no messages to process, the client waits for 45 seconds > > (which is the XA transaction timeout) before closing the request and the > XA > > transaction and starting a new iteration. > > > > I have analyzed how this process is actually being done using a tcpdump, > > and here is what I found: > > > > —— > > > > MQ Client --[XA_START]--> MQ Server > > > > MQ Client <--[XA_START_REPLY]-- MQ Server > > > > > > MQ Client --[SPI (SUBSCRIBE)]--> MQ Server > > > > MQ Client <--[SPI REPLY]-- MQ Server > > > > > > MQ Client --[REQUEST_MSGS]--> MQ Server > > > > MQ Client --[REQUEST_MSGS]--> MQ Server > > > > MQ Client <--[NOTIFICATION]-- MQ Server > > > > > > MQ Client --[MQCLOSE]--> MQ Server > > > > MQ Client <--[MQCLOSE_REPLY]-- MQ Server > > > > > > MQ Client --[XA_END]--> MQ Server > > > > MQ Client <--[XA_END_REPLY]-- MQ Server > > > > > > MQ Client --[XA_COMMIT]--> MQ Server > > > > MQ Client <--[XA_COMMIT_REPLY]-- MQ Server > > > > —— > > > > This is process is on a durable subscription, so each of the iterations > > (even when there is no message to consume) ends up generating I/O on the > MQ > > server (I think, mostly on the SYSTEM.DURABLE.SUBSCRIBER.QUEUE queue, > where > > MQ keeps trace of its subscribers). > > > > Having a somewhat high number of subscribers doing this process, the MQ > > server ends up I/O bound, with +4000 IOPS on the MQ logs disks (even > > outside of business hours, when there are no messages to consume). This > > process also consumes ~3 CPU outside of business hours. > > > > I am a bit puzzled that using XA on pub/sub scales so bad, and I am > > wondering if there is any way to implement this solution without doing so > > much subscribe operations. > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
