Hi Larry,
Thank you for your message! Your reply makes sense to me and I've tried
what you suggested to test it with a queue like:
from("aws2-sqs://aramark-notifications?maxMessagesPerPoll=1&concurrentConsumers=5&amazonSQSClient=#sqsClient")
but it does not change, I still see the messages with a delay of 5 seconds
between each of them.
If I turn on the trace with
(logging.level.org.apache.camel.component.aws2.sqs=TRACE). I see that the
next polling is only triggered after
the Delete message of a consumed message is sent. And in the logs I see "1
message received" but no polling will occur until the Delete message is
sent which occurs only after the sleep of 5 secs. Only 1 message is read
per polling and I should see at least 5 of them since concurrentConsumers=5.
It looks to me there is still only 1 concurrent consumer and I don't
understand why.
Arnaud
On Mon, Jan 31, 2022 at 3:27 PM Larry Shields <larry.shie...@gmail.com>
wrote:
> Hi Arnaud,
>
> I think what may be happening is that you first consumer is grabbing all of
> the messages from the queue that are available. The default message poll
> size is unlimited so your other 4 consumers are polling for nothing on the
> queue. What you might want to try is setting the maxMessagesPerPoll to a
> value. This will reduce the number of messages a consumer retrieves for
> each poll.
>
> The other thing I have done is set Greedy=true when using concurrent
> consumers if you're looking to increase throughput. This will cause the
> consumer to make another poll immediately without waiting for the next
> delay if the previous poll returned at least 1 message.
>
> Example:
>
> aws-sqs://my-queue?greedy=true&maxMessagesPerPoll=10&concurrentConsumers=5
>
> So in this example, each consumer will pull no more than 10 messages per
> poll to the SQS Queue. So if there are less than 10 messages on your
> queue, the other consumers aren't going to really get an messages.
>
> On Mon, Jan 31, 2022 at 7:15 AM Arnaud Level <arn...@toro-intl.com> wrote:
>
> > Hi Narsi,
> > It's a fifo but I actually tried the simple code I've posted on a
> standard
> > one too and observed the same behavior.
> >
> > On Mon, Jan 31, 2022 at 1:11 PM Narsi Reddy Nallamilli <
> > narsi.nallami...@gmail.com> wrote:
> >
> > > Hi Arnaud,
> > >
> > > Is your AWS queue type fifo or standard?
> > >
> > > On Mon, 31 Jan, 2022, 17:31 Arnaud Level, <arn...@toro-intl.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > (Camel version: camel-aws2-sqs-starter: 3.12.0)
> > > >
> > > > I am trying to use and understand concurentConsumers with a SQS
> queue:
> > > >
> > > >
> > >
> >
> from("aws2-sqs://queuexxx?concurrentConsumers=5&amazonSQSClient=#sqsClient&
> > > > waitTimeSeconds=20")
> > > > .process(exchange -> {
> > > > System.out.println("Message received...");
> > > > })
> > > > .process(exchange -> {
> > > > try {
> > > > Thread.sleep(5000);
> > > > } catch (InterruptedException e) {
> > > > e.printStackTrace();
> > > > }});
> > > >
> > > > With the above queue, if I send 3 messages at the same time, I have
> to
> > > wait
> > > > 5 seconds to see the second message ("Message received...") and 5
> more
> > > > seconds to see the third one. My understanding of concurentConsumers
> is
> > > > that with a value of 5 I would see the 3 messages at the same time
> > since
> > > 3
> > > > threads will consume them in parallel. If I add the Thread.sleep in a
> > > seda
> > > > route, I'm having this behavior (= The 3 messages are read at the
> same
> > > > time).
> > > >
> > > > Turning on the Camel logs it seems that the next polling is done only
> > > after
> > > > the Delete for the previous message is sent (which is with a delay of
> > > 5s).
> > > >
> > > > I would understand the above behavior with concurentConsumers=1 but I
> > > don't
> > > > with concurentConsumers=5. Could someone tell me what I've
> > misunderstood
> > > ?
> > > >
> > > > Thank you in advance!
> > > > Arnaud
> > > >
> > >
> >
>
--
Arnaud Level
Head of Project Delivery
TORO DEVELOPMENT LTD.
Skype: *toro_arnaud*
T: *+34 636 927 162*
M: *+886 920 565 531*
Còrsega 371, 3a planta 08037 Barcelona, Spain
www.toro-intl.com
------------------------------
Legal warning:
This message and its attachments are intended exclusively for its addressee
and may contain confidential information subject to professional secrecy.
Reproduction or distribution without the express permission of TORO
DEVELOPMENT SL is not allowed. If you believe that it has been sent to you
in error, please reply to the sender that you received the message in error.
According to the Data Protection Act, we inform you that your personal
information and email are part of a file owned by TORO DEVELOPMENT SL, the
purpose of the file being, commercial management and sending commercial
communications about our products and/or services. If you wish, you may
exercise rights of access, rectification, cancellation and opposition of
your data by sending a message to the following email address
*cont...@toro-eu.com
<cont...@toro-eu.com>*, indicating the "Subject" as the right you want to
exercise.
Aviso legal:
Este mensaje y sus archivos adjuntos van dirigidos exclusivamente a su
destinatario, pudiendo contener información confidencial sometida a secreto
profesional. No está permitida su reproducción o distribución sin la
autorización expresa de TORO DEVELOPMENT S.L.. Si usted no es el
destinatario final por favor elimínelo e infórmenos por esta vía. De
acuerdo con la LOPD, le informamos que sus datos personales y dirección de
correo electrónico forman parte de un fichero, cuyo responsable es TORO
DEVELOPMENT S.L., siendo la finalidad del fichero, la gestión de carácter
comercial y el envío de comunicaciones comerciales sobre nuestros productos
y/o servicios. Si lo desea, podrá usted ejercitar los derechos de acceso,
rectificación, cancelación y oposición de sus datos enviando un mensaje a
la siguiente dirección de correo electrónico a *cont...@toro-eu.com
<cont...@toro-eu.com>*, indicando en la línea de “Asunto” el derecho que
desea ejercitar.
