Many thanks to all of you guys for the valuable feedback. I'm going to evaluate
suggested approaches and will communicate results back here.
Have a great weekend.
-----Original Message-----
From: Ken Giusti <kgiu...@redhat.com>
Sent: woensdag 14 oktober 2020 20:05
To: users <users@qpid.apache.org>
Subject: Re: Non-blocking Sender with no Receivers
On Wed, Oct 14, 2020 at 11:05 AM Petrenko, Vadim <vadim.petre...@ns.nl>
wrote:
> Hi Ken,
>
> Yes, we're pre-settling messages already:
> connectionfactory.myFactoryLookupSender =
> amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> (dockerized Qpid with an exposed port).
>
> Making the sending client first check for available credits is a
> possibility, but not the ideal option we'd like to use. These clients
> are not in our control, it's other teams that develop their
> applications using our Qpid network. So we can't really enforce them. Only
> advise.
>
> I also noticed some strange behavior which I don't understand. Maybe
> you guys can explain.
> I've taken the Sender and Receiver from qpid-jms-examples, connected
> them to our router network and observed how this works.
> 1. When I start the Sender, it gets blocked because there is no
> receiver (in the logs I see "Holding Message send until credit is
> available"). This is as expected.
> 2. I start the Receiver, messages begin to flow, all good.
> 3. I kill the Receiver process, but the Sender is still sending messages.
> To nowhere. It doesn't timeout either.
>
> I then proceeded debugging AmqpFixedProducer, which works with
> credits, and noticed that on step 1 number of credits it gets via
> getEndpoint().getCredit() is 0. So it holds the message.
> On the next step Receiver gives it 250 credits. After that, even
> though the Receiver process is killed, the number of credits is still
> 250 and doesn't get decremented. The Sender continues to send. I
> expect that credits would be exhausted at some point.
> Is this as it should be?
>
If the sender's link is anonymous as Robbie described then yes credit will be
replenished regardless of the presence of your receiver.
If the sender's link is not anonymous I would expect that 250 outstanding
credits will be exhausted eventually. Once the router learns of the loss of all
receivers it should no longer replenish credit to the sender.
The router doesn't have a way to revoke credit btw - any extra credit
outstanding at the sender when the receivers go away will remain until the
sender either uses them or disconnects.
The 250 value is configurable - use the "linkCapacity" attribute on the
"listener" configuration entity. For example this changes the maximum
outstanding credit limit from 250 (default) down to 10 :
listener {
host: 0.0.0.0
port: amqp
authenticatePeer: no
saslMechanisms: ANONYMOUS
linkCapacity: 10
}
>
> Thanks.
>
>
> -----Original Message-----
> From: Ken Giusti <kgiu...@redhat.com>
> Sent: woensdag 14 oktober 2020 15:18
> To: users <users@qpid.apache.org>
> Subject: Re: Non-blocking Sender with no Receivers
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <vadim.petre...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a
> > way that a sender can connect to it at any moment and start sending
> > messages to a topic.
> > There might be no receivers at this moment, so the messages would be
> > just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving
> > messages currently being sent, then drop off. The sender will just
> > continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living
> > and there is no need to buffer it or persist. If no one consumes
> > this data in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they
> > appear and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure
> > that producers can only send messages to a router if at least one
> > consumer is available to receive them. Because Dispatch Router does
> > not store messages, this credit-based flow control prevents
> > producers from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to
> > implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on
> the link and if there is no credit simply drop the message rather than
> sending it?
> Getting the value of available credit depends on the client API you
> are using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being
> dropped you'll want the sender to settle the message after it is sent
> (a.k.a. "pre-settled).
>
>
>
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd
> > voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > vertrouwelijke informatie bevatten. Openbaarmaking,
> > vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud
> > van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet
> > toegestaan.
> > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> >
>
>
> --
> -K
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> additional commands, e-mail: users-h...@qpid.apache.org
>
>
--
-K
________________________________
Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik
door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie
bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking
van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is
uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent,
wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de
hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
