Il 08/01/2019 17:56, Gordon Sim ha scritto:
On 08/01/19 16:10, Alessandro Crotti - SMC Consulting wrote:
Hello,
I need help to understand which is the best way to handle this scenario.
*General information*:
1. I consume messages from an *Azure* queue that collects "status"
messages. Each message rappresents a passing of state for a specific
entity instance.
2. I have entity instance E1 with initial status S1
3. E1 status flow is:
1. from status S1 -> S2
2. from status S2 -> S3
3. Is not valid other kind of status passage (for example S1 -> S3)
*Standard scenario*: when I consume messages where the status flow is
valid there are no problems
* In the queue there are both messages with "E1 - S2" and "E1 - S3"
* I receive message with "E1 - S2" -> E1 status will be changed in S2
* Then I receive message with "E1 - S3" -> E1 status will be
changed in S3
* CORRECT
*Particular scenario*: if my message consumer receives unsorted
messages, I could lose information.
* In the queue there are both messages with "E1 - S2" and "E1 - S3"
* I receive message with "E1 - S3" -> NOT VALID, E1 status will still
be S1
o The message ackType is MODIFIED_FAILED because it is a trasient
error. If in next consuming attempts I have consumed message "E1
- S2" successfully, also this message can be consumed
successfully
* If I have maximum 10 retries before deadletter.
* If I would consume 10 times message "E1 - S3" before "E1 - S2" ->
Message "E1 - S3" would go in Deadletter and I would lose that
information
Basically my scenario is composed by "consecutive" messages because
they must follow a specific state diagram. Each status is important
it must be consumed before the next; it wouldn't be acceptable just
to take the latest one.
My question now is: how do AMQP and QPID handle this kind of scenario?
AMQP doesn't directly specify anything about required ordering. It
does have two standard properties - group-id and group-id-sequence -
intended for situations where you require strict ordered processing of
particular messages sequences. However there is nothing explicitly
defined. The Qpid brokers support JMS message groups, but that doesn't
help you if you need to use Azure queues (not sure what Azure offers
in this regard).
Do you have just one consumer of the queue? If so then provided the
queue guarantees ordered delivery you don't need any further grouping.
If the queueing you are using doesn't provide ordering guarantees you
could I guess try something where in your second scenario you neither
accept nor reject "E1 - S3", but hold on to it until you receive "E1 -
S2" and then process them in order and accept them both. That really
only works if you have only one consumer though (or have message
grouping semantics in which case you likely wouldn't need this
approach anyway).
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
I have more sessions/consumers, for this reason I have more parallel
consumers which consume these messages.
What do you mean with your sentence: ...neither accept nor reject "E1 -
S3", but hold on to it until you receive "E1 - S2"...
How can I hold on the message?
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
