On 08/01/19 16:10, Alessandro Crotti - SMC Consulting wrote:
Hello,
I need help to understand which is the best way to handle this scenario.
*General information*:
1. I consume messages from an *Azure* queue that collects "status"
messages. Each message rappresents a passing of state for a specific
entity instance.
2. I have entity instance E1 with initial status S1
3. E1 status flow is:
1. from status S1 -> S2
2. from status S2 -> S3
3. Is not valid other kind of status passage (for example S1 -> S3)
*Standard scenario*: when I consume messages where the status flow is
valid there are no problems
* In the queue there are both messages with "E1 - S2" and "E1 - S3"
* I receive message with "E1 - S2" -> E1 status will be changed in S2
* Then I receive message with "E1 - S3" -> E1 status will be changed in S3
* CORRECT
*Particular scenario*: if my message consumer receives unsorted
messages, I could lose information.
* In the queue there are both messages with "E1 - S2" and "E1 - S3"
* I receive message with "E1 - S3" -> NOT VALID, E1 status will still
be S1
o The message ackType is MODIFIED_FAILED because it is a trasient
error. If in next consuming attempts I have consumed message "E1
- S2" successfully, also this message can be consumed successfully
* If I have maximum 10 retries before deadletter.
* If I would consume 10 times message "E1 - S3" before "E1 - S2" ->
Message "E1 - S3" would go in Deadletter and I would lose that
information
Basically my scenario is composed by "consecutive" messages because they
must follow a specific state diagram. Each status is important it must
be consumed before the next; it wouldn't be acceptable just to take the
latest one.
My question now is: how do AMQP and QPID handle this kind of scenario?
AMQP doesn't directly specify anything about required ordering. It does
have two standard properties - group-id and group-id-sequence - intended
for situations where you require strict ordered processing of particular
messages sequences. However there is nothing explicitly defined. The
Qpid brokers support JMS message groups, but that doesn't help you if
you need to use Azure queues (not sure what Azure offers in this regard).
Do you have just one consumer of the queue? If so then provided the
queue guarantees ordered delivery you don't need any further grouping.
If the queueing you are using doesn't provide ordering guarantees you
could I guess try something where in your second scenario you neither
accept nor reject "E1 - S3", but hold on to it until you receive "E1 -
S2" and then process them in order and accept them both. That really
only works if you have only one consumer though (or have message
grouping semantics in which case you likely wouldn't need this approach
anyway).
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
