On 09/18/2014 12:31 PM, Flavio Percoco wrote:
On 09/17/2014 10:36 PM, Joe Gordon wrote:
My understanding of Zaqar is that it's like SQS. SQS uses distributed
queues, which have a few unusual properties [0]:
Message Order
Amazon SQS makes a best effort to preserve order in messages, but due to
the distributed nature of the queue, we cannot guarantee you will
receive messages in the exact order you sent them. If your system
requires that order be preserved, we recommend you place sequencing
information in each message so you can reorder the messages upon receipt.
Zaqar guarantees FIFO. To be more precise, it does that relying on the
storage backend ability to do so as well. Depending on the storage used,
guaranteeing FIFO may have some performance penalties.
Would it be accurate to say that at present Zaqar does not use
distributed queues, but holds all queue data in a storage mechanism of
some form which may internally distribute that data among servers but
provides Zaqar with a consistent data model of some form?
[...]
As of now, Zaqar fully relies on the storage replication/clustering
capabilities to provide data consistency, availability and fault
tolerance.
Is the replication synchronous or asynchronous with respect to client
calls? E.g. will the response to a post of messages be returned only
once the replication of those messages is confirmed? Likewise when
deleting a message, is the response only returned when replicas of the
message are deleted?
However, as far as consuming messages is concerned, it can
guarantee once-and-only-once and/or at-least-once delivery depending on
the message pattern used to consume messages. Using pop or claims
guarantees the former whereas streaming messages out of Zaqar guarantees
the later.
From what I can see, pop provides unreliable delivery (i.e. its similar
to no-ack). If the delete call using pop fails while sending back the
response, the messages are removed but didn't get to the client.
What do you mean by 'streaming messages'?
[...]
Based on our short conversation on IRC last night, I understand you're
concerned that FIFO may result in performance issues. That's a valid
concern and I think the right answer is that it depends on the storage.
If the storage has a built-in FIFO guarantee then there's nothing Zaqar
needs to do there. In the other hand, if the storage does not have a
built-in support for FIFO, Zaqar will cover it in the driver
implementation. In the mongodb driver, each message has a marker that is
used to guarantee FIFO.
That marker is a sequence number of some kind that is used to provide
ordering to queries? Is it generated by the database itself?
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
