Excerpts from Flavio Percoco's message of 2014-09-04 02:11:15 -0700: > Hey Clint, > > Thanks for reading, some comments in-line: > > On 09/04/2014 10:30 AM, Clint Byrum wrote: > > Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700: > > [snip] > > >> - Concern on should we really reinvent a queue system rather than > >> piggyback on one > >> > >> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message > >> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack > >> flavor on top. [0] > >> > > > > I think Zaqar is more like SMTP and IMAP than AMQP. You're not really > > trying to connect two processes in real time. You're trying to do fully > > asynchronous messaging with fully randomized access to any message. > > > > Perhaps somebody should explore whether the approaches taken by large > > scale IMAP providers could be applied to Zaqar. > > > > Anyway, I can't imagine writing a system to intentionally use the > > semantics of IMAP and SMTP. I'd be very interested in seeing actual use > > cases for it, apologies if those have been posted before. > > > >> Some things that differentiate Zaqar from SQS is it's capability for > >> supporting different protocols without sacrificing multi-tenantcy and > >> other intrinsic features it provides. Some protocols you may consider > >> for Zaqar are: STOMP, MQTT. > >> > >> As far as the backend goes, Zaqar is not re-inventing it either. It sits > >> on top of existing storage technologies that have proven to be fast and > >> reliable for this task. The choice of using NoSQL technologies has a lot > >> to do with this particular thing and the fact that Zaqar needs a storage > >> capable of scaling, replicating and good support for failover. > >> > > > > What's odd to me is that other systems like Cassandra and Riak are not > > being discussed. There are well documented large scale message storage > > systems on both, and neither is encumbered by the same licensing FUD > > as MongoDB. > > FWIW, they both have been discussed. As far as Cassandra goes, we raised > the red flag after reading reading this post[0]. The post itself may be > obsolete already but I don't think I have enough knowledge about > Cassandra to actually figure this out. Some folks have come to us asking > for a Cassandra driver and they were interested in contributing/working > on one. I really hope that will happen someday, although it'll certainly > happen as an external driver. Riak, on the other hand, was certainly a > good candidate. What made us go with MongoDB and Redis is they're both > good for the job, they are both likely already deployed in OpenStack > clouds and we have enough knowledge to provide support and maintenance > for both drivers.
It seems like Cassandra is good for when you're going to be writing all the time but only reading once. I would agree that this makes it less attractive for a generalized messaging platform, since you won't know how users will consume the messages, and if they are constantly reading then you'll have terrible performance. > >> # Use Cases > >> > >> In addition to the aforementioned concerns and comments, I also would > >> like to share an etherpad that contains some use cases that other > >> integrated projects have for Zaqar[0]. The list is not exhaustive and > >> it'll contain more information before the next meeting. > >> > >> [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases > >> > > > > Just taking a look, there are two basic applications needed: > > > > 1) An inbox. Horizon wants to know when snapshots are done. Heat wants > > to know what happened during a stack action. Etc. > > > > 2) A user-focused message queue. Heat wants to push data to agents. > > Swift wants to synchronize processes when things happen. > > > > To me, #1 is Zaqar as it is today. #2 is the one that I worry may not > > be served best by bending #1 onto it. > > Push semantics are being developed. We've had enough discussions that > have helped preparing the ground for it. However, I believe both use > cases could be covered by Zaqar as-is. > > Could you elaborate a bit more on #2? Especially on why you think Zaqar > as is can't serve this specific case? The difference between 1 and 2 is that 2 is a true queueing problem. The message should go away when it has been consumed, and the volume may be rather high. With 1, you have a storage problem, and a database makes a lot more sense. If users can stick to type 2 problems, they'll be able to stay much more lightweight because they won't need a large data store that supports random access. _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev