Excerpts from Flavio Percoco's message of 2014-03-19 03:01:19 -0700: > FWIW, I think there's a value on having an sqlalchemy driver. It's > helpful for newcomers, it integrates perfectly with the gate and I > don't want to impose other folks what they should or shouldn't use in > production. Marconi may be providing a data API but it's still > non-opinionated and it wants to support other drivers - or at least provide > a nice way to implement them. Working on sqlalchemy instead of amqp (or > redis) was decided in the incubation meeting. > > But again, It's an optional driver that we're talking about here. As > of now, our recommended driver is mongodb's and as I already mentioned > in this email, we'll start working on an amqp's one, which will likely > become the recommended one. There's also support for redis. > > As already mentioned, we have plans to complete the redis driver and > write an amqp based one and let them both live in the code base. > Having support for different storage dirvers makes marconi's sharding > feature more valuable. > >
Just to steer this back to technical development discussions a bit: I suggest the sqla driver be removed. It will never be useful as a queue backend. It will confuse newcomers because they'll see the schema and think that it will work and then use it, and then they find out that SQL is just not suitable for queueing about the time that they're taking a fire extinguisher to their rack. "Just use Redis" is pretty interesting as a counter to the concerns MongoDB's license situation. Redis, AFAIK, does not have many of the features that make MongoDB attractive for backing a queue. The primary one that I would cite is sharding. While MongoDB will manage sharding for you, Redis works more like Memcached when you want to partition[1]. This is particularly problematic for an operational _storage_ product as that means if you want to offline a node, you are going to have to consider what kind of partitioning Marconi has used, and how it will affect the availability and durability of the data. All of this to say, if Marconi is going to be high scale, I agree that SQL can't be used, and even that MongoDB, on technical abilities alone, makes some sense. But I think what might be simpler is if Marconi just shifted focus to make the API more like AMQP, and used AMQP on its backend. This allows cloud operators to deploy what they're used to for OpenStack, and would still give users something they're comfortable with (an HTTP API) to consume it. [1] http://redis.io/topics/partitioning _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev