(There have been many more replies since I started writing this.) Great stuff on the wiki. My $0.02:
+1 vote for Python/C modules for needed 'performance'. I might be the only one thinking that Python is up to the task, but I wanted to throw that out there. +0 for Erlang, although I *love* it and there are a ton of libraries out there, I'm really on the fence with how many people could actually contribute to the project. A lot of ramp-up and learning does not a stable project make (if the release schedule really is as tight as indicated on the Wiki). C++ and I have a history and I'd rather everything be written in C (yeah, I'm one of those people). For truly complex tasks C + Python work very well together and can produce some amazing results. That being said, C++ is a solid choice. Languages aside, it looks like you've got a decent start going, Eric, with Scale Stack. I do worry about reinventing the wheel a bit, although I'm not clear on the goals of your project so I could be wrong. Could this service just be an aggregation of some other proven technologies? The first thing that came to mind when this service was suggested was using a proven scalable key/value store to provide a queue-like interface and then all we'd have to do is provide the interface on top of it. HTTP + Authentication + Direct to customer's data store cluster = Queue Service? Voldemort, Keyspace, Scalaris, and tons of other projects could be researched to provide a scalable back-end for the queuing system. Maybe I don't quite understand which problem Scale Stack is solving? HTTP API + -----Original Message----- From: "Eric Day" <[email protected]> Sent: Thursday, February 17, 2011 5:21pm To: [email protected] Subject: [Openstack] Queue Service, next steps Thanks to everyone who gave great feedback on the first queue service thread. I've updated the wiki page to include the suggestions. http://wiki.openstack.org/QueueService With a decent vision of what we want to build, the next step is figuring out how. In a previous thread it was suggested that the preferred languages for OpenStack projects are Python, C, and C++. Since there is an emphasis on speed and efficiency for the queue service, I suggest we use C++. I expect this service to be CPU bound and would benefit being able to leverage multiple cores efficiently (within the same process), so I don't think Python is a good fit. I think C++ is a better fit than C due to the need for modular interfaces. While this can obviously be done in C, C++ APIs are more concise and much less error prone. The OO style will also make it easier for Python developers who also want to learn and assist with C++ projects. Erlang is not on the preferred lists, but I would also put it out there as an option. While it may be a great fit for a project like this, I worry it won't attract the developer resources since Erlang isn't really a first-class language yet. If we decide to take the C++ path, I propose using a modular application framework I've been working on over the past year (mostly in my spare time). It provides a simple module programming interface with dependency tracking (kind of like Linux kernel modules). It already provides a multi-threaded event module (currently based on libevent, but this is pluggable) with simple networking abstractions built on top of it. We should be able to dive in an start writing the HTTP protocol module and queue processing modules. You can check out the current project at: https://launchpad.net/scalestack http://scalestack.org/ The intention of using a framework like this is so we can easily reuse the other modules (auth, HTTP, logging, ...) for other OpenStack services in the future. Much like we use Eventlet, WSGI, etc. (and eventually openstack-common) for Python, we could prefer using the modules in Scale Stack for lower level projects. Thoughts? -Eric _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
