Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-02-21 Thread Eric Windisch
It has been a pretty long month. I absorbed the community's comments, made 
improvements where they've been raised, and have also refactored to  
use fully asynchronous messaging.

I won't say the driver isn't without bugs, but it is again fully-functional and 
feature complete. I've been working with Russell Bryant and his helpful reviews 
this morning to polish up for inclusion, if it can get the approvals. 

-- 
Eric Windisch


On Tuesday, February 7, 2012 at 4:05 PM, Eric Windisch wrote:

 The ZeroMQ RPC driver is now feature-complete. I'm cleaning up for a 
 merge-proposal! 
 
 -- 
 Eric Windisch 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-02-07 Thread Eric Windisch
The ZeroMQ RPC driver is now feature-complete. I'm cleaning up for a 
merge-proposal! 

-- 
Eric Windisch___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-30 Thread Rosa, Andrea
Hi 

In my opinion there is another point to consider: at this moment it's possible, 
in rabbitmq by auth-mechanism-ssl plugin, to use client and server 
authentication through certificates.
I don't know 0MQ, so maybe the answer to my question is addressed by 0MQ 
documentation (I'll have a look), but is there some feature to easily implement 
this security feature?

Best Regards
--
Andrea Rosa



-Original Message-
From: openstack-bounces+andrea.rosa=hp@lists.launchpad.net
[mailto:openstack-bounces+andrea.rosa=hp@lists.launchpad.net] On
Behalf Of Yun Mao
Sent: 27 January 2012 22:17
To: Alexis Richardson
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

Sorry to bring back a rather quiet thread from 3 days ago.

How fast do we need to queueing component to be? My observation from
Amazon EC2 us-east-1 is about 2 VMs provisioned per second on average.
Let's say there are 100 messages exchanged for the workload per second
per VM (which I believe is over estimated), and the peak time workload
is 100x higher. Then we need a queue that can do 20,000 messages per
second at the peak rate. Either Rabbit or 0MQ should handle this very
easily. So I'm assuming performance is not a concern.

Now if we go brokerless completely for all messages, that's an obvious
gain as we get rid of one type source of failure. Can that be
achieved? My impression after quickly skimming through the 0MQ
document is that those direct connection can be brokerless but things
more like broadcast can't be. I might very much be wrong at this, and
I would appreciate a lot if someone could help to explain.

Thanks,

Yun

On Wed, Jan 25, 2012 at 11:56 AM, Alexis Richardson
ale...@rabbitmq.com wrote:
 Eric

 Understood ;-)

 I am all in favour of community experimentation.

 1:1 messaging is a core use case for RabbitMQ.  Unlike regular
 queueing systems which use queues for shared topics, Rabbit is
 designed to support very large numbers of short lived queues as well
 as long lived queues.  These can be private or shared.  In other
 words: queues are buffers.  ZeroMQ goes one step further and
 co-locates the consumer with the buffer, and the routing logic with
 the producer.  The cases for which this is useful are discussed on the
 web site.

 alexis



 On Wed, Jan 25, 2012 at 4:49 PM, Eric Windisch e...@cloudscaling.com
wrote:
 Alexis,

 It is also obvious that the link I provided is a particularly biased
source,
 so it should be taken with a grain of salt. I only mentioned Qpid
because
 Nova just received a driver for it, I didn't know the differences in
such
 detail.

 One of the problems Nova has is that it registers N queues for N
hosts, with
 one host pulling from each queue (1:1). This is why ZeroMQ is a good
fit,
 whereby messages can be sent directly to those hosts. There are also
a small
 (but active) number of N to N queues which remain centralized and for
which
 running Rabbit or Qpid is a good fit.

 It would be interesting exercise to allow the ZeroMQ driver to defer
back to
 the Kombu or Qpid driver for those messages which must remain
centralized.

 --
 Eric Windisch

 On Wednesday, January 25, 2012 at 1:18 AM, Alexis Richardson wrote:

 On Wed, Jan 25, 2012 at 4:46 AM, Eric Windisch
e...@cloudscaling.com
 wrote:

 Sorry, I had originally sent only to Yun Mao. Sending to list.

 ---

 Rather than attempt to answer this, I defer to the ZeroMQ guide. It
should
 be noted that the designers of AMPQ, iMatix, designed and build
ZeroMQ.
 (RabbitMQ and QUID implement AMQP)


 Hold on a second there...

 There has been a LOT of muddle and fud (fuddle?) around AMQP. Let
 me try to clear that up.

 Qpid's default option is AMQP 0-10. While feature-rich, 0-10 is not
 widely used and was described by the AMQP chairman as too long and
 complicated not long after it was published. See also commentary on
 the web, on the subject of its length. Rabbit does not and will not
 support this version, and other folks have not implemented it either.

 WHEREAS --

 RabbitMQ implements AMQP 0-91, a 40 page spec. It's the one most
people use.

 0-9-1 is the version of AMQP that is used across a very large number
 of use cases, that is quite easy to implement. It was created by all
 the implementers of AMQP that existed at time of writing including
 Rabbit, Redhat, JPMorgan, and of course iMatix. Pieter @iMatix was
 the spec editor, and did a fantastic job. 0-9-1 provides
 interoperable messaging as witnessed by the large number (100s) of
 clients and add-ons that have been created by the community. There
 have also been several servers implemented, that all just work with
 those clients. For example Kaazing, StormMQ, and OpenAMQ. I believe
 that Qpid also supports it, which might be important for this
 community (Redhat guys please note).

 This is what Pieter said: Read AMQP/0.9.1, it is a beautiful, clean,
 minimalist work essentially created by cooperation in the working
 group

Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-30 Thread Eric Windisch
On Mon, Jan 30, 2012 at 5:05 AM, Rosa, Andrea andrea.r...@hp.com wrote:

 Hi

 In my opinion there is another point to consider: at this moment it's
 possible, in rabbitmq by auth-mechanism-ssl plugin, to use client and
 server authentication through certificates.
 I don't know 0MQ, so maybe the answer to my question is addressed by 0MQ
 documentation (I'll have a look), but is there some feature to easily
 implement this security feature?


The ZeroMQ driver will neither encrypt nor authenticate packets in the
Essex release. With ZeroMQ, it will be the requirement of the serialization
protocol to manage this. Currently, data is simply pickled. Perhaps for
Folsom we can create a blueprint for the signing  verification of messages.

-- 
Regards,
Eric Windisch
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-27 Thread Yun Mao
Sorry to bring back a rather quiet thread from 3 days ago.

How fast do we need to queueing component to be? My observation from
Amazon EC2 us-east-1 is about 2 VMs provisioned per second on average.
Let's say there are 100 messages exchanged for the workload per second
per VM (which I believe is over estimated), and the peak time workload
is 100x higher. Then we need a queue that can do 20,000 messages per
second at the peak rate. Either Rabbit or 0MQ should handle this very
easily. So I'm assuming performance is not a concern.

Now if we go brokerless completely for all messages, that's an obvious
gain as we get rid of one type source of failure. Can that be
achieved? My impression after quickly skimming through the 0MQ
document is that those direct connection can be brokerless but things
more like broadcast can't be. I might very much be wrong at this, and
I would appreciate a lot if someone could help to explain.

Thanks,

Yun

On Wed, Jan 25, 2012 at 11:56 AM, Alexis Richardson ale...@rabbitmq.com wrote:
 Eric

 Understood ;-)

 I am all in favour of community experimentation.

 1:1 messaging is a core use case for RabbitMQ.  Unlike regular
 queueing systems which use queues for shared topics, Rabbit is
 designed to support very large numbers of short lived queues as well
 as long lived queues.  These can be private or shared.  In other
 words: queues are buffers.  ZeroMQ goes one step further and
 co-locates the consumer with the buffer, and the routing logic with
 the producer.  The cases for which this is useful are discussed on the
 web site.

 alexis



 On Wed, Jan 25, 2012 at 4:49 PM, Eric Windisch e...@cloudscaling.com wrote:
 Alexis,

 It is also obvious that the link I provided is a particularly biased source,
 so it should be taken with a grain of salt. I only mentioned Qpid because
 Nova just received a driver for it, I didn't know the differences in such
 detail.

 One of the problems Nova has is that it registers N queues for N hosts, with
 one host pulling from each queue (1:1). This is why ZeroMQ is a good fit,
 whereby messages can be sent directly to those hosts. There are also a small
 (but active) number of N to N queues which remain centralized and for which
 running Rabbit or Qpid is a good fit.

 It would be interesting exercise to allow the ZeroMQ driver to defer back to
 the Kombu or Qpid driver for those messages which must remain centralized.

 --
 Eric Windisch

 On Wednesday, January 25, 2012 at 1:18 AM, Alexis Richardson wrote:

 On Wed, Jan 25, 2012 at 4:46 AM, Eric Windisch e...@cloudscaling.com
 wrote:

 Sorry, I had originally sent only to Yun Mao. Sending to list.

 ---

 Rather than attempt to answer this, I defer to the ZeroMQ guide. It should
 be noted that the designers of AMPQ, iMatix, designed and build ZeroMQ.
 (RabbitMQ and QUID implement AMQP)


 Hold on a second there...

 There has been a LOT of muddle and fud (fuddle?) around AMQP. Let
 me try to clear that up.

 Qpid's default option is AMQP 0-10. While feature-rich, 0-10 is not
 widely used and was described by the AMQP chairman as too long and
 complicated not long after it was published. See also commentary on
 the web, on the subject of its length. Rabbit does not and will not
 support this version, and other folks have not implemented it either.

 WHEREAS --

 RabbitMQ implements AMQP 0-91, a 40 page spec. It's the one most people use.

 0-9-1 is the version of AMQP that is used across a very large number
 of use cases, that is quite easy to implement. It was created by all
 the implementers of AMQP that existed at time of writing including
 Rabbit, Redhat, JPMorgan, and of course iMatix. Pieter @iMatix was
 the spec editor, and did a fantastic job. 0-9-1 provides
 interoperable messaging as witnessed by the large number (100s) of
 clients and add-ons that have been created by the community. There
 have also been several servers implemented, that all just work with
 those clients. For example Kaazing, StormMQ, and OpenAMQ. I believe
 that Qpid also supports it, which might be important for this
 community (Redhat guys please note).

 This is what Pieter said: Read AMQP/0.9.1, it is a beautiful, clean,
 minimalist work essentially created by cooperation in the working
 group to improve AMQP/0.8. I edited AMQP/0.9.1, based on a hundred or
 more fixes made by the best individual brains in that group. Alexis is
 right - this is not disappearing. (Ref - comments here:
 http://it.toolbox.com/blogs/open-source-smb/whats-the-future-of-amqp-44450)

 I agree with Pieter. We also like the way that 0-9-1 can play nicely
 with 0mq and other protocols. Note that Rabbit supports these via
 plugins.

 alexis






 http://zguide.zeromq.org/page:all#Why-We-Needed-MQ


 --
 Eric Windisch

 On Tuesday, January 24, 2012 at 5:20 PM, Yun Mao wrote:

 Hi I'm curious and unfamiliar with the subject. What's the benefit of
 0MQ vs Kombu? Thanks,

 Yun

 On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com
 

Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-25 Thread Thierry Carrez
Duncan McGreggor wrote:
 FWIW, I'd love to see this land in E3...

It won't land in E3, but can still be in Essex.

In particular if it's just an alternate code path that doesn't affect
existing code, it's a good candidate for FFe and landing early E4.

Cheers,

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-25 Thread Eric Windisch
Alexis, 

It is also obvious that the link I provided is a particularly biased source, so 
it should be taken with a grain of salt. I only mentioned Qpid because Nova 
just received a driver for it, I didn't know the differences in such detail.

One of the problems Nova has is that it registers N queues for N hosts, with 
one host pulling from each queue (1:1). This is why ZeroMQ is a good fit, 
whereby messages can be sent directly to those hosts. There are also a small 
(but active) number of N to N queues which remain centralized and for which 
running Rabbit or Qpid is a good fit.

It would be interesting exercise to allow the ZeroMQ driver to defer back to 
the Kombu or Qpid driver for those messages which must remain centralized.

-- 
Eric Windisch


On Wednesday, January 25, 2012 at 1:18 AM, Alexis Richardson wrote:

 On Wed, Jan 25, 2012 at 4:46 AM, Eric Windisch e...@cloudscaling.com 
 (mailto:e...@cloudscaling.com) wrote:
  Sorry, I had originally sent only to Yun Mao. Sending to list.
  
  ---
  
  Rather than attempt to answer this, I defer to the ZeroMQ guide. It should
  be noted that the designers of AMPQ, iMatix, designed and build ZeroMQ.
  (RabbitMQ and QUID implement AMQP)
  
 
 
 Hold on a second there...
 
 There has been a LOT of muddle and fud (fuddle?) around AMQP. Let
 me try to clear that up.
 
 Qpid's default option is AMQP 0-10. While feature-rich, 0-10 is not
 widely used and was described by the AMQP chairman as too long and
 complicated not long after it was published. See also commentary on
 the web, on the subject of its length. Rabbit does not and will not
 support this version, and other folks have not implemented it either.
 
 WHEREAS --
 
 RabbitMQ implements AMQP 0-91, a 40 page spec. It's the one most people use.
 
 0-9-1 is the version of AMQP that is used across a very large number
 of use cases, that is quite easy to implement. It was created by all
 the implementers of AMQP that existed at time of writing including
 Rabbit, Redhat, JPMorgan, and of course iMatix. Pieter @iMatix was
 the spec editor, and did a fantastic job. 0-9-1 provides
 interoperable messaging as witnessed by the large number (100s) of
 clients and add-ons that have been created by the community. There
 have also been several servers implemented, that all just work with
 those clients. For example Kaazing, StormMQ, and OpenAMQ. I believe
 that Qpid also supports it, which might be important for this
 community (Redhat guys please note).
 
 This is what Pieter said: Read AMQP/0.9.1, it is a beautiful, clean,
 minimalist work essentially created by cooperation in the working
 group to improve AMQP/0.8. I edited AMQP/0.9.1, based on a hundred or
 more fixes made by the best individual brains in that group. Alexis is
 right - this is not disappearing. (Ref - comments here:
 http://it.toolbox.com/blogs/open-source-smb/whats-the-future-of-amqp-44450)
 
 I agree with Pieter. We also like the way that 0-9-1 can play nicely
 with 0mq and other protocols. Note that Rabbit supports these via
 plugins.
 
 alexis
 
 
 
 
 
 
  http://zguide.zeromq.org/page:all#Why-We-Needed-MQ
  
  
  --
  Eric Windisch
  
  On Tuesday, January 24, 2012 at 5:20 PM, Yun Mao wrote:
  
  Hi I'm curious and unfamiliar with the subject. What's the benefit of
  0MQ vs Kombu? Thanks,
  
  Yun
  
  On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com 
  (mailto:e...@cloudscaling.com)
  wrote:
  
  Per today's meeting, I am proposing the ZeroMQ RPC driver for a
  feature-freeze exception.
  
  I am making good progress on this blueprint, it adds a new optional module
  and service without modifying any existing code or modules. I have been
  pushing to complete this work by E3, so I am close to completion, but cannot
  finish by tonight's deadline.
  
  The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and QPID
  for messaging within Nova. Currently, the code passes unit tests but fails
  on smoketests. I expect to have the code viable for a merge proposal in less
  than a week, tomorrow if I'm smart, lucky, and the store doesn't sell out of
  RedBull. A two week grace would give me a nice buffer.
  
  Thanks,
  Eric Windisch
  
  
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net 
  (mailto:openstack@lists.launchpad.net)
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
  
  
  
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net 
  (mailto:openstack@lists.launchpad.net)
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
  
 
 
 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack

[Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Eric Windisch
Per today's meeting, I am proposing the ZeroMQ RPC driver for a feature-freeze 
exception. 

I am making good progress on this blueprint, it adds a new optional module and 
service without modifying any existing code or modules. I have been pushing to 
complete this work by E3, so I am close to completion, but cannot finish by 
tonight's deadline.

The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and QPID for 
messaging within Nova. Currently, the code passes unit tests but fails on 
smoketests. I expect to have the code viable for a merge proposal in less than 
a week, tomorrow if I'm smart, lucky, and the store doesn't sell out of 
RedBull. A two week grace would give me a nice buffer.

Thanks,
Eric Windisch

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Duncan McGreggor
FWIW, I'd love to see this land in E3...

d

On 24 Jan 2012 - 16:08, Eric Windisch wrote:
 Per today's meeting, I am proposing the ZeroMQ RPC driver for a
 feature-freeze exception.

 I am making good progress on this blueprint, it adds a new optional
 module and service without modifying any existing code or modules. I
 have been pushing to complete this work by E3, so I am close to
 completion, but cannot finish by tonight's deadline.

 The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and
 QPID for messaging within Nova. Currently, the code passes unit tests
 but fails on smoketests. I expect to have the code viable for a merge
 proposal in less than a week, tomorrow if I'm smart, lucky, and the
 store doesn't sell out of RedBull. A two week grace would give me a
 nice buffer.

 Thanks,
 Eric Windisch


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Yun Mao
Hi I'm curious and unfamiliar with the subject. What's the benefit of
0MQ vs Kombu? Thanks,

Yun

On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com wrote:
 Per today's meeting, I am proposing the ZeroMQ RPC driver for a
 feature-freeze exception.

 I am making good progress on this blueprint, it adds a new optional module
 and service without modifying any existing code or modules. I have been
 pushing to complete this work by E3, so I am close to completion, but cannot
 finish by tonight's deadline.

 The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and QPID
 for messaging within Nova. Currently, the code passes unit tests but fails
 on smoketests. I expect to have the code viable for a merge proposal in less
 than a week, tomorrow if I'm smart, lucky, and the store doesn't sell out of
 RedBull. A two week grace would give me a nice buffer.

 Thanks,
 Eric Windisch


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Duncan McGreggor
Under specific architectures, 0MQ can process millions of messages per
second vs. RabbitMQ's many thousands.

RabbitMQ is a messaging system; 0MQ is a messaging framework. There's a
pretty good write-up on some of the basic differences here:
  http://www.zeromq.org/docs:welcome-from-amqp

Note that the creators of 0MQ (iMatix) were also the guys that invented
AMQP... so they've learned many excellent lessons ;-)

d


On 24 Jan 2012 - 20:20, Yun Mao wrote:
 Hi I'm curious and unfamiliar with the subject. What's the benefit of
 0MQ vs Kombu? Thanks,

 Yun

 On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com wrote:
  Per today's meeting, I am proposing the ZeroMQ RPC driver for a
  feature-freeze exception.
 
  I am making good progress on this blueprint, it adds a new optional module
  and service without modifying any existing code or modules. I have been
  pushing to complete this work by E3, so I am close to completion, but cannot
  finish by tonight's deadline.
 
  The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and QPID
  for messaging within Nova. Currently, the code passes unit tests but fails
  on smoketests. I expect to have the code viable for a merge proposal in less
  than a week, tomorrow if I'm smart, lucky, and the store doesn't sell out of
  RedBull. A two week grace would give me a nice buffer.
 
  Thanks,
  Eric Windisch
 
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to ? ? : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help ? : https://help.launchpad.net/ListHelp
 

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Zhongyue Luo
I assume the messages will be delivered directly to the destination rather
than piling up on a queue server?

On Wed, Jan 25, 2012 at 10:49 AM, Duncan McGreggor dun...@dreamhost.comwrote:

 Under specific architectures, 0MQ can process millions of messages per
 second vs. RabbitMQ's many thousands.

 RabbitMQ is a messaging system; 0MQ is a messaging framework. There's a
 pretty good write-up on some of the basic differences here:
  http://www.zeromq.org/docs:welcome-from-amqp

 Note that the creators of 0MQ (iMatix) were also the guys that invented
 AMQP... so they've learned many excellent lessons ;-)

 d


 On 24 Jan 2012 - 20:20, Yun Mao wrote:
  Hi I'm curious and unfamiliar with the subject. What's the benefit of
  0MQ vs Kombu? Thanks,
 
  Yun
 
  On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com
 wrote:
   Per today's meeting, I am proposing the ZeroMQ RPC driver for a
   feature-freeze exception.
  
   I am making good progress on this blueprint, it adds a new optional
 module
   and service without modifying any existing code or modules. I have been
   pushing to complete this work by E3, so I am close to completion, but
 cannot
   finish by tonight's deadline.
  
   The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and
 QPID
   for messaging within Nova. Currently, the code passes unit tests but
 fails
   on smoketests. I expect to have the code viable for a merge proposal
 in less
   than a week, tomorrow if I'm smart, lucky, and the store doesn't sell
 out of
   RedBull. A two week grace would give me a nice buffer.
  
   Thanks,
   Eric Windisch
  
  
   ___
   Mailing list: https://launchpad.net/~openstack
   Post to ? ? : openstack@lists.launchpad.net
   Unsubscribe : https://launchpad.net/~openstack
   More help ? : https://help.launchpad.net/ListHelp
  
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Eric Windisch
Sorry, I had originally sent only to Yun Mao. Sending to list.
---
Rather than attempt to answer this, I defer to the ZeroMQ guide. It should be 
noted that the designers of AMPQ, iMatix, designed and build ZeroMQ. (RabbitMQ 
and QUID implement AMQP)
http://zguide.zeromq.org/page:all#Why-We-Needed-MQ


-- 
Eric Windisch


On Tuesday, January 24, 2012 at 5:20 PM, Yun Mao wrote:

 Hi I'm curious and unfamiliar with the subject. What's the benefit of
 0MQ vs Kombu? Thanks,
 
 Yun
 
 On Tue, Jan 24, 2012 at 7:08 PM, Eric Windisch e...@cloudscaling.com 
 (mailto:e...@cloudscaling.com) wrote:
  Per today's meeting, I am proposing the ZeroMQ RPC driver for a
  feature-freeze exception.
  
  I am making good progress on this blueprint, it adds a new optional module
  and service without modifying any existing code or modules. I have been
  pushing to complete this work by E3, so I am close to completion, but cannot
  finish by tonight's deadline.
  
  The ZeroMQ driver will provide an alternative to Kombu (RabbitMQ) and QPID
  for messaging within Nova. Currently, the code passes unit tests but fails
  on smoketests. I expect to have the code viable for a merge proposal in less
  than a week, tomorrow if I'm smart, lucky, and the store doesn't sell out of
  RedBull. A two week grace would give me a nice buffer.
  
  Thanks,
  Eric Windisch
  
  
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net 
  (mailto:openstack@lists.launchpad.net)
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
  
 
 
 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] ZeroMQ RPC Driver - FF-Exception request

2012-01-24 Thread Eric Windisch



On Tuesday, January 24, 2012 at 6:05 PM, Zhongyue Luo wrote:

 I assume the messages will be delivered directly to the destination rather 
 than piling up on a queue server?
 
 
 


Although the blueprint doesn't specify this level of detail, the intention had 
originally been to deliver a queue-server model in Essex with a 
distributed-model in Folsom.  The first-cut code which passes tests does pile 
onto a queue server.  I now have code which implements a distributed model, but 
it is in a raw state. There are a number of edge-cases a distributed model can 
face, especially in regard to threading mechanisms (i.e. eventlet).

My current blockers for having this code ready for a merge-proposal are not 
related to a single-server or distributed model. Once I address these problems, 
there can be a decision on how much time is left to debug a distributed model 
within a FFe time-frame, or if this can be accepted as a separate blueprint 
within E4. If I get a solid 1-2 weeks or more for a FFe, I'll be more confident 
in the distributed model as there is more room to test, debug, and fix.

By the way, I must note in this thread, that I haven't done this alone.  A good 
amount of this work, i.e. the first-first-cut, was completed by Zed Shaw and 
was made possible through his QAL. Duncan McGreggor has been on hand, helpful, 
and has contributed some commits as well.

Regards,Eric Windisch___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp