Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-17 Thread Doug Hellmann
Thanks, Josh, I’ll subscribe to the issue to keep up to date.

On Nov 16, 2014, at 6:58 PM, Joshua Harlow harlo...@outlook.com wrote:

 I started the following issue on kombu's github page (to see if there is any 
 interest on there side to such an effort):
 
 https://github.com/celery/kombu/issues/430
 
 It's about seeing if the kombu folks would be ok with a 'rpc' subfolder in 
 there repository that can start to contain 'rpc' like functionality that now 
 exists in oslo.messaging (I don't see why they would be against this kind of 
 idea, since it seems to make sense IMHO).
 
 Let's see what happens,
 
 -Josh
 
 Doug Hellmann wrote:
 
 On Nov 13, 2014, at 7:02 PM, Joshua Harlow harlo...@yahoo-inc.com
 mailto:harlo...@yahoo-inc.com wrote:
 
 Don't forget my executor which isn't dependent on a larger set of
 changes for asyncio/trollious...
 
 https://review.openstack.org/#/c/70914/
 
 The above will/should just 'work', although I'm unsure what thread
 count should be by default (the number of green threads that is set at
 like 200 shouldn't be the same number used in that executor which uses
 real python/system threads). The neat thing about that executor is
 that it can also replace the eventlet one, since when eventlet is
 monkey patching the threading module (which it should be) then it
 should behave just as the existing eventlet one; which IMHO is pretty
 cool (and could be one way to completely remove the eventlet usage in
 oslo.messaging).
 
 Good point, thanks for reminding me.
 
 
 As for the kombu discussions, maybe its time to jump on the #celery
 channel (where the kombu folks hang out) and start talking to them
 about how we can work better together to move some of our features
 into kombu (and also depreciate/remove some of the oslo.messaging
 features that now are in kombu). I believe
 https://launchpad.net/~asksol is the main guy there (and also the main
 maintainer of celery/kombu?). It'd be nice to have these
 cross-community talks and at least come up with some kind of game
 plan; hopefully one that benefits both communities…
 
 I would like that, but won’t have time to do it myself this cycle. Maybe
 we can find another volunteer from the team?
 
 Doug
 
 
 -Josh
 
 https://launchpad.net/~asksol
 
 *From:* Doug Hellmann d...@doughellmann.com
 mailto:d...@doughellmann.com
 *To:* OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 mailto:openstack-dev@lists.openstack.org
 *Sent:* Wednesday, November 12, 2014 12:22 PM
 *Subject:* [openstack-dev] [oslo] oslo.messaging outcome from the summit
 
 The oslo.messaging session at the summit [1] resulted in some plans to
 evolve how oslo.messaging works, but probably not during this cycle.
 
 First, we talked about what to do about the various drivers like
 ZeroMQ and the new AMQP 1.0 driver. We decided that rather than moving
 those out of the main tree and packaging them separately, we would
 keep them all in the main repository to encourage the driver authors
 to help out with the core library (oslo.messaging is a critical
 component of OpenStack, and we’ve lost several of our core reviewers
 for the library to other priorities recently).
 
 There is a new set of contributors interested in maintaining the
 ZeroMQ driver, and they are going to work together to review each
 other’s patches. We will re-evaluate keeping ZeroMQ at the end of
 Kilo, based on how things go this cycle.
 
 We also talked about the fact that the new version of Kombu includes
 some of the features we have implemented in our own driver, like
 heartbeats and connection management. Kombu does not include the
 calling patterns (cast/call/notifications) that we have in
 oslo.messaging, but we may be able to remove some code from our driver
 and consolidate the qpid and rabbit driver code to let Kombu do more
 of the work for us.
 
 Python 3 support is coming slowly. There are a couple of patches up
 for review to provide a different sort of executor based on greenio
 and trollius. Adopting that would require some application-level
 changes to use co-routines, so it may not be an optimal solution even
 though it would get us off of eventlet. (During the Python 3 session
 later in the week we talked about the possibility of fixing eventlet’s
 monkey-patching to allow us to use the new eventlet under python 3.)
 
 We also talked about the way the oslo.messaging API uses URLs to get
 some settings and configuration options for others. I thought I
 remembered this being a conscious decision to pass connection-specific
 parameters in the URL, and “global” parameters via configuration
 settings. It sounds like that split may not have been implemented as
 cleanly as originally intended, though. We identified documenting URL
 parameters as an issue for removing the configuration object, as well
 as backwards-compatibility. I don’t think we agreed on any specific
 

Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-17 Thread Ilya Pekelny
 Flavio Percoco wrote:

 Still, I'd like us to learn from
 previous experiences and have a better plan for this driver (and
 future cases like this one).


Hi, all!

As one of a just joined ZeroMQ maintainers I have a growing plan of ZeroMQ
refactoring and development. At the most abstract view our plan is to
remove single broker and implement peer-2-peer model in the messaging
driver. Now exists a blueprint with this goal
https://blueprints.launchpad.net/oslo.messaging/+spec/reduce-central-broker.
I maintain a patch and a spec which I had inherited from Aleksey Kornienko.
For now this blueprint is the first step in the planning process. I believe
we can split this big work in a set of specs and if is needed in several
related blueprints. With these specs and BPs our plan should become
obvious. I wrote a mail in the dev mail list with short overview to the
coming work.

Please, feel free to discuss it all with me and correct me on this big road.

On Mon, Nov 17, 2014 at 4:45 PM, Doug Hellmann d...@doughellmann.com
wrote:

 Thanks, Josh, I’ll subscribe to the issue to keep up to date.

 On Nov 16, 2014, at 6:58 PM, Joshua Harlow harlo...@outlook.com wrote:

  I started the following issue on kombu's github page (to see if there is
 any interest on there side to such an effort):
 
  https://github.com/celery/kombu/issues/430
 
  It's about seeing if the kombu folks would be ok with a 'rpc' subfolder
 in there repository that can start to contain 'rpc' like functionality that
 now exists in oslo.messaging (I don't see why they would be against this
 kind of idea, since it seems to make sense IMHO).
 
  Let's see what happens,
 
  -Josh
 
  Doug Hellmann wrote:
 
  On Nov 13, 2014, at 7:02 PM, Joshua Harlow harlo...@yahoo-inc.com
  mailto:harlo...@yahoo-inc.com wrote:
 
  Don't forget my executor which isn't dependent on a larger set of
  changes for asyncio/trollious...
 
  https://review.openstack.org/#/c/70914/
 
  The above will/should just 'work', although I'm unsure what thread
  count should be by default (the number of green threads that is set at
  like 200 shouldn't be the same number used in that executor which uses
  real python/system threads). The neat thing about that executor is
  that it can also replace the eventlet one, since when eventlet is
  monkey patching the threading module (which it should be) then it
  should behave just as the existing eventlet one; which IMHO is pretty
  cool (and could be one way to completely remove the eventlet usage in
  oslo.messaging).
 
  Good point, thanks for reminding me.
 
 
  As for the kombu discussions, maybe its time to jump on the #celery
  channel (where the kombu folks hang out) and start talking to them
  about how we can work better together to move some of our features
  into kombu (and also depreciate/remove some of the oslo.messaging
  features that now are in kombu). I believe
  https://launchpad.net/~asksol is the main guy there (and also the main
  maintainer of celery/kombu?). It'd be nice to have these
  cross-community talks and at least come up with some kind of game
  plan; hopefully one that benefits both communities…
 
  I would like that, but won’t have time to do it myself this cycle. Maybe
  we can find another volunteer from the team?
 
  Doug
 
 
  -Josh
 
  https://launchpad.net/~asksol
 
 
  *From:* Doug Hellmann d...@doughellmann.com
  mailto:d...@doughellmann.com
  *To:* OpenStack Development Mailing List (not for usage questions)
  openstack-dev@lists.openstack.org
  mailto:openstack-dev@lists.openstack.org
  *Sent:* Wednesday, November 12, 2014 12:22 PM
  *Subject:* [openstack-dev] [oslo] oslo.messaging outcome from the
 summit
 
  The oslo.messaging session at the summit [1] resulted in some plans to
  evolve how oslo.messaging works, but probably not during this cycle.
 
  First, we talked about what to do about the various drivers like
  ZeroMQ and the new AMQP 1.0 driver. We decided that rather than moving
  those out of the main tree and packaging them separately, we would
  keep them all in the main repository to encourage the driver authors
  to help out with the core library (oslo.messaging is a critical
  component of OpenStack, and we’ve lost several of our core reviewers
  for the library to other priorities recently).
 
  There is a new set of contributors interested in maintaining the
  ZeroMQ driver, and they are going to work together to review each
  other’s patches. We will re-evaluate keeping ZeroMQ at the end of
  Kilo, based on how things go this cycle.
 
  We also talked about the fact that the new version of Kombu includes
  some of the features we have implemented in our own driver, like
  heartbeats and connection management. Kombu does not include the
  calling patterns (cast/call/notifications) that we have in
  oslo.messaging, but we may be able to remove some code from our driver
  and consolidate the qpid and rabbit 

Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-16 Thread Joshua Harlow
I started the following issue on kombu's github page (to see if there is 
any interest on there side to such an effort):


https://github.com/celery/kombu/issues/430

It's about seeing if the kombu folks would be ok with a 'rpc' subfolder 
in there repository that can start to contain 'rpc' like functionality 
that now exists in oslo.messaging (I don't see why they would be against 
this kind of idea, since it seems to make sense IMHO).


Let's see what happens,

-Josh

Doug Hellmann wrote:


On Nov 13, 2014, at 7:02 PM, Joshua Harlow harlo...@yahoo-inc.com
mailto:harlo...@yahoo-inc.com wrote:


Don't forget my executor which isn't dependent on a larger set of
changes for asyncio/trollious...

https://review.openstack.org/#/c/70914/

The above will/should just 'work', although I'm unsure what thread
count should be by default (the number of green threads that is set at
like 200 shouldn't be the same number used in that executor which uses
real python/system threads). The neat thing about that executor is
that it can also replace the eventlet one, since when eventlet is
monkey patching the threading module (which it should be) then it
should behave just as the existing eventlet one; which IMHO is pretty
cool (and could be one way to completely remove the eventlet usage in
oslo.messaging).


Good point, thanks for reminding me.



As for the kombu discussions, maybe its time to jump on the #celery
channel (where the kombu folks hang out) and start talking to them
about how we can work better together to move some of our features
into kombu (and also depreciate/remove some of the oslo.messaging
features that now are in kombu). I believe
https://launchpad.net/~asksol is the main guy there (and also the main
maintainer of celery/kombu?). It'd be nice to have these
cross-community talks and at least come up with some kind of game
plan; hopefully one that benefits both communities…


I would like that, but won’t have time to do it myself this cycle. Maybe
we can find another volunteer from the team?

Doug



-Josh

https://launchpad.net/~asksol

*From:* Doug Hellmann d...@doughellmann.com
mailto:d...@doughellmann.com
*To:* OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
*Sent:* Wednesday, November 12, 2014 12:22 PM
*Subject:* [openstack-dev] [oslo] oslo.messaging outcome from the summit

The oslo.messaging session at the summit [1] resulted in some plans to
evolve how oslo.messaging works, but probably not during this cycle.

First, we talked about what to do about the various drivers like
ZeroMQ and the new AMQP 1.0 driver. We decided that rather than moving
those out of the main tree and packaging them separately, we would
keep them all in the main repository to encourage the driver authors
to help out with the core library (oslo.messaging is a critical
component of OpenStack, and we’ve lost several of our core reviewers
for the library to other priorities recently).

There is a new set of contributors interested in maintaining the
ZeroMQ driver, and they are going to work together to review each
other’s patches. We will re-evaluate keeping ZeroMQ at the end of
Kilo, based on how things go this cycle.

We also talked about the fact that the new version of Kombu includes
some of the features we have implemented in our own driver, like
heartbeats and connection management. Kombu does not include the
calling patterns (cast/call/notifications) that we have in
oslo.messaging, but we may be able to remove some code from our driver
and consolidate the qpid and rabbit driver code to let Kombu do more
of the work for us.

Python 3 support is coming slowly. There are a couple of patches up
for review to provide a different sort of executor based on greenio
and trollius. Adopting that would require some application-level
changes to use co-routines, so it may not be an optimal solution even
though it would get us off of eventlet. (During the Python 3 session
later in the week we talked about the possibility of fixing eventlet’s
monkey-patching to allow us to use the new eventlet under python 3.)

We also talked about the way the oslo.messaging API uses URLs to get
some settings and configuration options for others. I thought I
remembered this being a conscious decision to pass connection-specific
parameters in the URL, and “global” parameters via configuration
settings. It sounds like that split may not have been implemented as
cleanly as originally intended, though. We identified documenting URL
parameters as an issue for removing the configuration object, as well
as backwards-compatibility. I don’t think we agreed on any specific
changes to the API based on this part of the discussion, but please
correct me if your recollection is different.

We also learned that there is a critical bug [2] related to heartbeats
for RabbitMQ, and we have a few patches up to 

Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-13 Thread Flavio Percoco

On 12/11/14 15:22 -0500, Doug Hellmann wrote:

The oslo.messaging session at the summit [1] resulted in some plans to evolve 
how oslo.messaging works, but probably not during this cycle.

First, we talked about what to do about the various drivers like ZeroMQ and the 
new AMQP 1.0 driver. We decided that rather than moving those out of the main 
tree and packaging them separately, we would keep them all in the main 
repository to encourage the driver authors to help out with the core library 
(oslo.messaging is a critical component of OpenStack, and we’ve lost several of 
our core reviewers for the library to other priorities recently).

There is a new set of contributors interested in maintaining the ZeroMQ driver, 
and they are going to work together to review each other’s patches. We will 
re-evaluate keeping ZeroMQ at the end of Kilo, based on how things go this 
cycle.


I'd like to thank the folks that have stepped up for this driver. It's
great to see that there's some interest in cleaning it up and
maintaining it.

That said, if at the end of Kilo the zmq driver is still not in a
usable/maintainable mode, I'd like us to be more strict with the plans
forward for it. We asked for support in the last 3 summits with bad
results for the previous 2 releases.

I don't mean to sound rude and I do believe the folks that have
stepped up will do a great job. Still, I'd like us to learn from
previous experiences and have a better plan for this driver (and
future cases like this one).



We also talked about the fact that the new version of Kombu includes some of 
the features we have implemented in our own driver, like heartbeats and 
connection management. Kombu does not include the calling patterns 
(cast/call/notifications) that we have in oslo.messaging, but we may be able to 
remove some code from our driver and consolidate the qpid and rabbit driver 
code to let Kombu do more of the work for us.


This sounds great. Please, whoever is going to work on this, feel add
me to the reviews.


Python 3 support is coming slowly. There are a couple of patches up for review 
to provide a different sort of executor based on greenio and trollius. Adopting 
that would require some application-level changes to use co-routines, so it may 
not be an optimal solution even though it would get us off of eventlet. (During 
the Python 3 session later in the week we talked about the possibility of 
fixing eventlet’s monkey-patching to allow us to use the new eventlet under 
python 3.)

We also talked about the way the oslo.messaging API uses URLs to get some 
settings and configuration options for others. I thought I remembered this 
being a conscious decision to pass connection-specific parameters in the URL, 
and “global” parameters via configuration settings. It sounds like that split 
may not have been implemented as cleanly as originally intended, though. We 
identified documenting URL parameters as an issue for removing the 
configuration object, as well as backwards-compatibility. I don’t think we 
agreed on any specific changes to the API based on this part of the discussion, 
but please correct me if your recollection is different.


I prefer URL parameters to specify options. As of now, I think we
treat URL parameters and config options as two different things. Is
this something we can change and translate URL parameters to config
options?

I guess if we get to that point, we'd end up asking ourselves: Why
shouldn't we use just config options in that case?

I think one - historical (?) - answer to that is that we once thought
about not using oslo.config in oslo.messaging.

Looking forward to have more feedback on this point, I unfortunately
missed this session because I had to attend another one.
Flavio

--
@flaper87
Flavio Percoco


pgpkNq0wtkvFS.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-13 Thread Joshua Harlow
On Nov 13, 2014, at 12:38 AM, Flavio Percoco fla...@redhat.com wrote:

 On 12/11/14 15:22 -0500, Doug Hellmann wrote:
 The oslo.messaging session at the summit [1] resulted in some plans to 
 evolve how oslo.messaging works, but probably not during this cycle.
 
 First, we talked about what to do about the various drivers like ZeroMQ and 
 the new AMQP 1.0 driver. We decided that rather than moving those out of the 
 main tree and packaging them separately, we would keep them all in the main 
 repository to encourage the driver authors to help out with the core library 
 (oslo.messaging is a critical component of OpenStack, and we’ve lost several 
 of our core reviewers for the library to other priorities recently).
 
 There is a new set of contributors interested in maintaining the ZeroMQ 
 driver, and they are going to work together to review each other’s patches. 
 We will re-evaluate keeping ZeroMQ at the end of Kilo, based on how things 
 go this cycle.
 
 I'd like to thank the folks that have stepped up for this driver. It's
 great to see that there's some interest in cleaning it up and
 maintaining it.
 
 That said, if at the end of Kilo the zmq driver is still not in a
 usable/maintainable mode, I'd like us to be more strict with the plans
 forward for it. We asked for support in the last 3 summits with bad
 results for the previous 2 releases.
 
 I don't mean to sound rude and I do believe the folks that have
 stepped up will do a great job. Still, I'd like us to learn from
 previous experiences and have a better plan for this driver (and
 future cases like this one).
 
 
 We also talked about the fact that the new version of Kombu includes some of 
 the features we have implemented in our own driver, like heartbeats and 
 connection management. Kombu does not include the calling patterns 
 (cast/call/notifications) that we have in oslo.messaging, but we may be able 
 to remove some code from our driver and consolidate the qpid and rabbit 
 driver code to let Kombu do more of the work for us.
 
 This sounds great. Please, whoever is going to work on this, feel add
 me to the reviews.
 
 Python 3 support is coming slowly. There are a couple of patches up for 
 review to provide a different sort of executor based on greenio and 
 trollius. Adopting that would require some application-level changes to use 
 co-routines, so it may not be an optimal solution even though it would get 
 us off of eventlet. (During the Python 3 session later in the week we talked 
 about the possibility of fixing eventlet’s monkey-patching to allow us to 
 use the new eventlet under python 3.)
 
 We also talked about the way the oslo.messaging API uses URLs to get some 
 settings and configuration options for others. I thought I remembered this 
 being a conscious decision to pass connection-specific parameters in the 
 URL, and “global” parameters via configuration settings. It sounds like that 
 split may not have been implemented as cleanly as originally intended, 
 though. We identified documenting URL parameters as an issue for removing 
 the configuration object, as well as backwards-compatibility. I don’t think 
 we agreed on any specific changes to the API based on this part of the 
 discussion, but please correct me if your recollection is different.
 
 I prefer URL parameters to specify options. As of now, I think we
 treat URL parameters and config options as two different things. Is
 this something we can change and translate URL parameters to config
 options?

I'd rather go completely with config and have something like 
https://review.openstack.org/#/c/130047/ which allows for users that don't have 
a CLI accessible (aka from other libraries) to actually use oslo.messaging (for 
ex, taskflow). I believe url parameters could work, its just that config 
already provides typing (ints, bools, lists) and descriptions and urls have 
none of this (they also don't have a nested structure, aka grouping, which I 
believe some of oslo.messaging is using?).

 
 I guess if we get to that point, we'd end up asking ourselves: Why
 shouldn't we use just config options in that case?
 
 I think one - historical (?) - answer to that is that we once thought
 about not using oslo.config in oslo.messaging.

So would https://review.openstack.org/#/c/130047/ make that better?

A user that doesn't have access to oslo.config options would then just have to 
provide a dictionary of equivalent options. As described in that review (and 
pretty obvious) is that not everyone in the python world uses oslo.config 
options and therefore we need/must have a compatiblity layer for users to use 
if they so choose.

This could then look like the following when used: 
http://paste.ubuntu.com/8982657/

In all honesty I just want one way that works, if thats URLs or config (IMHO 
the only way config will actually work is if we have a interface in oslo.config 
like described in 130047 for people that want to use oslo.messaging that are 
not 

Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-13 Thread Doug Hellmann

On Nov 13, 2014, at 3:38 AM, Flavio Percoco fla...@redhat.com wrote:

 On 12/11/14 15:22 -0500, Doug Hellmann wrote:
 The oslo.messaging session at the summit [1] resulted in some plans to 
 evolve how oslo.messaging works, but probably not during this cycle.
 
 First, we talked about what to do about the various drivers like ZeroMQ and 
 the new AMQP 1.0 driver. We decided that rather than moving those out of the 
 main tree and packaging them separately, we would keep them all in the main 
 repository to encourage the driver authors to help out with the core library 
 (oslo.messaging is a critical component of OpenStack, and we’ve lost several 
 of our core reviewers for the library to other priorities recently).
 
 There is a new set of contributors interested in maintaining the ZeroMQ 
 driver, and they are going to work together to review each other’s patches. 
 We will re-evaluate keeping ZeroMQ at the end of Kilo, based on how things 
 go this cycle.
 
 I'd like to thank the folks that have stepped up for this driver. It's
 great to see that there's some interest in cleaning it up and
 maintaining it.
 
 That said, if at the end of Kilo the zmq driver is still not in a
 usable/maintainable mode, I'd like us to be more strict with the plans
 forward for it. We asked for support in the last 3 summits with bad
 results for the previous 2 releases.
 
 I don't mean to sound rude and I do believe the folks that have
 stepped up will do a great job. Still, I'd like us to learn from
 previous experiences and have a better plan for this driver (and
 future cases like this one).

Absolutely. It seems that each time we ask for help, a new set of contributors 
step up. This is, I think, the third cycle where that has been the case? Three 
being widely recognized as a magic number (check your retry loops), either “the 
third time’s the charm” or “three strikes and you’re out” may apply in this 
case as well. :-) Of course, their success depends a great deal on *us* to 
review the changes, as well.

 
 
 We also talked about the fact that the new version of Kombu includes some of 
 the features we have implemented in our own driver, like heartbeats and 
 connection management. Kombu does not include the calling patterns 
 (cast/call/notifications) that we have in oslo.messaging, but we may be able 
 to remove some code from our driver and consolidate the qpid and rabbit 
 driver code to let Kombu do more of the work for us.
 
 This sounds great. Please, whoever is going to work on this, feel add
 me to the reviews.
 
 Python 3 support is coming slowly. There are a couple of patches up for 
 review to provide a different sort of executor based on greenio and 
 trollius. Adopting that would require some application-level changes to use 
 co-routines, so it may not be an optimal solution even though it would get 
 us off of eventlet. (During the Python 3 session later in the week we talked 
 about the possibility of fixing eventlet’s monkey-patching to allow us to 
 use the new eventlet under python 3.)
 
 We also talked about the way the oslo.messaging API uses URLs to get some 
 settings and configuration options for others. I thought I remembered this 
 being a conscious decision to pass connection-specific parameters in the 
 URL, and “global” parameters via configuration settings. It sounds like that 
 split may not have been implemented as cleanly as originally intended, 
 though. We identified documenting URL parameters as an issue for removing 
 the configuration object, as well as backwards-compatibility. I don’t think 
 we agreed on any specific changes to the API based on this part of the 
 discussion, but please correct me if your recollection is different.
 
 I prefer URL parameters to specify options. As of now, I think we
 treat URL parameters and config options as two different things. Is
 this something we can change and translate URL parameters to config
 options?
 
 I guess if we get to that point, we'd end up asking ourselves: Why
 shouldn't we use just config options in that case?
 
 I think one - historical (?) - answer to that is that we once thought
 about not using oslo.config in oslo.messaging.

That’s true, and another reason was to handle cases like an arbitrary number of 
connections (ceilometer needed more than one, though I don’t remember if it was 
truly arbitrary). Normally the messaging settings in the config file are in one 
group, and the application doesn’t know the name of the group. If we use the 
config file to pull in all of the settings the application would need to know 
the group name(s) because the options themselves are (a) not part of the API 
and (b) not necessarily registered on the global config object (the library 
might use the config filter to hide the option settings). I think we decided 
that URLs were less ugly than exposing the config group names, though they do 
introduce some other complexity. 

As Josh points out in his message in this thread, the thing 

Re: [openstack-dev] [oslo] oslo.messaging outcome from the summit

2014-11-13 Thread Joshua Harlow
Don't forget my executor which isn't dependent on a larger set of changes for 
asyncio/trollious...
https://review.openstack.org/#/c/70914/
The above will/should just 'work', although I'm unsure what thread count should 
be by default (the number of green threads that is set at like 200 shouldn't be 
the same number used in that executor which uses real python/system threads). 
The neat thing about that executor is that it can also replace the eventlet 
one, since when eventlet is monkey patching the threading module (which it 
should be) then it should behave just as the existing eventlet one; which IMHO 
is pretty cool (and could be one way to completely remove the eventlet usage in 
oslo.messaging).

As for the kombu discussions, maybe its time to jump on the #celery channel 
(where the kombu folks hang out) and start talking to them about how we can 
work better together to move some of our features into kombu (and also 
depreciate/remove some of the oslo.messaging features that now are in kombu). I 
believe https://launchpad.net/~asksol is the main guy there (and also the main 
maintainer of celery/kombu?). It'd be nice to have these cross-community talks 
and at least come up with some kind of game plan; hopefully one that benefits 
both communities...

-Josh
 From: Doug Hellmann d...@doughellmann.com
 To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org 
 Sent: Wednesday, November 12, 2014 12:22 PM
 Subject: [openstack-dev] [oslo] oslo.messaging outcome from the summit
   
The oslo.messaging session at the summit [1] resulted in some plans to evolve 
how oslo.messaging works, but probably not during this cycle.

First, we talked about what to do about the various drivers like ZeroMQ and the 
new AMQP 1.0 driver. We decided that rather than moving those out of the main 
tree and packaging them separately, we would keep them all in the main 
repository to encourage the driver authors to help out with the core library 
(oslo.messaging is a critical component of OpenStack, and we’ve lost several of 
our core reviewers for the library to other priorities recently).

There is a new set of contributors interested in maintaining the ZeroMQ driver, 
and they are going to work together to review each other’s patches. We will 
re-evaluate keeping ZeroMQ at the end of Kilo, based on how things go this 
cycle.

We also talked about the fact that the new version of Kombu includes some of 
the features we have implemented in our own driver, like heartbeats and 
connection management. Kombu does not include the calling patterns 
(cast/call/notifications) that we have in oslo.messaging, but we may be able to 
remove some code from our driver and consolidate the qpid and rabbit driver 
code to let Kombu do more of the work for us.

Python 3 support is coming slowly. There are a couple of patches up for review 
to provide a different sort of executor based on greenio and trollius. Adopting 
that would require some application-level changes to use co-routines, so it may 
not be an optimal solution even though it would get us off of eventlet. (During 
the Python 3 session later in the week we talked about the possibility of 
fixing eventlet’s monkey-patching to allow us to use the new eventlet under 
python 3.)

We also talked about the way the oslo.messaging API uses URLs to get some 
settings and configuration options for others. I thought I remembered this 
being a conscious decision to pass connection-specific parameters in the URL, 
and “global” parameters via configuration settings. It sounds like that split 
may not have been implemented as cleanly as originally intended, though. We 
identified documenting URL parameters as an issue for removing the 
configuration object, as well as backwards-compatibility. I don’t think we 
agreed on any specific changes to the API based on this part of the discussion, 
but please correct me if your recollection is different.

We also learned that there is a critical bug [2] related to heartbeats for 
RabbitMQ, and we have a few patches up to propose fixes in different ways (see 
the bottom of [1]). This highlighted again the fact that we have a shortage of 
reviewers for oslo.messaging.

Doug

[1] https://etherpad.openstack.org/p/kilo-oslo-oslo.messaging
[2] https://bugs.launchpad.net/nova/+bug/856764


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

  ___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev