Thanks for sharing this information. For the future, I think this type of analysis and discussion is something that is great to have on the mailing list instead of just a private group. I wish I had seen it sooner.
The code in nova.rpc seems useful enough that it very well may be used elsewhere. I know it's more than what is needed for notifications, but it does support what is needed for notifications (and more). I like the idea of moving the whole thing instead of having separate messaging code for just notifications. -- Russell Bryant On 04/03/2012 09:20 AM, Swaminathan Venkataraman wrote: > Hi, > I'm actively working on the notification part. I did some analysis on > the code and dependencies and was planning to submit a blueprint by end > of the week. We can use that to finalize the interface for the > notification. The rpc implementation is rich (compared to just what we > need for notifications) because nova uses it for all rpc related > communications. The idea that I was working with was to just move what > we need for notifications. In that scenario we do not really need all of > rpc in openstack-common. If we do want a common implementation that all > openstack components can use to communicate the middleware, it might > make to sense to move the whole of rpc to openstack-common. > > Thoughts? > > > Anyways, here is the analysis and some of the comments I got... > > Cheers, > Venkat > > * > > > ---------- Forwarded message ---------- > From: *Swaminathan Venkataraman* <venkat...@gmail.com > <mailto:venkat...@gmail.com>> > Date: Mon, Mar 19, 2012 at 8:31 PM > Subject: Re: [Openstack] Notifiers > To: Monsyne Dragon <mdra...@rackspace.com > <mailto:mdra...@rackspace.com>> > Cc: Mark McLoughlin <mar...@redhat.com <mailto:mar...@redhat.com>>, > Jason Kölker <jason.koel...@rackspace.com > <mailto:jason.koel...@rackspace.com>>, Jay Pipes <jaypi...@gmail.com > <mailto:jaypi...@gmail.com>> > > > I did some analysis on notifier and rpc in nova and there are a > bunch of dependencies that have to be sorted out before we can move > them to openstack-common. Here are some of the details. > > o notifier and rpc use flags, utils, logging, context, db, > exception, from nova. > o > o The modules in notfier and rpc use FLAGS from flags.py which is > an instance of NovaConfigOpts. They mainly use it to register > the config options and access them. Given that, it seems like we > could use CommonConfigOpts directly to register the options. > This will eliminate the dependency on flags and flagfile. > o > o There are three functions that are used from utils - utcnow, > import_object, and to_primitive. There is a utils in > openstack-common which already contains utcnow and > import_object. The code also macthes line to line with the > implementation in nova. The to_primitive function is missing in > openstack-common. One option could be to move this function > alone to openstack-common which should eliminate the dependency > on the nova based utils. > o > o notifier and api use log from nova. In fact they work with an > instance of NovaContextAdapter which in turn is an instance of > LoggerAdapter. NovaContextAdapter is used to pass the context, > the instance uuid, and the nova version to the logger. The > modules in openstack-common are using the python logging module > directly. So, if we need notifier to be able to print contextual > information we will have to add this functionality to > openstack-common. > o > o Both nova and openstack-common have an implementation of > RequestContext. The one in Nova is richer and both notifier and > rpc use functionality from RequestContext in nova. The other > difference is that the RequestContext in nova uses a weak > refernce store to save the context information. I did see a > couple of instances where the context information was deleted > from the store, but I'm not sure whether it is being accessed. > So, should the context in openstack-common be enhanced? > o > o db from nova is used only by capacity_notifier. It looks like it > sends events that are only related to compute manager events. > So, should this be part of openstack-common? > I've not looked at exception. I'll also have to look at rpc in more > detail. Please do let me know if this is the right direction. > > thanks, > Venkat > > ---------- Forwarded message ---------- > From: *Mark McLoughlin* <mar...@redhat.com <mailto:mar...@redhat.com>> > Date: Tue, Mar 20, 2012 at 8:05 PM > Subject: Re: [Openstack] Notifiers > To: Swaminathan Venkataraman <venkat...@gmail.com > <mailto:venkat...@gmail.com>> > Cc: Monsyne Dragon <mdra...@rackspace.com > <mailto:mdra...@rackspace.com>>, Jason Kölker > <jason.koel...@rackspace.com <mailto:jason.koel...@rackspace.com>>, Jay > Pipes <jaypi...@gmail.com <mailto:jaypi...@gmail.com>> > > ---------- Forwarded message ---------- > From: *Mark McLoughlin* <mar...@redhat.com <mailto:mar...@redhat.com>> > Date: Tue, Mar 20, 2012 at 1:25 PM > Subject: Re: [Openstack] Notifiers > To: Swaminathan Venkataraman <venkat...@gmail.com > <mailto:venkat...@gmail.com>> > Cc: Monsyne Dragon <mdra...@rackspace.com > <mailto:mdra...@rackspace.com>>, Jason Kölker > <jason.koel...@rackspace.com <mailto:jason.koel...@rackspace.com>>, Jay > Pipes <jaypi...@gmail.com <mailto:jaypi...@gmail.com>> > > > Hi Venkat, > > Could you file a bug or blueprint against openstack-common with all this > great info? > > Cheers, > Mark. > > On Tue, 2012-03-20 at 19:37 +0530, Swaminathan Venkataraman wrote: >> Sure Mark, but here is a bit more of analysis that I did. I'll file a >> blueprint because I'm not sure if this is a bug. >> >> >> There is an exception module defined in openstack-common. This has a >> class named openstackException which is similar to NovaException and I >> guess is to be subclassed to define exceptions that go in >> openstack-common. openstack-common also defines a decorator for >> wrapping methods to catch exceptions, but it does not try to send the >> exception to the notification system like the one in nova.exception >> does. Based on this we could use exceptions in openstack-common in >> notifier and rpc. We'll still have to figure out what to do with >> exceptions which are common (like ClassNotFoundexception). We'll have >> to duplicate them or modules in nova will have to import two different >> exception modules with different hierarchies, if we move common >> exceptions to openstack-common. The ideal scenario is to have one >> single hierarchy for exceptions, but I guess that this is a bigger >> decision. > > Firstly, I wouldn't consider the openstack.common.exception as > particularly complete or final. I'd be happy for us to completely > re-think the approach to exceptions in openstack-common. > > The wrap_exception() decorator could continue to live in nova for now. > > exception.ClassNotFound is only raised by utils.import_object, but > openstack-common has its own version of that method and raises its own > exception.NotFound. So, no problem there. > > I think its best to leave rabbit_notifier.py in nova for the moment. > Perhaps move it to the nova.rpc module. > >> The driver for notification is decided based on >> FALGS.notification_driver. In the current scenario, since glance and >> nova use different methods to do notifications, they can end up with >> different drivers. If we do make the notifier common and if we want to >> provide the capability for each of the openstack components to use >> different drivers, we cannot just use the driver defined in the >> configuration. One way is to pass the notifier as a parameter or >> provide the configuration to be used to get the notifier and use >> different configuration parameters for different components. > > I think we'll want the log, list, no-op and test notifiers in > openstack-common. > > The way to handle configuration is to do what glance does - have a > Notifier class which accepts a ConfigOpts as a constructor parameter. > The class defines a "strategy" or "driver" option and loads the > appropriate driver based on that option. > > IMHO, the best way to approach moving stuff like this to > openstack-common is to start with an (almost) fresh slate - don't be too > worried about what Nova or Glance currently does, but instead try and > define the best API to meet their requirements. > > Cheers, > Mark. > > > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp