This depends on what initialize is supposed to be doing. If it's just a one-time sync with a back-end, then I think calling it once in each child process might not be what we want.
I left a comment on Terry's patch. I think we should just use the callback manager to have a pre-fork and post-fork even to let drivers/plugins do whatever is appropriate for them. On Mon, Jun 8, 2015 at 1:00 PM, Robert Kukura <kuk...@noironetworks.com> wrote: > From a driver's perspective, it would be simpler, and I think sufficient, > to change ML2 to call initialize() on drivers after the forking, rather > than requiring drivers to know about forking. > > -Bob > > > On 6/8/15 2:59 PM, Armando M. wrote: > > Interestingly, [1] was filed a few moments ago: > > [1] https://bugs.launchpad.net/neutron/+bug/1463129 > > On 2 June 2015 at 22:48, Salvatore Orlando <sorla...@nicira.com> wrote: > >> I'm not sure if you can test this behaviour on your own because it >> requires the VMware plugin and the eventlet handling of backend response. >> >> But the issue was manifesting and had to be fixed with this mega-hack >> [1]. The issue was not about several workers executing the same code - the >> loopingcall was always started on a single thread. The issue I witnessed >> was that the other API workers just hang. >> >> There's probably something we need to understand about how eventlet can >> work safely with a os.fork (I just think they're not really made to work >> together!). >> Regardless, I did not spent too much time on it, because I thought that >> the multiple workers code might have been rewritten anyway by the pecan >> switch activities you're doing. >> >> Salvatore >> >> >> [1] https://review.openstack.org/#/c/180145/ >> >> On 3 June 2015 at 02:20, Kevin Benton <blak...@gmail.com> wrote: >> >>> Sorry about the long delay. >>> >>> >Even the LOG.error("KEVIN PID=%s network response: %s" % >>> (os.getpid(), r.text)) line? Surely the server would have forked before >>> that line was executed - so what could prevent it from executing once in >>> each forked process, and hence generating multiple logs? >>> >>> Yes, just once. I wasn't able to reproduce the behavior you ran into. >>> Maybe eventlet has some protection for this? Can you provide small sample >>> code for the logging driver that does reproduce the issue? >>> >>> On Wed, May 13, 2015 at 5:19 AM, Neil Jerram <neil.jer...@metaswitch.com >>> > wrote: >>> >>>> Hi Kevin, >>>> >>>> Thanks for your response... >>>> >>>> On 08/05/15 08:43, Kevin Benton wrote: >>>> >>>>> I'm not sure I understand the behavior you are seeing. When your >>>>> mechanism driver gets initialized and kicks off processing, all of that >>>>> should be happening in the parent PID. I don't know why your child >>>>> processes start executing code that wasn't invoked. Can you provide a >>>>> pointer to the code or give a sample that reproduces the issue? >>>>> >>>> >>>> https://github.com/Metaswitch/calico/tree/master/calico/openstack >>>> >>>> Basically, our driver's initialize method immediately kicks off a green >>>> thread to audit what is now in the Neutron DB, and to ensure that the other >>>> Calico components are consistent with that. >>>> >>>> I modified the linuxbridge mech driver to try to reproduce it: >>>>> http://paste.openstack.org/show/216859/ >>>>> >>>>> In the output, I never received any of the init code output I added >>>>> more >>>>> than once, including the function spawned using eventlet. >>>>> >>>> >>>> Interesting. Even the LOG.error("KEVIN PID=%s network response: %s" % >>>> (os.getpid(), r.text)) line? Surely the server would have forked before >>>> that line was executed - so what could prevent it from executing once in >>>> each forked process, and hence generating multiple logs? >>>> >>>> Thanks, >>>> Neil >>>> >>>> The only time I ever saw anything executed by a child process was >>>>> actual >>>>> API requests (e.g. the create_port method). >>>>> >>>> >>>> >>>> >>>> On Thu, May 7, 2015 at 6:08 AM, Neil Jerram < >>>>> neil.jer...@metaswitch.com >>>>> <mailto:neil.jer...@metaswitch.com>> wrote: >>>>> >>>>> Is there a design for how ML2 mechanism drivers are supposed to >>>>> cope >>>>> with the Neutron server forking? >>>>> >>>>> What I'm currently seeing, with api_workers = 2, is: >>>>> >>>>> - my mechanism driver gets instantiated and initialized, and >>>>> immediately kicks off some processing that involves communicating >>>>> over the network >>>>> >>>>> - the Neutron server process then forks into multiple copies >>>>> >>>>> - multiple copies of my driver's network processing then continue, >>>>> and interfere badly with each other :-) >>>>> >>>>> I think what I should do is: >>>>> >>>>> - wait until any forking has happened >>>>> >>>>> - then decide (somehow) which mechanism driver is going to kick off >>>>> that processing, and do that. >>>>> >>>>> But how can a mechanism driver know when the Neutron server forking >>>>> has happened? >>>>> >>>>> Thanks, >>>>> Neil >>>>> >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> < >>>>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Kevin Benton >>>>> >>>>> >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> >>> >>> -- >>> Kevin Benton >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Kevin Benton
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev