[Openstack] quantum-rootwrap
https://review.openstack.org/#/c/11524/ Patches posted. Review solicited. Thanks in advance... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] quantum-rootwrap
Hi Dan and ttx. I've posted what I sincerely hope is the last rev of changes for bug https://bugs.launchpad.net/quantum/+bug/1037815 I believe this addresses the last round of cleanups and questions, with the exception of amotoki's request for other agent support, eg L3. Since I don't have a good way to test that, I'd like to lobby for getting the fixes to the mechanism in, and treating extra agent cleanup as further bugs to be fixed later. I don't believe I'm breaking anything with the current patch, so it wouldn't make anything worse to install it. Trying to get all this sorted before end of week. Feedback appreciated. TIA... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: j...@redhat.com >Date: Tue, 14 Aug 2012 20:23:52 -0400 > >>From: Dan Wendlandt >>Date: Tue, 14 Aug 2012 15:22:31 -0700 >> >> >>jrd, my feeling is that we'd need a patch for this under review this > week to understand the >>magnitude of the changes if we want to consider if for a > feature-freeze exception. Thanks. > >Got it. > > Update: Dan and ttx, Gary has uploaded a patch set addressing the fix of quantum-rootwrap for me, until I finish getting my credentials sorted out so that I can push them myself. See https://review.openstack.org/11472 There's a couple of review comments, which I will address. What's your inclination? Have we come close enough to the deadline to justify getting this in for f-3, or should we hold off and treat it as a bug later? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Dan Wendlandt >Date: Tue, 14 Aug 2012 15:22:31 -0700 > >On Tue, Aug 14, 2012 at 1:54 AM, Thierry Carrez > wrote: > >Dan Wendlandt wrote: >> On Mon, Aug 13, 2012 at 12:51 PM, Vishvananda Ishaya >> mailto:vishvana...@gmail.com>> wrote: >> >> This is up to dan, I suppose, but the rootwrap stuff seems like >> something worth granting a ffe to… >> >> >> I wasn't going to mention it, as the urgency of a nearby deadline > can be >> helpful :) >> >> But yes, I'd grant an ffe to something this important, especially >> because it applies across all uses of quantum. > >On one hand it's a change that impacts almost all use cases, so >definitely not something that is simple or self-contained. On the > other, >it's quite easy to trace back issues to this. In summary, if it's the >only exception in Quantum, it's not really a problem :) > > [warning: a trick is included in the last paragraph] > >ttx, I caught it I'm on to your project management jedi mind tricks :) > >jrd, my feeling is that we'd need a patch for this under review this week > to understand the >magnitude of the changes if we want to consider if for a feature-freeze > exception. Thanks. Got it. I apologise that this is taking longer than I hoped. I'm still spending more time on learning curve than getting real productive stuff done. I suppose some of that's to be expected, but still. I spent most of today getting my test env up to snuff, getting more unit test infra written, and then trying to work out how to debug the unit tests. My python-debugging techniques are rusty, so I have to feel my way through. I'll work with the local RH contingent tomorrow. If I can get my new unit tests to behave, I *hope* to be able to run the rest of the suite through in relatively short time.If you (Dan and ttx) start feeling like we're just too tight, please say so, and we'll work out plan B. Until then, I'm going to keep banging on this to pull it together. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: j...@redhat.com >Date: Fri, 10 Aug 2012 11:52:49 -0400 [...] >Very much, thanks. More news as it happens... Here's where I've got to so far I've ported/transliterated code from nova/cinder to manage rootwrap filter defs the same way in quantum. I've plowed through most of the quantum filter defs which were embedded in the agent code, and changed them to newer format, in /etc/quantum/rootwrap.d/* Current headache is getting my test environment back to working condition, and then contriving enough tests to prove that the code changes are working. Once I get that done, I'll do a cleanup pass and get a changeset posted for review. We're getting close to the tomorrow deadline. I will work with Gary and Bob and Chris to try to get this stuff nailed ASAP, or figure out plan B if it looks like that's just too much of a stretch. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Thierry Carrez >Date: Fri, 10 Aug 2012 17:38:52 +0200 > >j...@redhat.com wrote: >> Apologies for the not-very coherent description. Please let me know >> if you think I'm off in the weeds or missing important bits. > >One other thing I spotted when I evaluated how broken quantum-rootwrap >was is at quantum/agent/linux/dhcp.py:181 where a command is called with >environment set. That works with sudo but rootwrap clears the >environment. This call therefore needs a specific filter (and probably >never worked with rootwrap). Hmmm. I hadn't even considered that. Ok, I'll keep that in mind as I'm in there. > >There is one filter defined for dnsmasq (the DnsmasqFilter which is >defined at quantum/rootwrap/filters.py:71) but it sets a slightly >different set of env variables. So it might need to be adapted (or the >call adjusted). > Fair enough. I don't even know what that does yet, so will need to work that out and figure out how to adjust. >Hope this helps. Very much, thanks. More news as it happens... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Thierry Carrez >Date: Thu, 09 Aug 2012 16:32:23 +0200 > [...] > >> My goal is by end of today , or tomorrow morning latest, to have at >> least a reasonably complete understanding of the changes necessary to >> get the quantum-rootwrap facility up to parity with nova/cinder. If I >> get to that deadline and I'm not there, I'll probably punt, as it >> becomes too much of a hail-mary to get the stuff stabilized and >> reviewed etc by tues. > >That sounds reasonable. > Ok, here's what I think I know, and what I propose to do with it: Fix quantum/bin/quantum-rootwrap to mimic changes to nova/cinder w/r/t conf file. This will introduce the notion of /etc/quantum/rootwrap.conf and allow for specifying path to filter specs. Fix quantum/rootwrap/filters.py likewise; update KillFilter (maybe more?) Fix quantum/rootwrap/wrapper.py likewise; load from files and construct filter datastructures Create etc/quantum/quantum-rootwrap.conf etc/quantum/rootwrap.d/ Move the filter specs from the various agent pieces in quantum/rootwrap to matching files in etc/quantum/filters.d. Update them while I'm at it. This probably means that those files in quantum/rootwrap go away. Alternate implementation: Collect all those pieces into a consolidated quantum.filters file and stick that in there. There appears to be no analog of nova/tests/test_nova_rootwrap.py for quantum, so I'll likely need to write something for that. It looks like the various .ini files in etc/quantum/plugins all set root helper for each agent. Keep that structure for now, revisit later. That likely means I'll need a way to propagate the a default root helper setting from the conf to each agent. Devstack appears to frob configs in nova and cinder, but copy the quantum configs verbatim. So I'm hoping I can get away without modifying devstack. Things I don't know yet: Python compatibility? I'm running 2.7; I don't believe anything I'm doing would break in earlier ones, but I gather that that will need to be tested before I'm done. Will I need to hair up the filter match code? I don't think so, but I haven't gotten enough working yet to tell. Hoping I can leave it as is. Apologies for the not-very coherent description. Please let me know if you think I'm off in the weeds or missing important bits. Thanks in advance... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Thierry Carrez >Date: Thu, 09 Aug 2012 16:32:23 +0200 > >j...@redhat.com wrote: >>>* Switch to rootwrap_config and deprecate root_helper >>>This would fully align quantum-rootwrap with nova-rootwrap. However > I'm >>>not sure it's reasonable to deprecate root_helper=sudo in Folsom, > given >>>how little tested quantum-rootwrap seems to be on Folsom. Maybe just >>>introducing rootwrap_config but leaving the deprecation message out ? >>>You can have a look at: >>> > https://github.com/openstack/cinder/commit/2b2c97eb5ca332ce7d1f83e4fd2e81fabe0acb66 >>> >> >> Ok. I did talk through this issue with Bob yesterday, but I'd be >> lying if I said I understood it all yet. >> >> Let me ask this: Since, as you say, there's not a lot of evidence of >> traffic through quantum-rootwrap, is there an obvious downside to >> deprecating root_helper=sudo at this stage? I'm not advocating either >> way, just trying to get up to speed on all the parts of the issue. > >Well, since there is not a lot of evidence of traffic through the >rootwrap, that means almost everyone is using root_helper=sudo. Marking >it deprecated, and recommending that everyone switches to the (untested >yet) rootwrap doesn't sound that much like a great idea. > Ah. Ok, got it. >I think we should deprecate root_helper=sudo when we are confident that >most people are using rootwrap and are satisfied with it. > Yes, concur. Thanks. Onward... ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Thierry Carrez >Date: Thu, 09 Aug 2012 10:34:17 +0200 > >j...@redhat.com wrote: >>> From: Dan Wendlandt >>> If someone (Bob?) has the immediate cycles to make rootwrap work in > Folsom with low to medium >>> risk of disruption, I'd be open to exploring that, even if it meant > inconsistent usage in quantum >>> vs. nova/cinder. >> >> Hi Dan. I've been working with Bob, getting myself up to speed on >> quantum. I've just talked it over with Bob, and I'll take a crack at >> this one. My approach is going to be to get the quantum rootwrap >> stuff up to parity with nova. It sounded like some further work might >> get done in this area for Grizzly, but for the short term, this ought >> to be fairly non-disruptive. > >There are a number of changes: > >* Switch to configuration-based filters >This should be relatively straightforward, although Quantum makes use of >root_helper in *many* more places than Nova/Cinder do. You can have a >look at: > > https://github.com/openstack/cinder/commit/d2d3c9cba4a647724f75c036a1985a10c966da35 Yes, I believe that's one of the changesets I've already been looking at. Glad to know I'm not off in the weeds :-) > >* Switch to rootwrap_config and deprecate root_helper >This would fully align quantum-rootwrap with nova-rootwrap. However I'm >not sure it's reasonable to deprecate root_helper=sudo in Folsom, given >how little tested quantum-rootwrap seems to be on Folsom. Maybe just >introducing rootwrap_config but leaving the deprecation message out ? >You can have a look at: > > https://github.com/openstack/cinder/commit/2b2c97eb5ca332ce7d1f83e4fd2e81fabe0acb66 > Ok. I did talk through this issue with Bob yesterday, but I'd be lying if I said I understood it all yet. Let me ask this: Since, as you say, there's not a lot of evidence of traffic through quantum-rootwrap, is there an obvious downside to deprecating root_helper=sudo at this stage? I'm not advocating either way, just trying to get up to speed on all the parts of the issue. >* Add missing filters, fix incomplete ones >You have to audit all uses of root_helper and add the corresponding >filter. In some cases the filter is there but the parameters are wrong >(kill, missing -HUP as an allowed signal). I also spotted one call that >sets environment before calling root_helper: that needs to use a >specific filter since rootwrap filters the environment out (see how >DnsmasqFilter works). > Ok. I haven't seen those, or didn't know what I was looking at, but I'll keep attention out for that stuff. >* Testing >The fact that nobody filed bugs around quantum-rootwrap being unusable >tends to show nobody actually uses Quantum with it (hence my suggestion >to remove it). If we are to ship that option, it needs to be tested one >way or another. Yes. Honestly, this is the part which I feel most unsure about. But I've decided to try to get my head around the code first, and then understand the testing implications. I will doubtless have more questions about that. > >I don't think it would be that disruptive (given that quantum-rootwrap >doesn't really work right now anyway). It is, however, a significant >amount of work to complete before the F3 cut Tuesday at end of day. >Corner-case missing filters can be treated as bugs post-F3 though. > Ok, understood. My goal is by end of today , or tomorrow morning latest, to have at least a reasonably complete understanding of the changes necessary to get the quantum-rootwrap facility up to parity with nova/cinder. If I get to that deadline and I'm not there, I'll probably punt, as it becomes too much of a hail-mary to get the stuff stabilized and reviewed etc by tues. >I'm available to help you and answer any question on the design of the >rootwrap you may have. Thanks very much. I will certainly have more questions as I proceed. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Quantum] Removing quantum-rootwrap
>From: Dan Wendlandt >Date: Wed, 8 Aug 2012 10:28:37 -0700 > >On Wed, Aug 8, 2012 at 9:22 AM, Thierry Carrez > wrote: > >Robert Kukura wrote: >> On 08/08/2012 09:31 AM, Thierry Carrez wrote: >>> Quantum currently contains bin/quantum-rootwrap, a copy of > nova-rootwrap >>> supposed to control its privilege escalation to run commands as > root. >>> >>> However quantum-rootwrap is currently non-functional, missing a lot > of >>> filter definitions that are necessary for it to work correctly. >> >> Is missing definitions the only issue? Those may need updating for > F-3, >> but this can certainly be done. > >Those are the only issues I spotted. Making Quantum compatible with the >latest version of rootwrap as shipped in Nova/Cinder, though, is a lot >more work. > >>> Quantum >>> is generally run with root_helper=sudo and a wildcard sudoers file. >> >> What is your basis for this statement? The packaging of Essex Quantum >> for Fedora and RHEL/EPEL do configure root_helper to use >> quantum-rootwrap. If another distribution doesn't do this, I would >> consider that a distribution bug, not an upstream problem. > >Given that quantum-rootwrap is currently non-working, I suspected that >everyone running Quantum *on Folsom* was using sudo and not the >rootwrap. If most people do that, it probably means it's a it early to >deprecate root_helper=sudo support in Folsom. > >>> That >>> means Quantum is not ready to deprecate in Folsom (and remove in >>> Grizzly) its ability to run with root_helper=sudo, like Nova and > Cinder do. >> >> What's involved in deprecating this ability in Folsom? Is it that >> difficult? If Nova and Cinder are doing it, why shouldn't Quantum? > >As a quick grep will show, there is much more adherence to root_helper >in Quantum than in Nova/Cinder, where it was used in a single place. >It's definitely doable, but I'd say a bit dangerous (and too late) 4 >days before F3. I certainly won't have enough time for it... > >> I do have an issue with Folsom dropping a capability that is being > used >> in Essex. If the existing rootwrap really does more harm than good, > this >> might be justified, but I don't think you can argue nobody has used > it. > >Fair point, it was definitely used in Essex. > >We have three options at this point: > >* Remove it (but is it acceptable to "lose" functionality compared to >Essex, even if Essex is not a "core" release for Quantum ?) > >* Just fix it by adding missing filters (but then accept that >quantum-rootwrap doesn't behave like nova-rootwrap and cinder-rootwrap, >which is bad for consistency) > >* Align quantum-rootwrap with nova-rootwrap and deprecate usage of >root_helper, by overhauling how root_helper is pervasively used >throughout Quantum code (lots of work, and introducing a lot of >disruption that late in the cycle) > >Personally I think only the first two options are realistic. So this >boils down to losing functionality from Essex vs. hurting Folsom core >consistency. > >If someone (Bob?) has the immediate cycles to make rootwrap work in Folsom > with low to medium >risk of disruption, I'd be open to exploring that, even if it meant > inconsistent usage in quantum >vs. nova/cinder. > Hi Dan. I've been working with Bob, getting myself up to speed on quantum. I've just talked it over with Bob, and I'll take a crack at this one. My approach is going to be to get the quantum rootwrap stuff up to parity with nova. It sounded like some further work might get done in this area for Grizzly, but for the short term, this ought to be fairly non-disruptive. >I also think we need to develop basic guidelines that should be enforced > by reviewers with >respect to correctly using rootwrap moving forward. Is there a quick > pointer we have for >developers and reviewers to use? > >Dan > > > >-- >Thierry Carrez (ttx) >Release Manager, OpenStack > >___ >Mailing list: https://launchpad.net/~openstack >Post to : openstack@lists.launchpad.net >Unsubscribe : https://launchpad.net/~openstack >More help : https://help.launchpad.net/ListHelp > >-- >~~~ >Dan Wendlandt >Nicira, Inc: www.nicira.com >twitter: danwendlandt >~~~ > > >--
Re: [Openstack] [DELTACLOUD-INTERNAL] Plan for adding RHEVm and vSphere support to Cloud-Init
>From: jvlcek >Date: Wed, 18 Jul 2012 15:42:33 -0400 > >On 07/18/2012 03:00 PM, Itamar Heim wrote: >> On 07/18/2012 09:47 PM, jvlcek wrote: >>> On 07/18/2012 01:47 PM, Itamar Heim wrote: >>>> On 07/17/2012 10:03 PM, jvlcek wrote: > > I just wanted to let folks know what is happening regarding cloud-init > support for RHEVm and vSphere and what OSs. > > Currently cloud-init only supports Ec2. A version is available > for Fedora and EPEL. > > I am working on updating upstream to also support sourcing > user data for RHEVm and vSphere, using the same technique we do > with Audrey from a Delta-Cloud launch. >>>> >>>> Can you please provide some more background on this? >>>> which feature set of cloud-init do you see this covering and how would >>>> the information will be passed (I assume some would be via the vm >>>> payload). >>>> but going forward, i was envisioning for ovirt/rhev to better >>>> integrate with cloud-init via something like the meta data service (vm >>>> payload of type network). >>>> >>>> so there is custom vm payload, which integration via the current >>>> payload mechanism would be great (since it will be the userdata for >>>> network payload as well). >>>> but for some things, iiuc cloud-init correctly, they are not supposed >>>> to be via userdata, rather via specific named fields. >>>> >>>> Thanks, >>>> Itamar >>>> > > Once I have this pushed upstream I plan to push it to Fedora and then > out to RHEL targeting release 6.4 > > See: BZ 838659 - https://bugzilla.redhat.com/show_bug.cgi?id=838659 > > Please let me know if this plan conflicts with any known needs or > if I am on the right track. >>>> >>> >>> >>> I am adding a new data source to cloud-init allowing user data to be >>> picked up by >>> cloud-init running on a launching RHEVm and vSphere instance the same >>> way we >>> currently do for Audrey as launched from DeltaCloud. >> >> why are you using the floppy method for rhev and not the iso one? >> (I hope you are using 3.1 vm payload feature rather than the custom >> hook approach)? >> >>> >>> That being: on RHEVm the user data is made available via floppy and on >>> vSphere >>> via a cdrom. >>> >>> I am not familiar with what you are envisioning for ovirt/rhev. How soon >>> would that >>> ovirt/rhev pieces be available? Perhaps we should discuss enhancing what >>> I am doing >>> and when that could happen. >> >> this will take at least a few months. > >MFojtik and/or Lutter; > see below question regarding Delta-Cloud support for RHEVm3.1 > >Itamar, > >I have been using the floppy method as the 3.1 iso method was not available >when I started working on this. > >A RHEVm3.1 test system was set up just a few days ago in Westford >which I could start to use for testing the iso method. > >Do you think it makes sense to support both or just move to supporting only >3.1 iso method for Cloud-Init? My 2 kopeks is that we should go with what we got now (3.0 support) then treat 3.1 as an enhancement, possibly fairly soon. Unless somebody (Itamar?) thinks that 3.0 will dissapear entirely once 3.1 is available, we're going to want it to work with both anyhow. This way, you can get the support for RHEV and Vsphere in, and stabilized, then deal with the enhancement later. > >Is support for the RHEVm3.1 iso method available in Delta-Cloud? >(cc-ing Delta-Cloud) > >Can you point me at examples of how the instance will access the data >from the 3.1 iso method? > >Thank you for the help and input! >Joe > > > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
