This stuff all makes me happy. I think I can continue to deal with flags with these changes. If we can create a best-practices for naming of flags, where they should go, and how they fit into conf files, I think I'm content to stick with gflags.
I approved the patch, but if anyone else disagrees/has additional suggestions, speak up! On Thu, Jul 29, 2010 at 10:58 AM, Andy Smith <[email protected]> wrote: > Wow! A whole mailing list I wasn't seeing! > > A quick rundown of what it seems we want out of the configuration system: > > 1. Command-line parsing that doesn't choke as easily on flags that are > not currently defined. > 2. Ability to refer to flags that are specific to another service for > service interoperability. > 3. Ability to deal with new flags at runtime for things such as > pluggable backends. > > I'm a little sad to see #1 go just in case people typo a flag, but perhaps > that > can be fixed at a later date by doing a check to see whether a flag is > very > similarly named to another flag that we do know about. > > Anyway, I had a good idea in mind for how to solve all this so I went ahead > and made a branch: > > https://code.launchpad.net/~termie/nova/mega_flags > > The DECLARE syntax is still perhaps a little verbose in that you have to > declare > all the flags you intend to use, but I think that it is probably better > that you > say upfront everything you are expecting to use so that people reading the > file > know what you depend on. > > > > On Thu, Jul 29, 2010 at 7:19 PM, Vishvananda Ishaya <[email protected] > > wrote: > >> +termie since he wasn't subscribed >> >> On Wed, Jul 28, 2010 at 2:44 PM, Chris Behrens < >> [email protected]> wrote: >> >>> ++ >>> >>> We should list requirements and look for what we need. >>> >>> >>> On Jul 28, 2010, at 2:41 PM, Monty Taylor wrote: >>> >>> > On 07/28/2010 02:28 PM, Eric Day wrote: >>> >> ++ >>> >> >>> >> I'm all for using an existing solution if one exists. I've not looked >>> >> enough to make calls either way though. I want to figure out *what* >>> >> we are looking for in features to make those decisions. >>> > >>> > ++ >>> > >>> >> On Wed, Jul 28, 2010 at 01:37:18PM -0700, Monty Taylor wrote: >>> >>> So I know I haven't convinced everyone to love bzr yet ... but as >>> they >>> >>> are a large python project with command line and config file options >>> - >>> >>> and plugins - perhaps looking at the infrastructure/design they use >>> >>> might be a good idea? >>> >>> >>> >>> Also, the work derks did with cement might be of help. >>> >>> >>> >>> I believe both are designed to do things similar to how you are >>> >>> discussing them below (although different, of course - we're all >>> python >>> >>> devs, there's no way we're going to actually do things the same. :) ) >>> >>> >>> >>> Monty >>> >>> >>> >>> (what Eric is saying makes sense to me - but I don't have a whole >>> bunch >>> >>> of stake either way here- I am a fan of reusing solutions that exist >>> >>> where possible though of course) >>> >>> >>> >>> On 07/28/2010 01:24 PM, Eric Day wrote: >>> >>>> Hi Vish, >>> >>>> >>> >>>> If we want to keep things modular and have runtime module selection >>> >>>> like you mention, we probably need to rethink flags. Using gflags >>> >>>> may not be an option unless we can somehow make 'undefok=' a global >>> >>>> option. In other project (that was not in Python, so no code to >>> help), >>> >>>> the flow is: >>> >>>> >>> >>>> * Enforce the use of module names in the options. For example, for >>> >>>> generic queue module options use --queue.*, for libvirt module >>> >>>> options, use --libvirt.*. If we want to make this seamless, we >>> >>>> would probably need to use something else instead gflags or create >>> >>>> a wrapper to enforce the required behavior. >>> >>>> >>> >>>> * Import the core option manager, first thing that happens when >>> >>>> starting a binary. >>> >>>> >>> >>>> * Parse all options, separating each out into the modules they >>> belong >>> >>>> to. We don't know what is valid yet, but we can at least group by >>> module. >>> >>>> >>> >>>> * Load any required modules via normal 'import' lines. They can >>> verify >>> >>>> options for their module space. >>> >>>> >>> >>>> * Have some core flags that specify which modules to load, for >>> example, >>> >>>> use rabbit vs fakerabbit. Then 'import' the selected optional >>> modules. >>> >>>> >>> >>>> * As optional modules load, let them verify the module namespace >>> >>>> options just like the required modules did. >>> >>>> >>> >>>> * Any options for modules that were not loaded are just ignored. >>> >>>> >>> >>>> Thoughts on this? It has worked out quite well in the other C++ >>> project >>> >>>> for me, and with Python it would be even easier to put together. :) >>> >>>> >>> >>>> -Eric >>> >>>> >>> >>>> On Wed, Jul 28, 2010 at 11:13:40AM -0700, Vishvananda Ishaya wrote: >>> >>>>> I'm having some annoyances with gflags which I'd like to air out >>> here. >>> >>>>> Maybe we can come to a consensus about how to move forward with >>> them. I >>> >>>>> find gflags annoying in the following ways: >>> >>>>> a) flags are irritating for global settings. Settings that apply >>> to the >>> >>>>> project as a whole have to be set in multiple places so that the >>> binaries >>> >>>>> all get them properly. This can be fixed somewhat by a shared >>> flagfile. >>> >>>>> For example: >>> >>>>> /etc/nova/nova-manage.conf: >>> >>>>> --flagfile=/etc/nova/nova-common.conf # shared settings >>> >>>>> --otherflag=true #manage specific settings >>> >>>>> The problem here is that the shared settings can only include >>> settings >>> >>>>> that are imported by EVERY binary, or one of the binaries will >>> choke. So >>> >>>>> if you have a flag that 4 of 5 binaries use, you either have to >>> set it in >>> >>>>> four flagfiles or put it in common with an ugly undefok= line. >>> This all >>> >>>>> seems nasty to me. Other possibilities include moving truly >>> >>>>> common/settings related flags into the common flags.py so that >>> they are >>> >>>>> available to all binaries. It all seems a bit hackish. >>> >>>>> b) including files for flags only >>> >>>>> There are places where we need access to a flag, but we aren't >>> actually >>> >>>>> making calls in the file. Pyflakes and pylint complain about >>> unused >>> >>>>> imports. Perhaps we fix this by moving these flags into common >>> flagfile? >>> >>>>> c) dependency injection >>> >>>>> This is related to the issue above. If we are dynamically >>> loading >>> >>>>> specific drivers (for example the auth driver or a datastore >>> backend) as >>> >>>>> specified by a flag, the import is often done later than the >>> parent file >>> >>>>> is imported. Therefore using flags to configure settings for the >>> driver >>> >>>>> will fail, because the binary recognizing the flags is dependent >>> on the >>> >>>>> file that contains the flags being imported. Workarounds here >>> include >>> >>>>> finding a different method for dependency injection, hacking >>> flags to >>> >>>>> search for flags in injected dependencies somehow, or configuring >>> drivers >>> >>>>> differently than the rest of the system. >>> >>>>> So I see 3 options for moving forward >>> >>>>> 1) ditch gflags completely and use a different method for >>> specifying >>> >>>>> settings >>> >>>>> 2) use a combination of some kind of settings file for general >>> >>>>> configuration, and flags for specific runtime settings/hacks >>> >>>>> 3) find good standard practices/workarounds for the above issues >>> >>>>> Thoughts? >>> >>>>> Vish >>> >>>> >>> >>>>> _______________________________________________ >>> >>>>> Mailing list: https://launchpad.net/~nova >>> >>>>> Post to : [email protected] >>> >>>>> Unsubscribe : https://launchpad.net/~nova >>> >>>>> More help : https://help.launchpad.net/ListHelp >>> >>>> >>> >>>> >>> >>>> _______________________________________________ >>> >>>> Mailing list: https://launchpad.net/~nova >>> >>>> Post to : [email protected] >>> >>>> Unsubscribe : https://launchpad.net/~nova >>> >>>> More help : https://help.launchpad.net/ListHelp >>> >>>> >>> >> >>> > >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~nova >>> > Post to : [email protected] >>> > Unsubscribe : https://launchpad.net/~nova >>> > More help : https://help.launchpad.net/ListHelp >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~nova >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~nova >>> More help : https://help.launchpad.net/ListHelp >>> >> >> >
_______________________________________________ Mailing list: https://launchpad.net/~nova Post to : [email protected] Unsubscribe : https://launchpad.net/~nova More help : https://help.launchpad.net/ListHelp
