Re: [Openstack] fake_flags + common breakout
On 05/04/2012 12:45 PM, Eric Windisch wrote: >> >> I guess another question is, why do you need to set ZeroMQ related flags >> in fake_flags? I think those should only be settings that apply for >> *all* unit tests. I would just register your flags in your unit tests. >> >> https://github.com/openstack/nova/blob/master/nova/tests/rpc/test_qpid.py#L69 > > The fake_rabbit flag doesn't, but this is otherwise a good point. For > now, I've just hard-coded the flag into the test. Yeah, the fake_rabbit flag used to apply to a lot more than it does now. It was applicable to all tests before nova.rpc.impl_fake was added during Essex development. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] fake_flags + common breakout
> > I guess another question is, why do you need to set ZeroMQ related flags > in fake_flags? I think those should only be settings that apply for > *all* unit tests. I would just register your flags in your unit tests. > > https://github.com/openstack/nova/blob/master/nova/tests/rpc/test_qpid.py#L69 > > > The fake_rabbit flag doesn't, but this is otherwise a good point. For now, I've just hard-coded the flag into the test. The specific flag was to set the MatchMaker. I was forcing messages to use a "Localhost" matchmaker. I could override the flag from the test itself, but there might be reasons why someone would want to override this. In fact, I have overridden this to test the various matchmakers. It is also more transparent. To be honest, this specific requirement will be much lessened once I write independent tests for the matchmaker classes. However, it would also be *nice* to be able to override various settings to test them without modifying the modules or tests themselves. I'd like if I could bump up/down the number of ZeroMQ IO threads to use during tests (it defaults to 1). While modifying the test/module isn't much worse than modifying fake_flags, I find it cleaner. -- Eric Windisch___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] fake_flags + common breakout
On 05/04/2012 12:16 PM, Eric Windisch wrote: > I added a flag in my branch which uses a flag defined in the ZeroMQ driver. > > This was fine before, but is a problem now, because tests/__init__.py > doesn't do rpc.register_opts. Instead, it does > FLAGS.register_opts(rpc.rpc_opts), which only loads the common RPC flags. > > Of course, the flags file *defines* the RPC driver to be used, so there > would be no benefit of running rpc.register_opts until after fake_flags > is loaded. > > The easiest solutions I see are: > A. load fake_flags.py, then rpc.register_opts, then run a fake_rpc_flags.py. > B. import rpc, add rpc.register_opts to fake_flags.py, then add any > flags we want. > C. not support testing flags on RPC drivers, have a common "testing" flag. I guess another question is, why do you need to set ZeroMQ related flags in fake_flags? I think those should only be settings that apply for *all* unit tests. I would just register your flags in your unit tests. https://github.com/openstack/nova/blob/master/nova/tests/rpc/test_qpid.py#L69 It's kind of a hack, but hopefully we can kill this whole register_opts() thing eventually anyway, as I mentioned in another message. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] fake_flags + common breakout
On 05/04/2012 12:08 PM, Russell Bryant wrote: > On 05/04/2012 11:53 AM, Eric Windisch wrote: >> Russell, >> >> FYI, with the flags patch, it is no longer possible to set >> rpc-implementation dependent flags in fake_flags. >> >> Even Rabbit has a flag in there (fake_rabbit), but Rabbit flags are >> currently global, so it works. That won't be true for long… We're going >> to have to fix this. >> >> One option is to initialize the RPC layer from fake_flags.py and then >> set the options. > > Each place that fake_flags is imported, rpc gets initialized first. See > these lines of code, and the fake_flags import shortly after: > > https://github.com/openstack/nova/blob/master/nova/tests/__init__.py#L63 > https://github.com/openstack/nova/blob/master/bin/nova-dhcpbridge#L103 One other related note ... I heard some rumblings from markmc about getting all projects to standardize on a global config object from openstack.common.cfg. If that happens in nova, then we can get rid of this nova.rpc.register_opts() business completely. >> Another option, for now, might just be to push this problem into the >> implementations and have a global "testing" flag that is >> implementation/backend independent. This is uglier on the >> implementation/driver side, but cleaner on the unit tests. This is >> basically what 'fake_rabbit' is now, anyway. > > As far as I can tell, this isn't actually a problem with the uses of > fake_flags right now. What problem did you hit? -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] fake_flags + common breakout
I added a flag in my branch which uses a flag defined in the ZeroMQ driver. This was fine before, but is a problem now, because tests/__init__.py doesn't do rpc.register_opts. Instead, it does FLAGS.register_opts(rpc.rpc_opts), which only loads the common RPC flags. Of course, the flags file *defines* the RPC driver to be used, so there would be no benefit of running rpc.register_opts until after fake_flags is loaded. The easiest solutions I see are: A. load fake_flags.py, then rpc.register_opts, then run a fake_rpc_flags.py. B. import rpc, add rpc.register_opts to fake_flags.py, then add any flags we want. C. not support testing flags on RPC drivers, have a common "testing" flag. -- Eric Windisch On Friday, May 4, 2012 at 6:08 PM, Russell Bryant wrote: > On 05/04/2012 11:53 AM, Eric Windisch wrote: > > Russell, > > > > FYI, with the flags patch, it is no longer possible to set > > rpc-implementation dependent flags in fake_flags. > > > > Even Rabbit has a flag in there (fake_rabbit), but Rabbit flags are > > currently global, so it works. That won't be true for long… We're going > > to have to fix this. > > > > One option is to initialize the RPC layer from fake_flags.py and then > > set the options. > > > > > Each place that fake_flags is imported, rpc gets initialized first. See > these lines of code, and the fake_flags import shortly after: > > https://github.com/openstack/nova/blob/master/nova/tests/__init__.py#L63 > https://github.com/openstack/nova/blob/master/bin/nova-dhcpbridge#L103 > > > Another option, for now, might just be to push this problem into the > > implementations and have a global "testing" flag that is > > implementation/backend independent. This is uglier on the > > implementation/driver side, but cleaner on the unit tests. This is > > basically what 'fake_rabbit' is now, anyway. > > > > > As far as I can tell, this isn't actually a problem with the uses of > fake_flags right now. What problem did you hit? > > -- > Russell Bryant > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] fake_flags + common breakout
On 05/04/2012 11:53 AM, Eric Windisch wrote: > Russell, > > FYI, with the flags patch, it is no longer possible to set > rpc-implementation dependent flags in fake_flags. > > Even Rabbit has a flag in there (fake_rabbit), but Rabbit flags are > currently global, so it works. That won't be true for long… We're going > to have to fix this. > > One option is to initialize the RPC layer from fake_flags.py and then > set the options. Each place that fake_flags is imported, rpc gets initialized first. See these lines of code, and the fake_flags import shortly after: https://github.com/openstack/nova/blob/master/nova/tests/__init__.py#L63 https://github.com/openstack/nova/blob/master/bin/nova-dhcpbridge#L103 > Another option, for now, might just be to push this problem into the > implementations and have a global "testing" flag that is > implementation/backend independent. This is uglier on the > implementation/driver side, but cleaner on the unit tests. This is > basically what 'fake_rabbit' is now, anyway. As far as I can tell, this isn't actually a problem with the uses of fake_flags right now. What problem did you hit? -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
