On 08/16/2017 04:34 PM, Ademar Reis wrote: > On Wed, Aug 16, 2017 at 02:28:47PM -0400, Cleber Rosa wrote: >> >> It's not covered by the fallback mechanism. What I mean is that Avocado >> should probably have, as one of the test parameter system providers >> which are enabled by default, one that looks for parameters in the test >> data dir. Something like: >> >> 1. test => examples/tests/sleeptest.py >> 2. test's datadir (current concept) => examples/tests/sleeptest.py.data >> 3. test's file-based param => examples/tests/sleeptest.py.data/params >> >> The code handling line 3, would be one of the test parameter system >> providers. Just one that is enabled by default. > > OK, this makes sense to me and is aligned with what I propose, we're > in sync. >
OK. >> >>>> >>>>> Users who don't want any specificity and/or have a small test base >>>>> with a low chance of clashes could simply ignore the prefix both >>>>> when creating parameters and when making calls to params.get(). >>>>> >>>> >>>> Yes. >>>> >>>>>> >>>>>> The second item will probably mean the definition of a new class to >>>>>> the ``avocado.core.plugin_interfaces`` module, together with a new >>>>>> dispatcher(-like) implementation in ``avocado.core.dispatcher``. >>>>>> >>>>>> Parameters availability: local .vs. external >>>>>> ============================================ >>>>>> >>>>>> Right now, all parameters are given to the test at instantiation time. >>>>>> Let's say that in this scenario, all parameters are *local* to the >>>>>> test. Local parameters have the benefit that the test is self >>>>>> contained and doesn't need any external communication. >>>>>> >>>>>> In theory, this is also a limitation, because all parameters must be >>>>>> available before the test is started. Even if other parameter system >>>>>> implementations are possible, with a local approach, there would be a >>>>>> number of limitations. For long running tests, that may depend on >>>>>> parameters generated during the test, this would be a blocker. Also, >>>>>> if a huge number of parameters would be available (think of a huge >>>>>> cloud or database of parameters) they would all have to be copied to >>>>>> the test at instantiation time. Finally, it also means that the >>>>>> source of parameters would need to be able to iterate over all the >>>>>> available parameters, so that they can be copied, which can be a >>>>>> further limitation for cascading implementations. >>>>>> >>>>>> An external approach to parameters, would be one that the test holds a >>>>>> handle to a broker of parameter providers. The parameter resolution >>>>>> would be done at run time. This avoids the copying of parameters, but >>>>>> requires runtime communication with the parameter providers. This can >>>>>> make the test execution much more fragile and dependent on the external >>>>>> communication. Even by minimizing the number of communication >>>>>> endpoints by communicating with the test runner only, it can still add >>>>>> significant overhead, latency and point of failures to the test >>>>>> execution. >>>>>> >>>>>> I believe that, at this time, the limitations imposed by local >>>>>> parameter availability do *not* outweigh the problems that an external >>>>>> approach can bring. In the future, if advanced use cases require an >>>>>> external approach to parameters availability, this can be reevaluated. >>>>> >>>>> If I understand your point correctly, this is an implementation >>>>> detail. It depends on what the "contract" is between the test runner >>>>> (parameter provider) and the test being run (the user of >>>>> params.get()). >>>>> >>>> >>>> It is an implementation detail, but one that will ultimately reflect on >>>> what is available to test writers. >>>> >>>>> For example, should users assume parameters are dynamic and can >>>>> change during the lifetime of a test, an therefore two identical >>>>> calls to params.get() might return different values? Should it be >>>>> possible to change params (something like params.put()) at runtime? >>>>> >>>>> (IMO the answer should be no to both questions). >>>>> >>>>> If you have something different in mind, then it would be >>>>> interesting to see some real use-cases. >>>>> >>>> >>>> Defining what users could expect is what I was trying to get to, but you >>>> were way more direct here. I mentioned that iterating through the >>>> available parameters wouldn't be possible with the "external" approach, >>>> because test writers would be affected by those choices. >>>> >>>> When I said that the "limitations imposed by local parameter >>>> availability do *not* outweigh the problems that an external approach >>>> can bring", I basically stood by the "local parameter availability" >>>> approach, which mean to users that it'd be possible to: >>>> >>>> 1) Iterate through parameters >>>> 2) Change parameters >>>> 3) Remove parameters >>>> 4) Add parameters >>>> >>>> Now, it's not because it's possible that it should be supported or >>>> encouraged. For the record, #1 is already possible with the current >>>> implementation: >>>> >>>> http://avocado-framework.readthedocs.io/en/53.0/api/core/avocado.core.html?highlight=avocadoparams#avocado.core.varianter.AvocadoParams.iteritems >>>> >>>> And so is, to some extent #4 (and depending on how you look at it, #2): >>>> >>>> http://avocado-framework.readthedocs.io/en/53.0/api/core/avocado.core.html?highlight=avocadoparams#avocado.core.varianter.Varianter.add_default_param >>>> >>> >>> Right. But these are avocado.core APIs (implementation details) that >>> should not be exposed or supported. >>> >> >> The second one is not exposed to test writers, the first one is. >> >>> I've been assuming params.get() is the only params-related API >>> exposed to test writers. Is that correct? >>> >> >> No, all the methods in AvocadoParams are available: >> >> http://avocado-framework.readthedocs.io/en/53.0/api/core/avocado.core.html?highlight=avocadoparams#avocado.core.varianter.AvocadoParams >> >> So it's possible to do things like: >> >> def test(): >> for o, k, v in self.params.iteritems(): >> self.log.debug("%s %s %s", o, k, v) >> >> Now, some clarifications: I got confused and pasted the wrong link to >> add_default_param(). That is a Varianter method, so it's *not* >> available to test writers. I got confused because I was staring at the >> AvocadoParams __init__ parameter "default_params". And and instance of >> AvocadoParams is what you get as the "self.params" on a test. >> >> Hopefully that last paragraph is not too confusing. > > You once again linked to avocado.core documentation, but the object > is available to test writers. I imagine you understand we have a > leak here: an internal object is being exposed as part of the API to > test writers. > I did link to the "avocado.core" documentation because an instance of that (internal only) class is what users currently have access to: >>> from avocado import Test >>> class MyTest(Test): def test(self): pass >>> test = MyTest() >>> test.params <AvocadoParams *: * ([]),0: /test/* ([])> > So let me change my question: > > I've been assuming params.get() is the only params-related API we > *want* (or plan) to expose to test writers. Is that correct? > That's correct. And that's why I kept bringing back the discussion about params.iteritems(). > Thanks. > - Ademar > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
signature.asc
Description: OpenPGP digital signature