Hi Ronny, Benjamin, all,
On Tue, Sep 29, 2009 at 00:33 +0200, Ronny Pfannschmidt wrote:
> Hi,
>
> after some refactoring in the tests for the new serializer and some chat
> with Benjamin it got appearant that some form of chaining the
> pytest_generate_tests is helpfull and necessart.
I am missing concise code examples to showcase what is missing and to
understand concretely the issues and what you propose. From what
i understand - for a function like this:
def test_function(arg1, arg2):
...
it's currently not easily possible to have a funcarg generator
for arg1 and arg2 independently. I believe that this could be
helped through a new attribute on the request object, like:
for call in request.calls:
# use call.id, call.param, call.funcargs
request.addcall(...)
If both the 'arg1' and 'arg2' generator are implenented
this way they would collaborate to create m*n calls of
the function.
Another bit maybe is how to explicitely set funcarg factories, something
like e.g.:
@py.test.mark.genfuncarg(arg1=arg1factory, arg2=arg2factory)
def test_function(arg1, arg2):
...
This can be played with today through a custom pytest_generate_tests hook.
I think the factories should be called with the request object.
Experimentation and real-life usage is neccessary to settle details.
It'd be great if we settled best/proven practises for
parametrized/generated tests and did a plugin with docs on it.
Any other comments? Or am i missing your points entirely?
cheers,
holger
> Since there are various details lurking around,
> this is a initial try to collect the needs for propperly chaining
> generate_tests calls.
> I will handle each aspect in a separate section.
> For each aspect i will add all items i currenly consider
> intresting/usefull, that might be incomplete and/or more than
> reasonable.
>
> Necessary Scopes
> ----------------
>
> conftest/plugin
> Required to allow comfortable whole-project generators
> package/module
> the same goes for sub-groups of the tests
> functions
> doing special cases at the module level hook tends to be a mess
> see the current serializer tests for reasons
note that you can also have funcargs at class level.
> Additional dimensions of variation that could proof usefull
> ------------------------------------------------------------
>
> funcargs
> choose test generators based on the arguments a function explicitly
> takes,
> implicitly used funcargs via request.getfuncargvalue cant be
> considered there
> * marks
> using decorators to add values and the way to add them to the call
>
>
> Chaining order
> --------------
>
> conftest plugin -> package -> subpackage -> module -> function
>
>
> building the test id's
> -------------------------
>
> for just chaining the tests one can simply append to a tuple of id's for
> each successive add_call invokation per hook
>
> if variation on funcargs is considered one could show those like
> additional keyword parameters
>
> funcarg generators shouldn't support chaining themself, just use the
> closest matching test generator
>
> dealing with the param argument of addcall
> -------------------------------------------
>
> * provide the value of the most close scope to request objecs
> * previous values of params could be accessible via linking to the
> precedessor of the current call or by providing a list ordered by scope
> * for funcarg variation it might be helpfull to supple the funcarg
> function with the param the funcarg related generator created
>
>
> dealing with the funcargs argument of addcall
> ---------------------------------------------
>
> * values from more close scope should override vales for less close
> scope
> * warnings could be printed for name-clashes/overrides
>
>
> how to actually combine the generators
> --------------------------------------
>
> Note: early never released versions of the generate_tests feature
> provided implied combinations of all matching test generators, however
> that was fragile and got dumped for 'match most closest hook'
>
> i thenk we should prefer a explicit combination where for each addcall
> of an outer scope, the complete generate_tests function of the inner
> scope has to be run
>
> this results in a clean tree of invocations and better controll over the
> actual combinations
>
>
> regards Ronny Pfannschmidt
>
> _______________________________________________
> py-dev mailing list
> [email protected]
> http://codespeak.net/mailman/listinfo/py-dev
>
--
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu
_______________________________________________
py-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/py-dev
