On Tue, Apr 23, 2013 at 09:56 -0700, Leah Klearman wrote: > Holger, > > The only reason I can think of why py.test would perform teardown / > finalization differently depending on the nextitem is if class, module, and > package level fixtures are being used.
Yes, i tried to say something equivalent in my previous mail. > If only method level fixtures are > used, then they should always get created and always get torn down for each > test case. Is that not how it works? Effectively it should be like this and probably independently of the value of nextitem. If you find issues let us know. > We can't re-run just the 'call' phase because the failure of the previous > 'call' phase may have left behind a partial situation (see automated > testing before we had setup and teardown and fixtures). Sure, good point. cheers, holger > -Leah > > > On Tue, Apr 23, 2013 at 12:16 AM, holger krekel <[email protected]> wrote: > > > Hey Leah, > > > > On Mon, Apr 22, 2013 at 15:08 -0700, Leah Klearman wrote: > > > Hey Holger, > > > > > > Thank you for the timely patch. I haven't seen Bob online today, but > > > hopefully he'll be able to test it very soon. > > > > > > A word of warning: your calling of runtestprotocol() is not quite right > > > > and might lead to problems. "nextitem" should really be the item that > > > > is going to be run next. So if you re-run three times the first two > > > > invocations should have nextitem=item. > > > > > > > > > I agree that that would be ideal, but it only re-runs the test if it > > fails, > > > so I > > > would have to know the future in order to know what value to send > > nextitem=. > > > Since hopefully most tests pass the first try, sending nextitem instead > > of > > > item seems to be the better answer. > > > > FYI pytest perform teardown/finalization according to nextitem: if your > > next item uses different fixtures, the "item" ones may be torn down > > (it also depends on the caching scope of the fixtures). > > > > Do you generally want to re-run the setup or could you also settle > > on just re-running the "call" phase of a test? The latter one would > > be easier and probably not disrupt fixture mechanics. > > > > best, > > holger > > > > > Thanks! > > > -Leah > > > > > > > > > On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <[email protected]> > > wrote: > > > > > > > Hi Leah, Bob, > > > > > > > > On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: > > > > > Hi Holger and other py.test mavens, > > > > > > > > > > Bob has reported a problem with my py.test plugin > > pytest-rerunfailures > > > > [1] > > > > > not re-running the setup before rerunning the test. > > > > > > > > > > Looking at my code, it has pytest_runtest_protocol() [2] looping > > > > > on _pytest.runner.runtestprotocol() [3], which in turn runs the > > setup, > > > > the > > > > > test, and the teardown. > > > > > > > > > > [1] https://github.com/klrmn/pytest-rerunfailures > > > > > [2] > > > > > > > > > > > https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plugin.py#L46 > > > > > [3] > > > > > > > > > > > https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a03f/_pytest/runner.py?at=default#cl-65 > > > > > > > > > > I haven't taken the hours needed to get my head fully into py.test > > plugin > > > > > development mode, but I'm not sure I can implement a fix at my layer. > > > > > > > > > > > > > > > I'm hoping someone here will have some insight. > > > > > > > > It's a bit intricate. A function item keeps around some fixture state > > > > and was so far not intended to be run multiple times. I went ahead and > > > > tried to improve the behaviour to better allow re-running. Please try > > > > with > > > > > > > > pip install -i http://pypi.testrun.org -U pytest > > > > > > > > which should give you pytest-2.3.5.dev16 at least. This is bound to be > > > > released soon so quick feedback is welcome. If you still have problems > > > > please try to send a minimal test file which shows undesired behaviour. > > > > > > > > A word of warning: your calling of runtestprotocol() is not quite right > > > > and might lead to problems. "nextitem" should really be the item that > > > > is going to be run next. So if you re-run three times the first two > > > > invocations should have nextitem=item. > > > > > > > > best, > > > > holger > > > > > > > > > > > > > Thanks, > > > > > -Leah > > > > > > > > > > > > > > > On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < > > > > [email protected]>wrote: > > > > > > > > > > > I just verified this behaviour myself with a simple test [1]. I > > see it > > > > > > with both funcargs and fixtures, but I'm not sure if it has to do > > with > > > > the > > > > > > plugin, or the way py.test works. It does inject the value into the > > > > test > > > > > > method, but it doesn't rerun the fixture, so it seems like it is > > > > caching > > > > > > the first run of the fixture and using that on subsequent runs. > > > > > > > > > > > > I'm not sure if this is something that the plugin can have any > > effect > > > > on, > > > > > > or if it's just the way fixtures work. It is specified for this > > fixture > > > > > > that it is scope='function', and perhaps py.test makes that happen > > by > > > > > > checking the function name, which is, of course, the same for each > > > > run. I > > > > > > did try removing the scope argument from the fixture but that had > > no > > > > > > effect. > > > > > > > > > > > > Do you have any thoughts about this, @klrmn < > > https://github.com/klrmn > > > > >? > > > > > > > > > > > > [1] https://gist.github.com/bobsilverberg/5385035 > > > > > > > > > > > > — > > > > > > Reply to this email directly or view it on GitHub< > > > > > > https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-16363644 > > > > > > > > > > > . > > > > > > > > > > > > > > > _______________________________________________ > > > > > Pytest-dev mailing list > > > > > [email protected] > > > > > http://mail.python.org/mailman/listinfo/pytest-dev > > > > > > > > > > _______________________________________________ Pytest-dev mailing list [email protected] http://mail.python.org/mailman/listinfo/pytest-dev
