Stateful prove

2007-11-29 Thread Andy Armstrong
I'd like # Run all tests $ prove --state=save -rb t/*.t # Re-run failures from above $ prove --state=fail -rbv # Re-run failures and remember failures $ prove --state=save,fail -rbv Using the third form repeatedly would run only the test programs that failed previously - so as you fixed thin

Re: Stateful prove

2007-11-29 Thread Adrian Howard
On 29 Nov 2007, at 14:56, Andy Armstrong wrote: I'd like # Run all tests $ prove --state=save -rb t/*.t # Re-run failures from above $ prove --state=fail -rbv # Re-run failures and remember failures $ prove --state=save,fail -rbv Using the third form repeatedly would run only the test progr

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 15:04, Adrian Howard wrote: On 29 Nov 2007, at 14:56, Andy Armstrong wrote: I'd like # Run all tests $ prove --state=save -rb t/*.t # Re-run failures from above $ prove --state=fail -rbv # Re-run failures and remember failures $ prove --state=save,fail -rbv Using the thir

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 15:08, Andy Armstrong wrote: $ prove --state=fail,all,save ? (option order matters, 'fail' adds the failed tests to the run queue, 'all' adds everything, list is de-duped so the failures run first, saved for next time) And, of course, $ prove --state=pass to make sur

Re: Stateful prove

2007-11-29 Thread Adrian Howard
On 29 Nov 2007, at 15:08, Andy Armstrong wrote: [snip] One I'd like even more would be to run tests in order of most- recently-failed. It's something I've hacked together several times at various companies with T::H::Straps. $ prove --state=fail,all,save ? (option order matters, 'fail' add

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 16:05, Adrian Howard wrote: I think that's subtly different from most-recently-failed order. "-state=fail,all,save" gives me failing tests first. If I'm reading you right it doesn't apply ordering within the groups of tests. Running all tests in most-recently-failed order

Re: Stateful prove

2007-11-29 Thread Ovid
--- Andy Armstrong <[EMAIL PROTECTED]> wrote: > Yup. I'm working on it now so I'll incorporate something like that. > Thanks :) While you're at it, we can eliminate this bug (this is related to a stateful prove): http://rt.cpan.org/Public/Bug/Display.html?id=210

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 16:18, Ovid wrote: Yup. I'm working on it now so I'll incorporate something like that. Thanks :) While you're at it, we can eliminate this bug (this is related to a stateful prove): http://rt.cpan.org/Public/Bug/Display.html?id=2101 Someone wants to be

Re: Stateful prove

2007-11-29 Thread Eric Wilhelm
# from Andy Armstrong # on Thursday 29 November 2007 06:56: ># Run all tests >$ prove --state=save -rb t/*.t I would rather just '--save-state', why do the "=foo,bar" thing ? Just always save? Then --only-failed will always work without needing to re-run with the save flag? I guess an env var

Re: Stateful prove

2007-11-29 Thread Andy Lester
On Nov 29, 2007, at 10:22 AM, Andy Armstrong wrote: prove --shuffle=$seed Or perhaps App::Prove can cache the last random seed used and use that if someone wants to repeat the last shuffle: prove --last-shuffle Good call. I'll do that too then. I really really really don't like the i

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 17:28, Andy Lester wrote: Good call. I'll do that too then. I really really really don't like the idea of basing it on a given seed. I want to be able to know state about the tests that have run, the order they were in, etc, without having to go through prove to genera

Re: Stateful prove

2007-11-29 Thread Andy Lester
On Nov 29, 2007, at 11:30 AM, Andy Armstrong wrote: I really really really don't like the idea of basing it on a given seed. I want to be able to know state about the tests that have run, the order they were in, etc, without having to go through prove to generate the sequence for me. M

Re: Stateful prove

2007-11-29 Thread Eric Wilhelm
# from Andy Armstrong # on Thursday 29 November 2007 09:08: >On 29 Nov 2007, at 17:10, Eric Wilhelm wrote: >>> # Run all tests >>> $ prove --state=save -rb t/*.t >> >> I would rather just '--save-state', why do the "=foo,bar" thing ? > >Because it allows you to express what you'd like to have happ

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 17:10, Eric Wilhelm wrote: # Run all tests $ prove --state=save -rb t/*.t I would rather just '--save-state', why do the "=foo,bar" thing ? Because it allows you to express what you'd like to have happen with the saved state. -- Andy Armstrong, Hexten

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 17:42, Eric Wilhelm wrote: If the option parser preserved the order of the flags, you wouldn't have to cram that all into an '=foo,bar' opterand. --save-state --no-save-state --failed-only --failed-first (Yes, needs more thought in the "abbreviation and short uniqueness

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 16:05, Adrian Howard wrote: I like this coz my preferred way of working is to run all of the tests all of the time, rather than just the particular test script that I'm working on at the moment. That way if I have some dumb code that breaks something else in the system I g

Re: Stateful prove

2007-11-29 Thread Eric Wilhelm
# from Andy Armstrong # on Thursday 29 November 2007 12:26: >Eric: I *really* like have all the state selectors in a single switch. >   Please have a look at it and tell me if you find it awful. How does an alias play-along with that? Can I say "--state=save" in the alias and --state=hot on the

Re: Stateful prove

2007-11-29 Thread A. Pagaltzis
* Andy Armstrong <[EMAIL PROTECTED]> [2007-11-29 16:10]: > $ prove --state=fail,all,save > > ? > > (option order matters, 'fail' adds the failed tests to the run > queue, 'all' adds everything, list is de-duped so the failures > run first, saved for next time) Why this mental overhead? Do they rea

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 22:10, Eric Wilhelm wrote: How does an alias play-along with that? Can I say "--state=save" in the alias and --state=hot on the command-line? You can as of r887 :) -- Andy Armstrong, Hexten

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 22:09, A. Pagaltzis wrote: (option order matters, 'fail' adds the failed tests to the run queue, 'all' adds everything, list is de-duped so the failures run first, saved for next time) Why this mental overhead? Do they really have to be ordered? Is there any way that any othe

Re: Stateful prove

2007-11-29 Thread A. Pagaltzis
* Andy Armstrong <[EMAIL PROTECTED]> [2007-11-29 23:25]: > On 29 Nov 2007, at 22:09, A. Pagaltzis wrote: >> Why this mental overhead? Do they really have to be ordered? >> Is there any way that any other order would do anything >> remotely useful? Would `save,fail,all` be useful? Or >> `all,save,fa

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 22:34, A. Pagaltzis wrote: Aha. I see; those were examples that hadn’t come up before. So ordering is useful to avoid an explosion of options. At the same time, there is a bunch of aspects where order sensitivity clearly does not make sense: does save,todo,failed do something u

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 22:38, Andy Armstrong wrote: Fixed - although I have test failures on our smoke boxes at the moment. Fixed also. -- Andy Armstrong, Hexten

Re: Stateful prove

2007-11-29 Thread Michael G Schwern
Andy Armstrong wrote: > I'd like > > # Run all tests > $ prove --state=save -rb t/*.t > > # Re-run failures from above > $ prove --state=fail -rbv > > # Re-run failures and remember failures > $ prove --state=save,fail -rbv > > Using the third form repeatedly would run only the test programs th

Re: Stateful prove

2007-11-29 Thread Eric Wilhelm
# from A. Pagaltzis # on Thursday 29 November 2007 14:34: >>  failed,todo,all >> >> would do something sensible and distinct from >> >>  todo,failed,all > >Aha. I see; those were examples that hadn’t come up before. So >ordering is useful to avoid an explosion of options. Or ... WE COULD HAVE AN

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 29 Nov 2007, at 23:36, Michael G Schwern wrote: When does state get cleared? When you save a new state over the top of it. In practice you don't need to explicitly clear it I think. I could add an erase option - or you could just rm .prove. -- Andy Armstrong, Hexten

Re: Stateful prove

2007-11-29 Thread Andy Armstrong
On 30 Nov 2007, at 00:25, Eric Wilhelm wrote: Aha. I see; those were examples that hadn’t come up before. So ordering is useful to avoid an explosion of options. Or ... WE COULD HAVE AN ORDERING OPTION PARSER! SPEAK UP. We could, indeed, have an ordering option parser. But I suspect ordere