Re: The spewing problem.
Michael G Schwern said:
The problem I'm hearing over and over again is Test::Builder is spewing
crap all over my screen and obscuring the first, real failure. So now
that the problem is clearly stated, how do we solve it without making
all that spew (which can be useful) totally unavailable?
Ok, a shot in the dark...
1. Redirect test output to a temporary log file (which should get cleaned
up during make clean al.)
2. Let the harness be aware of this log file, and tail(1) it as things
get written to it, printing whatever's relevant/asked for to the
terminal.
3. If there's an error detected, write a warning at the end of the test
run, pointing to the log file (and perhaps line number in it.)
- Salve
--
#!/usr/bin/perl
sub AUTOLOAD{$AUTOLOAD=~/.*::(\d+)/;seek(DATA,$1,0);print# Salve Joshua Nilsen
getc DATA}$='};{';@_=unpack(C*,unpack(u*,':4@,$'.# [EMAIL
PROTECTED]
'2!--5-(50P%$PL,!0X354UC-PP%/0\`'.\n));eval {'@_'}; __END__ is near! :)
Re: The spewing problem.
--- Salve J Nilsen [EMAIL PROTECTED] wrote: Ok, a shot in the dark... 1. Redirect test output to a temporary log file (which should get cleaned up during make clean al.) 2. Let the harness be aware of this log file, and tail(1) it as things get written to it, printing whatever's relevant/asked for to the terminal. 3. If there's an error detected, write a warning at the end of the test run, pointing to the log file (and perhaps line number in it.) How is this simpler than 'bail on fail' or 'die on fail'? Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Fri, Jan 11, 2008 at 06:12:57PM -0800, Roman Shaposhnik wrote: My main beef with stack helpers is that they have to be executed inside the kernel and are subject to the usual DTrace restrictions. I would like to see a possibility of offloading more work to user space in a generic fashion. For example, we have a libcollector.so facility that is capable of reconstructing stacks under the most challenging of situations (inside the function preamble, etc.) it is way more accurate in what it reports back than ustack() it is also capable of more stack unwinding than jstack(). Of course, it runs in user space. All respect to libcollector.so, but the conditions under which DTrace must gather a stack are far more constrained than those of a user-land program. The DTrace code from probe context can't do I/O or block -- it can only load and store. Even looping is a bit of a stretch. Adam -- Adam Leventhal, FishWorkshttp://blogs.sun.com/ahl
Re: [dtrace-discuss] DTrace in Perl: What probes should we have?
On Jan 12, 2008, at 12:26 PM, Adam Leventhal wrote: On Fri, Jan 11, 2008 at 06:12:57PM -0800, Roman Shaposhnik wrote: My main beef with stack helpers is that they have to be executed inside the kernel and are subject to the usual DTrace restrictions. I would like to see a possibility of offloading more work to user space in a generic fashion. For example, we have a libcollector.so facility that is capable of reconstructing stacks under the most challenging of situations (inside the function preamble, etc.) it is way more accurate in what it reports back than ustack() it is also capable of more stack unwinding than jstack(). Of course, it runs in user space. All respect to libcollector.so, but the conditions under which DTrace must gather a stack are far more constrained than those of a user-land program. The DTrace code from probe context can't do I/O or block -- it can only load and store. Even looping is a bit of a stretch. I don't disagree at all! And I'm not trying to say that *stack() should be replaced. I'm merely trying to explore whether the best of both worlds approach is doable. What we've done for Project D-Light clearly shows that for some applications it is doable. At this point I really would like to explore whether something like that could be generalized. See my earlier email to Eric. I understand his argument perfectly well. But I'm still not convinced that it makes the idea void. If convincing me is perceived as a waste of time -- that's ok. I guess I should get back to you when I earn my Solaris kernel guru degree :-) If on the other hand, I'm not explaining what I want clearly -- I can try again. Thanks, Roman.
Re: The spewing problem.
Ovid said:
--- Salve J Nilsen [EMAIL PROTECTED] wrote:
Ok, a shot in the dark...
1. Redirect test output to a temporary log file (which should get
cleaned up during make clean al.)
2. Let the harness be aware of this log file, and tail(1) it as
things get written to it, printing whatever's relevant/asked for to
the terminal.
3. If there's an error detected, write a warning at the end of the
test run, pointing to the log file (and perhaps line number in it.)
How is this simpler than 'bail on fail' or 'die on fail'?
Perhaps it's not easier to implement, no. But the user gets at least
access to the entire set of errors and warnings while still getting the
high-level test report in the terminal.
If the goal here is to reduce the amount of pointless tests run because
of some failure in an earlier test (DIE_ON_FAILURE=1), then this
suggestion may be a bit off the mark. But if the goal is to get access to
the first failing diagnostics, then just keeping them around in a file
ought to be enough (and that would also give you the option to do
something with the test output you might not otherwise, e.g. post-test
analysis or statistics gathering.)
- Salve
--
#!/usr/bin/perl
sub AUTOLOAD{$AUTOLOAD=~/.*::(\d+)/;seek(DATA,$1,0);print# Salve Joshua Nilsen
getc DATA}$='};{';@_=unpack(C*,unpack(u*,':4@,$'.# [EMAIL
PROTECTED]
'2!--5-(50P%$PL,!0X354UC-PP%/0\`'.\n));eval {'@_'}; __END__ is near! :)
Re: What should it's name be?
Smylers wrote: Names normally descend in specificity from left to right, but Test::More::Even doesn't really make sense. How about Test::More::Still, Test::More::Stuff, Test::More::Crap, Test::More::More :) -- Michael Peters Developer Plus Three, LP
Re: What should it's name be?
--- Smylers [EMAIL PROTECTED] wrote: Ovid writes: Forget all the silly debates. I don't want to wait or sit around while we argue over this silly stuff. You mean you don't want flaming for the apostrofly in the subject? OK then, I won't mention it. Yeah, I saw that right after I sent it. I cringed, but then, I have enough typos that I am getting used to this :) But somewhere that train of thought led me to Test::Most -- which implies more than 'more', and could work if you feel like claiming this module will meet most testing needs, or be useful in most test programs or something? You know, I really like that name. I might just go with that. I had settled on Test::Extended because I'm trying to 'extend' what is traditionally done with test suites and maybe keep it as a permanent experimentation ground, but Test::Most sounds good. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Fwd: CPAN Upload: O/OV/OVID/Test-Most-0.01.tar.gz
Let me know if there are any problems. I'm sure there will be :) Cheers, Ovid --- PAUSE [EMAIL PROTECTED] wrote: Date: Sun, 13 Jan 2008 18:07:17 +0100 Subject: CPAN Upload: O/OV/OVID/Test-Most-0.01.tar.gz To: Curtis Ovid Poe [EMAIL PROTECTED] From: PAUSE [EMAIL PROTECTED] The uploaded file Test-Most-0.01.tar.gz has entered CPAN as file: $CPAN/authors/id/O/OV/OVID/Test-Most-0.01.tar.gz size: 6960 bytes md5: 473e72107f9a1e48c9f1c0f7db936179 No action is required on your part Request entered by: OVID (Curtis Ovid Poe) Request entered on: Sun, 13 Jan 2008 17:07:03 GMT Request completed: Sun, 13 Jan 2008 17:07:17 GMT Thanks, -- paused, v937 -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Re: The spewing problem.
Matisse Enzer wrote: On Jan 12, 2008, at 10:24 PM, Michael G Schwern wrote: Matisse Enzer wrote: I just want to be able to run a test suite with a switch that makes the entire test run stop after the first failure is reported. Ok, it's nice to want things, but why do you want it? Almost entirely because when I'm developing on a code base with a large test suite I often want to stop the test run as fast as possible when a failure occurs - currently I do a control-C but would prefer if I could use a switch when I run the tests to have it just stop right after the first failure. Ok, why do you want to stop it as fast as possible when a failure occurs? -- 164. There is no such thing as a were-virgin. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/?page_id=3
Re: Fwd: CPAN Upload: O/OV/OVID/Test-Most-0.01.tar.gz
--- Ovid [EMAIL PROTECTED] wrote: Let me know if there are any problems. I'm sure there will be :) For those who can't wait, the docs are at http://use.perl.org/~Ovid/journal/35381. I forgot to credit Smylers with the name in the CPAN version. This will be corrected in the next release. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
Re: The spewing problem.
On Jan 13, 2008, at 9:17 AM, Michael G Schwern wrote: Matisse Enzer wrote: On Jan 12, 2008, at 10:24 PM, Michael G Schwern wrote: Matisse Enzer wrote: I just want to be able to run a test suite with a switch that makes the entire test run stop after the first failure is reported. Ok, it's nice to want things, but why do you want it? Almost entirely because when I'm developing on a code base with a large test suite I often want to stop the test run as fast as possible when a failure occurs - currently I do a control-C but would prefer if I could use a switch when I run the tests to have it just stop right after the first failure. Ok, why do you want to stop it as fast as possible when a failure occurs? So I can more quickly focus on fixing that first test failure. If you are going to suggest some other workflow for developing - looking at all the failures to see where the root problem is, that's fine, and I often do that too. Sometimes though I just want to immediately focus on the first test failure, mainly when I am doing test-first development and the test suite takes more than a few seconds to run to completion. -M --- Matisse Enzer [EMAIL PROTECTED] http://www.matisse.net/ - http://www.eigenstate.net/
Re: The spewing problem.
Michael G Schwern wrote: Ok, why do you want to stop it as fast as possible when a failure occurs? I have a 45 minute test suite and I want to work on the first failure as soon as possible. I also have multiple desktops and am doing other things in another desktop, so I want to know as soon as the failure happens so that I can start working on it: make test || echo -e \a Would be nice if that would beep after the first failure instead of after 45 minutes and the whole thing is done. -- Michael Peters Developer Plus Three, LP
Re: The spewing problem.
On 13 Jan 2008, at 18:06, Michael Peters wrote: Ok, why do you want to stop it as fast as possible when a failure occurs? I have a 45 minute test suite and I want to work on the first failure as soon as possible. I also have multiple desktops and am doing other things in another desktop, so I want to know as soon as the failure happens so that I can start working on it: make test || echo -e \a Would be nice if that would beep after the first failure instead of after 45 minutes and the whole thing is done. OK, it's not the same as what's been asked for but would it be useful for prove to have a don't start any new test programs after = N failures option? -- Andy Armstrong, Hexten
Re: The spewing problem.
On Sunday 13 January 2008 04:27:38 Ovid wrote: --- Salve J Nilsen [EMAIL PROTECTED] wrote: 1. Redirect test output to a temporary log file (which should get cleaned up during make clean al.) 2. Let the harness be aware of this log file, and tail(1) it as things get written to it, printing whatever's relevant/asked for to the terminal. 3. If there's an error detected, write a warning at the end of the test run, pointing to the log file (and perhaps line number in it.) How is this simpler than 'bail on fail' or 'die on fail'? It doesn't conflate output TAP results from tests with interpret TAP results from tests. -- c
Re: The spewing problem.
Michael Peters wrote: Michael G Schwern wrote: Ok, why do you want to stop it as fast as possible when a failure occurs? I have a 45 minute test suite and I want to work on the first failure as soon as possible. I also have multiple desktops and am doing other things in another desktop, so I want to know as soon as the failure happens so that I can start working on it: make test || echo -e \a Would be nice if that would beep after the first failure instead of after 45 minutes and the whole thing is done. I keep digging away at this because I'm looking for a problem other than I want to see the first failure. And that's what I'm hearing from you and from Matisse and everyone else. Yours is a little different, it's I want to be alerted on first failure. You see how this is distinct from halt on first failure? It gives me a lot more room for different solutions that don't involve just cutting off all the following information. -- Look at me talking when there's science to do. When I look out there it makes me glad I'm not you. I've experiments to be run. There is research to be done On the people who are still alive. -- Jonathan Coulton, Still Alive
Re: The spewing problem.
On Jan 13, 2008, at 10:40 AM, Michael G Schwern wrote: You see how this is distinct from halt on first failure? It gives me a lot more room for different solutions that don't involve just cutting off all the following information. I do see (and respect) that difference, but stopping the test run is indeed what I want. Certainly if here is a general hook for what to do after the first failure that is fine, as long as i can easily make that action BAIL_OUT or the moral equivalent. --- Matisse Enzer [EMAIL PROTECTED] http://www.matisse.net/ - http://www.eigenstate.net/
Re: The spewing problem.
Michael G Schwern wrote: Paul Johnson wrote: This is something that I too have asked for in the past. I've even hacked up my own stuff to do it, though obviously not as elegantly as you or Geoff. Here's my use case. I have a bunch of tests that generally pass. I hack something fundamental and run my tests. Loads of them fail. Diagnostics spew over my screen. Urgh, I say. Now I could scroll back through them. When faced with a tough problem it's often useful to go back check that it's actually the problem and not a solution posing as a problem. Make Test::Builder die on failure is a solution, and it's not a particularly good one. It's hard to implement in Test::Builder and there's all the loss of information issues I've been yelping bout. The problem I'm hearing over and over again is Test::Builder is spewing crap all over my screen and obscuring the first, real failure. So now that the problem is clearly stated, how do we solve it without making all that spew (which can be useful) totally unavailable? Whatever the answer is, invoking a solution that requires universal education of humans isn't a viable answer. I keep seeing use this method, or that method... None of them look trivial. Whatever method is used, should involve 3 keystrokes or less. This shouldn't be any more complicated than -g (where g in my case stands for goat as in feinting goat) Adam K
Re: The spewing problem.
Adam Kennedy wrote: This shouldn't be any more complicated than -g (where g in my case stands for goat as in feinting goat) Ok, I'll bite. Why a goat and why is it feinting? -- ...they shared one last kiss that left a bitter yet sweet taste in her mouth--kind of like throwing up after eating a junior mint. -- Dishonorable Mention, 2005 Bulwer-Lytton Fiction Contest by Tami Farmer
Re: The spewing problem.
* Michael G Schwern [EMAIL PROTECTED] [2008-01-14 00:35]: Why a goat and why is it feinting? Because http://video.google.com/videoplay?docid=5306665151934403100 Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: What should it's name be?
# from Smylers # on Sunday 13 January 2008 04:29: Names normally descend in specificity from left to right, but Test::More::Even doesn't really make sense. I'm glad Test::More::Better got missed. I call dibs. --Eric -- So malloc calls a timeout and starts rummaging around the free chain, sorting things out, and merging adjacent small free blocks into larger blocks. This takes 3 1/2 days. --Joel Spolsky --- http://scratchcomputing.com ---
Re: What should it's name be?
* Eric Wilhelm [EMAIL PROTECTED] [2008-01-14 03:15]: I'm glad Test::More::Better got missed. I call dibs. Test::More::Better::Bigger::Badder? Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: What should it's name be?
Aristotle Pagaltzis wrote: * Eric Wilhelm [EMAIL PROTECTED] [2008-01-14 03:15]: I'm glad Test::More::Better got missed. I call dibs. Test::More::Better::Bigger::Badder? Regards, Test::More::Bigger::Longer::Uncut signature.asc Description: OpenPGP digital signature
Re: The spewing problem.
# from chromatic # on Sunday 13 January 2008 10:28: How is this simpler than 'bail on fail' or 'die on fail'? It doesn't conflate output TAP results from tests with interpret TAP results from tests. Yes. -- I arise in the morning torn between a desire to improve the world and a desire to enjoy the world. This makes it hard to plan the day. --E.B. White --- http://scratchcomputing.com ---
Re: The spewing problem.
Matisse Enzer wrote: Ok, why do you want to stop it as fast as possible when a failure occurs? So I can more quickly focus on fixing that first test failure. I use make test 21 | less Works for individual tests too make perl -Mblib t/testname.t 21 | less Sam.
Re: What should it's name be?
I know I am a bit late to the party but what about Test::Anything ? Gabor
Re: The spewing problem.
On Jan 13, 2008, at 7:13 PM, Sam Vilain wrote: Matisse Enzer wrote: Ok, why do you want to stop it as fast as possible when a failure occurs? So I can more quickly focus on fixing that first test failure. I use make test 21 | less Works for individual tests too make perl -Mblib t/testname.t 21 | less I don't see how this stops running the test suite upon the first failure, can you explain please? --- Matisse Enzer [EMAIL PROTECTED] http://www.matisse.net/ - http://www.eigenstate.net/
