Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Geoffrey Young <[EMAIL PROTECTED]> wrote: > > > There are two usual rebuttals. > > the third being "just add it and let me decide" > > :) Right in one! I'll go fix that diagnostic thing now. Unfortunately, I think I'll have to violate encapsulation :( As a side note: after reading ev

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Ovid wrote: > I'll go fix that diagnostic thing now. Unfortunately, I think I'll > have to violate encapsulation :( If you know how to fix it let me know, because other than enumerating each testing module you might use and lex-wrapping all the functions they export, I'm not sure how to do it. T

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
The whole idea of halting on first failure was introduced to me by some XUnit folks. Their rationale was not to avoid spewing output, they had no such problem since it's all done via a GUI, but that once one failure has happened the failing code might hose the environment and all following results

Test::Builder statistics

2008-01-12 Thread Ovid
My first attempt at determining the most popular testing modules left out Test.pm. Whoops! I've fixed that. Out of almost 60,000 test programs, it turns out Test.pm is used 8,937 times. Now that I have a file which lists how many times each test module is used, I can start examining my extracte

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > The whole idea of halting on first failure was introduced to me by > some XUnit > folks ... As any field scientist knows, there's no such thing as > uncontaminated data. As any tester knows, a "one size fits all suit" often doesn't fit. Let peop

Re: Call for Speakers: Perl QA Hackathon in Oslo (important update)

2008-01-12 Thread Salve J Nilsen
A friendly reminder... Salve J Nilsen said: Salve J Nilsen said: Hey guys. Oslo.pm is planning a Perl QA Workshop/Hackathon in Oslo, Saturday April 4th to Monday April 7th, 2008. http://perl-qa.hexten.net/wiki/index.php/OsloQAWorkshop2008 We're about to get a significant amount of fundi

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Aristotle Pagaltzis
* Michael G Schwern <[EMAIL PROTECTED]> [2008-01-12 12:00]: > Ovid wrote: > > I'll go fix that diagnostic thing now. Unfortunately, I > > think I'll have to violate encapsulation :( > > If you know how to fix it let me know, because other than > enumerating each testing module you might use and le

Preserving diagnostics when dieing on test failure

2008-01-12 Thread Ovid
Here's my test: #!/usr/bin/perl use strict; use warnings; use lib '.'; use MyTestMore 'no_plan', 'fail'; ok 1; ok 1; is 2, 3, 'bad'; ok 1; Here's my output: ok 1 ok 2 not ok 3 - bad # Failed test 'bad' # at /System/Library/Perl/5.8.6/Test/More.pm line 329.

Dieing on failure and preserving line numbers

2008-01-12 Thread Ovid
Looks like I have it now. I removed Hook::LexWrap, handled the $Test::Builder::Level myself and did some curious work to preserve the exact return value behavior of Test::Builder::diag. Remove Hook::LexWrap and change the 'if fail' test in the import method to the following: if ('fail' eq $_[

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Ovid wrote: > --- Michael G Schwern <[EMAIL PROTECTED]> wrote: > >> The whole idea of halting on first failure was introduced to me by >> some XUnit >> folks ... As any field scientist knows, there's no such thing as >> uncontaminated data. > > As any tester knows, a "one size fits all suit" ofte

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Aristotle Pagaltzis wrote: > * Michael G Schwern <[EMAIL PROTECTED]> [2008-01-12 12:00]: >> Ovid wrote: >>> I'll go fix that diagnostic thing now. Unfortunately, I >>> think I'll have to violate encapsulation :( >> If you know how to fix it let me know, because other than >> enumerating each testin

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Roman Shaposhnik
On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: > excellent stuff Andy! > > To start with the obvious :) > > Due to the lack of ustack helper, I am hoping to add a stack probe Two things: first of all DTrace could really benefit from a general purpose mechanism of hooking up custom sta

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Sven Dowideit
excellent stuff Andy! To start with the obvious :) Due to the lack of ustack helper, I am hoping to add a stack probe - I really could do with being able to write a dtrace that shows me a perl stack trace on file system access - yesterday I spent a borish day using inotifywait and some perl t

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Bryan Cantrill
On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: > > excellent stuff Andy! > > > > To start with the obvious :) > > > > Due to the lack of ustack helper, I am hoping to add a stack probe > > Two things: first of all DTr

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Roman Shaposhnik
On Fri, 2008-01-11 at 17:32 -0800, Bryan Cantrill wrote: > On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > > On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: > > > excellent stuff Andy! > > > > > > To start with the obvious :) > > > > > > Due to the lack of ustack helpe

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Eric Schrock
On Fri, Jan 11, 2008 at 06:23:14PM -0800, Roman Shaposhnik wrote: > > I guess we are talking about two different kinds of generality here. > What I have in mind is the generality that would allow DTrace to be > able to offload some of the heavylifting to the userspace. I DO > understand the rami

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread John Levon
On Sat, Jan 12, 2008 at 02:03:42AM +, Andy Armstrong wrote: > > http://blogs.sun.com/levon/entry/python_and_dtrace_in_build > > I read that Bryan. I got the impression - maybe incorrectly - that it > relies on the Python interpreter recursing when it enters a Python > function - so that

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > PS Couldn't you have the TAP harness kill the test process on first > failure? I then have even less control over the diagnostics than I would if Test::Builder handled this responsibility. It's also an improper separation of concerns. Test::Bu

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > > Set a flag that T::B should quit when the next test result is > > about to be recorded? > > I guess it works, but it leaves you dead halfway through another test > function which is weird. The code I sent is conceptually similar, but since a f

Re: Preserving diagnostics when dieing on test failure

2008-01-12 Thread Michael G Schwern
Ovid wrote: > So we can preserve diagnostics, but we need help in cleaning up those > damned line numbers. Hook::LexWrap didn't have the magic I thought it > would. ok() is now inside a wrapper so you're one level further down then it thinks. Just add one to $Level and then take it back off agai

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Roman Shaposhnik
On Sat, 2008-01-12 at 01:30 +, John Levon wrote: > On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > > > > Due to the lack of ustack helper, I am hoping to add a stack probe > > > > Two things: first of all DTrace could really benefit from a > > general purpose mechanism o

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Roman Shaposhnik
On Jan 11, 2008, at 6:26 PM, Eric Schrock wrote: On Fri, Jan 11, 2008 at 06:23:14PM -0800, Roman Shaposhnik wrote: I guess we are talking about two different kinds of generality here. What I have in mind is the generality that would allow DTrace to be able to offload some of the heavylift

Re: Dieing on failure and preserving line numbers

2008-01-12 Thread Paul Johnson
On Sat, Jan 12, 2008 at 04:48:23AM -0800, Ovid wrote: > Looks like I have it now. I really like the direction this is going. This is something that I too have asked for in the past. I've even hacked up my own stuff to do it, though obviously not as elegantly as you or Geoff. Here's my use case

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread John Levon
On Fri, Jan 11, 2008 at 04:39:25PM -0800, Roman Shaposhnik wrote: > > Due to the lack of ustack helper, I am hoping to add a stack probe > > Two things: first of all DTrace could really benefit from a > general purpose mechanism of hooking up custom stack helpers > with the rest of the system.

Re: Dieing on failure and preserving line numbers

2008-01-12 Thread Ovid
--- Paul Johnson <[EMAIL PROTECTED]> wrote: > I don't want it on all the time. I don't want to change anything in > my > code to do it. I don't (yet) care that it doesn't run my cleanup > code > or whatever. I just want to easily find out what the first error > was, > and then have a short edit

The spewing problem.

2008-01-12 Thread Michael G Schwern
Paul Johnson wrote: > This is something that I too have asked for in the past. I've even > hacked up my own stuff to do it, though obviously not as elegantly as > you or Geoff. Here's my use case. > > I have a bunch of tests that generally pass. I hack something > fundamental and run my tests.

Re: Test::Builder statistics

2008-01-12 Thread Michael G Schwern
Ovid wrote: > My first attempt at determining the most popular testing modules left > out Test.pm. Whoops! I've fixed that. > > Out of almost 60,000 test programs, it turns out Test.pm is used 8,937 > times. Now that I have a file which lists how many times each test > module is used, I can sta

Re: [dtrace-discuss] DTrace in Perl: What probes should we have?

2008-01-12 Thread Sven Dowideit
Roman Shaposhnik wrote: On Sat, 2008-01-12 at 11:21 +1100, Sven Dowideit wrote: excellent stuff Andy! To start with the obvious :) Due to the lack of ustack helper, I am hoping to add a stack probe Two things: first of all DTrace could really benefit from a general purpose mechanis

Re: The spewing problem.

2008-01-12 Thread chromatic
On Saturday 12 January 2008 07:03:16 Michael G Schwern wrote: > The problem I'm hearing over and over again is "Test::Builder is spewing > crap all over my screen and obscuring the first, real failure".  So now > that the problem is clearly stated, how do we solve it without making all > that spew

Re: The spewing problem.

2008-01-12 Thread Andy Armstrong
On 12 Jan 2008, at 18:33, chromatic wrote: The problem I'm hearing over and over again is "Test::Builder is spewing crap all over my screen and obscuring the first, real failure". So now that the problem is clearly stated, how do we solve it without making all that spew (which can be usefu

Re: The spewing problem.

2008-01-12 Thread Sam Vilain
Michael G Schwern wrote: > Paul Johnson wrote: >> This is something that I too have asked for in the past. I've even >> hacked up my own stuff to do it, though obviously not as elegantly as >> you or Geoff. Here's my use case. >> >> I have a bunch of tests that generally pass. I hack something >

Re: The spewing problem.

2008-01-12 Thread Matisse Enzer
I just want to be able to run a test suite with a switch that makes the entire test run stop after the first failure is reported. --- Matisse Enzer <[EMAIL PROTECTED]> http://www.matisse.net/ - http://www.eigenstate.net/

Re: The spewing problem.

2008-01-12 Thread Michael G Schwern
Matisse Enzer wrote: > I just want to be able to run a test suite with a switch that makes the > entire test run stop after the first failure is reported. Ok, it's nice to want things, but why do you want it? -- 100. Claymore mines are not filled with yummy candy, and it is wrong to tell n

Re: The spewing problem.

2008-01-12 Thread Matisse Enzer
On Jan 12, 2008, at 10:24 PM, Michael G Schwern wrote: Matisse Enzer wrote: I just want to be able to run a test suite with a switch that makes the entire test run stop after the first failure is reported. Ok, it's nice to want things, but why do you want it? Almost entirely because when