Re: Sparse Test Output

2006-10-27 Thread chromatic
On Thursday 26 October 2006 07:37, Paul Beckingham wrote: To each his own, but my thoughts were not that it takes time to   parse, or that there is any unreasonable performance issue here -   just that it is so completely *unnecessary* to say ok lots of times. Crazy thought: don't call ok()

Re: Sparse Test Output

2006-10-27 Thread Nik Clayton
Michael Peters wrote: Jonathan Rockway wrote: That said, I am willing to clean it up when I have time, and hopefully be able to provide a libtap that makes TAP support easy for any language. There is already a libtap - http://www.onlamp.com/pub/a/onlamp/2006/01/19/libtap.html Indeed. Is

Re: Sparse Test Output

2006-10-26 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: TAPx::Parser has not yet been optimized so some profiling information would likely prove interesting. I would much rather improve the parser than change the protocol. This has been a concern of mine for quite some time as I've noticed this, too.

Re: Sparse Test Output

2006-10-26 Thread Nicholas Clark
On Wed, Oct 25, 2006 at 03:53:58PM -0700, Michael G Schwern wrote: I am totally unconvinced. I would love to see some profiling on this. I would hope that the act of running 50,000 tests would swamp the cost of parsing 50,000 tests. I don't have the exact numbers any more, but had

Re: Sparse Test Output

2006-10-26 Thread Paul Beckingham
This raises the question, why are you storing the raw results rather than storing the results after having been run through a TAP parser? I am storing all test results, from all test runs, to drive a tinderbox report with drill-down capability. I can (and do) post- process the TAP output

Re: Sparse Test Output

2006-10-26 Thread Chris Dolan
On Oct 26, 2006, at 9:37 AM, Paul Beckingham wrote: and waiting for the test output/TAP to be parsed takes quite a while. Just my $0.02 worth. -=Chris To each his own, but my thoughts were not that it takes time to parse, or that there is any unreasonable performance issue here - just

Re: Sparse Test Output

2006-10-26 Thread Michael G Schwern
Paul Beckingham wrote: and waiting for the test output/TAP to be parsed takes quite a while. Just my $0.02 worth. -=Chris To each his own, but my thoughts were not that it takes time to parse, or that there is any unreasonable performance issue here - just that it is so completely

Re: Sparse Test Output

2006-10-26 Thread Michael Peters
Jonathan Rockway wrote: That said, I am willing to clean it up when I have time, and hopefully be able to provide a libtap that makes TAP support easy for any language. There is already a libtap - http://www.onlamp.com/pub/a/onlamp/2006/01/19/libtap.html -- Michael Peters Developer Plus

Re: Sparse Test Output

2006-10-26 Thread Jonathan Rockway
There is already a libtap Ah! Thanks for pointing that out. I was planning a libtap to *parse* TAP, whereas this libtap *emits* TAP. If necessary, the name could be changed to libparsetap or something :) -- package JAPH;use Catalyst qw/-Debug/;($;=JAPH)-config(name = do { $,.=reverse

Re: Sparse Test Output

2006-10-26 Thread A. Pagaltzis
* Jonathan Rockway [EMAIL PROTECTED] [2006-10-26 21:15]: If necessary, the name could be changed to libparsetap or something :) Maybe Yet Another Parser for TAP – libyaptap. :^) Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/

Re: Sparse Test Output

2006-10-25 Thread Paul Beckingham
I'm with Adrian. Printing out ok 100,000 times shouldn't be a big deal unless you're reading the TAP via some sort of IP over clay tablets protocol. But... My test estimate is two orders of magnitude larger, so it actually is a big deal to capture and store those results. But I would

Re: Sparse Test Output

2006-10-25 Thread Christopher H. Laco
Paul Beckingham wrote: I'm with Adrian. Printing out ok 100,000 times shouldn't be a big deal unless you're reading the TAP via some sort of IP over clay tablets protocol. But... My test estimate is two orders of magnitude larger, so it actually is a big deal to capture and store those

Re: Sparse Test Output

2006-10-25 Thread Michael G Schwern
Christopher H. Laco wrote: I'm in the same boat. Recently, I've started testing my environment when things go wrong. (I blame Andy). I have one test alone that has a test count of 500,000+. That's a lot of oks to be processed, when I only want the ones that didn't pass. Now, add in a few

Re: Sparse Test Output

2006-10-25 Thread Christopher H. Laco
Michael G Schwern wrote: Christopher H. Laco wrote: I'm in the same boat. Recently, I've started testing my environment when things go wrong. (I blame Andy). I have one test alone that has a test count of 500,000+. That's a lot of oks to be processed, when I only want the ones that didn't

Re: Sparse Test Output

2006-10-25 Thread Michael G Schwern
Paul Beckingham wrote: I'm with Adrian. Printing out ok 100,000 times shouldn't be a big deal unless you're reading the TAP via some sort of IP over clay tablets protocol. But... My test estimate is two orders of magnitude larger, so it actually is a big deal to capture and store those

Re: Sparse Test Output

2006-10-25 Thread Michael G Schwern
Michael G Schwern wrote: It does add significant overhead. Here's the example of one of Regexp::Common's tests. 0 windhund /private/var/local/cpan_shell/build/Regexp-Common-2.120$ time perl -Ilib ~/tmp/strip_ok t/number/integer.t 1..23534 real0m4.882s user0m5.469s sys

Re: Sparse Test Output

2006-10-24 Thread Michael G Schwern
Adrian Howard wrote: On 5 Oct 2006, at 15:11, Paul Beckingham wrote: Recently I was required to create another flavor of test harness that runs tests, then captures and stores output. The nature of my testing means that I am running millions of tests, and the resultant captured output is