Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Smylers
Andy Armstrong writes: > Michael Peters wrote: > > > Why can't you just use test names? > > > > ok($is_ok, 'HTML nested bullet list, open tags (bug 5497)'); > > You certainly could - but I tend to think of them as descriptions > rather than names and therefore not entirely ideal as the primar

Re: Tracking tests over time

2007-07-12 Thread Andy Armstrong
Michael Peters wrote: > By the way. Do you have a link to show this or is it just something that > prints > to the console when you run the tests? They had an accident with the mail server that's obviously supposed to be relaying the tests and about 12 months worth spewed forth onto the list toda

Re: Tracking tests over time

2007-07-12 Thread Michael Peters
Andy Armstrong wrote: > I've just noticed that the MediaWiki team have a test harness that > tracks the life of individual assertions over time: > > * HTML nested bullet list, open tags (bug 5497) [Has never passed] > * HTML nested ordered list, open tags (bug 5497) [Has never passed] >

Re: Tracking tests over time

2007-07-12 Thread Andy Armstrong
Ian Malpass wrote: > Requiring that test numbering doesn't change seems like an awfully large > burden. It would mean you couldn't add extra values to a loop with a > test in it, for example (unless it happened to be at the end of your > test). It would be difficult to enforce, easy to forget/ignor

Re: Tracking tests over time

2007-07-12 Thread Ian Malpass
Andy Armstrong wrote: With TAP that'd require either adding a persistent identifier to tests, requiring that the test numbering doesn't change over time (new tests added at the end) or that there's some heuristic that attempts to track assertions that have been renumbered. Requiring that test

Re: [tapx-dev] Trailing backslash in TAP lines

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > Is the intention that > > ok 1 THIS IS NOT A \# TODO > > should be parsed as a normal test with the description "THIS IS NOT A # > TODO"? > > It doesn't seem to be doing that currently. That's interpreted as a TODO. Crap. Th

Re: [tapx-dev] Trailing backslash in TAP lines

2007-07-12 Thread Andy Armstrong
Ovid wrote: > - Original Message > From: Steffen Schwigon <[EMAIL PROTECTED]> > >> Should I correct my test output or should TAP::Parser handle lines >> with trailing backslash differently? > > I think that TAP::Parser's behavior is incorrect. The problem will be found > in the TAP::Pa

Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Andy Armstrong
Michael Peters wrote: > Why can't you just use test names? > > ok($is_ok, 'HTML nested bullet list, open tags (bug 5497)'); You certainly could - but I tend to think of them as descriptions rather than names and therefore not entirely ideal as the primary key. I don't really want to be forced n

Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Michael Peters
Andy Armstrong wrote: > I've just noticed that the MediaWiki team have a test harness that > tracks the life of individual assertions over time: > > * HTML nested bullet list, open tags (bug 5497) [Has never passed] > * HTML nested ordered list, open tags (bug 5497) [Has never passed]

Tracking tests over time

2007-07-12 Thread Andy Armstrong
I've just noticed that the MediaWiki team have a test harness that tracks the life of individual assertions over time: * HTML nested bullet list, open tags (bug 5497) [Has never passed] * HTML nested ordered list, open tags (bug 5497) [Has never passed] * Fuzz testing: image with bog

Re: [tapx-dev] Forking tests

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > Can you confirm that this works OK with your runtests? Yup. Works fine. It's me not forcing the shared memory value to be a number, so it looks like we had strange behavior due to padded NULLs. Not sure if this should be

Re: [tapx-dev] Forking tests

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > Taking out the shared memory calls and setting the test number > explicitly makes it work under runtests for me. Crap. I get the same results. I'm suspecting that the Test::Class problem is unrelated to this one (though I w

Re: [tapx-dev] Forking tests

2007-07-12 Thread Andy Armstrong
Ovid wrote: > I can never get it to work with the Test::Class code, using either prove or > runtests. In the regular .t style program, it succeeds if I use a leading > plan, but fails if I use a trailing plan with runtests: > > use Test::More tests => 3; > > Versus: > > use Test::More 'no

Re: [tapx-dev] Forking tests

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > > Parse errors: No plan found in TAP output > > D'oh - sorry - should have read the output from your runtests properly :) > > So we're losing the plan somehow. I can never get it to work with the Test::Class code, using eith

Re: [tapx-dev] Forking tests

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > Interestingly runtests is happy for me: > > [13:38] andy $ runtests -v forkingtest.pl > forkingtest...ok 1 - mytest called > ok 2 - in the child > ok 3 - in the parent > 1..3 > All 3 subtests passed > > Test Summary Repor

Re: [tapx-dev] Forking tests

2007-07-12 Thread Andy Armstrong
Andy Armstrong wrote: > Test Summary Report > --- > forkingtest.pl (Wstat: 0 Tests: 3 Failed: 0) > Parse errors: No plan found in TAP output > Files=1, Tests=3, 0 wallclock secs ( 0.03 cusr + 0.02 csys = 0.05 CPU) D'oh - sorry - should have read the output from your runtests p

Re: [tapx-dev] Forking tests

2007-07-12 Thread Andy Armstrong
Ovid wrote: > - Original Message > From: Andy Armstrong <[EMAIL PROTECTED]> > >> What happens if you explicitly set the test number to what it should be? > >>$test->current_test( 3 ); # < hard wired > > Exact same behavior, which implies (to me) that my use of s

Re: [tapx-dev] Forking tests

2007-07-12 Thread Andy Armstrong
Ovid wrote: > Basically, if I fork a child, when we return, Test::Builder will not have the > new test count, so I explicitly use shared memory [1]. While the above runs > just fine with prove, runtests (from the TAP::Parser distribution), fails > with the following: > > ok 1 - mytest called

Re: [tapx-dev] Forking tests

2007-07-12 Thread Ovid
- Original Message From: Andy Armstrong <[EMAIL PROTECTED]> > What happens if you explicitly set the test number to what it should be? >$test->current_test( 3 ); # < hard wired Exact same behavior, which implies (to me) that my use of shared memory is correct.

Re: [tapx-dev] Forking tests

2007-07-12 Thread Andy Armstrong
Ovid wrote: > Am I doing something really stupid here? Could I be using shared memory incorrectly? What happens if you explicitly set the test number to what it should be? if ( !defined( my $child_pid = fork() ) ) { diag "Cannot fork: $!"; } elsif ( !$child_pid ) { ok

Re: Forking tests

2007-07-12 Thread Ovid
- Original Message From: Adam Kennedy <[EMAIL PROTECTED]> > See the tests for Process::Backgroundable for a method I was using to > spawn a process via IPC::Run3 and backgrounding it, rather than forking... I've already got a workaround in place. My main concern is what's causing the e

Forking tests

2007-07-12 Thread Ovid
Hi all, I'm at wit's end about this and any guidance would be appreciated. I have a class which needs to maintain state based on pid. So how do I test that? I fork, of course (can't use threads as our Perl is built without thread support). That's when things go bad. As far as I can tell, Ta