Re: "todo" tests in the TAP Plan
- Original Message From: Michael G Schwern <[EMAIL PROTECTED]> > > Ah, crud. I need to support it then. Bummer. I'll try to get a release > > out there when I can, then. > > Don't bother, its a poorly designed feature and likely unused. I don't want > to see it pushed forward into TAP. OK, I'll ignore them then. Cheers, Ovid -- Buy the book -- http://www.oreilly.com/catalog/perlhks/ Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
Re: "todo" tests in the TAP Plan
On 8 Sep 2006, at 01:52, Michael G Schwern wrote: Adrian Howard wrote: Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? http://perl-qa.yi.org/index.php/TAP_version I meant in the context of Ovid's TAPx::Parser code. Rather than adding the old style Test.pm syntax to the existing parser, have an "old" and a "current" parser. Adrian
Re: "todo" tests in the TAP Plan
Adrian Howard wrote: Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? http://perl-qa.yi.org/index.php/TAP_version
Re: "todo" tests in the TAP Plan
On 6 Sep 2006, at 14:33, Ovid wrote: - Original Message From: Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> Hmm, that's curious. However, if it's undocumented I would argue against supporting it right now. What benefit does it gain us? This comes from the Good Old Test.pm module: $ perl -MTest -e 'plan tests => 10, todo => [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? Adrian
Re: "todo" tests in the TAP Plan
On Wednesday 06 September 2006 22:50, Michael G Schwern wrote: > Shlomi Fish wrote: > > "t/sample-tests/todo" in the Test-Harness distribution reads: > > > > <<< > > print < > 1..5 todo 3 2; > > ok 1 > > ok 2 > > not ok 3 > > ok 4 > > ok 5 > > DUMMY_TEST > > > > As one can see, the "1..5" plan is followed by the "todo 3 2;" > > directive. This is supposed to indicate something about plan ahead todo > > tests. (Instead of the "# TODO" directives in the individual tests' > > outputs. Now: > > > > 1. t/sample-tests/todo is being run by Test-Harness, which seems to think > > the "not ok 3" is a todo test. > > > > 2. This todo-enabled plan is not documented in: > > > > http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod > > > > 3. TAPx-Parser version 0.20 cannot handle it. > > > > -- > > > > I'd like to know what I should do about this feature, because right now > > I'm trying to convert Test-Run to use TAPX::Harness, and this is giving > > me problems. > > These "old style" todo tests were never really documented until I stumbled > on them inside Test::Harness and Test.pm, nobody really understood, aren't > very useful and weren't really used. > > They were deprecated a while ago. Since they were never really documented > or used I just hid the deprecation notice inside Test::Harness as it was > only interesting to folks hacking on it. This notice used to appear in > Test::Harness but it appears to have been removed. > > =begin _deprecated > > Alternatively, you can specify a list of what tests are todo as part > of the test header. > > 1..23 todo 5 12 23 > > This only works if the header appears at the beginning of the test. > > This style is B. > > =end _deprecated > > > Don't push them forward into TAP. Any feature which relies on static test > numbering is broken. Just let them die. OK, thanks. I'll forward-port this sample-tests script to the new "# TODO" syntax , and would not take measurements to handle this situation. Thanks for the heads up. Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ Chuck Norris wrote a complete Perl 6 implementation in a day but then destroyed all evidence with his bare hands, so no one will know his secrets.
Re: "todo" tests in the TAP Plan
Ovid wrote: $ perl -MTest -e 'plan tests => 10, todo => [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. By "it" do you mean Test.pm or Test.pm's todo feature? The former I can believe. The latter... I'd be interested to know of a single use in the wild. And if there is anyone using it they'd likely be better served by moving to Test::Legacy and inline TODO tests. Or backporting inline TODO tests to Test.pm. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Don't bother, its a poorly designed feature and likely unused. I don't want to see it pushed forward into TAP.
Re: "todo" tests in the TAP Plan
Shlomi Fish wrote: "t/sample-tests/todo" in the Test-Harness distribution reads: <<< print
Re: "todo" tests in the TAP Plan
- Original Message From: Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> > > Hmm, that's curious. However, if it's undocumented I would argue against > > supporting it right now. What benefit does it gain us? > > This comes from the Good Old Test.pm module: > > $ perl -MTest -e 'plan tests => 10, todo => [2,4];' > 1..10 todo 2 4; > > As there are quite some test scripts out there that use it, staying > compatible with it sounds like a prerequisite. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Cheers, Ovid -- Buy the book -- http://www.oreilly.com/catalog/perlhks/ Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
Re: "todo" tests in the TAP Plan
Selon Ovid <[EMAIL PROTECTED]>: > > As one can see, the "1..5" plan is followed by the "todo 3 2;" directive. > > This is supposed to indicate something about plan ahead todo tests. > > [...] > > I'd like to know what I should do about this feature, because right now I'm > > trying to convert Test-Run to use TAPX::Harness, and this is giving me > > problems. > > Hmm, that's curious. However, if it's undocumented I would argue against > supporting it right now. What benefit does it gain us? This comes from the Good Old Test.pm module: $ perl -MTest -e 'plan tests => 10, todo => [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. > We can only allow the todoList on a leading plan because one on a trailing > plan means we can't know the test results until the test run has completed > (which would suck in the case of infinite streams). > > However, even allowing this on a leading plan is still problematic because > either the programmer needs to know the test numbers in advance to add that > list or the code somehow would need to deduce them. Then, we'd have to > started defining the semantics of what happens if we have that leading > todoList in case of one of those tests having a SKIP or TODO directive. I'm > guessing that means that leading todoList test numbers would be ignored in > that case? I would suggest that this is not a feature we want unless we can > nail down its intent and semantics quite carefully. Test.pm could only handle predefined test numbers (it doesn't have a "no_plan" option like Test::More). -- Sébastien Aperghis-Tramoni Close the world, txEn eht nepO.
Re: "todo" tests in the TAP Plan
On Sep 6, 2006, at 3:59 AM, Ovid wrote: Hmm, that's curious. However, if it's undocumented I would argue against supporting it right now. What benefit does it gain us? The flip to that is that we've always said that Test::Harness is the reference implementation. In a way, it is documented. -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:petdance
Re: "todo" tests in the TAP Plan
- Original Message
From: Shlomi Fish <[EMAIL PROTECTED]>
> "t/sample-tests/todo" in the Test-Harness distribution reads:
>
> <<<
> print < 1..5 todo 3 2;
> ok 1
> ok 2
> not ok 3
> ok 4
> ok 5
> DUMMY_TEST
> >>>
>
> As one can see, the "1..5" plan is followed by the "todo 3 2;" directive.
> This is supposed to indicate something about plan ahead todo tests. (Instead
> of the "# TODO" directives in the individual tests' outputs. Now:
>
> 1. t/sample-tests/todo is being run by Test-Harness, which seems to think
> the "not ok 3" is a todo test.
>
> 2. This todo-enabled plan is not documented in:
>
> http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod
>
> 3. TAPx-Parser version 0.20 cannot handle it.
>
> --
>
> I'd like to know what I should do about this feature, because right now I'm
> trying to convert Test-Run to use TAPX::Harness, and this is giving me
> problems.
Hmm, that's curious. However, if it's undocumented I would argue against
supporting it right now. What benefit does it gain us?
It also makes the grammar a tad more complicated (though not overly so). This
no longer works:
tap::= plan tests | tests plan
plan ::= '1..' positiveInteger "\n"Instead we'd have to do this:
tap ::= plan todoList "\n" tests | tests plan "\n"?
plan ::= '1..' positiveInteger
todoList ::= 'todo ' positiveInteger { ' ' positiveInteger }
We can only allow the todoList on a leading plan because one on a trailing plan
means we can't know the test results until the test run has completed (which
would suck in the case of infinite streams).
However, even allowing this on a leading plan is still problematic because
either the programmer needs to know the test numbers in advance to add that
list or the code somehow would need to deduce them. Then, we'd have to started
defining the semantics of what happens if we have that leading todoList in case
of one of those tests having a SKIP or TODO directive. I'm guessing that means
that leading todoList test numbers would be ignored in that case? I would
suggest that this is not a feature we want unless we can nail down its intent
and semantics quite carefully.
Does anyone know the rational behind it?
Cheers,
Ovid
--
Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
"todo" tests in the TAP Plan
"t/sample-tests/todo" in the Test-Harness distribution reads: <<< print <>> As one can see, the "1..5" plan is followed by the "todo 3 2;" directive. This is supposed to indicate something about plan ahead todo tests. (Instead of the "# TODO" directives in the individual tests' outputs. Now: 1. t/sample-tests/todo is being run by Test-Harness, which seems to think the "not ok 3" is a todo test. 2. This todo-enabled plan is not documented in: http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod 3. TAPx-Parser version 0.20 cannot handle it. -- I'd like to know what I should do about this feature, because right now I'm trying to convert Test-Run to use TAPX::Harness, and this is giving me problems. Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ Chuck Norris wrote a complete Perl 6 implementation in a day but then destroyed all evidence with his bare hands, so no one will know his secrets.
