Re: todo tests in the TAP Plan
- Original Message From: Michael G Schwern [EMAIL PROTECTED] Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Don't bother, its a poorly designed feature and likely unused. I don't want to see it pushed forward into TAP. OK, I'll ignore them then. Cheers, Ovid -- Buy the book -- http://www.oreilly.com/catalog/perlhks/ Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
Re: todo tests in the TAP Plan
On 8 Sep 2006, at 01:52, Michael G Schwern wrote: Adrian Howard wrote: Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? http://perl-qa.yi.org/index.php/TAP_version I meant in the context of Ovid's TAPx::Parser code. Rather than adding the old style Test.pm syntax to the existing parser, have an old and a current parser. Adrian
Re: todo tests in the TAP Plan
On 6 Sep 2006, at 14:33, Ovid wrote: - Original Message From: Sébastien Aperghis-Tramoni [EMAIL PROTECTED] Hmm, that's curious. However, if it's undocumented I would argue against supporting it right now. What benefit does it gain us? This comes from the Good Old Test.pm module: $ perl -MTest -e 'plan tests = 10, todo = [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? Adrian
Re: todo tests in the TAP Plan
Adrian Howard wrote: Maybe this is the right time to think about mechanisms supporting different versions of the TAP protocol? http://perl-qa.yi.org/index.php/TAP_version
Re: todo tests in the TAP Plan
- Original Message
From: Shlomi Fish [EMAIL PROTECTED]
t/sample-tests/todo in the Test-Harness distribution reads:
print DUMMY_TEST;
1..5 todo 3 2;
ok 1
ok 2
not ok 3
ok 4
ok 5
DUMMY_TEST
As one can see, the 1..5 plan is followed by the todo 3 2; directive.
This is supposed to indicate something about plan ahead todo tests. (Instead
of the # TODO directives in the individual tests' outputs. Now:
1. t/sample-tests/todo is being run by Test-Harness, which seems to think
the not ok 3 is a todo test.
2. This todo-enabled plan is not documented in:
http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod
3. TAPx-Parser version 0.20 cannot handle it.
--
I'd like to know what I should do about this feature, because right now I'm
trying to convert Test-Run to use TAPX::Harness, and this is giving me
problems.
Hmm, that's curious. However, if it's undocumented I would argue against
supporting it right now. What benefit does it gain us?
It also makes the grammar a tad more complicated (though not overly so). This
no longer works:
tap::= plan tests | tests plan
plan ::= '1..' positiveInteger \nInstead we'd have to do this:
tap ::= plan todoList \n tests | tests plan \n?
plan ::= '1..' positiveInteger
todoList ::= 'todo ' positiveInteger { ' ' positiveInteger }
We can only allow the todoList on a leading plan because one on a trailing plan
means we can't know the test results until the test run has completed (which
would suck in the case of infinite streams).
However, even allowing this on a leading plan is still problematic because
either the programmer needs to know the test numbers in advance to add that
list or the code somehow would need to deduce them. Then, we'd have to started
defining the semantics of what happens if we have that leading todoList in case
of one of those tests having a SKIP or TODO directive. I'm guessing that means
that leading todoList test numbers would be ignored in that case? I would
suggest that this is not a feature we want unless we can nail down its intent
and semantics quite carefully.
Does anyone know the rational behind it?
Cheers,
Ovid
--
Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
Re: todo tests in the TAP Plan
Selon Ovid [EMAIL PROTECTED]: As one can see, the 1..5 plan is followed by the todo 3 2; directive. This is supposed to indicate something about plan ahead todo tests. [...] I'd like to know what I should do about this feature, because right now I'm trying to convert Test-Run to use TAPX::Harness, and this is giving me problems. Hmm, that's curious. However, if it's undocumented I would argue against supporting it right now. What benefit does it gain us? This comes from the Good Old Test.pm module: $ perl -MTest -e 'plan tests = 10, todo = [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. We can only allow the todoList on a leading plan because one on a trailing plan means we can't know the test results until the test run has completed (which would suck in the case of infinite streams). However, even allowing this on a leading plan is still problematic because either the programmer needs to know the test numbers in advance to add that list or the code somehow would need to deduce them. Then, we'd have to started defining the semantics of what happens if we have that leading todoList in case of one of those tests having a SKIP or TODO directive. I'm guessing that means that leading todoList test numbers would be ignored in that case? I would suggest that this is not a feature we want unless we can nail down its intent and semantics quite carefully. Test.pm could only handle predefined test numbers (it doesn't have a no_plan option like Test::More). -- Sébastien Aperghis-Tramoni Close the world, txEn eht nepO.
Re: todo tests in the TAP Plan
- Original Message From: Sébastien Aperghis-Tramoni [EMAIL PROTECTED] Hmm, that's curious. However, if it's undocumented I would argue against supporting it right now. What benefit does it gain us? This comes from the Good Old Test.pm module: $ perl -MTest -e 'plan tests = 10, todo = [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Cheers, Ovid -- Buy the book -- http://www.oreilly.com/catalog/perlhks/ Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
Re: todo tests in the TAP Plan
Shlomi Fish wrote: t/sample-tests/todo in the Test-Harness distribution reads: print DUMMY_TEST; 1..5 todo 3 2; ok 1 ok 2 not ok 3 ok 4 ok 5 DUMMY_TEST As one can see, the 1..5 plan is followed by the todo 3 2; directive. This is supposed to indicate something about plan ahead todo tests. (Instead of the # TODO directives in the individual tests' outputs. Now: 1. t/sample-tests/todo is being run by Test-Harness, which seems to think the not ok 3 is a todo test. 2. This todo-enabled plan is not documented in: http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod 3. TAPx-Parser version 0.20 cannot handle it. -- I'd like to know what I should do about this feature, because right now I'm trying to convert Test-Run to use TAPX::Harness, and this is giving me problems. These old style todo tests were never really documented until I stumbled on them inside Test::Harness and Test.pm, nobody really understood, aren't very useful and weren't really used. They were deprecated a while ago. Since they were never really documented or used I just hid the deprecation notice inside Test::Harness as it was only interesting to folks hacking on it. This notice used to appear in Test::Harness but it appears to have been removed. =begin _deprecated Alternatively, you can specify a list of what tests are todo as part of the test header. 1..23 todo 5 12 23 This only works if the header appears at the beginning of the test. This style is Bdeprecated. =end _deprecated Don't push them forward into TAP. Any feature which relies on static test numbering is broken. Just let them die.
Re: todo tests in the TAP Plan
Ovid wrote: $ perl -MTest -e 'plan tests = 10, todo = [2,4];' 1..10 todo 2 4; As there are quite some test scripts out there that use it, staying compatible with it sounds like a prerequisite. By it do you mean Test.pm or Test.pm's todo feature? The former I can believe. The latter... I'd be interested to know of a single use in the wild. And if there is anyone using it they'd likely be better served by moving to Test::Legacy and inline TODO tests. Or backporting inline TODO tests to Test.pm. Ah, crud. I need to support it then. Bummer. I'll try to get a release out there when I can, then. Don't bother, its a poorly designed feature and likely unused. I don't want to see it pushed forward into TAP.
Re: todo tests in the TAP Plan
On Wednesday 06 September 2006 22:50, Michael G Schwern wrote: Shlomi Fish wrote: t/sample-tests/todo in the Test-Harness distribution reads: print DUMMY_TEST; 1..5 todo 3 2; ok 1 ok 2 not ok 3 ok 4 ok 5 DUMMY_TEST As one can see, the 1..5 plan is followed by the todo 3 2; directive. This is supposed to indicate something about plan ahead todo tests. (Instead of the # TODO directives in the individual tests' outputs. Now: 1. t/sample-tests/todo is being run by Test-Harness, which seems to think the not ok 3 is a todo test. 2. This todo-enabled plan is not documented in: http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod 3. TAPx-Parser version 0.20 cannot handle it. -- I'd like to know what I should do about this feature, because right now I'm trying to convert Test-Run to use TAPX::Harness, and this is giving me problems. These old style todo tests were never really documented until I stumbled on them inside Test::Harness and Test.pm, nobody really understood, aren't very useful and weren't really used. They were deprecated a while ago. Since they were never really documented or used I just hid the deprecation notice inside Test::Harness as it was only interesting to folks hacking on it. This notice used to appear in Test::Harness but it appears to have been removed. =begin _deprecated Alternatively, you can specify a list of what tests are todo as part of the test header. 1..23 todo 5 12 23 This only works if the header appears at the beginning of the test. This style is Bdeprecated. =end _deprecated Don't push them forward into TAP. Any feature which relies on static test numbering is broken. Just let them die. OK, thanks. I'll forward-port this sample-tests script to the new # TODO syntax , and would not take measurements to handle this situation. Thanks for the heads up. Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ Chuck Norris wrote a complete Perl 6 implementation in a day but then destroyed all evidence with his bare hands, so no one will know his secrets.
todo tests in the TAP Plan
t/sample-tests/todo in the Test-Harness distribution reads: print DUMMY_TEST; 1..5 todo 3 2; ok 1 ok 2 not ok 3 ok 4 ok 5 DUMMY_TEST As one can see, the 1..5 plan is followed by the todo 3 2; directive. This is supposed to indicate something about plan ahead todo tests. (Instead of the # TODO directives in the individual tests' outputs. Now: 1. t/sample-tests/todo is being run by Test-Harness, which seems to think the not ok 3 is a todo test. 2. This todo-enabled plan is not documented in: http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod 3. TAPx-Parser version 0.20 cannot handle it. -- I'd like to know what I should do about this feature, because right now I'm trying to convert Test-Run to use TAPX::Harness, and this is giving me problems. Regards, Shlomi Fish - Shlomi Fish [EMAIL PROTECTED] Homepage:http://www.shlomifish.org/ Chuck Norris wrote a complete Perl 6 implementation in a day but then destroyed all evidence with his bare hands, so no one will know his secrets.
