Re: TAPx::Parser 0.02
This is cool, Ovid! I think you're definitely on the right track.
Thoughts:
- I'd like an option to automatically s{ \A \s* - \s+ }{} all test
descriptions. I bet a lot of people would end up doing this
themselves, including myself.
- Speaking of that step, the underscore in your tokenizing method
(_lex) denotes it as private. What if I wanted to massage the tokens
after that phase? Specifying after_lexing hooks might be useful.
- I'm not sure how closely you're trying to follow a traditional,
multi-phase parser. If the lexer's job is to simply turn TAP output
into data structures, why does it die on the semantic error of a
forgotten plan? (10-lex.t line 183) Sounds like that's the job of the
parser.
- I mentioned this in another reply, but I'll phrase it differently
here: Having the lexer and parser die immediately on error is easy,
yes. But having the components recording _all_ errors for later
examination by the user would be very, very helpful.
Your tests are very descriptive well organized.
On 7/8/06, Ovid [EMAIL PROTECTED] wrote:
Hi all,
The next version of my TAP parser is at
http://users.easystreet.com/ovid/downloads/TAPx-Parser-0.02.tar.gz
It's still not complete, but it's a lot further along than it was. Some notes
from Changes:
0.028 June, 2006
- Moved some lexer responsibility to the parser. This will allow us
to eventually parse streams.
- Properly track passed/failed tests, even accounting for TODO.
- Added support for comments and unknown lines.
- Allow explicit and inferred test numbers to be mixed.
- Allow escaped hashe marks in the test description.
- Renamed to TAPx::Parser. Will probably rename it again.
Note that it does not yet handle streams, but that's kind of low on my list
right now (though it is on my list).
Just to give you an idea of what it can do, for this test:
ok 1 - input file opened
We get this:
my $test = shift @results;
isa_ok $test, $TEST;
ok $test-is_test, '... and it should identify itself as a test';
is $test-ok, 'ok', '... and it should have the correct ok()';
ok $test-is_ok, '... and the correct boolean version of ok()';
is $test-number, 1, '... and have the correct test number';
is $test-description, '- input file opened',
'... and the correct description';
ok !$test-directive, '... and not have a directive';
ok !$test-explanation, '... or a directive explanation';
ok !$test-is_skip, '... and it is not a SKIPped test';
ok !$test-is_todo, '... nor a TODO test';
is $test-as_string, 'ok 1 - input file opened',
'... and its string representation should be correct';
There are still some bugs, but it's definitely getting there.
Cheers,
Ovid
-- If this message is a response to a question on a mailing list, please send
follow up questions to the list.
Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
--
Ian Langworth
Re: TAPx::Parser 0.02
- Original Message
From: Ian Langworth [EMAIL PROTECTED]
This is cool, Ovid! I think you're definitely on the right track.
Thanks!
Thoughts:
- I'd like an option to automatically s{ \A \s* - \s+ }{} all test
descriptions. I bet a lot of people would end up doing this
themselves, including myself.
Hmm, I can probably add that in fairly easily.
- Speaking of that step, the underscore in your tokenizing method
(_lex) denotes it as private. What if I wanted to massage the tokens
after that phase? Specifying after_lexing hooks might be useful.
Can you show me some example syntax of what you'd want to do? The problem is,
the lexing and parsing are very tightly coupled. Allowing people to massage
the lexer output directly is asking for trouble.
- I'm not sure how closely you're trying to follow a traditional,
multi-phase parser. If the lexer's job is to simply turn TAP output
into data structures, why does it die on the semantic error of a
forgotten plan? (10-lex.t line 183) Sounds like that's the job of the
parser.
Originally I had a clearer distinction between lexing and parsing, but there
were three reasons I abandonded this:
1. TAP is line based and allows non-parseable junk.
2. Context is very important and much of that is easy to track while lexing.
3. Due to some lexing ambiguities raised by precedence issues, I found that
the lexer would need much finer-grained tokens then I was creating and that
made the grammar more complicated.
However, you are correct that the lexer should not be dying. In fact, neither
should the parser. I decided last night that all errors should be recorded, as
you were wanting.
Your tests are very descriptive well organized.
Really? I thought I was a bit sloppy with 'em :)
Cheers,
Ovid
Re: TAPx::Parser 0.02
Ways to tweak the TAP structure before handing it to the parser? The
removal of leading hyphens thing in the previous replies, sanitizing
sensitive information (maybe the non-employees shouldn't see IP
addresses in the output when they see the parsed result), who knows.
My opinion is that playing with the tokenizer output shouldn't be
discouraged. In someone breaks the structure, however, that's their
problem. You're the one with vetitive power in the end, though :-)
On 7/9/06, Ovid [EMAIL PROTECTED] wrote:
- Original Message
From: Ian Langworth [EMAIL PROTECTED]
This is cool, Ovid! I think you're definitely on the right track.
Thanks!
Thoughts:
- I'd like an option to automatically s{ \A \s* - \s+ }{} all test
descriptions. I bet a lot of people would end up doing this
themselves, including myself.
Hmm, I can probably add that in fairly easily.
- Speaking of that step, the underscore in your tokenizing method
(_lex) denotes it as private. What if I wanted to massage the tokens
after that phase? Specifying after_lexing hooks might be useful.
Can you show me some example syntax of what you'd want to do? The problem is,
the lexing and parsing are very tightly coupled. Allowing people to massage
the lexer output directly is asking for trouble.
- I'm not sure how closely you're trying to follow a traditional,
multi-phase parser. If the lexer's job is to simply turn TAP output
into data structures, why does it die on the semantic error of a
forgotten plan? (10-lex.t line 183) Sounds like that's the job of the
parser.
Originally I had a clearer distinction between lexing and parsing, but there
were three reasons I abandonded this:
1. TAP is line based and allows non-parseable junk.
2. Context is very important and much of that is easy to track while lexing.
3. Due to some lexing ambiguities raised by precedence issues, I found that
the lexer would need much finer-grained tokens then I was creating and that
made the grammar more complicated.
However, you are correct that the lexer should not be dying. In fact, neither
should the parser. I decided last night that all errors should be recorded, as
you were wanting.
Your tests are very descriptive well organized.
Really? I thought I was a bit sloppy with 'em :)
Cheers,
Ovid
--
Ian Langworth
TAPx::Parser 0.02
Hi all, The next version of my TAP parser is at http://users.easystreet.com/ovid/downloads/TAPx-Parser-0.02.tar.gz It's still not complete, but it's a lot further along than it was. Some notes from Changes: 0.028 June, 2006 - Moved some lexer responsibility to the parser. This will allow us to eventually parse streams. - Properly track passed/failed tests, even accounting for TODO. - Added support for comments and unknown lines. - Allow explicit and inferred test numbers to be mixed. - Allow escaped hashe marks in the test description. - Renamed to TAPx::Parser. Will probably rename it again. Note that it does not yet handle streams, but that's kind of low on my list right now (though it is on my list). Just to give you an idea of what it can do, for this test: ok 1 - input file opened We get this: my $test = shift @results; isa_ok $test, $TEST; ok $test-is_test, '... and it should identify itself as a test'; is $test-ok, 'ok', '... and it should have the correct ok()'; ok $test-is_ok, '... and the correct boolean version of ok()'; is $test-number, 1, '... and have the correct test number'; is $test-description, '- input file opened', '... and the correct description'; ok !$test-directive, '... and not have a directive'; ok !$test-explanation, '... or a directive explanation'; ok !$test-is_skip, '... and it is not a SKIPped test'; ok !$test-is_todo, '... nor a TODO test'; is $test-as_string, 'ok 1 - input file opened', '... and its string representation should be correct'; There are still some bugs, but it's definitely getting there. Cheers, Ovid -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
