Re: More on the TAP::Grammar
On 7/7/06, Ovid <[EMAIL PROTECTED]> wrote: This long-winded (as usual) explanation brings me around to my actual question: is there really any need to have the lexing and parsing stages clearly delineated? I can't see why, but from you mentioning several times that "that's a parser's job, not a lexer's", I assume you have some reason behind this. Am I missing something? Is there some desire that one might take the lexed tokens and direct them to another parser, for example? I started writing a single parser/lexer and found it to be a bit too convoluted. Separting the parsing and lexing is turning out simpler. Sounds like you're having the opposite experience. Possibly because you're using a real grammar and I'm using ad hoc code. Maybe I'm just gunshy after working with Test::Harness for so long. That's ok, the strength of different implementations is they take different approaches. Interface-wise, I'm looking at TAP::Harness' bits in terms of what behaviors a user might want to override. I'm trying to draw clear lines of responsibility so a user can play with the bits they care about without having to touch stuff they don't. Its very unlikely they'll want to override the lexer, unless they're writing a TAP extension, so I'm walling that off.
Re: More on the TAP::Grammar
- Original Message From: Ovid <[EMAIL PROTECTED]> > Also, since I had not seen the TAP::Harness work before, I didn't > know that a separate parser was already being planned. My only > motivation for writing TAP::Parser was my assumption that it was > needed and was not going to otherwise be written. If you already > are planning your own, mine would just be more wasted space on > the CPAN :) Never mind. I just read the QA wiki a bit more closely and see that you're encouraging other versions. Cheers, Ovid
Re: More on the TAP::Grammar
- Original Message From: Michael G Schwern <[EMAIL PROTECTED]> On 7/4/06, Ovid <[EMAIL PROTECTED]> wrote: > > Because we're discarding anything which does not look like a plan or a test > > line > > Don't discard them, just pass them through unaltered. Don't want to > lose any information. The /^#/ lines should be marked as comments. > Anything else is marked as junk but still passed through. I'll make that change. > > 1. The lexer croaks() if the planned tests and the actual tests don't > > match. > > I assume this behavior should be handled elsewhere? > > Doesn't sound like a lexer's job. Sounds more like the parser's. ... > But again, parser's job. ... > Also sounds like the parser's job. ... > Also, parser's job. | Originally I had the behavior the lexer and parser quite separate, but due to some precedence issues in the lexing, I scrapped my original design and took something which is conceptually similar to the Parse::RecDescent approach in that I "blurred" the difference between the lexing and parsing steps. I see in http://perl-qa.yi.org/index.php/TAP::Harness that there is a clearer separation of the lexing and parsing, but I deliberately abandoned that as not being needed in this case because it was overkill. Once I took that approach, things were much simpler to write. This long-winded (as usual) explanation brings me around to my actual question: is there really any need to have the lexing and parsing stages clearly delineated? I can't see why, but from you mentioning several times that "that's a parser's job, not a lexer's", I assume you have some reason behind this. Am I missing something? Is there some desire that one might take the lexed tokens and direct them to another parser, for example? Also, since I had not seen the TAP::Harness work before, I didn't know that a separate parser was already being planned. My only motivation for writing TAP::Parser was my assumption that it was needed and was not going to otherwise be written. If you already are planning your own, mine would just be more wasted space on the CPAN :) Cheers, Ovid
Re: TAP Grammar
On 7/5/06, Chris Dolan <[EMAIL PROTECTED]> wrote: On Jul 3, 2006, at 6:33 PM, chromatic wrote: > On Monday 03 July 2006 09:01, Jonathan T. Rockway wrote: > >> That said, I would be interested. I'm still trying to page all the >> perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any >> additional >> examples would helpful, interesting, and fun. For me, anyway :) > > Jerry Gay just wrote a PGE TAP grammar: > > <http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg> > > -- c I hacked a rudimentary test case for that grammar, borrowing heavily from Punie: https://svn.clotho.com/public/parrot-experiments/TAP/ excellent! i have the beginnings of a TAP parser in my working copy, i'll look at what you've got and synchronize them up. thanks for the test cases, it should be a simple matter to migrate them into tests in the parrot repo. i'll also make the makefile platform-independent, as it won't run on windows as it is. To run it, edit the first line of the Makefile to specify the path to PARROT_DIR and run "make". My tests found a few bugs in Jerry's grammar. My fixed version of that grammar is here: https://svn.clotho.com/public/parrot-experiments/TAP/grammars/TAP.pg i've applied your changes (with one minor mod: s/\n/$$/) to parrot revision 13168.) thanks and keep the patches coming! ~jerry
Re: TAP Grammar
On Jul 3, 2006, at 6:33 PM, chromatic wrote: On Monday 03 July 2006 09:01, Jonathan T. Rockway wrote: That said, I would be interested. I'm still trying to page all the perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any additional examples would helpful, interesting, and fun. For me, anyway :) Jerry Gay just wrote a PGE TAP grammar: <http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg> -- c I hacked a rudimentary test case for that grammar, borrowing heavily from Punie: https://svn.clotho.com/public/parrot-experiments/TAP/ To run it, edit the first line of the Makefile to specify the path to PARROT_DIR and run "make". My tests found a few bugs in Jerry's grammar. My fixed version of that grammar is here: https://svn.clotho.com/public/parrot-experiments/TAP/grammars/TAP.pg Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: More on the TAP::Grammar
On 7/4/06, Ovid <[EMAIL PROTECTED]> wrote:
Because we're discarding anything which does not look like a plan or a test line
Don't discard them, just pass them through unaltered. Don't want to
lose any information. The /^#/ lines should be marked as comments.
Anything else is marked as junk but still passed through.
Currently there are some assumptions I'm making, some of which I think are
wrong.
1. The lexer croaks() if the planned tests and the actual tests don't match.
I assume this behavior should be handled elsewhere?
Doesn't sound like a lexer's job. Sounds more like the parser's.
2. Croaks if numbered and non-numbered tests are mixed:
ok 1 - This is a test.
ok This is only a test
not ok 3 - Had this been an actual emergency ...
ok still don't have a number
This is perfectly ok. A virtual number is assigned.
This would be bad:
ok 1
ok
ok
ok 3
But again, parser's job.
3. Croaks if the tests are numbered but the numbers are out of sequence.
Also sounds like the parser's job.
4. Croaks if planned tests are a test number are less than one.
1..0 is a skip all.
Also, parser's job.
Questions:
1. Are non-integers allowed for test numbers? I know this has been proposed
for groups of tests.
It may have been proposed, but it is not currently allowed.
2. Is there a plan at some point to allow for test failure output to be
officially incorporated in TAP?
Yes. It will not be in the form of comments.
I know we discard that *now*, but in the future, people with a GUI are going to
want to click on a failure and see:
not ok 3 - Had this been an actual emergency ...
# Failed test 'Had this been an actual emergency ...'
# in foo.t at line 5.
# got: '42'
# expected: 'answer'
Instead of:
not ok 3 - No more info, sucks to be you.
They can still see that if you pass the comments through the
lexer/parser unaltered and allow the GUI to do whatever they like with
it.
Next, what interface do you want? I'm thinking something like this:
my $parser = TAP::Parser->new({
newlines => 1, # add newlines to output
I don't understand what this option would do. What does a parser output?
version => $optional_version_number
});
$parser->parse($tap); # croaks if it can't
What is $tap? A string? Then it can't parse a stream like we do now.
print $parser->plan;
foreach my $line ( $parser->test_lines ) {
print $line;
}
What work is the parser doing here if all you get is the original line
spat back?
Re: TAP Grammar
On 7/3/06, Ovid <[EMAIL PROTECTED]> wrote: - Original Message From: Michael G Schwern <[EMAIL PROTECTED]> > Diagnostic information *is* unreliable in TAP. > Do not parse it. It is now being discarded. Don't discard it, just pass it straight through. Consider the use case of wanting to do colorized TAP output. Or a TAP viewer with collapsable/expandable comment sections. > Test::Harness handles directives case insensitively. I don't see any > particular reason to stop. OK. What about 'ok' and 'not ok'? It seems inconsistent to allow one to be case-insensitive and not the other. Might be inconsistent but that's the way Test::Harness works. I'd rather err on the side of being more strict when possible. We can always losen it up later. Besides, its never been a problem before. YAGNI.
More on the TAP::Grammar
Hi all,
I'm going to be heading out town tonight and will be gone for the next two
days, but don't take my silence to mean that I've dropped this. I have started
on the TAP::Parser module and have the lexer *mostly* working. There are a
couple of extra tests I need to write for it and it will be done. Also,
because the grammar itself will be context-free, much of the "awareness" of the
parser is being pushed into the lexer (for example, it croaks if it finds more
than one plan).
Because we're discarding anything which does not look like a plan or a test
line, the following:
TAP version 1
1..3
ok 1 - This is a test.
... and bogus info
ok 2 - This is only a test
# this will be discarded
!!! More garbage!
not ok 3 - Had this been an actual emergency ...
# Failed test 'Had this been an actual emergency ...'
# in foo.t at line 5.
# got: '42'
# expected: 'answer'
Gets reduced down to the following:
1..3
ok 1 - This is a test.
ok 2 - This is only a test
not ok 3 - Had this been an actual emergency ...
Currently there are some assumptions I'm making, some of which I think are
wrong.
1. The lexer croaks() if the planned tests and the actual tests don't match.
I assume this behavior should be handled elsewhere?
2. Croaks if numbered and non-numbered tests are mixed:
ok 1 - This is a test.
ok This is only a test
not ok 3 - Had this been an actual emergency ...
ok still don't have a number
3. Croaks if the tests are numbered but the numbers are out of sequence.
4. Croaks if planned tests are a test number are less than one.
Questions:
1. Are non-integers allowed for test numbers? I know this has been proposed
for groups of tests.
2. Is there a plan at some point to allow for test failure output to be
officially incorporated in TAP? I know we discard that *now*, but in the
future, people with a GUI are going to want to click on a failure and see:
not ok 3 - Had this been an actual emergency ...
# Failed test 'Had this been an actual emergency ...'
# in foo.t at line 5.
# got: '42'
# expected: 'answer'
Instead of:
not ok 3 - No more info, sucks to be you.
Next, what interface do you want? I'm thinking something like this:
my $parser = TAP::Parser->new({
newlines => 1, # add newlines to output
version => $optional_version_number
});
$parser->parse($tap); # croaks if it can't
print $parser->plan;
foreach my $line ( $parser->test_lines ) {
print $line;
}
I'm sure there's more information I can capture and provide for you, so tell me
what you want (if I can easily provide it, I will).
Cheers,
Ovid
-- If this message is a response to a question on a mailing list, please send
follow up questions to the list.
Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
Re: TAP Grammar
I would say that even if nobody else is interested, if you're interested, do it :) That said, I would be interested. I'm still trying to page all the perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any additional examples would helpful, interesting, and fun. For me, anyway :) Regards, Jonathan Rockway Ovid wrote: Hi all, I would still like to be in a position to write a grammar for TAP, but I've heard no answers to my questions. Should I assume that a formal grammar is not wanted/desired at this point? Cheers, Ovid
Re: TAP Grammar
On Monday 03 July 2006 09:01, Jonathan T. Rockway wrote: > That said, I would be interested. I'm still trying to page all the > perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any additional > examples would helpful, interesting, and fun. For me, anyway :) Jerry Gay just wrote a PGE TAP grammar: <http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg> -- c
Re: TAP Grammar
- Original Message From: Michael G Schwern <[EMAIL PROTECTED]> > Diagnostic information *is* unreliable in TAP. > Do not parse it. It is now being discarded. > Test::Harness does not allow a leading space. There's no reason to start. OK. Thanks. > Test::Harness handles directives case insensitively. I don't see any > particular reason to stop. OK. What about 'ok' and 'not ok'? It seems inconsistent to allow one to be case-insensitive and not the other. > We discussed this at YAPC and all agreed: the first extension to TAP > should be putting a version number into TAP. About the only other > thing we agreed on is that the version should be a simple integer. So > the format is open. I'll ignore this for now, then. > > More that one person has expressed interest in a TAP::Parser and all > > we need is for those officially supporting/maintaining TAP to say > > "make it so". > What community do you program for? Just do it, man. Get approval / > fix it later. Doing it now. Tougher than I thought! Due to some weird semantics involved, I've already jettisoned HOP::Lexer and have rolled my own :( Not sure when it will be done. Cheers, Ovid
Re: TAP Grammar
On 7/3/06, Ovid <[EMAIL PROTECTED]> wrote: Currently, the way that Test::Harness::TAP reads, I should properly discard anything which is not a plan, test or diagnostic output. However, test failure output and programmer supplied diagnostic output need to be disambiguated or diagnostic information is unreliable. Diagnostic information *is* unreliable in TAP. There is no way in TAP to provide structured diagnostics. None. Zero. All you have is unparsable diagnostics (comments). There is *no format* to those diagnostics. What Test::Builder is outputting is *not reliable*. Do not parse it. A diagnostic format is something which needs to be added to TAP, but nothing currently exists (we tossed around some ideas at YAPC). Do not let this stop you from writing a formal TAP grammar. You can go ahead and spec it out as it currently stands. Writing a TAP diagnostic format is orthoganal to writing a grammar for the current version of TAP. Also, it's ambiguous from the documentation whether or not whitespace is allowed before the qr/(?:not )ok.*\n/. I think it's implied that no leading whitespace is allowed but that could be incorrect. Test::Harness does not allow a leading space. There's no reason to start. Also, regarding directives, the POD shows both "# skip" and "# SKIP". I assume only the latter was intended, but I bet that will break things. Test::Harness handles directives case insensitively. I don't see any particular reason to stop. Also, with discussion of upgrades to TAP in the future, unless there's a guarantee that all upgrades will be backwards compatible to the current implementation, we need version numbers in the output. Again, I've not heard any response to this. We discussed this at YAPC and all agreed: the first extension to TAP should be putting a version number into TAP. About the only other thing we agreed on is that the version should be a simple integer. So the format is open. Since anything not understood is ignored (and I'm liking that feature) we don't have to worry much about backwards compatibility. So probably something as simple as... TAP Version 2 1..2 ok 1 ok 2 though I'm sure there's plenty of proposals floating around out there. More that one person has expressed interest in a TAP::Parser and all we need is for those officially supporting/maintaining TAP to say "make it so". What community do you program for? Just do it, man. Get approval / fix it later.
Re: TAP Grammar
- Original Message > From: chromatic <[EMAIL PROTECTED]> > > On Monday 03 July 2006 09:12, Ovid wrote: > > I am tempted, but there are some problems with parsing TAP output as it > > currently stands. I can write a grammar for it, but the grammar can easily > > produce some output which doesn't make sense. > Isn't this the syntactic/semantic problem common to all grammars? No, this is a different issue. There are some problems that I'm not worrying about simply because they can't readily be solved, but there are others which are caused by either ambiguity in the spec or lack of any official declaration as to what to do. Currently, the way that Test::Harness::TAP reads, I should properly discard anything which is not a plan, test or diagnostic output. However, test failure output and programmer supplied diagnostic output need to be disambiguated or diagnostic information is unreliable. I've started hacking on TAP::Parser now and it turns out to be a thorny issue. To make it easier, I'm relying on the following quote fom the POD: Any output line that is not a plan, a test line or a diagnostic is incorrect. How a harness handles the incorrect line is undefined. Test::Harness silently ignores incorrect lines, but will become more stringent in the future. In other words, that means that any diag() or test failure messages will automatically be grouped together. This behavior is incorrect as different things need to be different. All you have to do is write "diag 'read() failed'" to potentially confuse someone reading the output. Also, it's ambiguous from the documentation whether or not whitespace is allowed before the qr/(?:not )ok.*\n/. I think it's implied that no leading whitespace is allowed but that could be incorrect. Also, regarding directives, the POD shows both "# skip" and "# SKIP". I assume only the latter was intended, but I bet that will break things. Also, with discussion of upgrades to TAP in the future, unless there's a guarantee that all upgrades will be backwards compatible to the current implementation, we need version numbers in the output. Again, I've not heard any response to this. More that one person has expressed interest in a TAP::Parser and all we need is for those officially supporting/maintaining TAP to say "make it so". Cheers, Ovid -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
Re: TAP Grammar
On Jul 3, 2006, at 5:52 AM, Ovid wrote: Hi all, I would still like to be in a position to write a grammar for TAP, but I've heard no answers to my questions. Should I assume that a formal grammar is not wanted/desired at this point? No no no, please do. I will be glad to put it in TAP distro. -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:petdance
Re: TAP Grammar
On Monday 03 July 2006 09:12, Ovid wrote: > I am tempted, but there are some problems with parsing TAP output as it > currently stands. I can write a grammar for it, but the grammar can easily > produce some output which doesn't make sense. Isn't this the syntactic/semantic problem common to all grammars? -- c
Re: TAP Grammar
- Original Message From: Jonathan T. Rockway <[EMAIL PROTECTED]> > I would say that even if nobody else is interested, if you're > interested, do it :) I am tempted, but there are some problems with parsing TAP output as it currently stands. I can write a grammar for it, but the grammar can easily produce some output which doesn't make sense. Thus, I'm stuck in the awkard position of needing fundamental issues resolved before I can proceed in a confident manner. Cheers, Ovid -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
TAP Grammar
Hi all, I would still like to be in a position to write a grammar for TAP, but I've heard no answers to my questions. Should I assume that a formal grammar is not wanted/desired at this point? Cheers, Ovid -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
