Re: Pod::Critic?

[jerry gay]

On 6/6/07, Ian Malpass <[EMAIL PROTECTED]> wrote:

I've been documenting lately, so I'm a bit POD-focussed at the moment :)
I was pondering creating Pod::Critic, as a documentation analogue of
Perl::Critic.

Clearly it's not so easy to give hard-and-fast rules about
documentation, but I thought it might be useful as a framework for
enforcing internal documentation standards and "house styles", even if
it didn't ship with many policies itself.

Policies I've thought of:

 * Has NAME
 * Has SYNOPSIS
 * Has copyright details
 * Has license details
 * Method docs have examples
 * No spelling errors (borrowing Pod::Spell)

Other more vague/less useful ones, perhaps:

 * Module names are links
 * Method names are in C<> sequences

Some or all of these may not be relevant for particular instances, and I
suspect it'll be less common to use an out-of-the-box set of policies.

Anyone got any thoughts?

Ian


this would be a help to the parrot project. we have coding standards
and have used perl::critic to good effect. we've also been working on
documentation standards, and a flexible and configurable tool that
makes it easy to enforce those standards would be most welcome.

i can think of a number of policies that we would like to enforce, and
i would be happy to provide them (and get them working) if this
project takes off.
~jerry

Re: YAML?

[jerry gay]

On 3/19/07, Mark Fowler <[EMAIL PROTECTED]> wrote:


On 19 Mar 2007, at 13:03, Adrian Howard wrote:

> Nope. It's to add some simple YAML-ish output in places to help
> clarify things. It's also a very small subset of YAML rather than
> the full YAML spec.
> See 

Why not JSON?  http://www.json.org


+1


It's

a) Already implemented in a large number of libraries on a large
number of programming languages.
b) Has wider take up than YAML, or at least more mindshare in the
world, and when you're pushing a standard, mindshare is what you want.
c) Doesn't require a separate standard (where, defining a subset of
YAML does.  It's probably not enough to just go with simple examples)
d) Still human readable and computer readable.


might i add,
e) it's *incredibly easy* both to produce, and to parse
f) YAML is overkill, if JSON meets the requirements.
g) it works on parrot already (selfish, i know)

~jerry

Re: What the 'runtests' output looks like

[jerry gay]

On 1/15/07, Ovid <[EMAIL PROTECTED]> wrote:

Hi all,

If you've not checked out TAPx::Parser lately, here's what the
'runtests' output looks like:

  http://publius-ovidius.livejournal.com/222624.html


it's nice to see color output (too bad it won't work on
windows--cmd.exe's problem, not yours.)

i'd like to see skipped tests, failed todo, and unexpectedly
successful todo tests in colors other than pass or fail. have a look
at the pugs smoke output for some ideas: http://smoke.pugscode.org/.
you'll see yellow for unexpected success, darker green for expected
todo failures, and lighter green for skipped tests. there may be other
color schemes out there for displaying test results, but if there are,
i'm not aware of them.

in any case, different things should appear different, and similar
things should appear the same. this will help to preserve the
principle of least surprise.
~jerry

Re: TAPx::Parser 0.05_03

[jerry gay]

On 1/8/07, Ovid <[EMAIL PROTECTED]> wrote:

It's on its way to the CPAN now.  Thanks for lots of feedback.  I'd
really love to hear from Windows users as to whether or not it seems to
work OK there.


28 regression tests are failing, otherwise, all tests are passing.

t/010-regression.t   28  7168   636   28  1 24 46 68 91 114 136 158 181 203 226
 249 272 295 317 339 362 385 408 431
 454 477 500 523 546 569 592 615

they're all failing the same way, too:
t/010-regression...NOK 615/0#   Failed test ''too_many' should parse suc
cessfully'
#   at t/010-regression.t line 2257.
# open3: IO::Pipe: Can't spawn-NOWAIT: No such file or directory at C:\usr\local
\perl-5.8.8\cpan\build\TAPx-Parser-0.50_03\blib\lib/TAPx/Parser/Source.pm line 1
03

i'll see if i can figure it out.
~jerry

Re: Comment about BAIL_OUT

[jerry gay]

On 1/4/07, Andy Lester <[EMAIL PROTECTED]> wrote:


On Jan 4, 2007, at 11:21 PM, Eric Wilhelm wrote:

>> No.  You either have tests that are ordered, or you don't.
>
> Stated as if it were some sort of immutable law of the universe!

It is as far as Test::Harness goes.  Test::Harness doesn't have any
sort of idea of what connects tests together.


don't we have BEGIN, INIT, and CHECK blocks to deal with this?
~jerry

Re: TAP 2.0

[jerry gay]

On 9/29/06, Ovid <[EMAIL PROTECTED]> wrote:

got:  this is line 1
  this is line 2
  this is line 3
expected: this is line 1
  this is line b
  this is line 3


i prefer this syntax, as it's easier to parse in languages without
builtin heredocs. however, it bothers me that with this syntax it's
difficult to tell if and how many leading spaces the first line of got
or expected output contains. may i propose instead that ' got:', '
   expected:', etc. appear on their own line, and a fixed number of
spaces precede the output. that makes it easy to parse both visually
and programmaticly.

got:
   this is line 1
  this is line 2
 this is line 3

in this example, there are five spaces before 'got:', and ten before
the output. additionally, each line of output contains two spaces
(some before, and some after.) it's not easy to visually parse
differences in trailing spaces, but i'm not sure it's worth addressing
that problem, as any fix is likely to be quite ugly, and unnecessary
in the majority of cases.

~jerry

Re: Time to pack

[jerry gay]

On 9/28/06, Ovid <[EMAIL PROTECTED]> wrote:

Can someone tell me why this test fails?




your pack template is one item long. try C<'u*'> instead.
~jerry

TAP parsing and standard errors -- changing the problem (was Re: Merging STDOUT and STDERR)

[jerry gay]

it is obvious that, in the general case, managing multiple streams on
multiple filehandles using existing technologies is a *hard* problem.
it does, however (and thanks for the hint barrie) seem possible to
solve the problem for special cases--those you can control.

On 9/22/06, Barrie Slaymaker <[EMAIL PROTECTED]> wrote:

If you can patch the source process' program, you can try disabling
buffering and having it tag each chunk of output to a single stream, or
having it wait for "user" (i.e. parent process) input after each
significant bit of output are two decent approaches.


first of all, i think this is calling for a TAP Emitter
(TAPx::Emitter?), which new harnesses can use to emit TAP. if there's
an emitter that plays friendly with TAPx::Parser, especially in the
case of streamed TAP (TAPx::Emitter::Streams?) using one of the
methods barrie suggests, or some other equally solid approach, you
will have solved the problem for some subset of the problem space.

although it is not your final goal, (work everywhere perfectly now) it
does achieve some subgoals, and it is a direction to move forward.
open the door for others to use your shiny new TAP parser in the way
they want by providing a means to do so. there's no denying it will be
seen as a Good Thing.

~jerry

Re: Merging STDOUT and STDERR

[jerry gay]

On 9/21/06, Ovid <[EMAIL PROTECTED]> wrote:

OK, I'm stuck.  I've read through IPC::Open3 and friends, I've looked through 
CPANPLUS, I've tried to solve this problem but no matter how hard I try, while 
I can fetch both STDOUT and STDERR, I cannot guarantee that they're in synch.  
That's my big problem and

So I'm going to head over to Perlmonks and ask there.  It seems like, unless 
the source of the data ensures that everything is going to the same filehandle, 
I can't reliably solve this problem.  Lots of folks have offered suggestions 
and for that I am very grateful, but what about the 'must be in synch' problem? 
 That's the one I'm struggling with.


do you have a test that clearly shows the failure? if so, i can try to
code up an IPC::Run solution. it shouldn't take me very long to
determine if it's feasible.
~jerry

Re: New TAP Grammar

[jerry gay]

On 9/14/06, Ovid <[EMAIL PROTECTED]> wrote:

- Original Message 
From: jerry gay <[EMAIL PROTECTED]>

> thanks for the responses. i've done a second take on the TAP grammar
> in perl6 format, with the help of patrick michaud. i'm pretty sure
> it's correct now, at least as per your previous grammar. have a look:

I'll update my grammar as I can and I'll send it along and I'll try to 
incorporate your work.  It's great to see a Perl6 version.


:)


Do Perl6 grammars support semantics along with syntax?  Each test number should 
actually be one greater than the previous test number, but a traditional EBNF 
can't support that.


yes, they do support semantics (via closures.) you can keep a state
variable around that lets you know the last test's number. it can be
done in perl6 or parrot by writing a custom rule, since rules and subs
are the same thing. i wouldn't do that in the parser, though, it has a
bad code smell to me. i think it would better go in a semantic
analyser.


~ there are some places i took liberties (eg. \d versus [:digit:], \N
> verses [:print:] (are comments, directives, and description really
> limited to printable characters?).)

Hmm, never thought about that.  I don't think there's anything really wrong 
with non-printable characters (are null bytes a problem?), but I'm not sure how 
to represent that.


it can be represented in perl6 quite easily: C<< <+print & \N> >> is
one way, but i don't think there's trouble in non-printables in tap
comments.



> as an aside, patrick asked why C<1..0> was valid. i explained
> C, and he asked why it wasn't C<1..> or the more perl6ish
> C<1..*>. i know TAP isn't perl-only, but i have to say, C<1..*> is
> really growing on me... 'one to whatever.' it looks and sounds better
> than 'one to zero.' anyone else have thoughts on that?

'1..0' actually means 'skip all'.  '1..' or '1..*' would be useful for 
indicating an infinite stream.  'no_plan' TAP has the plan at the end.


silly me, i knew that. thanks for the reminder.
~jerry

Re: New TAP Grammar

[jerry gay]

thanks for the responses. i've done a second take on the TAP grammar
in perl6 format, with the help of patrick michaud. i'm pretty sure
it's correct now, at least as per your previous grammar. have a look:

grammar TAP;

## Ovid's TAP grammar, translated, corrected, and rendered idiomatic
## NOTE: not yet extended to deal with 'Bail out!' etc.
token tap {   |   * }
token plan{ <'1..'> \d+ \n }
token lines   { + }
token line{  |  }
token test{
   
   [ <' '> (<[1..9]> \d*) ]?  ## assumes a single space, not \h+
   \h* ? \h* ? \n
 }
token status  { <'not '>? <'ok'> }
token description {  <-[#\n]>+ }
token directive   {  <'# '> [:i todo | skip ] \N* }
token comment { <'#'> \N* \n }

notes:
~ there are some places i took liberties (eg. \d versus [:digit:], \N
verses [:print:] (are comments, directives, and description really
limited to printable characters?).) this leaves fewer named rules,
which is a significant optimization. if it is incorrect, it can be
modified to match the semantics of ovid's grammar more closely.
~ removed unnecessary indirections, like the 'tests' rule
~ reduced backtracking
~ ovid defines lines/line, but never uses them. this is fixed
~ not yet running on parrot against the straps tests, so some
assumptions about whitespace may be incorrect
~ assumed ordering of comments/tests is significant, otherwise 'lines'
can be replaced with the body of 'line'

overall, i think it looks really clean in perl6, and it will parse
really nicely. $ will be an array of all the lines, and
each element of $ will have either a  or a 
key.

as an aside, patrick asked why C<1..0> was valid. i explained
C, and he asked why it wasn't C<1..> or the more perl6ish
C<1..*>. i know TAP isn't perl-only, but i have to say, C<1..*> is
really growing on me... 'one to whatever.' it looks and sounds better
than 'one to zero.' anyone else have thoughts on that?

~jerry

Re: New TAP Grammar

[jerry gay]

On 9/14/06, Ovid <[EMAIL PROTECTED]> wrote:

I'm not particularly gifted with grammars, so corrections welcome.

The corrected TAP grammar:

 digit  ::= [:digit:]
 character  ::= ([:print:] - "\n")
 positiveInteger::= ( digit - '0' ) {digit}
 nonNegativeInteger ::= digit {digit}

 tap ::= plan tests | tests plan {comment}
 plan::= '1..' nonNegativeInteger "\n"
 lines   ::= line {lines}
 line::= (comment | test) "\n"
 tests   ::= test {test}
 test::= status positiveInteger? description? directive?
 status  ::= 'not '? 'ok '
 description ::= (character - (digit '#')) {character - '#'}
 directive   ::= '#' ( 'TODO' | 'SKIP' ) ' ' {character}
 comment ::= '#' {character}


i translated this grammar to a perl 6 grammar. i know it has some bugs
in it (probably 'rule' vs. 'token' among others) but it's a good
start. from translating it, there are a few things that stick out.

first, the grammar:

grammar TAP;

token digit  { \d }
token character  { <+print-[\n]> }
token positiveInteger{ <+digit-[0]> * }
token nonNegativeInteger { + }

rule  tap { [   |   ? ] }
rule  plan{ <'1..'>  \n }
rule  lines   { + }
rule  line{ [  |  ] \n }
rule  tests   { + }
rule  test{  ? ? ? }
rule  status  { [ not \s ]? ok }
rule  description { <-[#]-digit> <-[#]>* }
rule  directive   { <'#'> [ :ignorecase todo | skip ] * }
rule  comment { <'#'> * }

~ i thought descriptions started with a hyphen C<->, not an octothorpe C<#>.
~ doesn't  belong before  ??
  C<# SKIP comment> rather than C<- comment #SKIP>
~ schwern already mentioned 'skip_all'
~ missing 'Bail out!'

in any case, it looks pretty good, and i'll try to keep my parrot TAP
parser grammar in line with the TAPx::Parser grammar as it develops.

~jerry
btw, i love writing grammars in perl6. what kind of a geek does that
make me? eh, who cares. grammars rock!

Re: post-YAPC::Europe CPANTS news

[jerry gay]

On 9/6/06, chromatic <[EMAIL PROTECTED]> wrote:

On Wednesday 06 September 2006 02:53, Thomas Klausner wrote:

> - buildtool_not_executable
> Check if the buildtool (Makefile.PL, Build.PL) are not executable
> (and thus need to be called with 'perl Build.PL' thereby specifying
> which exact version of Perl you want)

I'm not sure of the value of this one; how does an author make the buildtool
executable on Windows, for example?  I have the impression (not using
Windows) that users must always call the tool with 'perl Build.PL' on such
platforms.

Am I mistaken?


on windows the command interpreter uses file associations instead of
C<#!>. files with extension C<.pl> are usually associated with
C, and invoked without specifying C on the command
line. this is user-modifiable, so it's possible to configure files
with the C<.pl> extension to be associated with another executable, or
none at all.
~jerry

soliciting TAP comments from language designers (where to begin?)

[jerry gay]

as i work on parrot, i happen to know a few language designers. i know
they're interested in providing feedback on the proposed TAP changes, but
(perhaps because i'm new to the list) i don't see a good thread to point
them to. is there a summary of changes, or something i've missed that would
be a good link to send to folks for their feedback?

~jerry

Re: designing a test suite for multiple implementations (tools thread)

[jerry gay]

On 8/14/06, chromatic <[EMAIL PROTECTED]> wrote:


Ugh, so this means editing the canonical test repository for every
status-changing update to every implementation?  Yuck.  This seems like a
problem for implementation-specific harnesses and reporting tools.



this is what prompted me to start this thread. the problem is years of test
suite design, which point to keeping todo() info with in the testfiles, with
each subtest. moving todo() info out of these test files leads to fragile
test harnesses, as adding a test to the middle of a file will change test
numbers. if test descriptions are used, then unique descriptions for each
test are required. et cetera.

this is all hashed out in the link to the #perl6 conversation larry provided
previously. the conclusion was to keep todo() info with the subtests and
deal with the result (for now.) i *hope* there's a better way. i repost the
link for ease of reference:
http://colabti.de/irclogger/irclogger_log/perl6?date=2006-08-07,Mon&sel=110#l193

~jerry

Re: designing a test suite for multiple implementations (tools thread)

[jerry gay]

[parrot-porters and p6l have been removed from this thread, as discussion of
testing tools is not directly related to those lists]

On 8/11/06, Agent Zhang <[EMAIL PROTECTED]> wrote:


On 8/12/06, jerry gay <[EMAIL PROTECTED]> wrote:

this last testing rule i mentioned becomes somewhat problematic when
> designing a test suite for use by multiple perl6 implementations. it
> is obvious that some implementations of perl6 will not pass all tests,
> and will need to mark certain tests as todo() and skip() in order to
> run the suite successfully.
>

IIRC, for the p6-on-p5 implementation, clkao has implemented some kind
of magic (in Makefile.PL?) to copying passing test files from Pugs
test suit to v6-alpha's own directory. Of course, it's far from ideal,
but i think it's a good enough solution to the current (early) stage
of development.



yes, it's a good step, but it's not ideal. i'm hoping discussion with those
familiar with test suite design, test tools, and test libraries can help us
find what we want, find the tools we can use to help, and develop new tools
we need in order to get results.

I've added the plan for the neutral todo mechanism to Pugs' TASKS

file, getting help from many others on #perl6. The new todo marks look
like this:

   todo :pugs<6.28.0>, :p6p5<0.110>, :parrot<1.00>;
   is $got, $expected;  # affected by the preceding todo call



or equivalently,


is $got, $expected, todo :pugs<6.2.13>, :p6p5<0.110>;



this is a proposed api change to the test library for perl6. i expect that
since these are the first tests written in perl6, they'll be used my many as
a starting point for writing their own tests in the future. i'd like to hear
from the perl-qa folks on what they think about supporting this feature: is
this an acceptable proposal, or is there some other (better) way?

~jerry

Re: designing a test suite for multiple implementations

[jerry gay]

On 8/11/06, Larry Wall <[EMAIL PROTECTED]> wrote:

Just to avoid repeating some of the discussion, here's a link to #perl6:


http://colabti.de/irclogger/irclogger_log/perl6?date=2006-08-07,Mon&sel=110#l193

The discussion goes on and off for most of the rest of the page,
so you probably want to search for and highlight "todo" if you're
using firefox.


thanks for the link, i didn't know #perl6 was logged. for those of you
who may not know, on that channel i'm known as "[particle]".


Anyway, we thrashed the internal/external question pretty hard,
and the rough consensus was to leave todos with the tests for now,
but move to a todo() function that is easy to isolate, visually
if nothing else.  My own take on it is that we can just define the
"official" tests to be the parts outside of todo() calls for now.


that take seems to me like a particularly interesting form of
blindness. however, it is a blindness that we (perl6 language
implementors) would all share. using this definition, it seems we the
blind should all be able to eventually describe this "official"
elephant.


But Jerry raises a lot of good questions that need to be answered over
the long haul.  Certainly by the time we have a real 6.0 we'll need to
have worked out the policy and polity issues.  For now, my gut feeling
is that we'll need at least four times as many tests as we have so far,
and rapid test development is best served by exercising the least amount
of control.  Trying to lock it down too early will simply stall the
process for most of the million monkeys who are currently writing tests.
Version control is our best friend here, at least for now.  Trouble
tickets are probably premature unless they're *very* lightweight.


true, my questions are mainly "long haul." i agree that control to the
test repo should not *yet* be limited, so we can maintain our
(seemingly quickening) pace of perl6 development. i don't intend to
fix what isn't broken.

the timely questions are those related to design, and implementation
thereof. it is here where i hope suggestions like darren's (thanks!)
and those from others with test design experience can lend a hand.
~jerry

designing a test suite for multiple implementations

[jerry gay]

recently, perl 6 development has taken the form of a multi-method
dispatch. that is, multiple implementations are under active
development. this includes pugs (in haskell,) v6 (in perl5,)
v6-Compiler (in perl6,) and perl6 (on parrot.)  hopefully, each of
these returns the same result, a working[1] perl6 implementation.

it makes sense for these implementations to share perl6 tests written
from the perl6 specification. it is also important for this to happen,
as it will help solidify the specification, allow implementations to
check results against each other, share new and corrected tests
easily, and be the canonical reference for answering questions about
the spec. it is also important that only perl6 specification tests be
shared; implementation-specific tests belong solely with the
implementation..

few have ever created a test suite of this magnitude before--i
certainly haven't, and don't expect anyone will do it alone. the
overarching question i have is: how is an test suite targeted for
multiple implementations designed, developed, managed, and controlled?

i believe we can quickly address the "developed" question with the
word "organically." tests will be added, changed, or removed as
needed. organization of tests in the suite will change over time as
refactoring of any large body of tests or code is inevitable. this is
the simple, practical solution we usually expect when working with
perl.

as for "controlled", i expect restricted access to the test
repository, as a protected body of tests is important to more people
as soon as multiple implemenations are using it. providing a method of
submitting and applying patches will allow anyone the ability to
propose a new test for inclusion.

for "managed," i have a few ideas. currently, the suite lives in the
pugs repo. this is a fine first approximation, but i believe it will
soon be time to move this suite[3]. the question is, should it be
moved into their own repository, or into the repo of the "official"
perl6 implementation (if such a beast will indeed exist,) or should it
live in some other place i haven't thought of yet? the options here
are limited, and i believe straightforward. it should be easy to come
to an agreement and take action.

i've left the "designed" question for last, because i think it's the
most difficult.  perl6 test suite has already taken steps towards
targeting multiple implementations. the suite has been designed with a
set of "sanity" tests, which are required to pass successfully in
order to run the perl6 "Test" module, which is written in perl6. this
allows the remainder of the suite's test files to be written in perl6,
making their representation independent of implementation. all the
perl6 implementations i've mentioned above use this test suite, and
pass some subset of the tests. this is a big win, and has allowed
newer implementations to grow quickly as the course has already been
laid.

but unsolved problems remain. perl has a long history of testing.
there are widely held beliefs on how to design and develop test
suites. simple rules, like keeping like tests together, using good
test descriptions, testing components in isolation, performing
integrated tests, and marking tests as todo() or skip() when not
working as expected.

this last testing rule i mentioned becomes somewhat problematic when
designing a test suite for use by multiple perl6 implementations. it
is obvious that some implementations of perl6 will not pass all tests,
and will need to mark certain tests as todo() and skip() in order to
run the suite successfully.

this is further complicated by the fact that perl6 is designed to be
executed in multiple platforms. therefore, a test may succeed for a
particular subset of implementations on a particular subset of
platforms per implementation, but fail elsewhere.

so, what should todo() information would look like, and where it
should go? it's a tough problem, because there's a lot of pain, and it
has to be shared somehow. if it's decided we are testing the
specification, we should design implementation-independent tests. from
this, it follows that implementation details (like todo()) should not
exist in the test. instead, todo() info would be contained in a config
file, or in a test harness, or by some other implementation-specific
method. this is not ideal, since changes to the shared test suite may
invalidate the todo() information in a particular implementation as
test numbers or descriptions change.

keeping todo info directly with the test means a particular
implementation has this information with the test (as is currently
best practice,) but it has it's downsides as well. test files become
crowded with todo() statements and conditional logic for
implementation- and platform-dependent todo()s. each time a new
implementation is updated, the shared test suite will be modified,
requiring (imo) unnecessary test suite changes.

i should point out that implementation-specific tests must exist, but
they mus

Re: TAPx::Parser 0.10 (now with stream support)

[jerry gay]

On 7/24/06, Ovid <[EMAIL PROTECTED]> wrote:

http://users.easystreet.com/ovid/downloads/TAPx-Parser-0.10.tar.gz

Not having a 'net connection in my flat yet meant that I was mostly flying 
blind and had to try and recall what was said on the list from memory.  As a 
result, I'm sure that what I've put together doesn't quite match what may be 
expected.  However, I've gotten enough stuff done that I think this could 
possibly serve as a preliminary TAP::Harness parser.  Any thoughts on this?


i've only started to review this, but i did notice a doc/code mismatch
in TAPx::Parser.pm:

   # the following items are hashes because they are sparse arrays
   results   => [],# TAP parser results
   skipped   => [],#
   todo  => [],#
   passed=> [],#
   failed=> [],#
   actual_failed => [],# how many tests really failed
   actual_passed => [],# how many tests really passed
   parse_errors  => [],# perfect TAP should have none

these don't seem to be hashes, as the comment suggests.
~jerry

Re: TAP diagnostic syntax proposal

[jerry gay]

On 7/13/06, Jonathan Rockway <[EMAIL PROTECTED]> wrote:


> are you serious?  listen to what they people here are saying - we _all_
> read the raw TAP output, all the time, and not because we're TAP
> developers interested in the underlying implementations.  as users, the
> (current) raw TAP diagnostics helps us figure out why a test failed, and
> if it doesn't make sense due to bad wording or reversed expectations
> then it's that much harder than it needs to be.
>

No, you're reading diagnostic output that your program prints.  "got"
and "expected" aren't part of TAP, they're comments in the TAP stream
that some emitters emit and that harnesses print.  Look at the TAP
grammer here:

http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg


wow, my code is being used in a flame war! *blush* :-)


There's no "got" or "expected" part of TAP, there's "ok 1" "not ok 3",
"1..34", and "# this is a comment".  So if we're going to add this to
TAP, which I think is the whole point of this thread, then it doesn't
matter what it's called.  Your tap parser just needs to pick out a
token, and throw a certain event if it finds that.  It's up to the TAP
harness (a level or so above the lexer/parser) to print "got: foo"
"expected: foo".  So if you don't like "got" or "expected", that's fine,
just change the harness.  I'm sure in some instances there won't be even
be a line like that printed -- there will be a HTML table or something.
That's the advantage of making these part of the TAP protocol; you won't
have to worry about the semantics of "got" or "expected"... that will
all be handled for you by the parser and then be presented nicely.

To summarize, right now "got" and "expected" are artifacts of Test::More
(etc.), not the TAP protocol.


the grammar file you referenced was the product of a short hacking
session, and should not be billed as the official TAP grammar (i don't
know if such a document even exists.)

this thread has raised some good points, which you helped me organize
here: it's important to consider the syntax, semantics, and pragmatics
of TAP. after all, it is intended to be a use-anywhere test protocol
(and i hope to implement it for parrot in a way such that any
high-level language targeting parrot can easily use the same TAP
library.)

~ Syntax: TAP 2 (or whatever it'll be called) must have a small,
clearly specified grammar. this leads to better interoperation between
TAP implementations, due to fewer syntax errors. this also leads to
fewer semantic and pragmatic errors as well.

~ Semantics: TAP 2 must allow test module writers to express enough
relevant information (through test descriptions, diagnostic output,
tags, etc.) in order to describe a test result accurately--this
includes custom diagnostic output specific to the problem domain.

~ Pragmatics: TAP 2 must allow users to understand all relevant
information. if there's language or a method of presentation or other
WTDI that is understood by a wider audience, it must be used over one
understood by a smaller audience.

i hope this categorization makes sense, and you find it helpful.
~jerry

Re: TAP diagnostic syntax proposal

[jerry gay]

On 7/11/06, Jonathan T. Rockway <[EMAIL PROTECTED]> wrote:

Ian Langworth wrote:

> I mentioned YAML with a pretense I failed to mention -- that we
> wouldn't parse the YAML. That's already been done, and there are
> plenty of parsers.


I agree with this.  YAML has been done and done again, in every
language.  It works, it's tested.  I don't think we need EVERY feature
of YAML; Syck is enough for my everyday use, anyway.


if i recall correctly, syck doesn't handle utf-8/16. does/will tap
care about that?

oh, and while i'm thinking about it, i think it would be wise to
include a tap version in the tap header, so as this protocol changes,
parsers will have a chance to handle different tap versions
gracefully.
~jerry

new! parrot tap parser

[jerry gay]

at the chicago hackathon, i decided to create a simple tap grammar
using perl 6 regexes. you can find the example grammar at:
   http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg

that spawned interest from chris dolan on creating a parser using
parrot's parser grammar engine (pge.) today, i've committed a working
implementation of a tap parser to the parrot repository. feel free to
check it out from http://svn.perl.org/parrot/trunk/, look in
languages/tap/. there you should find a readme with a small example of
how to get it working. it's still very much a work in progress, but
i'll be happy to answer any questions you may have.

andy lester pointed me here, saying there's tapalicious discussion
going on. so, i've just signed up, to join the fun.

we're hoping to extend this tap parser to be available for use by
high-level languages implemented on parrot. of course, before this
occurs, we need to know what hlls may want to do with a tap parser
(perhaps emit xml/html/text/etc?) any uses you can think of would be
greatly appreciated.

~jerry

Re: TAP Grammar

[jerry gay]

On 7/5/06, Chris Dolan <[EMAIL PROTECTED]> wrote:

On Jul 3, 2006, at 6:33 PM, chromatic wrote:

> On Monday 03 July 2006 09:01, Jonathan T. Rockway wrote:
>
>> That said, I would be interested.  I'm still trying to page all the
>> perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any
>> additional
>> examples would helpful, interesting, and fun.  For me, anyway :)
>
> Jerry Gay just wrote a PGE TAP grammar:
>
> <http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg>
>
> -- c

I hacked a rudimentary test case for that grammar, borrowing heavily
from Punie:
   https://svn.clotho.com/public/parrot-experiments/TAP/


excellent! i have the beginnings of a TAP parser in my working copy,
i'll look at what you've got and synchronize them up. thanks for the
test cases, it should be a simple matter to migrate them into tests in
the parrot repo. i'll also make the makefile platform-independent, as
it won't run on windows as it is.


To run it, edit the first line of the Makefile to specify the path to
PARROT_DIR and run "make".  My tests found a few bugs in Jerry's
grammar.  My fixed version of that grammar is here:
   https://svn.clotho.com/public/parrot-experiments/TAP/grammars/TAP.pg


i've applied your changes (with one minor mod: s/\n/$$/) to parrot
revision 13168.)
thanks and keep the patches coming!
~jerry