Re: test harness presentation layer plugins

[Eric Wilhelm]

# from Michael G Schwern
# on Tuesday 31 March 2009 15:22:

>    want:  foo bar
>    have:  foo bar
>    display: |
>       want: foo\N{SPACE}bar
>       have: foo\N{KLINGON MOMENT OF SILENCE}bar
>
>But it's really more useful for more complicated formatting.
>  Something simple like making subtle whitespace differences visible
> should be a feature of the TAP consumer.

My thoughts exactly.  If the Test::* modules causing this itch are truly 
only doing formatting, structured diagnostics should be plenty of input 
for them.

And this 'display:' key is actually a bit unsettling because now you've 
got preformatted (encoded) formatting coming from the test.  But, I can 
see a case where not all is() are created equal -- so perhaps the 
producer would simply tag the result, allowing that to be dispatched to 
a formatter:

  want:  foo bar
  have:  foo*bar
  tag:   utf-klingon

Then you have some dispatcher on the harness (presumably a builtin or 
one plugin which itself has plugins) to read the 'tag:' and decide 
which code does the formatting -- TAP::Formatter::Dispatch or so.

Aside:  I'm assuming that the 'have: foo bar' in Schwern's example 
includes a unicode nbsp or something.

--Eric
-- 
The opinions expressed in this e-mail were randomly generated by
the computer and do not necessarily reflect the views of its owner.
--Management
---
http://scratchcomputing.com
---

Re: test harness presentation layer plugins

[David Golden]

On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern  wrote:
> But it's really more useful for more complicated formatting.  Something simple
> like making subtle whitespace differences visible should be a feature of the
> TAP consumer.

Which is why it will be nice when the "consumer" isn't just getting
text output on a filehandle.

/me wants structured TAP capture and easy Harness plugins to change
how things are presented.

I hope I can finally get CPAN Testers 2.0 far along this year so that
next year at a QA hackathon, I can spend more time with the
TAP/Harness team.

-- David

Re: test harness presentation layer plugins

[Michael G Schwern]

Ricardo SIGNES wrote:
> I'm mostly sending this email because I had an idea, and it's late, and I 
> don't
> want to forget.
> 
> Today it occured to me that many Test:: extensions are more about better
> diagnostic output than better test comparison.
> 
> This stinks:
> 
>   want: foo bar
>   have: foo bar
> 
> So you write some plugin that does the same old comparison but shows:
> 
>   want: foo\N{SPACE}bar
>   have: foo\N{KLINGON MOMENT OF SILENCE}bar
> 
> What if display types could be provided informationally in the TAP stream.  
> The
> stream could include:
> 
>   want: foo bar
>   have: foo bar
>   presentation: Some::Plugin
> 
> With the plugin installed, the presentation layer could fix the output to be
> useful.  Without it, you still have the data displayed and could presumably
> re-display the TAP stream later with the presenter plugin in place.
> 
> I am not sure whether this is a good idea, or already well-established as a 
> way
> to deal with this annoyance.  Good night.

Your annoyance is valid, but you've reversed the presentation responsibility.

The test should not be dictating presentation.  If it has such clear ideas
about how the data is to be presented it might as well just format it itself.
   And that's what we want to avoid, because the test author is a poor judge
of how you want the data presented.  It also means the test and the
presentation have to coordinate what Some::Plugin means.

The whole point of the structured diagnostics is to allow the user of the test
to decide how it's to be displayed.  You should simply tell your TAP reader
how you want diagnostics formatted.  For a textual reader you have to do this
before the test is run, but imagine a GUI reader where you can use a
contextual menu to change the display of a single assertion after the fact.

That said, there is a suggested key to give the test control.  "display"
allows the test to suggest "please display the test diagnostics like this",
but it's just some text.  So...

want:  foo bar
have:  foo bar
display: |
   want: foo\N{SPACE}bar
   have: foo\N{KLINGON MOMENT OF SILENCE}bar

But it's really more useful for more complicated formatting.  Something simple
like making subtle whitespace differences visible should be a feature of the
TAP consumer.


-- 
But there's no sense crying over every mistake.
You just keep on trying till you run out of cake.
-- Jonathan Coulton, "Still Alive"