Re: test harness presentation layer plugins
Ricardo SIGNES wrote: > * Michael G Schwern [2009-03-31T18:22:50] >>> What if display types could be provided informationally in the TAP stream. >>> The stream could include: >>> >>> want: foo bar >>> have: foo bar >>> presentation: Some::Plugin >>> >>> With the plugin installed, the presentation layer could fix the output to be >>> useful. Without it, you still have the data displayed and could presumably >>> re-display the TAP stream later with the presenter plugin in place. >> Your annoyance is valid, but you've reversed the presentation responsibility. >> >> The test should not be dictating presentation. > > It isn't dictating. It's suggesting a useful default. > > Until cpan testers are submitting TAP harness, which would allow the author to > redisplay the TAP, this would lower the prereqs for installing a distribution > (you don't need a presentation-like test plugin to test the module). The presence of a presentation keyword doesn't lower the prereqs over not having it. At best it keeps it flat. By making it a suggestion (which is fine) it adds a "recommended" dependency on the plugin module. Even if it is "a useful default" what does "Some::Plugin" mean and to whom? TAP::Harness? What about some other TAP consumer? The alternative is to come up with a generic set of display modes understood by TAP parsers which we're no where near ready for. The simple way out, which the OP probably didn't realize, is to simply say "Presentation" which makes it unofficial [1] and then you can do whatever you want without worrying about the far reaching consequences. I will confess, what I'm feeling here is that this is going in the direction of xUnit where the display modes are locked into the testing system the author has chosen. >> The whole point of the structured diagnostics is to allow the user of the >> test >> to decide how it's to be displayed. You should simply tell your TAP reader >> how you want diagnostics formatted. For a textual reader you have to do this >> before the test is run, but imagine a GUI reader where you can use a >> contextual menu to change the display of a single assertion after the fact. > > I understand that -- but we don't have those yet, and even if we did, we're > not > getting the real TAP back, yet, from smokers. Which would be easier: writing a presentation plugin system, writing plugins, waiting for Test::Builder to support structured diagnostics, adding a presentation keyword to all applicable tests and getting smokers to install all the plugins; or changing the smoking modules to send back the TAP using the existing TAP::Harness::Archive and waiting for the smokers to upgrade normally? [1] I know, we broke down over how do designate an official vs unofficial key in Oslo. Ovid and I talked about it briefly in Birmingham and /^[a-z]/ == "official", everything else unofficial is on again at least for draft purposes. -- 7. Not allowed to add "In accordance with the prophesy" to the end of answers I give to a question an officer asks me. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/
Re: test harness presentation layer plugins
* Michael G Schwern [2009-03-31T18:22:50] > > What if display types could be provided informationally in the TAP stream. > > The stream could include: > > > > want: foo bar > > have: foo bar > > presentation: Some::Plugin > > > > With the plugin installed, the presentation layer could fix the output to be > > useful. Without it, you still have the data displayed and could presumably > > re-display the TAP stream later with the presenter plugin in place. > > Your annoyance is valid, but you've reversed the presentation responsibility. > > The test should not be dictating presentation. It isn't dictating. It's suggesting a useful default. Until cpan testers are submitting TAP harness, which would allow the author to redisplay the TAP, this would lower the prereqs for installing a distribution (you don't need a presentation-like test plugin to test the module). > The whole point of the structured diagnostics is to allow the user of the test > to decide how it's to be displayed. You should simply tell your TAP reader > how you want diagnostics formatted. For a textual reader you have to do this > before the test is run, but imagine a GUI reader where you can use a > contextual menu to change the display of a single assertion after the fact. I understand that -- but we don't have those yet, and even if we did, we're not getting the real TAP back, yet, from smokers. -- rjbs
Re: test harness presentation layer plugins
David Golden wrote: > On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern wrote: >> But it's really more useful for more complicated formatting. Something >> simple >> like making subtle whitespace differences visible should be a feature of the >> TAP consumer. > > Which is why it will be nice when the "consumer" isn't just getting > text output on a filehandle. > > /me wants structured TAP capture and easy Harness plugins to change > how things are presented. > > I hope I can finally get CPAN Testers 2.0 far along this year so that > next year at a QA hackathon, I can spend more time with the > TAP/Harness team. TAP::Harness already reads structured diagnostics, if it sees a "TAP version 13" header. Doesn't really do anything with it, but the data is available. It's Test::Builder that's dragging its feet. AndyA hacked in some basic diagnostic handling to Test::More with Test::More::Diagnostic if you want to play. http://search.cpan.org/dist/Test-More-Diagnostic -- You know what the chain of command is? It's the chain I go get and beat you with 'til you understand who's in ruttin' command here. -- Jayne Cobb, "Firefly"
Re: test harness presentation layer plugins
# from Michael G Schwern
# on Tuesday 31 March 2009 15:22:
> want: foo bar
> have: foo bar
> display: |
> want: foo\N{SPACE}bar
> have: foo\N{KLINGON MOMENT OF SILENCE}bar
>
>But it's really more useful for more complicated formatting.
> Something simple like making subtle whitespace differences visible
> should be a feature of the TAP consumer.
My thoughts exactly. If the Test::* modules causing this itch are truly
only doing formatting, structured diagnostics should be plenty of input
for them.
And this 'display:' key is actually a bit unsettling because now you've
got preformatted (encoded) formatting coming from the test. But, I can
see a case where not all is() are created equal -- so perhaps the
producer would simply tag the result, allowing that to be dispatched to
a formatter:
want: foo bar
have: foo*bar
tag: utf-klingon
Then you have some dispatcher on the harness (presumably a builtin or
one plugin which itself has plugins) to read the 'tag:' and decide
which code does the formatting -- TAP::Formatter::Dispatch or so.
Aside: I'm assuming that the 'have: foo bar' in Schwern's example
includes a unicode nbsp or something.
--Eric
--
The opinions expressed in this e-mail were randomly generated by
the computer and do not necessarily reflect the views of its owner.
--Management
---
http://scratchcomputing.com
---
Re: test harness presentation layer plugins
On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern wrote: > But it's really more useful for more complicated formatting. Something simple > like making subtle whitespace differences visible should be a feature of the > TAP consumer. Which is why it will be nice when the "consumer" isn't just getting text output on a filehandle. /me wants structured TAP capture and easy Harness plugins to change how things are presented. I hope I can finally get CPAN Testers 2.0 far along this year so that next year at a QA hackathon, I can spend more time with the TAP/Harness team. -- David
Re: test harness presentation layer plugins
Ricardo SIGNES wrote:
> I'm mostly sending this email because I had an idea, and it's late, and I
> don't
> want to forget.
>
> Today it occured to me that many Test:: extensions are more about better
> diagnostic output than better test comparison.
>
> This stinks:
>
> want: foo bar
> have: foo bar
>
> So you write some plugin that does the same old comparison but shows:
>
> want: foo\N{SPACE}bar
> have: foo\N{KLINGON MOMENT OF SILENCE}bar
>
> What if display types could be provided informationally in the TAP stream.
> The
> stream could include:
>
> want: foo bar
> have: foo bar
> presentation: Some::Plugin
>
> With the plugin installed, the presentation layer could fix the output to be
> useful. Without it, you still have the data displayed and could presumably
> re-display the TAP stream later with the presenter plugin in place.
>
> I am not sure whether this is a good idea, or already well-established as a
> way
> to deal with this annoyance. Good night.
Your annoyance is valid, but you've reversed the presentation responsibility.
The test should not be dictating presentation. If it has such clear ideas
about how the data is to be presented it might as well just format it itself.
And that's what we want to avoid, because the test author is a poor judge
of how you want the data presented. It also means the test and the
presentation have to coordinate what Some::Plugin means.
The whole point of the structured diagnostics is to allow the user of the test
to decide how it's to be displayed. You should simply tell your TAP reader
how you want diagnostics formatted. For a textual reader you have to do this
before the test is run, but imagine a GUI reader where you can use a
contextual menu to change the display of a single assertion after the fact.
That said, there is a suggested key to give the test control. "display"
allows the test to suggest "please display the test diagnostics like this",
but it's just some text. So...
want: foo bar
have: foo bar
display: |
want: foo\N{SPACE}bar
have: foo\N{KLINGON MOMENT OF SILENCE}bar
But it's really more useful for more complicated formatting. Something simple
like making subtle whitespace differences visible should be a feature of the
TAP consumer.
--
But there's no sense crying over every mistake.
You just keep on trying till you run out of cake.
-- Jonathan Coulton, "Still Alive"
test harness presentation layer plugins
I'm mostly sending this email because I had an idea, and it's late, and I don't
want to forget.
Today it occured to me that many Test:: extensions are more about better
diagnostic output than better test comparison.
This stinks:
want: foo bar
have: foo bar
So you write some plugin that does the same old comparison but shows:
want: foo\N{SPACE}bar
have: foo\N{KLINGON MOMENT OF SILENCE}bar
What if display types could be provided informationally in the TAP stream. The
stream could include:
want: foo bar
have: foo bar
presentation: Some::Plugin
With the plugin installed, the presentation layer could fix the output to be
useful. Without it, you still have the data displayed and could presumably
re-display the TAP stream later with the presenter plugin in place.
I am not sure whether this is a good idea, or already well-established as a way
to deal with this annoyance. Good night.
--
rjbs
