Event handling: One method per event or one method for all?
tl;dr version: Is it better for event handlers to have one method per type of
event? Or one method to accept all events? Or something else?
Currently event handlers (called EventWatchers) in Test::Builder2 implement
two methods: accept_event and accept_result. accept_result() is a special
case to handle results (which are special events) and accept_event() takes
care of everything else.
This means you wind up writing stuff like:
my %event_dispatch = (
"stream start" => "accept_stream_start",
"stream end"=> "accept_stream_end",
"set plan" => "accept_set_plan",
"log" => "accept_log",
);
sub accept_event {
my $self = shift;
my($event, $ec) = @_;
my $type = $event->event_type;
my $method = $event_dispatch{$type};
return unless $method;
$self->$method($event, $ec);
return;
}
Which isn't too bad, but it seems like make-work.
The alternative is for the event handlers to have a method for each event,
(ie. accept_stream_start, etc...) falling back to accept_event if there is no
specific handler.
my $method = type2method($event->event_type);
$handler->can($method) ?
$handler->$method($event) : $handler->accept_event($method);
I chose a single method for all events for two reasons:
1) Speed. The event coordinator doesn't have to call can() on every handler
and every event.
2) It makes it easy to write a handler that slurps up all events, like the
History object.
It turns out #1 can be made moot with some careful programming. #2 can be
handled by falling back to a generic handler.
So now the question is one of good event handler design, which I don't have
experience with. How does one design a good event handler? Is the pattern
one method per type of event? Or one method to do the dispatching? Or
something else?
--
Being faith-based doesn't trump reality.
-- Bruce Sterling
Re: Subtest design in Test::Builder 1.5
On 2011.10.26 12:55 PM, Eric Wilhelm wrote:
> To be specific, I picture the interface as one of:
>
> a) subtest_start() must return the subtest handler object
> (might be a new object, or just $self)
>
> or:
> b) subtest_start() must return the $invocant, $method, %args for
> construction of the subtest handler object
> (might be [$classname,'new', foo => 'bar'] or [$self, sub{shift}])
>
> (Where "return" might be in some way besides return() if warranted.)
>
> I think b is a better way to go if you want to keep more flexibility at
> the framework level. It might seem like a trivial difference, but it's
> really sausage or bacon. By deferring the execution, you will be able
> to put things before the construction/swap, add arguments to the method
> call, and other fun stuff within the framework's scope. If you go with
> a, the constructor already executed and the sausage is already ground.
Given that the event coordinator can't know anything about the interface of
the method they're calling on $classname, the event coordinator can't add
arguments. So I don't see the utility of B.
And given that the logic inside the event coordinator will look something like:
my $subtest_handler = $handler->accept_subtest_start;
The coordinator can do whatever it likes before triggering the handler as well
as after. And the handler can do whatever it likes to generate the subtest
handler.
I don't see the utility of the extra complexity of B in this case. It seems
it adds no extra flexibility, potential or real.
>> That's... almost Orwellian. Strictures are flexibility!
>
> For a long-term framework design, I think what you have seen with
> e.g. 'ok or diag' vs stop-on-failure is very telling. If the execution
> is ceded to the caller (sausage), you limit what features the framework
> can add in the future. But, whenever the "do this" is brought back
> into the scope of the framework in some composable fashion (i.e.
> deferred execution), the framework can add before/after/around (or even
> skip?) features later because you have that extra bit of control.
Sure, I get what you're saying. I couldn't help the 1984 dig. :-)
--
170. Not allowed to "defect" to OPFOR during training missions.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
Re: Do we need subtests in TAP?
On 10/25/11 11:56 PM, Michael G Schwern wrote: I keep looking at subtests and keeping thinking that if there wasn't a test count to manage, would we need subtests? Do we need all that complexity? If it's just about the test count, can it be managed a better way? I haven't followed this discussion in depth. But I would have to say that building in too much complexity to something that is at the bottom of the Perl testing toolchain does not sound like a flexible, maintainable way to go forward.
Re: Do we need subtests in TAP?
Adrian forgot to send this to the list. Original Message Subject: Re: Do we need subtests in TAP? Date: Wed, 26 Oct 2011 14:14:31 +0100 From: Adrian Howard To: Michael G Schwern Hey there, On 26 Oct 2011, at 04:56, Michael G Schwern wrote: > I understand wanting "blocks of tests" and the ability to make plans for just > those blocks, but do we need a discrete test state for that? For example, > Test::Class provides most of what subtests provide without special support. ... and to do that T::C contains a bunch of annoying special case code than is (I think) still wrong in an odd corner case. Everybody who wants to do the things T::C does will also have to do that work. T::C implemented with subtests is _much_ cleaner code. There may be other ways of getting that complexity out of T::C (and similar) and into Test::Builder of course - but I'm not 100% sure what you're suggesting... > It occurred to me because most other testing systems don't have subtests. > They have test methods, but those aren't treated as a sub-state of the test. Some do have different levels of hierarchy though. (e.g. JUnit's Test Case / Suite distinction). > In essence, a "subtest" is nothing more than a block of tests with a name and > it's own plan. The special TAP formatting seems unnecessary. I guess that's > the real question, do we need the special TAP formatting or do we just need > named test blocks with their own plan? One thing subtests TAP formatting gives you is a simple way to nest TAP streams from elsewhere. Any other system would mean you have to rewrite the nested stream (I think?) Cheers, Adrian -- Who invented the eponym?
Re: Do we need subtests in TAP?
> "David" == David Golden writes:
David> I wonder how many people are using subtests with a plan and how many
David> are replying on the implied "done_testIng" feature.
I'm teaching it now, and I find it very valuable.
subtest 'network tests' => sub {
$ENV{NETWORK_TESTS} or plan skip_all => 'skipping network tests';
...
};
subtest 'tolerate partial failure' => sub {
...
is($foo, $bar, 'foo is bar') or return;
...
tests that depend on $foo is $bar, and would all fail without
...
};
Far easier than the old way of counting all the skipped tests and
performing skips depending on success.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion
Re: Subtest design in Test::Builder 1.5
# from Michael G Schwern
# on Wednesday 26 October 2011 09:51:
>> Note that you should be able to allow e.g. "the new subtest object
>> is just a copy of me" and other advanced usage without ceding all of
>> the mechanics to the handler object.
>
>Ooh, good idea. I think that's the clincher right there. If a
> handler can effectively nullify the swap then flexibility is
> achieved. A handler can then deal with the subtest_start and
> subtest_end events in its own way.
>
>Perhaps a cleaner way would be for the event coordinator to ask the
> event handler if it wants to be swapped out in the event of a
> subtest. Just an attribute on a handler that defaults to true.
>
>OTGH I don't see why the exception is necessary. There's no harm in
> storing the same object twice in the stack and it eliminates a bunch
> of special case code in the EventCoordinator.
To be specific, I picture the interface as one of:
a) subtest_start() must return the subtest handler object
(might be a new object, or just $self)
or:
b) subtest_start() must return the $invocant, $method, %args for
construction of the subtest handler object
(might be [$classname,'new', foo => 'bar'] or [$self, sub{shift}])
(Where "return" might be in some way besides return() if warranted.)
I think b is a better way to go if you want to keep more flexibility at
the framework level. It might seem like a trivial difference, but it's
really sausage or bacon. By deferring the execution, you will be able
to put things before the construction/swap, add arguments to the method
call, and other fun stuff within the framework's scope. If you go with
a, the constructor already executed and the sausage is already ground.
>That's... almost Orwellian. Strictures are flexibility!
For a long-term framework design, I think what you have seen with
e.g. 'ok or diag' vs stop-on-failure is very telling. If the execution
is ceded to the caller (sausage), you limit what features the framework
can add in the future. But, whenever the "do this" is brought back
into the scope of the framework in some composable fashion (i.e.
deferred execution), the framework can add before/after/around (or even
skip?) features later because you have that extra bit of control.
--Eric
--
software: a hypothetical exercise which happens to compile.
---
http://scratchcomputing.com
---
Re: Subtest design in Test::Builder 1.5
On 2011.10.25 10:54 PM, Eric Wilhelm wrote: > I'm only working from intuition and my understanding of what you > described as the problem. If you try to implement a few scenarios of > special handler functionality using each design approach, that might > help clarify the issues. All I really have is "nested formatting" and "logger that wants to see everything". I'm not yet sure if the history will best be handled by a single object or children. The odd part about building a long term framework is I really don't know what people are going to use it for. > Note that you should be able to allow e.g. "the new subtest object is > just a copy of me" and other advanced usage without ceding all of the > mechanics to the handler object. Ooh, good idea. I think that's the clincher right there. If a handler can effectively nullify the swap then flexibility is achieved. A handler can then deal with the subtest_start and subtest_end events in its own way. Perhaps a cleaner way would be for the event coordinator to ask the event handler if it wants to be swapped out in the event of a subtest. Just an attribute on a handler that defaults to true. OTGH I don't see why the exception is necessary. There's no harm in storing the same object twice in the stack and it eliminates a bunch of special case code in the EventCoordinator. > The trouble with doing this with a stack in the coordinator instead is > that you may need a way for a subtest to access its parent handler for > context -- but that need doesn't dictate that the subtest handler must > store the parent handler. The default implementation EventWatchers inherit would do that, and there'd be a slot for it. Bets are off if you do it yourself. > For instance, perhaps you continually pass > the coordinator or a context in e.g. $handler->$method($coordinator) > and allow access via that with the proviso that $coordinator's > attributes must not be held in $handler. Already done. Good validation of the design that you came up with it independently. :) my $parent = $coordinator->give_me_my_parent($handler) is going to be interesting to implement. Again, I think handlers are going to need IDs. Well, events already have them so I can just turn that into a role. It'll probably be useful later. > My main concern here is > avoiding memory-loop and handler swapping bugs which are likely when > various handlers start interacting. Yeah, that's what has me worried, too. > It seems like keeping the stack/current control in the coordinator will > help more than hurt in future flexibility. Would you rather be blocked > from making a change because somebody did something you didn't think of > in a subtest_start() and the API had granted them implicit permission > to do that (aka "total control") or grow from a more conservative > position where more of the data and flow control is managed by your > coordinator object which grants flexibility via defined interfaces? That's... almost Orwellian. Strictures are flexibility! Ok, I'm convinced. Not by the last argument, but in general. Thanks! -- 101. I am not allowed to mount a bayonet on a crew-served weapon. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/
Re: Do we need subtests in TAP?
Hi,
On Wed, Oct 26, 2011 at 4:56 AM, Michael G Schwern wrote:
> I keep looking at subtests and keeping thinking that if there wasn't a test
> count to manage, would we need subtests? Do we need all that complexity? If
> it's just about the test count, can it be managed a better way?
>
> I understand wanting "blocks of tests" and the ability to make plans for just
> those blocks, but do we need a discrete test state for that? For example,
> Test::Class provides most of what subtests provide without special support.
I use subtests a lot nowadays, but mostly for grouping. I don't care
at all for the separate plan for subtests.
If tomorrow the sub subtest in Test::More became:
sub subtest {
diag("Testing ".shift);
shift->();
}
I probably would not notice :)
Bye,
--
Pedro Melo
@pedromelo
http://www.simplicidade.org/
http://about.me/melo
xmpp:m...@simplicidade.org
mailto:m...@simplicidade.org
Re: Do we need subtests in TAP?
On Tue, Oct 25, 2011 at 11:56 PM, Michael G Schwern wrote: > I keep looking at subtests and keeping thinking that if there wasn't a test > count to manage, would we need subtests? Do we need all that complexity? If > it's just about the test count, can it be managed a better way? I find several benefits: * Less visual clutter in non-verbose output * Indentation of verbose output * Test granularity I rarely care about test counts anymore -- the overhead of updating plan just isn't worth it to me. > I understand wanting "blocks of tests" and the ability to make plans for just > those blocks, but do we need a discrete test state for that? For example, > Test::Class provides most of what subtests provide without special support. I would suggest looking at Test::Routine as well -- it has been a god-send to me for testing Metabase and uses subtests. > It occurred to me because most other testing systems don't have subtests. > They have test methods, but those aren't treated as a sub-state of the test. That might work under an assertion-type system, where tests within the method are assertions and some harness is catching exceptions within the test methods. The downside of assertions is that the test method dies on the first assertion and you get no information about whether subsequent tests would have passed or failed. > In essence, a "subtest" is nothing more than a block of tests with a name and > it's own plan. I wonder how many people are using subtests with a plan and how many are replying on the implied "done_testIng" feature. > The special TAP formatting seems unnecessary. I guess that's > the real question, do we need the special TAP formatting or do we just need > named test blocks with their own plan? Because I really don't care about numbering, I don't really care if subtests have their own numbers or just continue the number sequence. I do like the visualization options of subtests and it would be weird if the numbering "jumped" when subtests were not verbose. -- David
