Re: Subtest design in Test::Builder 1.5
# from Michael G Schwern
# on Wednesday 26 October 2011 09:51:
Note that you should be able to allow e.g. the new subtest object
is just a copy of me and other advanced usage without ceding all of
the mechanics to the handler object.
Ooh, good idea. I think that's the clincher right there. If a
handler can effectively nullify the swap then flexibility is
achieved. A handler can then deal with the subtest_start and
subtest_end events in its own way.
Perhaps a cleaner way would be for the event coordinator to ask the
event handler if it wants to be swapped out in the event of a
subtest. Just an attribute on a handler that defaults to true.
OTGH I don't see why the exception is necessary. There's no harm in
storing the same object twice in the stack and it eliminates a bunch
of special case code in the EventCoordinator.
To be specific, I picture the interface as one of:
a) subtest_start() must return the subtest handler object
(might be a new object, or just $self)
or:
b) subtest_start() must return the $invocant, $method, %args for
construction of the subtest handler object
(might be [$classname,'new', foo = 'bar'] or [$self, sub{shift}])
(Where return might be in some way besides return() if warranted.)
I think b is a better way to go if you want to keep more flexibility at
the framework level. It might seem like a trivial difference, but it's
really sausage or bacon. By deferring the execution, you will be able
to put things before the construction/swap, add arguments to the method
call, and other fun stuff within the framework's scope. If you go with
a, the constructor already executed and the sausage is already ground.
That's... almost Orwellian. Strictures are flexibility!
For a long-term framework design, I think what you have seen with
e.g. 'ok or diag' vs stop-on-failure is very telling. If the execution
is ceded to the caller (sausage), you limit what features the framework
can add in the future. But, whenever the do this is brought back
into the scope of the framework in some composable fashion (i.e.
deferred execution), the framework can add before/after/around (or even
skip?) features later because you have that extra bit of control.
--Eric
--
software: a hypothetical exercise which happens to compile.
---
http://scratchcomputing.com
---
Re: Subtest design in Test::Builder 1.5
# from Michael G Schwern on Monday 24 October 2011 22:12: Plan A... 2) The old one is squirreled away in a stack. 3) The new handler is told how deeply it's nested. ...On the down side, every handler needs a delegator ... yucky. ... Plan B... when a subtest_start happens... 1) An event handler creates a new one of itself (the subtest handler) 2) The subtest handler is told how deeply nested it is. 3) The parent handler stores itself in the subtest handler. 4) The parent tells the event coordinator to remove itself and add the subtest handler instead. ... On the down side, the event coordinator needs a swap me out method for event handlers. I like the sound of plan B, except for the stores itself in combined with swap me out. Can the event coordinator keep a stack? At the point where the parent handler has to tell the coordinator swap me out, you could have the coordinator providing the functionality which you were thinking of getting from the delegator. How much freedom you want to allow the handlers in subtest_start() and subtest_end()? If a subtest handler is standard procedure, perhaps the construction/destruction/stack is 100% managed by the coordinator (or with some constrained flexibility like taking a classname and arguments from the return of subtest_start() -- or even pass in a messenger/knob if that fits better than using the return values.) The code to handle the creating and swapping of a subtest could be put in the event handler superclass. Would that be for convenience of reuse or because an essential part of the API is the freedom to override that method and do it differently? I think putting this in the coordinator would allow fewer surprises. --Eric -- Beware of bugs in the above code; I have only proved it correct, not tried it. --Donald Knuth --- http://scratchcomputing.com ---
Re: Subtest design in Test::Builder 1.5
On 2011.10.25 12:29 AM, Eric Wilhelm wrote: I like the sound of plan B, except for the stores itself in combined with swap me out. Any specific doubts? Can the event coordinator keep a stack? At the point where the parent handler has to tell the coordinator swap me out, you could have the coordinator providing the functionality which you were thinking of getting from the delegator. That's an interesting idea. It certainly could happen. The two problems are A) it constrains what an event watcher can do (see below) and B) the subtest watcher still has to communicate its results back to the parent watcher. B could be solved by the event coordinator doing the swap and then asking the subtest history to generate a result event summarizing the subtest. The parent watchers (which are now swapped in) will see that. Come to think of it, that sort of thing is nice and clean no matter who is doing the swapping. But I don't see a solution to A. What the event coordinator could do to help is keep track of the subtest depth and add that onto events. Then nothing else has to worry about setting or tracking that, they just ask the event. The code to handle the creating and swapping of a subtest could be put in the event handler superclass. Would that be for convenience of reuse or because an essential part of the API is the freedom to override that method and do it differently? I think putting this in the coordinator would allow fewer surprises. Both. For example, I can easily imagine a history or event logger that wants to record the whole test as a tree. Part of the point is to provide maximum flexibility to test module authors, so we're not stuck with design mistakes 10 years later, while still retaining a system that's easy to use if you do things as expected. -- 44. I am not the atheist chaplain. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/
Subtest design in Test::Builder 1.5
Subtests are the last major feature hurdle for Test::Builder 1.5. They're kind of a hacky mess in Test::Builder 1 which I don't want to bring forward. A subtest has to 1) create a separate state of the test 2) change the format of the output 3) communicate the result of the test back to the parent Currently this is done by 1) copying the existing TB singleton 2) creating a new TB object set to indent the output 3) jamming it in place of the TB singleton 4) running the subtests 5) restoring the original singleton 6) the subtest object tells the singleton how it turned out #3 is necessary because Test modules tend to get the singleton once and continue to use it, so the subtest object has to use the same object. In Test::Builder 1.5 the state of the test no longer resides in the Test::Builder object (this is a white lie, but becoming less so), but rather in event handlers such as the Formatter and History objects. This means subtests can continue to use the same Test::Builder object and the madness is not necessary. Instead, subtests would generate 'subtest_start' and 'subtest_end' events which would be handled by the event handlers. There's two ways to go on this: Plan A... each event handler that cares about subtests is actually a delegator to the real event handler. When a subtest_start happens... 1) The delegator creates a new instance of the real event handler is created. 2) The old one is squirreled away in a stack. 3) The new handler is told how deeply it's nested. 4) The delegator starts delegating to the newly created handler. When a subtest_end happens... 1) The delegator communicates the result of the subtest to the first handler 2) The delegator pops the stack and starts talking to the original handler On the up side, event handlers can be written without having to worry too much about nesting. They just need to be able to be nested. For TAP that just means indenting the output. For XML that just means not outputting headers. All pretty easy. On the down side, every handler needs a delegator clogging and complicating things. While it could be made pretty generic, it's yucky. I've known it sucks which has stopped me from doing anything about it. Plan B... when a subtest_start happens... 1) An event handler creates a new one of itself (the subtest handler) 2) The subtest handler is told how deeply nested it is. 3) The parent handler stores itself in the subtest handler. 4) The parent tells the event coordinator to remove itself and add the subtest handler instead. When a subtest_end happens... 1) The subtest handler tells its parent how things turned out 2) The subtest handler tells the event coordinator to remove itself and put the parent handler back 3) The subtest handler forgets about the parent to avoid a circular dep I like this because it retains the upside of Plan A, event handlers don't have to worry too much about nesting, without each event handler having a middleman. On the down side, the event coordinator needs a swap me out method for event handlers. This isn't too hard, I'll just have to add a unique ID method to event handlers. The code to handle the creating and swapping of a subtest could be put in the event handler superclass. I'm going to code up Plan B, and I'd like to hear people's thoughts on solving this problem. Rarely has my first stab at a solution been the right one in this project. Thanks for listening. -- Who invented the eponym?
