Re: Review: A new stab at a potential std.unittests
Jonathan M Davis Wrote: > > In particular, needing to pass LineInfo() to assertExcThrown!() to know the > file > and line number was disliked (though it was by far the best solution that I'd > been able to come up with). Not sure if this helps, but if you default-initialize template function parameters with __LINE__ and __FILE__ they get the line and file of where the template was instantiated.
Re: Review: A new stab at a potential std.unittests
On Friday, November 19, 2010 11:37:16 Sean Kelly wrote: > Jonathan M Davis Wrote: > > In particular, needing to pass LineInfo() to assertExcThrown!() to know > > the file and line number was disliked (though it was by far the best > > solution that I'd been able to come up with). > > Not sure if this helps, but if you default-initialize template function > parameters with __LINE__ and __FILE__ they get the line and file of where > the template was instantiated. Yes. The problem was that the function was a _variadic_ template. So, you couldn't have default arguments. LineInfo had default arguments in its opCall() which were the file and line number, so you passed LineInfo() as the first argument to the function, thereby not having to pass __FILE__ and __LINE__, but ideally, they would be default arguments of the original function as you suggest. It's just that the variadic part got in the way. The new version is neither variadic (since whole call is a single string rather than passing the function name and arguments separately), and it doesn't even need the __FILE__ and __LINE__ number as default arguments, since they're just used directly in the string that's mixed in. - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
Jonathan M Davis Wrote: > On Friday, November 19, 2010 11:37:16 Sean Kelly wrote: > > Jonathan M Davis Wrote: > > > In particular, needing to pass LineInfo() to assertExcThrown!() to know > > > the file and line number was disliked (though it was by far the best > > > solution that I'd been able to come up with). > > > > Not sure if this helps, but if you default-initialize template function > > parameters with __LINE__ and __FILE__ they get the line and file of where > > the template was instantiated. > > Yes. The problem was that the function was a _variadic_ template. So, you > couldn't have default arguments. This should work: void func(string x = __FILE__, T...)(T args); D allows defaulted template arguments to occur before non-defaulted ones.
Re: Review: A new stab at a potential std.unittests
On Friday 19 November 2010 11:59:18 Sean Kelly wrote: > Jonathan M Davis Wrote: > > On Friday, November 19, 2010 11:37:16 Sean Kelly wrote: > > > Jonathan M Davis Wrote: > > > > In particular, needing to pass LineInfo() to assertExcThrown!() to > > > > know the file and line number was disliked (though it was by far the > > > > best solution that I'd been able to come up with). > > > > > > Not sure if this helps, but if you default-initialize template function > > > parameters with __LINE__ and __FILE__ they get the line and file of > > > where the template was instantiated. > > > > Yes. The problem was that the function was a _variadic_ template. So, you > > couldn't have default arguments. > > This should work: > > void func(string x = __FILE__, T...)(T args); > > D allows defaulted template arguments to occur before non-defaulted ones. If so, then I obviously assumed that that wouldn't work. You certainly can't do that with the function arguments. I think that the new solution is still better though, because it allows you to write the function call like you normally would. You just have to put it in string form. That, and there were problems with passing the function as an alias. Still, it's definitely good to know that you can have default template arguments before non-defaulted ones. Thanks. - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
Sean Kelly, el 19 de noviembre a las 14:59 me escribiste:
> Jonathan M Davis Wrote:
>
> > On Friday, November 19, 2010 11:37:16 Sean Kelly wrote:
> > > Jonathan M Davis Wrote:
> > > > In particular, needing to pass LineInfo() to assertExcThrown!() to know
> > > > the file and line number was disliked (though it was by far the best
> > > > solution that I'd been able to come up with).
> > >
> > > Not sure if this helps, but if you default-initialize template function
> > > parameters with __LINE__ and __FILE__ they get the line and file of where
> > > the template was instantiated.
> >
> > Yes. The problem was that the function was a _variadic_ template. So, you
> > couldn't have default arguments.
>
> This should work:
>
> void func(string x = __FILE__, T...)(T args);
>
> D allows defaulted template arguments to occur before non-defaulted ones.
And what is func!("blah")(); is supposed to do, make x = "blah"? args[0]
= "blah"? both?
(I don't have a compiler at hand to try it =P)
--
Leandro Lucarella (AKA luca) http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05)
--
He used to do surgery
On girls in the eighties
But gravity always wins
Re: Review: A new stab at a potential std.unittests
On Friday 19 November 2010 12:39:20 Leandro Lucarella wrote:
> Sean Kelly, el 19 de noviembre a las 14:59 me escribiste:
> > Jonathan M Davis Wrote:
> > > On Friday, November 19, 2010 11:37:16 Sean Kelly wrote:
> > > > Jonathan M Davis Wrote:
> > > > > In particular, needing to pass LineInfo() to assertExcThrown!() to
> > > > > know the file and line number was disliked (though it was by far
> > > > > the best solution that I'd been able to come up with).
> > > >
> > > > Not sure if this helps, but if you default-initialize template
> > > > function parameters with __LINE__ and __FILE__ they get the line and
> > > > file of where the template was instantiated.
> > >
> > > Yes. The problem was that the function was a _variadic_ template. So,
> > > you couldn't have default arguments.
> >
> > This should work:
> >
> > void func(string x = __FILE__, T...)(T args);
> >
> > D allows defaulted template arguments to occur before non-defaulted ones.
>
> And what is func!("blah")(); is supposed to do, make x = "blah"? args[0]
> = "blah"? both?
>
> (I don't have a compiler at hand to try it =P)
T... is filled in be args, so if there are no args, T is empty. It can't
confuse
x for T...
- Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
Leandro Lucarella Wrote:
> Sean Kelly, el 19 de noviembre a las 14:59 me escribiste:
> >
> > This should work:
> >
> > void func(string x = __FILE__, T...)(T args);
> >
> > D allows defaulted template arguments to occur before non-defaulted ones.
>
> And what is func!("blah")(); is supposed to do, make x = "blah"? args[0]
> = "blah"? both?
void func(string x = __FILE__, T...)(T args)
{
writeln( x, ' ', args.length );
}
void main()
{
func!("hi")();
}
prints:
hi 0
Re: Review: A new stab at a potential std.unittests
Jonathan M Davis, el 19 de noviembre a las 13:24 me escribiste:
> On Friday 19 November 2010 12:39:20 Leandro Lucarella wrote:
> > Sean Kelly, el 19 de noviembre a las 14:59 me escribiste:
> > > Jonathan M Davis Wrote:
> > > > On Friday, November 19, 2010 11:37:16 Sean Kelly wrote:
> > > > > Jonathan M Davis Wrote:
> > > > > > In particular, needing to pass LineInfo() to assertExcThrown!() to
> > > > > > know the file and line number was disliked (though it was by far
> > > > > > the best solution that I'd been able to come up with).
> > > > >
> > > > > Not sure if this helps, but if you default-initialize template
> > > > > function parameters with __LINE__ and __FILE__ they get the line and
> > > > > file of where the template was instantiated.
> > > >
> > > > Yes. The problem was that the function was a _variadic_ template. So,
> > > > you couldn't have default arguments.
> > >
> > > This should work:
> > >
> > > void func(string x = __FILE__, T...)(T args);
> > >
> > > D allows defaulted template arguments to occur before non-defaulted ones.
> >
> > And what is func!("blah")(); is supposed to do, make x = "blah"? args[0]
> > = "blah"? both?
> >
> > (I don't have a compiler at hand to try it =P)
>
> T... is filled in be args, so if there are no args, T is empty. It can't
> confuse
> x for T...
Good point, thanks for the clarification.
--
Leandro Lucarella (AKA luca) http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05)
--
Sometimes you got to suffer a little in your youth to motivate you to
succeed later in life. Do you think if Bill Gates got laid in high
school, do you think there'd be a Microsoft? Of course not. You gotta
spend a lot of time stuffin your own locker with your underwear wedged
up your arse before you think "I'm gona take over the world with
computers! You'll see I'll show them."
Re: Review: A new stab at a potential std.unittests
On 19-nov-10, at 23:44, Sean Kelly wrote: Leandro Lucarella Wrote: Sean Kelly, el 19 de noviembre a las 14:59 me escribiste: This should work: void func(string x = __FILE__, T...)(T args); D allows defaulted template arguments to occur before non- defaulted ones. I wasn't aware that __FILE__ and __LINE__ did expand at the instantiation place, it is nice, but that seems to be the case only for D2 :(
Re: Review: A new stab at a potential std.unittests
On 2010-11-19 19:16, Jonathan M Davis wrote: Updated code: http://is.gd/hqPb2 Okay. As mentioned before, I have helper unit test functions which I use heavily in std.datetime and which are pretty much going to have to either end up as private helper functions in std.datetime or actually get added in a separate module for everyone to use (I'd prefer the latter). My last post on it seems to have met with positive feedback overall for the basic idea but not necessarily the execution. In particular, needing to pass LineInfo() to assertExcThrown!() to know the file and line number was disliked (though it was by far the best solution that I'd been able to come up with). Overall, it's worked quite well, but periodically (for reasons that I _still_ don't understand) passing the function as an alias didn't work properly (sometimes the template constraint which checked that calling the function - exactly how it was called in the code - compiled failed, whereas the template would compile if the constraint was removed, which makes no sense at all to me). So, there were enough problems in using it and enough dislike for how it worked, that I mucked around with it a bit to find a better way to do it, and I believe that I've found one. You end up passing the entire function call as a string (preferably a WYSIWYG string). So, instead of assertExcThrown!(Exception, myfunc)(LineInfo(), param1, param2); you get the much shorter and cleaner mixin(assertExcThrown!(Exception, `myfunc(param1, param2)`)); It mixes the whole thing in on that one line, so it doesn't affect the line count and the call is actually done at the local scope, so it's exactly as if you'd written the function call directly in the function (since that's how it gets compiled in). It also manages to deal with __FILE__ and __LINE__ internally this way without even needing to pass them as default parameters. The one downside is that - being a string mixin - it does come with higher compilation overhead. But that should improve as the compiler improves ( http://d.puremagic.com/issues/show_bug.cgi?id=1382 is likely the main culprit). The list of functions is unchanged, but a few of them became eponymous templates to be mixed in as strings rather than being templated function calls. It is bit annoying to have to use mixin to do it, but the result is much cleaner on the whole, I believe. So, it's a huge useability change. In any case, I'm presenting the updated for your review, so tell me what you think. - Jonathan M Davis P.S. My most recent update of std.datetime doesn't use the updated unit test functions yet, so the only examples of how to use them are in the docs and source of my unittest module - both of which are included in the link above. I'll likely post a version of std.datetime with the updated unit tests later today, so if you really want to see the functions used on a larger scale, you can check that out then. Why don't you use delegates instead of string mixins? For example, assertExcThrown, could take a delegate which calls the function you want to test instead of a string that represents the call. The mixin want be needed as well. Am I missing something? -- /Jacob Carlborg
Re: Review: A new stab at a potential std.unittests
I'm not particularly fond of this interface and think that a solution with a delegate / lazy or alias template parameter would be more convenient. However, until we have ast macros I do see the added value in this approach. Some remarks about the api, not a proper review of the code itself: - core.exception and std.string must be imported to use the module, relevant symbols should be selectively and publicy imported instead. - exception would be better abbreviated as ex instead of exc, I believe this is more conventional. (assertExThrown instead of assertExcThrown) - assertExcThrown should fail if an exception is thrown that is not T or a subclass of T. (catch and throw AssertError) - assertEqual and assertNotEqual is inconsistent in naming with the assertOpFoo class of functions I believe these assertions should be added: - assertExcThrown and assertExcNotThrown with a default T (should be Exception, not Throwable) - something equivalent to std.algorithm.equal, the latter is very useful in unittests - assertOpBinary, assertTrue and perhaps assertPred (where the predicate is an template alias parameter)
Re: Review: A new stab at a potential std.unittests
What about debug vs release compilation for this new module? We know we have assert for debug mode, and enforce for release mode (except the special assert false case). If I want assertExcThrown to be compiled in release mode it seems I'd need an enforced version of it, possibly called enforceExcThrown that uses enforce internally. I'd prefer if we could somehow mark assert's to be compiled in release mode instead of having to duplicate code for assert and enforce versions of these new unittest functions. On 11/20/10, Jacob Carlborg wrote: > On 2010-11-19 19:16, Jonathan M Davis wrote: >> Updated code: http://is.gd/hqPb2 >> >> Okay. As mentioned before, I have helper unit test functions which I use >> heavily >> in std.datetime and which are pretty much going to have to either end up >> as >> private helper functions in std.datetime or actually get added in a >> separate >> module for everyone to use (I'd prefer the latter). My last post on it >> seems to >> have met with positive feedback overall for the basic idea but not >> necessarily >> the execution. >> >> In particular, needing to pass LineInfo() to assertExcThrown!() to know >> the file >> and line number was disliked (though it was by far the best solution that >> I'd >> been able to come up with). Overall, it's worked quite well, but >> periodically >> (for reasons that I _still_ don't understand) passing the function as an >> alias >> didn't work properly (sometimes the template constraint which checked that >> calling the function - exactly how it was called in the code - compiled >> failed, >> whereas the template would compile if the constraint was removed, which >> makes no >> sense at all to me). So, there were enough problems in using it and enough >> dislike for how it worked, that I mucked around with it a bit to find a >> better >> way to do it, and I believe that I've found one. You end up passing the >> entire >> function call as a string (preferably a WYSIWYG string). So, instead of >> >> assertExcThrown!(Exception, myfunc)(LineInfo(), param1, param2); >> >> you get the much shorter and cleaner >> >> mixin(assertExcThrown!(Exception, `myfunc(param1, param2)`)); >> >> >> It mixes the whole thing in on that one line, so it doesn't affect the >> line count >> and the call is actually done at the local scope, so it's exactly as if >> you'd >> written the function call directly in the function (since that's how it >> gets >> compiled in). It also manages to deal with __FILE__ and __LINE__ >> internally >> this way without even needing to pass them as default parameters. The one >> downside is that - being a string mixin - it does come with higher >> compilation >> overhead. But that should improve as the compiler improves ( >> http://d.puremagic.com/issues/show_bug.cgi?id=1382 is likely the main >> culprit). >> >> The list of functions is unchanged, but a few of them became eponymous >> templates >> to be mixed in as strings rather than being templated function calls. It >> is bit >> annoying to have to use mixin to do it, but the result is much cleaner on >> the >> whole, I believe. So, it's a huge useability change. >> >> In any case, I'm presenting the updated for your review, so tell me what >> you >> think. >> >> - Jonathan M Davis >> >> >> P.S. My most recent update of std.datetime doesn't use the updated unit >> test >> functions yet, so the only examples of how to use them are in the docs and >> source of my unittest module - both of which are included in the link >> above. >> I'll likely post a version of std.datetime with the updated unit tests >> later >> today, so if you really want to see the functions used on a larger scale, >> you >> can check that out then. > > Why don't you use delegates instead of string mixins? For example, > assertExcThrown, could take a delegate which calls the function you want > to test instead of a string that represents the call. The mixin want be > needed as well. Am I missing something? > > -- > /Jacob Carlborg >
Re: Review: A new stab at a potential std.unittests
On Saturday 20 November 2010 08:03:52 Jacob Carlborg wrote:
> Why don't you use delegates instead of string mixins? For example,
> assertExcThrown, could take a delegate which calls the function you want
> to test instead of a string that represents the call. The mixin want be
> needed as well. Am I missing something?
Well, delegates wouldn't be a bad idea, but they're unwieldy too. Would you
rather write
assertExcThrown!Exception((){func(param1, param2);});
or
mixin(assertExcThrown!(Exception, `func(param1, param2)`));
The delegate would be less efficient, but you could reduce code duplication by
going that route, since then __FILE__ and __LINE__ could be default parameters
to assertExcThrown!(). It would also make it so that __FILE__ and __LINE__
could
be replace if you reall wanted to (which could be useful), but you could do
that
by changing it so that the mixin version of assertExcThrown!() took __FILE__
and
__LINE__ as default parameters instead of them being internal to the mixin.
Neither is pretty, but they both have their pros and cons. It looks like if I
do
keep the mixin version, I should probably move __FILE__ and __LINE__ to
parameters instead of mixing them in directly, so that it's more flexible. I
suppose that you could just make the function call parameter a lazy one and
templatize assertExcThrown!() on the return type of the function (I _think_
that
that would still work with void). That way, you could avoid having to type
(){;}
around the function call. In a way though, it is a bit silly to declare it as
lazy, since it _always_ gets called. That could be the best way to go though.
Honestly, it never even occurred to me to make it a delegate. I was originally
trying to pass the function and its arguments separately, and that didn't work
as well as I'd have liked, and the mixin solution is what I arrived at. I do
find
having to pass a delegate uglier than having to use a mixin, but that's
probably
pretty subjective and debatable. Using delegates will make for less efficient
unit
tests, but it would mean less code duplication, and thus faster compilation.
So,
it's not entirely a straightforward choice, I think.
The lazy solution sounds pretty good actually. Can anyone think of any real
downsides to that? So, it would look something like
assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t line =
__LINE__);
- Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
On Saturday 20 November 2010 10:16:55 Lutger Blijdestijn wrote: > I'm not particularly fond of this interface and think that a solution with > a delegate / lazy or alias template parameter would be more convenient. > However, until we have ast macros I do see the added value in this > approach. > > Some remarks about the api, not a proper review of the code itself: > > - core.exception and std.string must be imported to use the module, > relevant symbols should be selectively and publicy imported instead. I hadn't thought of that. Good idea. > - exception would be better abbreviated as ex instead of exc, I believe > this is more conventional. (assertExThrown instead of assertExcThrown) I'll think about it. I don't think that I've often seen it abbreviated at all. But if Ex really is more common, then that would be better. > - assertExcThrown should fail if an exception is thrown that is not T or a > subclass of T. (catch and throw AssertError) It currently catches the exact exception (well, throwable) that you tell it to. The others are let through exactly like they would be with an assert. It also means that you get to see exactly what the erroneous exception was so that you can deal with it. An AssertError would be less informative, maybe even misleading. You get an AssertError because it didn't throw any exception, whereas you get the wrong exception if it threw the wrong one. I suppose that I could see an argument that because it didn't throw that exact exception, it should throw an AssertError, but I think that it's worse, because you can't see what the unexpected exception was. It would be less helpful in debugging rather than more. > - assertEqual and assertNotEqual is inconsistent in naming with the > assertOpFoo class of functions Not quite. assertEqual() doesn't explicitly test opEqual(). It uses ==. So, it works with primitives as well as structs that didn't declare an opEquals(). assertOpCmp() on the other hand, explicitly calls opCmp() - so, for instance, assertOpCmp!"=="() specifically tests that opCmp() returned zero rather than using ==. However, assertOpOpAssign doesn't actually call opOpAssign(), so there arguably is an inconsistency there. I'll have to think about how it would be best to adjust those names. I think that assertEqual() should definitely stay assertEqual() - it doesn't explicitly call opEquals() and other unit testing frameworks frequent have a function named assertEquals(), so it will be more familiar to many. I also think that assertOpCmp!() should stay as it is, since it is explicitly calling opCmp(). The really question is what to rename assertOpOpAssign!() to, since like with assertEqual(), it would be too restrictive to directly call opOpAssign(), and I can't rename it assertOpAssign() because opAssign() is something else entirely. So, if a different name would be better, I'm going to have to find one, since I don't know what else to name it. > I believe these assertions should be added: > > - assertExcThrown and assertExcNotThrown with a default T (should be > Exception, not Throwable) That's not a bad idea, though personally, I tend to think that if someone used an Exception rather than a derived type that they aren't be specific enough (I'm sure that many would argue with me on that one though). It might work to just give it a default parameter (and I do agree that the default should be Exception rather than Throwable). > - something equivalent to std.algorithm.equal, the latter is very useful in > unittests I'm not sure that I'd noticed std.algorithm.equal() before. That's a good idea. assertEqual() could simply be templatized in the same manner as std.algorithm.equal(). > - assertOpBinary, assertTrue and perhaps assertPred (where the predicate is > an template alias parameter) I decided that assertTrue() was a waste of time, because that's what a bare assert does. The same goes for assertFalse(), because all you have to do is add ! in front of the expression. All of the functions that I have are there to either improve the output (such as printing out the actual values of the two values being compared in assertEqual()), or they get rid of boilerplate code (such as the try-catch block and other details in assertExcThrown!()). As for assertOpBinary, are you suggesting a function which does the given binary operation and then compares the result with the given expected result? That would be a good addition. As for assertPred. I don't know what that would do. Would that take a predicate and its parameters and then assert that the result was true? If that's what you're looking for, then assertPredTrue and assertPredFalse would be better. I think that you need to clarify quite what you meant though. I pretty much created the unit test functions which I needed for std.datetime. So, I'm sure that there are other good ones which could be added. And I think that we should do so. My main concern though has b
Re: Review: A new stab at a potential std.unittests
On Saturday 20 November 2010 10:23:36 Andrej Mitrovic wrote: > What about debug vs release compilation for this new module? > > We know we have assert for debug mode, and enforce for release mode > (except the special assert false case). If I want assertExcThrown to > be compiled in release mode it seems I'd need an enforced version of > it, possibly called enforceExcThrown that uses enforce internally. > > I'd prefer if we could somehow mark assert's to be compiled in release > mode instead of having to duplicate code for assert and enforce > versions of these new unittest functions. It's specifically versioned on unittest. So, if you have unit tests enable, they'll be compiled in. If you don't, then they won't. Their entire purpose is for improving unit tests, and they aren't really intended for use anywhere else. And since they all throw AssertError on failure, they would be exactly as useful as an assert would be if it were compiled in. Why would you be looking to use them outside of unit tests? What would you be trying to do with them? - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
On Saturday 20 November 2010 16:23:32 Jonathan M Davis wrote: > The lazy solution sounds pretty good actually. Can anyone think of any real > downsides to that? So, it would look something like > > assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t > line = __LINE__); Wait. No. I didn't think that one through enough. It doesn't work. You You don't actually get a delegate out of lazy, and what you need for assertExcThrown!() is a delegate. It would be great to be able to implicitly create a delegate like that, but lazy does it underneath the hood, so to speak, and when the function call is made, you still get the result of the function call, and by that point, it's too late, since to get the result of the function call, you obviously need to have called it. So, the question remains whether it would be better to pass a string which gets mixed in or a delegate which gets called. I think that the delegate solution is uglier to use, but that's debatable. It's less efficient at runtime, but it would be faster to compile and would result in less code bloat (though how much code bloat matters for unit tests is debatable, since they shouldn't end up in released binaries). I'm really not sure which is better. I like the mixin solution, but the delegate solution has its advantages as well. Anyone else have an opinion on whether (and why) the string mixin approach or the delegate approach is better? - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
On 2010-11-21 01:23, Jonathan M Davis wrote:
On Saturday 20 November 2010 08:03:52 Jacob Carlborg wrote:
Why don't you use delegates instead of string mixins? For example,
assertExcThrown, could take a delegate which calls the function you want
to test instead of a string that represents the call. The mixin want be
needed as well. Am I missing something?
Well, delegates wouldn't be a bad idea, but they're unwieldy too. Would you
rather write
assertExcThrown!Exception((){func(param1, param2);});
or
mixin(assertExcThrown!(Exception, `func(param1, param2)`));
I would go with the delegate, but when you format the code like that it
doesn't look any good (btw, no need for the extra pair of empty
parentheses). I think this looks better:
assertExcThrown!Exception({
func(param1, param2);
});
And BTW, D needs a better way to pass a delegates to a function,
something like the syntax that can be used in Scala:
assertExcThrown!Exception {
func(param1, param2);
}
I think that someone has talked about this syntax before on this mailing
list.
The delegate would be less efficient, but you could reduce code duplication by
going that route, since then __FILE__ and __LINE__ could be default parameters
to assertExcThrown!(). It would also make it so that __FILE__ and __LINE__ could
be replace if you reall wanted to (which could be useful), but you could do that
by changing it so that the mixin version of assertExcThrown!() took __FILE__ and
__LINE__ as default parameters instead of them being internal to the mixin.
Neither is pretty, but they both have their pros and cons. It looks like if I do
keep the mixin version, I should probably move __FILE__ and __LINE__ to
parameters instead of mixing them in directly, so that it's more flexible. I
suppose that you could just make the function call parameter a lazy one and
templatize assertExcThrown!() on the return type of the function (I _think_ that
that would still work with void). That way, you could avoid having to type (){;}
around the function call. In a way though, it is a bit silly to declare it as
lazy, since it _always_ gets called. That could be the best way to go though.
Honestly, it never even occurred to me to make it a delegate. I was originally
trying to pass the function and its arguments separately, and that didn't work
as well as I'd have liked, and the mixin solution is what I arrived at. I do
find
having to pass a delegate uglier than having to use a mixin, but that's probably
pretty subjective and debatable. Using delegates will make for less efficient
unit
tests, but it would mean less code duplication, and thus faster compilation. So,
it's not entirely a straightforward choice, I think.
I guess you're right and I would choose the delegates.
The lazy solution sounds pretty good actually. Can anyone think of any real
downsides to that? So, it would look something like
assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t line =
__LINE__);
- Jonathan M Davis
--
/Jacob Carlborg
Re: Review: A new stab at a potential std.unittests
On 2010-11-21 02:34, Jonathan M Davis wrote:
On Saturday 20 November 2010 16:23:32 Jonathan M Davis wrote:
The lazy solution sounds pretty good actually. Can anyone think of any real
downsides to that? So, it would look something like
assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t
line = __LINE__);
Wait. No. I didn't think that one through enough. It doesn't work. You You don't
actually get a delegate out of lazy, and what you need for assertExcThrown!() is
a delegate. It would be great to be able to implicitly create a delegate like
that, but lazy does it underneath the hood, so to speak, and when the function
call is made, you still get the result of the function call, and by that point,
it's too late, since to get the result of the function call, you obviously need
to have called it. So, the question remains whether it would be better to pass a
string which gets mixed in or a delegate which gets called.
Ok, now I'm not sure what you mean. If you don't get a delegate what do
you get? And what to you mean by "and by that point, it's too late", do
late for what?
I think that the delegate solution is uglier to use, but that's debatable. It's
less efficient at runtime, but it would be faster to compile and would result in
less code bloat (though how much code bloat matters for unit tests is debatable,
since they shouldn't end up in released binaries). I'm really not sure which is
better. I like the mixin solution, but the delegate solution has its advantages
as well.
Anyone else have an opinion on whether (and why) the string mixin approach or
the delegate approach is better?
- Jonathan M Davis
This works according to the documentation of assertExcThrown:
void foo (int, int)
{
throw new Throwable("");
}
void assertExcThrown (E = Throwable, T) (lazy T dg, string file =
__FILE__, size_t line = __LINE__)
{
bool thrown;
try
dg();
catch (E e)
thrown = true;
if (!thrown)
throw new AssertError(format("assertExcThrown() failed: No %s
was thrown.", E.stringof), file, line);
}
void main ()
{
assertExcThrown(foo(3, 4));
}
--
/Jacob Carlborg
Re: Review: A new stab at a potential std.unittests
On Sunday 21 November 2010 04:19:51 Jacob Carlborg wrote:
> On 2010-11-21 02:34, Jonathan M Davis wrote:
> > On Saturday 20 November 2010 16:23:32 Jonathan M Davis wrote:
> >> The lazy solution sounds pretty good actually. Can anyone think of any
> >> real downsides to that? So, it would look something like
> >>
> >> assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t
> >> line = __LINE__);
> >
> > Wait. No. I didn't think that one through enough. It doesn't work. You
> > You don't actually get a delegate out of lazy, and what you need for
> > assertExcThrown!() is a delegate. It would be great to be able to
> > implicitly create a delegate like that, but lazy does it underneath the
> > hood, so to speak, and when the function call is made, you still get the
> > result of the function call, and by that point, it's too late, since to
> > get the result of the function call, you obviously need to have called
> > it. So, the question remains whether it would be better to pass a string
> > which gets mixed in or a delegate which gets called.
>
> Ok, now I'm not sure what you mean. If you don't get a delegate what do
> you get? And what to you mean by "and by that point, it's too late", do
> late for what?
>
> > I think that the delegate solution is uglier to use, but that's
> > debatable. It's less efficient at runtime, but it would be faster to
> > compile and would result in less code bloat (though how much code bloat
> > matters for unit tests is debatable, since they shouldn't end up in
> > released binaries). I'm really not sure which is better. I like the
> > mixin solution, but the delegate solution has its advantages as well.
> >
> > Anyone else have an opinion on whether (and why) the string mixin
> > approach or the delegate approach is better?
> >
> > - Jonathan M Davis
>
> This works according to the documentation of assertExcThrown:
>
> void foo (int, int)
> {
> throw new Throwable("");
> }
>
> void assertExcThrown (E = Throwable, T) (lazy T dg, string file =
> __FILE__, size_t line = __LINE__)
> {
> bool thrown;
>
> try
> dg();
>
> catch (E e)
> thrown = true;
>
> if (!thrown)
> throw new AssertError(format("assertExcThrown() failed: No %s
> was thrown.", E.stringof), file, line);
> }
>
> void main ()
> {
> assertExcThrown(foo(3, 4));
> }
Okay. I obviously don't use lazy enough and am probably posting too often
without enough sleep. I was thinking that with a lazy parameter, whatever
delegate which was created for it was called and the value assigned to the
function argument before the function body was actually entered. So, internally
to the function, it didn't matter at all whether a parameter was lazy or not,
as
if the laziness of the parameters affected the outside of the function only and
not the inside. But that would completely defeat the point of lazy and makes no
sense at all.
Now, the code that you give works, and I think that it shows that using lazy is
exactly what assertExcThrown _should_ do. It allows for you to do a normal
function call and then have it explicitly called within the function block
without having to create a delegate or give the function call as a string. It
annoys me a bit to have the extra overhead of the delegate, and part of me does
think that it's silly to have a lazy parameter which is _always_ called, but it
does the job, and it creates less code bloat than the mixin solution, and if
speed is the main concern, the extra compilation overhead of the mixin solution
is probably greater than the overhead of the delegate, so if you expect to
compile and then immediately run the unit tests, the overall result of using a
delegate is likely faster. And the lazy parameter gets rid of any ugliness that
creating a delegate might have over the string mixin.
So, I'll change it to work with a lazy parameter instead. Thanks for your help.
- Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
On 2010-11-21 14:06, Jonathan M Davis wrote:
On Sunday 21 November 2010 04:19:51 Jacob Carlborg wrote:
On 2010-11-21 02:34, Jonathan M Davis wrote:
On Saturday 20 November 2010 16:23:32 Jonathan M Davis wrote:
The lazy solution sounds pretty good actually. Can anyone think of any
real downsides to that? So, it would look something like
assertExcThrown(E : Throwable, T)(lazy T, string file = __FILE__, size_t
line = __LINE__);
Wait. No. I didn't think that one through enough. It doesn't work. You
You don't actually get a delegate out of lazy, and what you need for
assertExcThrown!() is a delegate. It would be great to be able to
implicitly create a delegate like that, but lazy does it underneath the
hood, so to speak, and when the function call is made, you still get the
result of the function call, and by that point, it's too late, since to
get the result of the function call, you obviously need to have called
it. So, the question remains whether it would be better to pass a string
which gets mixed in or a delegate which gets called.
Ok, now I'm not sure what you mean. If you don't get a delegate what do
you get? And what to you mean by "and by that point, it's too late", do
late for what?
I think that the delegate solution is uglier to use, but that's
debatable. It's less efficient at runtime, but it would be faster to
compile and would result in less code bloat (though how much code bloat
matters for unit tests is debatable, since they shouldn't end up in
released binaries). I'm really not sure which is better. I like the
mixin solution, but the delegate solution has its advantages as well.
Anyone else have an opinion on whether (and why) the string mixin
approach or the delegate approach is better?
- Jonathan M Davis
This works according to the documentation of assertExcThrown:
void foo (int, int)
{
throw new Throwable("");
}
void assertExcThrown (E = Throwable, T) (lazy T dg, string file =
__FILE__, size_t line = __LINE__)
{
bool thrown;
try
dg();
catch (E e)
thrown = true;
if (!thrown)
throw new AssertError(format("assertExcThrown() failed: No %s
was thrown.", E.stringof), file, line);
}
void main ()
{
assertExcThrown(foo(3, 4));
}
Okay. I obviously don't use lazy enough and am probably posting too often
without enough sleep. I was thinking that with a lazy parameter, whatever
delegate which was created for it was called and the value assigned to the
function argument before the function body was actually entered. So, internally
to the function, it didn't matter at all whether a parameter was lazy or not, as
if the laziness of the parameters affected the outside of the function only and
not the inside. But that would completely defeat the point of lazy and makes no
sense at all.
Now, the code that you give works, and I think that it shows that using lazy is
exactly what assertExcThrown _should_ do. It allows for you to do a normal
function call and then have it explicitly called within the function block
without having to create a delegate or give the function call as a string. It
annoys me a bit to have the extra overhead of the delegate, and part of me does
think that it's silly to have a lazy parameter which is _always_ called, but it
does the job, and it creates less code bloat than the mixin solution, and if
speed is the main concern, the extra compilation overhead of the mixin solution
is probably greater than the overhead of the delegate, so if you expect to
compile and then immediately run the unit tests, the overall result of using a
delegate is likely faster. And the lazy parameter gets rid of any ugliness that
creating a delegate might have over the string mixin.
So, I'll change it to work with a lazy parameter instead. Thanks for your help.
- Jonathan M Davis
I don't know if the compiler can inline delegates or not but if it can I
think this case would be very easy for the compiler to inline the delegate.
--
/Jacob Carlborg
Re: Review: A new stab at a potential std.unittests
On Sunday 21 November 2010 05:44:15 Jacob Carlborg wrote: > I don't know if the compiler can inline delegates or not but if it can I > think this case would be very easy for the compiler to inline the delegate. It can't. That's one of the big issues with enforce. At the moment, it actually makes it so that if you use enforce in a function, that function can't be inlined. Assuming that that gets fixed, assertExcThrown!() might then be inlineable, depending on how picky the inliner is, at which point perhaps the delegate could be inlined, but I don't think that delegates will ever be inlineable unless the function they're called in is inlined, because the delegate to inline would change every time that the function is called. In any case, it's definitely true that the compiler may be able to better optimize assertExcThrown!(). But much as I'd like fast unit testing code, I'd much rather have useable and readily maintainable unit testing code than fast unit testing code. It would be nice to have both though. - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests
Jonathan M Davis wrote: > On Saturday 20 November 2010 10:16:55 Lutger Blijdestijn wrote: >> I'm not particularly fond of this interface and think that a solution >> with a delegate / lazy or alias template parameter would be more >> convenient. However, until we have ast macros I do see the added value in >> this approach. >> >> Some remarks about the api, not a proper review of the code itself: >> >> - core.exception and std.string must be imported to use the module, >> relevant symbols should be selectively and publicy imported instead. > > I hadn't thought of that. Good idea. > >> - exception would be better abbreviated as ex instead of exc, I believe >> this is more conventional. (assertExThrown instead of assertExcThrown) > > I'll think about it. I don't think that I've often seen it abbreviated at > all. But if Ex really is more common, then that would be better. I know of std.exception.enforceEx, and catch(Exception ex) is also regularly used in examples. >> - assertExcThrown should fail if an exception is thrown that is not T or >> a subclass of T. (catch and throw AssertError) > > It currently catches the exact exception (well, throwable) that you tell > it to. The others are let through exactly like they would be with an > assert. It also means that you get to see exactly what the erroneous > exception was so that you can deal with it. An AssertError would be less > informative, maybe even misleading. You get an AssertError because it > didn't throw any exception, whereas you get the wrong exception if it > threw the wrong one. Suppose you want assert that a FileNotFoundException is thrown. Now if you get an Exception then: - technically the unittest has passed because no AssertError has been thrown (splitting hairs) - if for whatever reason you catch this error in the unittest, things will get screwed up - You (or possibly some other script) won' t get the same command line output, it's then harder to correlate the exception with the assertion I agree about wanting to know the original Exception though. I think this is possible by setting the .next field of AssertError with that exception. ... > >> I believe these assertions should be added: >> >> - assertExcThrown and assertExcNotThrown with a default T (should be >> Exception, not Throwable) > > That's not a bad idea, though personally, I tend to think that if someone > used an Exception rather than a derived type that they aren't be specific > enough (I'm sure that many would argue with me on that one though). It > might work to just give it a default parameter (and I do agree that the > default should be Exception rather than Throwable). I agree but it doesn't matter for general use, people will want this and this is also practice in phobos (mostly through the use of enforce I think). ... > >> - assertOpBinary, assertTrue and perhaps assertPred (where the predicate >> is an template alias parameter) > > I decided that assertTrue() was a waste of time, because that's what a > bare assert does. The same goes for assertFalse(), because all you have to > do is add ! in front of the expression. All of the functions that I have > are there to either improve the output (such as printing out the actual > values of the two values being compared in assertEqual()), or they get rid > of boilerplate code (such as the try-catch block and other details in > assertExcThrown!()). Except that assert does not print the expression which resulted in the assertion like all other functions do, so assertTrue does improve the output too. > As for assertOpBinary, are you suggesting a function which does the given > binary operation and then compares the result with the given expected > result? That would be a good addition. > > As for assertPred. I don't know what that would do. Would that take a > predicate and its parameters and then assert that the result was true? If > that's what you're looking for, then assertPredTrue and assertPredFalse > would be better. I think that you need to clarify quite what you meant > though. bool isSorted(); int[] numbers = getSortedNumbers(); assertPred!isSorted(numbers); It's the same as assert(isSorted(numbers)), except it allows for improved output. Not very important, but I find it common to use such predicates for testing so it might help. Alternatively assertTrue and assertFalse could take an optional predicate, defaulting to the identity and negation respectively: assertTrue!isSorted(numbers);
Re: Review: A new stab at a potential std.unittests
Jacob Carlborg wrote:
> On 2010-11-21 01:23, Jonathan M Davis wrote:
>> On Saturday 20 November 2010 08:03:52 Jacob Carlborg wrote:
>>> Why don't you use delegates instead of string mixins? For example,
>>> assertExcThrown, could take a delegate which calls the function you want
>>> to test instead of a string that represents the call. The mixin want be
>>> needed as well. Am I missing something?
>>
>> Well, delegates wouldn't be a bad idea, but they're unwieldy too. Would
>> you rather write
>>
>> assertExcThrown!Exception((){func(param1, param2);});
>>
>> or
>>
>> mixin(assertExcThrown!(Exception, `func(param1, param2)`));
>
> I would go with the delegate, but when you format the code like that it
> doesn't look any good (btw, no need for the extra pair of empty
> parentheses). I think this looks better:
>
> assertExcThrown!Exception({
> func(param1, param2);
> });
>
> And BTW, D needs a better way to pass a delegates to a function,
> something like the syntax that can be used in Scala:
>
> assertExcThrown!Exception {
> func(param1, param2);
> }
This is possible, but too surprising:
assertExcThrown!Exception = {
func(param1, param2);
};
with lazy:
assertExcThrown!Exception = func(param1, param2);
Re: Review: A new stab at a potential std.unittests
On 2010-11-21 17:20, Lutger Blijdestijn wrote:
Jacob Carlborg wrote:
On 2010-11-21 01:23, Jonathan M Davis wrote:
On Saturday 20 November 2010 08:03:52 Jacob Carlborg wrote:
Why don't you use delegates instead of string mixins? For example,
assertExcThrown, could take a delegate which calls the function you want
to test instead of a string that represents the call. The mixin want be
needed as well. Am I missing something?
Well, delegates wouldn't be a bad idea, but they're unwieldy too. Would
you rather write
assertExcThrown!Exception((){func(param1, param2);});
or
mixin(assertExcThrown!(Exception, `func(param1, param2)`));
I would go with the delegate, but when you format the code like that it
doesn't look any good (btw, no need for the extra pair of empty
parentheses). I think this looks better:
assertExcThrown!Exception({
func(param1, param2);
});
And BTW, D needs a better way to pass a delegates to a function,
something like the syntax that can be used in Scala:
assertExcThrown!Exception {
func(param1, param2);
}
This is possible, but too surprising:
assertExcThrown!Exception = {
func(param1, param2);
};
with lazy:
assertExcThrown!Exception = func(param1, param2);
There's also the operator overload abuse which overloads opIn:
assertExcThrown!Exception in {
func(param1, param2);
};
--
/Jacob Carlborg
Re: Review: A new stab at a potential std.unittests
On Sunday 21 November 2010 08:11:06 Lutger Blijdestijn wrote:
> Jonathan M Davis wrote:
> > On Saturday 20 November 2010 10:16:55 Lutger Blijdestijn wrote:
> >> I'm not particularly fond of this interface and think that a solution
> >> with a delegate / lazy or alias template parameter would be more
> >> convenient. However, until we have ast macros I do see the added value
> >> in this approach.
> >>
> >> Some remarks about the api, not a proper review of the code itself:
> >>
> >> - core.exception and std.string must be imported to use the module,
> >> relevant symbols should be selectively and publicy imported instead.
> >
> > I hadn't thought of that. Good idea.
> >
> >> - exception would be better abbreviated as ex instead of exc, I believe
> >> this is more conventional. (assertExThrown instead of assertExcThrown)
> >
> > I'll think about it. I don't think that I've often seen it abbreviated at
> > all. But if Ex really is more common, then that would be better.
>
> I know of std.exception.enforceEx, and catch(Exception ex) is also
> regularly used in examples.z
Then, I'll change it to assertExThrown.
> >> - assertExcThrown should fail if an exception is thrown that is not T or
> >> a subclass of T. (catch and throw AssertError)
> >
> > It currently catches the exact exception (well, throwable) that you tell
> > it to. The others are let through exactly like they would be with an
> > assert. It also means that you get to see exactly what the erroneous
> > exception was so that you can deal with it. An AssertError would be less
> > informative, maybe even misleading. You get an AssertError because it
> > didn't throw any exception, whereas you get the wrong exception if it
> > threw the wrong one.
>
> Suppose you want assert that a FileNotFoundException is thrown. Now if you
> get an Exception then:
> - technically the unittest has passed because no AssertError has been
> thrown (splitting hairs)
> - if for whatever reason you catch this error in the unittest, things will
> get screwed up
> - You (or possibly some other script) won' t get the same command line
> output, it's then harder to correlate the exception with the assertion
>
> I agree about wanting to know the original Exception though. I think this
> is possible by setting the .next field of AssertError with that exception.
Actually, not only does AssertError not take a next as one if its constructor
parameters, but even if you set it directly before throwing it, it still
doesn't
print out the other exception. So, if you throw an AssertError in the case of
the wrong exception, the best that you're going to get is a message saying what
type of exception was thrown and maybe what it said, but you couldn't get its
stack trace, which could be far more important.
> >> I believe these assertions should be added:
> >>
> >> - assertExcThrown and assertExcNotThrown with a default T (should be
> >> Exception, not Throwable)
> >
> > That's not a bad idea, though personally, I tend to think that if someone
> > used an Exception rather than a derived type that they aren't be specific
> > enough (I'm sure that many would argue with me on that one though). It
> > might work to just give it a default parameter (and I do agree that the
> > default should be Exception rather than Throwable).
>
> I agree but it doesn't matter for general use, people will want this and
> this is also practice in phobos (mostly through the use of enforce I
> think).
I'll do it, but I do think that it's generally bad practice to throw an
Exception rather than a subtype of Exception (except perhaps in a script or
other similarly short program).
> >> - assertOpBinary, assertTrue and perhaps assertPred (where the predicate
> >> is an template alias parameter)
> >
> > I decided that assertTrue() was a waste of time, because that's what a
> > bare assert does. The same goes for assertFalse(), because all you have
> > to do is add ! in front of the expression. All of the functions that I
> > have are there to either improve the output (such as printing out the
> > actual values of the two values being compared in assertEqual()), or
> > they get rid of boilerplate code (such as the try-catch block and other
> > details in assertExcThrown!()).
>
> Except that assert does not print the expression which resulted in the
> assertion like all other functions do, so assertTrue does improve the
> output too.
Neither do any of the others. They print the value. So if
assertEqual(func("hello", "world"), 2); on line 115 of config.d failed because
func("hello", "world") returned a 3, you'd get
core.exception.asserter...@config.d(115): assertEquals() failed: actual [3],
expected [2].
whereas with assert(func("hello", "world") == 2); you'd get
core.exception.asserter...@config(115): unittest failure
assertEqual() is way more informative, but it doesn't print the actual
expression, just the result of the expression. Short of using string mixins, I
don't know how to
Re: Review: A new stab at a potential std.unittests [Update]
Most recent code: http://is.gd/hO8HP Per the suggestions in this thread, I changed the string mixin templates to functions using lazy. I also added assertOpBinary and made assertOpOpAssign() call opOpAssign() directly. Finally, I made assertEqual() and assertNotEqual() templatized on a predicate (like std.algorithm.equal()) as suggested. I did not add an assertPred(), because on further reflection, it didn't really make sense. Aside from the issue of unary vs binary predicates (which could be solved with assertUniPred() and assertBiPred()), what a predicate does varies so much from function to function, that testing one generically just doesn't make sense. For instance, find() is looking for equality whereas sort() is looking for a comparison. You could test that all elements in a range return true for a predicate like you might want with find(), but doing the same thing with sort()'s predicate makes no sense at all. So, I didn't add assertPred() or anything similar. Any more feedback on my potential std.unittests, or is looking good overall? I definitely think that it's better than when I first posted it, so the feedback thus far has definitely been helpful, and I do find these functions extremely useful in my own code. - Jonathan M Davis P.S. My most recent update for std.datetime does include the most recent version of unittests.d, so if you want to see them in more large scale use, you can look there.
Re: Review: A new stab at a potential std.unittests [Update]
On 2010-11-26 09:10, Jonathan M Davis wrote: Any more feedback on my potential std.unittests, or is looking good overall? I definitely think that it's better than when I first posted it, so the feedback thus far has definitely been helpful, and I do find these functions extremely useful in my own code. - Jonathan M Davis P.S. My most recent update for std.datetime does include the most recent version of unittests.d, so if you want to see them in more large scale use, you can look there. How about catching AssertErrors (if possible), collecting them and then at the end of the unit test output all failed unit tests. This will allow the unit tests after a failing unit test to also run. Feel free to have a look at my rspec inspired unit test "framework": http://dsource.org/projects/orange/browser/orange/test/UnitTester.d -- /Jacob Carlborg
Re: Review: A new stab at a potential std.unittests [Update]
On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote: > On 2010-11-26 09:10, Jonathan M Davis wrote: > > Any more feedback on my potential std.unittests, or is looking good > > overall? I definitely think that it's better than when I first posted > > it, so the feedback thus far has definitely been helpful, and I do find > > these functions extremely useful in my own code. > > > > - Jonathan M Davis > > > > > > P.S. My most recent update for std.datetime does include the most recent > > version of unittests.d, so if you want to see them in more large scale > > use, you can look there. > > How about catching AssertErrors (if possible), collecting them and then > at the end of the unit test output all failed unit tests. This will > allow the unit tests after a failing unit test to also run. > > Feel free to have a look at my rspec inspired unit test "framework": > http://dsource.org/projects/orange/browser/orange/test/UnitTester.d All of my functions work essentially the same as assert does. That means that once an assertion fails (or one of my functions fails) in a unittest block, that block has failed and is done. This has been discussed before, and the general consensus is that this is the correct and desired behavior. It can be a bit annoying if you have a whole bunch of unrelated assertions in a single unittest block, but if you're doing anything where each assertion relies on state with an assertion relying on previous assertions passing, and assertions didn't really stop the test from running, you'd get a ton of irrelevant errors after a single failure. My functions are intended to expand on assert, giving you better error messages on failures and reducing boiler plate code, not change how unit tests in D fundamentally work. And I don't want to introduce stuff into Phobos which changes how unit tests in D fundamentally work. I think that it's great as is. It's just that helper functions that produce better error messages than assert, and helper functions which reduce boiler plate code in common test cases can improve on assert and make for better unit tests, and I think that they would be a good addition to Phobos. Currently, if a unittest block fails, then none of the other unittest blocks in that module run, but unittest blocks in other modules do run. It's supposed to be changed so that all unittest blocks in a module are run even if one fails, but that change hasn't been made yet. Once that is made, you'll be able to split up your unittest blocks to more thoroughly to allow for more tests to run on a single failure if you'd like to. But it is by design that once you have an error in a unittest block, that unittest block does not continue to run. - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests [Update]
On 2010-11-27 21:36, Jonathan M Davis wrote: On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote: On 2010-11-26 09:10, Jonathan M Davis wrote: Any more feedback on my potential std.unittests, or is looking good overall? I definitely think that it's better than when I first posted it, so the feedback thus far has definitely been helpful, and I do find these functions extremely useful in my own code. - Jonathan M Davis P.S. My most recent update for std.datetime does include the most recent version of unittests.d, so if you want to see them in more large scale use, you can look there. How about catching AssertErrors (if possible), collecting them and then at the end of the unit test output all failed unit tests. This will allow the unit tests after a failing unit test to also run. Feel free to have a look at my rspec inspired unit test "framework": http://dsource.org/projects/orange/browser/orange/test/UnitTester.d All of my functions work essentially the same as assert does. That means that once an assertion fails (or one of my functions fails) in a unittest block, that block has failed and is done. This has been discussed before, and the general consensus is that this is the correct and desired behavior. It can be a bit annoying if you have a whole bunch of unrelated assertions in a single unittest block, but if you're doing anything where each assertion relies on state with an assertion relying on previous assertions passing, and assertions didn't really stop the test from running, you'd get a ton of irrelevant errors after a single failure. This how it works in my unit test library: For every test you call a testing function that takes a delegate, in this delegate you have your assertions. This function will call the delegate and catch all assert exceptions/errors and collected them for later outputting. Now this delegate will end execution if an exception is thrown and the remaining code in the delegate will not be run. If you want your test to stop after a failing assert you put all the asserts into one call to the testing function. If you want your test to continue after a failing assert you call the testing function multiple times. My functions are intended to expand on assert, giving you better error messages on failures and reducing boiler plate code, not change how unit tests in D fundamentally work. And I don't want to introduce stuff into Phobos which changes how unit tests in D fundamentally work. I think that it's great as is. It's just that helper functions that produce better error messages than assert, and helper functions which reduce boiler plate code in common test cases can improve on assert and make for better unit tests, and I think that they would be a good addition to Phobos. I guess your functions are like assert helpers and my library is at a higher level. Currently, if a unittest block fails, then none of the other unittest blocks in that module run, but unittest blocks in other modules do run. It's supposed to be changed so that all unittest blocks in a module are run even if one fails, but that change hasn't been made yet. Once that is made, you'll be able to split up your unittest blocks to more thoroughly to allow for more tests to run on a single failure if you'd like to. But it is by design that once you have an error in a unittest block, that unittest block does not continue to run. - Jonathan M Davis -- /Jacob Carlborg
Re: Review: A new stab at a potential std.unittests [Update]
On Saturday 27 November 2010 15:30:23 Jacob Carlborg wrote: > On 2010-11-27 21:36, Jonathan M Davis wrote: > > On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote: > >> On 2010-11-26 09:10, Jonathan M Davis wrote: > >>> Any more feedback on my potential std.unittests, or is looking good > >>> overall? I definitely think that it's better than when I first posted > >>> it, so the feedback thus far has definitely been helpful, and I do find > >>> these functions extremely useful in my own code. > >>> > >>> - Jonathan M Davis > >>> > >>> > >>> P.S. My most recent update for std.datetime does include the most > >>> recent version of unittests.d, so if you want to see them in more > >>> large scale use, you can look there. > >> > >> How about catching AssertErrors (if possible), collecting them and then > >> at the end of the unit test output all failed unit tests. This will > >> allow the unit tests after a failing unit test to also run. > >> > >> Feel free to have a look at my rspec inspired unit test "framework": > >> http://dsource.org/projects/orange/browser/orange/test/UnitTester.d > > > > All of my functions work essentially the same as assert does. That means > > that once an assertion fails (or one of my functions fails) in a > > unittest block, that block has failed and is done. This has been > > discussed before, and the general consensus is that this is the correct > > and desired behavior. It can be a bit annoying if you have a whole bunch > > of unrelated assertions in a single unittest block, but if you're doing > > anything where each assertion relies on state with an assertion relying > > on previous assertions passing, and assertions didn't really stop the > > test from running, you'd get a ton of irrelevant errors after a single > > failure. > > This how it works in my unit test library: > > For every test you call a testing function that takes a delegate, in > this delegate you have your assertions. This function will call the > delegate and catch all assert exceptions/errors and collected them for > later outputting. Now this delegate will end execution if an exception > is thrown and the remaining code in the delegate will not be run. > > If you want your test to stop after a failing assert you put all the > asserts into one call to the testing function. If you want your test to > continue after a failing assert you call the testing function multiple > times. I confess that I don't see any real difference between that and simply splitting up unittest blocks (assuming, of course, that subsequent unittest blocks in a module get run after one fails, which hasn't been implmented yet), and the delegate solution at least sounds more complicated. > > My functions are intended to expand on assert, giving you better error > > messages on failures and reducing boiler plate code, not change how unit > > tests in D fundamentally work. And I don't want to introduce stuff into > > Phobos which changes how unit tests in D fundamentally work. I think > > that it's great as is. It's just that helper functions that produce > > better error messages than assert, and helper functions which reduce > > boiler plate code in common test cases can improve on assert and make > > for better unit tests, and I think that they would be a good addition to > > Phobos. > > I guess your functions are like assert helpers and my library is at a > higher level. That's probably a good way to put it. I think that higher level libraries certainly have their place, but I don't think that it necessarily makes sense to put that sort of thing in Phobos. - Jonathan M Davis
Re: Review: A new stab at a potential std.unittests [Update]
On 2010-11-28 01:17, Jonathan M Davis wrote: On Saturday 27 November 2010 15:30:23 Jacob Carlborg wrote: On 2010-11-27 21:36, Jonathan M Davis wrote: On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote: On 2010-11-26 09:10, Jonathan M Davis wrote: Any more feedback on my potential std.unittests, or is looking good overall? I definitely think that it's better than when I first posted it, so the feedback thus far has definitely been helpful, and I do find these functions extremely useful in my own code. - Jonathan M Davis P.S. My most recent update for std.datetime does include the most recent version of unittests.d, so if you want to see them in more large scale use, you can look there. How about catching AssertErrors (if possible), collecting them and then at the end of the unit test output all failed unit tests. This will allow the unit tests after a failing unit test to also run. Feel free to have a look at my rspec inspired unit test "framework": http://dsource.org/projects/orange/browser/orange/test/UnitTester.d All of my functions work essentially the same as assert does. That means that once an assertion fails (or one of my functions fails) in a unittest block, that block has failed and is done. This has been discussed before, and the general consensus is that this is the correct and desired behavior. It can be a bit annoying if you have a whole bunch of unrelated assertions in a single unittest block, but if you're doing anything where each assertion relies on state with an assertion relying on previous assertions passing, and assertions didn't really stop the test from running, you'd get a ton of irrelevant errors after a single failure. This how it works in my unit test library: For every test you call a testing function that takes a delegate, in this delegate you have your assertions. This function will call the delegate and catch all assert exceptions/errors and collected them for later outputting. Now this delegate will end execution if an exception is thrown and the remaining code in the delegate will not be run. If you want your test to stop after a failing assert you put all the asserts into one call to the testing function. If you want your test to continue after a failing assert you call the testing function multiple times. I confess that I don't see any real difference between that and simply splitting up unittest blocks (assuming, of course, that subsequent unittest blocks in a module get run after one fails, which hasn't been implmented yet), and the delegate solution at least sounds more complicated. As you said, it hasn't been implemented yet and I wanted that behavior now. My functions are intended to expand on assert, giving you better error messages on failures and reducing boiler plate code, not change how unit tests in D fundamentally work. And I don't want to introduce stuff into Phobos which changes how unit tests in D fundamentally work. I think that it's great as is. It's just that helper functions that produce better error messages than assert, and helper functions which reduce boiler plate code in common test cases can improve on assert and make for better unit tests, and I think that they would be a good addition to Phobos. I guess your functions are like assert helpers and my library is at a higher level. That's probably a good way to put it. I think that higher level libraries certainly have their place, but I don't think that it necessarily makes sense to put that sort of thing in Phobos. - Jonathan M Davis -- /Jacob Carlborg
