On Friday, February 25, 2011 14:33:20 Magnus Lie Hetland wrote: > On 2011-02-25 20:04:10 +0100, Jonathan M Davis said: > > On Friday, February 25, 2011 07:30:50 Magnus Lie Hetland wrote: > >> Or, more generally, how do you test asserts (which is what I'm using in > >> my preconditions etc.)? > >> > >> As far as I can see, collectException() won't collect errors, which is > >> what assert() throws -- so what's the standard way of writing unit > >> tests for preconditions that use assert? (I.e., test that they will, in > >> fact, throw when you break them.) > > > > I think that the reality of the matter is the most of the time people > > _don't_ check them. And on some level, it doesn't make sense to. It's > > kind of like asking how people test their unit tests. Unit tests are > > already testing code. Do > > you want to be testing them on top of that? And if you do, do you test > > _that_ code? Where do you stop? > > I guess so. But you could say the same thing about other cases where > you throw an exception when you detect that something is wrong -- but > those are normally tested, right? Also, the difference here is that the > precondition is written as a general "test", whereas my actual test > would have specific cases. > > For example, I have a test that checks that I don't add the same object > twice to some structure, and the check involves some traversal -- code > that could potentially be wrong. I wanted to make sure that it wasn't > by explicitly adding the same object twice -- code (i.e., my unit test) > that most likely could not be wrong. > > But I do see your point. > > [snip] > > > And testing post-conditions and invariants in the manner that you're > > trying to do borders on impossible. What are you going to do, repeat the > > post-condition or > > invariant test on the result of the function or on the state of the > > object that the function was called on after the function was called? > > That's just doing the test twice. > > Right. > > > You might as well just re-read the post-conditions and invariants to > > make sure that you wrote them correctly. > > > > I do see value in testing pre-conditions if you're using exceptions > > rather than assertions (which means that you're not use in blocks). In > > that case, you're testing the API to make sure that it does what it's > > supposed to do. But if you're dealing with assertions, then it's really > > test code as opposed to API code, and I don't see the same value in > > testing that. You'd just be testing test > > code. > > OK. For the practical reason, I refer you to my explanation above. But > I guess it's a style issue -- and I'm fine with not testing these > things, by all means.
When using assertions, you're checking the logic of your program and they should _always_ be true. When using exceptions, it's something that can conceivably fail at runtime. And if you view your function or class/struct as being part of an API, then you don't know who or what code will be using it, so it generally makes sense to use exceptions. If you use a pre-condition and assert that input is valid, then you're really testing the code that's calling your function, not the function itself. So, using such assertions makes good sense when you control both the caller and the callee and it's something that should never happen. However, if you don't necessarily control the caller or if it's something that _could_ happen at runtime (even if it shouldn't), then an exception makes a lot more sense. I tend to view exceptions as part of the API and think that they should be tested. Assertions, on the other hand, aren't really part of the API, since they go away in release mode, and I therefore view them as test code. They're verifying that your logic is correct. So, on some level, it is indeed a stylistic thing, but where you choose to use exceptions and where you choose to use assertions can have a big effect on code that uses your code. > [snip] > > > Those changes _do_ make it so that you can use collectException to > > collect an Error (though it defaults to catching Exceptions only), but > > they also > > include assertThrown and assertNotThrown which effectively assert that > > the Exception or Error that you expected to be thrown (or not) from a > > particular expression or function call was indeed thrown (or not). > > So, you _can_ use that with AssertError to verify your pre-conditions. > > OK, thanks. > > > However, I would point out that catching Errors is generally a _bad_ > > idea. > > [snip lots of useful stuff] > > Thanks for educating me :D > > I guess the conclusion will be that I'll focus on keeping my > preconditions really simple. (And any utility functions I use in them > can then get unit tests of their own instead ;) That's probably a good way to handle it. I find that I often tend to have helper functions like that simply because I end up testing the same thing in a variety of places, and I don't want to duplicate the code. It also has the advantage of making it so that you can therefore explicitly test that that function works correctly rather than having to worry about how it's used - like dealing with AssertErrors. Personally, the only place that I've caught AssertErrors is in testing functions like assertThrown, because in that case, the fact that the function throws an AssertError is an integral part of its behavior and API. - Jonathan M Davis
