On Fri, Dec 05, 2014 at 03:55:22PM +0000, Chris via Digitalmars-d wrote: > On Friday, 5 December 2014 at 15:44:35 UTC, Wyatt wrote: > >On Friday, 5 December 2014 at 14:53:43 UTC, Chris wrote: > >> > >>As I said, I'm not against unit tests and I use them where they make > >>sense (difficult output, not breaking existing tested code). But I > >>often don't bother with them when they tell me what I already know. > >> > >>assert(addNumbers(1,1) == 2); > >> > >>I've found myself in the position when unit tests give me a false > >>sense of security.
This is an example of a poor unittest. Well, maybe *one* such case isn't a bad idea to stick in a unittest block somewhere (to make sure things haven't broken *outright*, but you'd notice that via other channels pretty quickly!). But this is akin to writing a unittest that computes the square root of a number in order to test a function that computes the square root of a number. Either it's already blindingly obvious and you're just wasting time, or the unittest is so complex that it proves nothing (you could be repeating exactly the same bugs as the code itself!). No, a better way to writing a unittest is to approach it from the user's (i.e., caller's) POV. Given this function (as a black box), what kind of behaviour do I expect from it? What if I give it unusual arguments, will it still give the correct result? It's well-known that most bugs happen on boundary conditions, not in the general output (which is usually easy to get right the first time). So, unittests should mainly focus on boundary and exceptional cases. For example, in testing a sqrt function, I wouldn't waste time testing sqrt(16) or sqrt(65536) -- at the most, I'd do just one such case and move on. But most of the testing should be on the exceptional cases, e.g., what happens with sqrt(17) if the function returns an int? That's one case. What about sqrt(1)? sqrt(0)? what happens if you hand it a negative number? > >Sure, you need to test the obvious things, > > Everywhere? For each function? It may be desirable but hard to > maintain. Also, unit tests break when you change the behavior of a > function, then you have to redesign the unit test for this particular > function. I prefer unit tests for bigger chunks. Usually, I don't even bother unittesting a function that isn't generic enough that I know it won't drastically change over time. Usually, it's when I start factoring out code in generic form that I really start working on the unittests. When I'm still in the experimental / exploratory stage, I'd throw in some tests to catch boundary conditions, but I wouldn't spend too much time on that. Most of the unittests should be aimed at preserving certain guarantees -- e.g., math functions should obey certain identities even around boundary values, API functions should always behave according to what external users would expect, etc.. But internal functions that are subject to a lot of changes -- I wouldn't do too much more than just stick in a few things that I know might be problematic (usually while writing the code itself). Any cases not caught by this will be caught at the API boundary when something starts failing API guarantees. Besides these, I'd add a unittest for each bug I fix -- for regression control. I'm not afraid of outright deleting unittests if the associated function has been basically gutted and rewritten from scratch, if said unittests are more concerned with implementation details. The ones concerned with overall behaviour would be kept. This is another reason it's better to put the unittest effort on the API level than on overly white-box dependent parts, since those are subject to frequent revisions. > >but I find the real gains come from being able to verify the > >behaviour of edge cases and pathological input; and, critically, > >ensuring that that behaviour doesn't change as you refactor. (My day > >job involves writing and maintaining legacy network libraries and > >parsers in pure C. D's clean and easy unit tests would be a godsend > >for me.) > > > >-Wyatt > > True, true. Unfortunately, the edge cases are usually spotted when > using the software, not in unit tests. They can be included later, but > new pathological input keeps coming up (especially if you write for > third party software). I guess it depends on the kind of application you write, but when writing unittests I tend to focus on what ways the code could break, rather than how it might work. Sure, you won't be able to come up with *all* the cases, and unittests sure don't guarantee 100% bug-free code, but generally you do catch the most frequent ones, which saves time dealing with the whole cycle of customer reports, generating bug fix change orders, QA testing, etc.. The ones that weren't caught early will eventually be found in the field, and they would be added to the growing body of unittests to control future regressions. > Now don't get me wrong, I wouldn't want to miss unit tests in D, but I > use them more carefully now, not everywhere. As with all things, I'm skeptical of blindly applying some methodology even when it's not applicable or of questionable benefit. So while I definitely highly recommend D unittests, I wouldn't go so far as to mandate that, for example, every function must have at least 3 test cases or something like that. While I do use unittests a lot in D because I find it helpful, I'm skeptical of going all-out TDD. Everything in real-life is context- and situation-dependent, and such overly-zealous rule application usually results in wasted efforts for only marginal benefits. T -- Тише едешь, дальше будешь.
