Re: Documented unittests & code coverage

2016-08-05 Thread Jonathan M Davis via Digitalmars-d
On Friday, August 05, 2016 14:10:02 Walter Bright via Digitalmars-d wrote: > On 8/5/2016 12:10 PM, Jonathan M Davis via Digitalmars-d wrote: > > [...] > > Yes, I pretty much agree with your entire post. I just wish that you'd agree that issues like these https://issues.dlang.org/show_bug.cgi?id=1

Re: Documented unittests & code coverage

2016-08-05 Thread Walter Bright via Digitalmars-d
On 8/5/2016 12:10 PM, Jonathan M Davis via Digitalmars-d wrote: [...] Yes, I pretty much agree with your entire post.

Re: Documented unittests & code coverage

2016-08-05 Thread Jonathan M Davis via Digitalmars-d
On Thursday, August 04, 2016 19:37:35 Walter Bright via Digitalmars-d wrote: > In adding some overflow detection to Phobos, I discovered that some > allocations were never called by the unittests. Adding a unittest for those > paths, I discovered those paths didn't work at all for any cases. > > I'

Re: Documented unittests & code coverage

2016-08-05 Thread Atila Neves via Digitalmars-d
On Friday, 5 August 2016 at 02:37:35 UTC, Walter Bright wrote: In adding some overflow detection to Phobos, I discovered that some allocations were never called by the unittests. Adding a unittest for those paths, I discovered those paths didn't work at all for any cases. I'm not giving up co

Re: Documented unittests & code coverage

2016-08-04 Thread Walter Bright via Digitalmars-d
In adding some overflow detection to Phobos, I discovered that some allocations were never called by the unittests. Adding a unittest for those paths, I discovered those paths didn't work at all for any cases. I'm not giving up coverage testing anytime soon, regardless of what some study claim

Re: Documented unittests & code coverage

2016-08-04 Thread Walter Bright via Digitalmars-d
On 8/4/2016 12:04 PM, Atila Neves wrote: What I take issue with is two things: 1. Code coverage metric targets (especially if the target is 100%). This leads to inane behaviours such as "testing" a print function (which itself was only used in testing) to meet the target. It's busywork that acc

Re: Documented unittests & code coverage

2016-08-04 Thread Atila Neves via Digitalmars-d
On Thursday, 4 August 2016 at 10:24:39 UTC, Walter Bright wrote: On 8/4/2016 1:13 AM, Atila Neves wrote: On Thursday, 28 July 2016 at 23:14:42 UTC, Walter Bright wrote: On 7/28/2016 3:15 AM, Johannes Pfau wrote: And as a philosophical question: Is code coverage in unittests even a meaningful

Re: Documented unittests & code coverage

2016-08-04 Thread Walter Bright via Digitalmars-d
On 8/4/2016 1:13 AM, Atila Neves wrote: On Thursday, 28 July 2016 at 23:14:42 UTC, Walter Bright wrote: On 7/28/2016 3:15 AM, Johannes Pfau wrote: And as a philosophical question: Is code coverage in unittests even a meaningful measurement? Yes. I've read all the arguments against code covera

Re: Documented unittests & code coverage

2016-08-04 Thread Atila Neves via Digitalmars-d
On Thursday, 28 July 2016 at 23:14:42 UTC, Walter Bright wrote: On 7/28/2016 3:15 AM, Johannes Pfau wrote: And as a philosophical question: Is code coverage in unittests even a meaningful measurement? Yes. I've read all the arguments against code coverage testing. But in my usage of it for 3

Re: Documented unittests & code coverage

2016-07-30 Thread Seb via Digitalmars-d
On Friday, 29 July 2016 at 05:49:01 UTC, Jonathan M Davis wrote: On Thursday, July 28, 2016 22:12:58 Walter Bright via Digitalmars-d wrote: As soon as we start taking the % coverage too seriously, we are in trouble. It's never going to be cut and dried what should be tested and what is unreason

Re: Documented unittests & code coverage

2016-07-29 Thread Jack Stouffer via Digitalmars-d
On Friday, 29 July 2016 at 07:01:35 UTC, Walter Bright wrote: The pilot reads the indicated value, interprets it in the context of what the other instruments say, APPLIES GOOD JUDGMENT, and flies the airplane. Continuing with this metaphor, in this situation you're not the pilot making the ju

Re: Documented unittests & code coverage

2016-07-29 Thread Steven Schveighoffer via Digitalmars-d
On 7/29/16 3:01 AM, Walter Bright wrote: On 7/28/2016 11:07 PM, Jack Stouffer wrote: you're making a decision on the user's behalf that coverage % is unimportant without knowing their circumstances. Think of it like the airspeed indicator on an airplane. There is no right or wrong airspeed. Th

Re: Documented unittests & code coverage

2016-07-29 Thread Chris via Digitalmars-d
On Friday, 29 July 2016 at 07:01:35 UTC, Walter Bright wrote: On 7/28/2016 11:07 PM, Jack Stouffer wrote: you're making a decision on the user's behalf that coverage % is unimportant without knowing their circumstances. Think of it like the airspeed indicator on an airplane. There is no righ

Re: Documented unittests & code coverage

2016-07-29 Thread Walter Bright via Digitalmars-d
On 7/28/2016 11:07 PM, Jack Stouffer wrote: you're making a decision on the user's behalf that coverage % is unimportant without knowing their circumstances. Think of it like the airspeed indicator on an airplane. There is no right or wrong airspeed. The pilot reads the indicated value, interp

Re: Documented unittests & code coverage

2016-07-28 Thread Walter Bright via Digitalmars-d
On 7/28/2016 10:49 PM, Jonathan M Davis via Digitalmars-d wrote: True, but particularly when you start doing stuff like trying to require that modules have 100% coverage - or that the coverage not be reduced by a change - it starts mattering - especially if it's done with build tools. The current

Re: Documented unittests & code coverage

2016-07-28 Thread Jack Stouffer via Digitalmars-d
On Friday, 29 July 2016 at 05:12:58 UTC, Walter Bright wrote: As soon as we start taking the % coverage too seriously, we are in trouble. It's never going to be cut and dried what should be tested and what is unreasonable to test, and I see no point in arguing about it. The % is a useful indi

Re: Documented unittests & code coverage

2016-07-28 Thread Jonathan M Davis via Digitalmars-d
On Thursday, July 28, 2016 22:12:58 Walter Bright via Digitalmars-d wrote: > As soon as we start taking the % coverage too seriously, we are in trouble. > It's never going to be cut and dried what should be tested and what is > unreasonable to test, and I see no point in arguing about it. > > The %

Re: Documented unittests & code coverage

2016-07-28 Thread Walter Bright via Digitalmars-d
On 7/28/2016 9:48 PM, Jonathan M Davis via Digitalmars-d wrote: gives some good examples of why having the unittest blocks themselves counted in the total percentage is problematic and can lead to dmd's code coverage tool listing than 100% coverage in a module that is fully tested. What's critica

Re: Documented unittests & code coverage

2016-07-28 Thread Jonathan M Davis via Digitalmars-d
On Thursday, July 28, 2016 16:14:42 Walter Bright via Digitalmars-d wrote: > On 7/28/2016 3:15 AM, Johannes Pfau wrote: > > And as a philosophical question: Is code coverage in unittests even a > > meaningful measurement? > > Yes. I've read all the arguments against code coverage testing. But in my

Re: Documented unittests & code coverage

2016-07-28 Thread Seb via Digitalmars-d
On Thursday, 28 July 2016 at 23:14:42 UTC, Walter Bright wrote: On 7/28/2016 3:15 AM, Johannes Pfau wrote: And as a philosophical question: Is code coverage in unittests even a meaningful measurement? Yes. I've read all the arguments against code coverage testing. But in my usage of it for 3

Re: Documented unittests & code coverage

2016-07-28 Thread Walter Bright via Digitalmars-d
On 7/28/2016 3:15 AM, Johannes Pfau wrote: And as a philosophical question: Is code coverage in unittests even a meaningful measurement? Yes. I've read all the arguments against code coverage testing. But in my usage of it for 30 years, it has been a dramatic and unqualified success in improvi

Re: Documented unittests & code coverage

2016-07-28 Thread Jonathan M Davis via Digitalmars-d
On Thursday, July 28, 2016 12:15:27 Johannes Pfau via Digitalmars-d wrote: > And as a philosophical question: Is code coverage in unittests even a > meaningful measurement? We write unittests to test the library code. But > if there's a line in a unittests which is never executed, this does not > d