On 5/13/16 5:16 PM, Adam D. Ruppe wrote:
On Friday, 13 May 2016 at 20:39:56 UTC, Steven Schveighoffer wrote:
Thoughts?
Yeah, I'm not a big fan of this either. A lot of in-comment examples
have been moved to unittests lately, and I think it is a net negative:
* Running it gives silent output
* Data representation in source isn't always instructive
* Assert just kinda looks weird
The one pro would be that it is automatically tested... but is it?
Consider the following:
---
import std.stdio;
///
unittest {
writeln("Hello, world!");
}
---
That passes the test, but if the user copy/pasted the example, it
wouldn't actually compile because of the missing import.
This isn't tested, even for DDOC examples. So I'm not super concerned
about it.
A unit test can use a private module symbol and that won't work in user
code either.
Certainly, some surrounding boilerplate is expected much of the time,
but the unittest doesn't even prove it actually runs with the same
user-expected surrounding code. It just proves it runs from the
implementation module: it can use private imports, private functions,
and more.
So it is a dubious win for automatic testing too.
The idea is to make sure examples actually compile and run in SOME context.
To get down to the lowest level, you can tell someone to copy and paste
an example in notepad, and run dmd on it, but if they haven't yet
downloaded dmd, then it won't work ;) It's impossible to test what the
user is going to do or know before hand.
D's documented unittests are somewhere in the middle... and I think
fails to capture the advantages of either extreme.
I'm not knocking documented unit tests. Certainly, without documented
unit tests, the situation before was that examples were riddled with
bugs. What I'm pointing out here is that DDOC examples have some
advantages that unittests cannot currently use.
A potential way to fix this may be marking a unit test as being a
complete example program that assumes the user has installed proper
access to the library. Then it won't compile unless you add the correct
imports, and it's treated as if it were in a separate module (no private
symbol access). This is probably the closest we can get to simulating a
user copying an example unit test into his own file and trying to run it.
-Steve
