On 23-mar-10, at 20:29, bearophile wrote:
Pelle M.:
I'm not sure I understand, could you explain?<
That was my best explanation, sorry.
I am not experienced with unittest frameworks, and would like to
understand what the D system lacks.<
I think two times in the past I have written a list of those lacking
things. To give a good answer to your question I have to write a
lot, and it's not nice to write a lot when the words get ignored. So
first devs have to agree that a problem exists, then later we can
design things to improve the situation. Otherwise it's just a waste
of my energy, like trying to talk in vacuum.
actually there are some hooks in tango (and I believe similarly in
phobos) to do what you want
the module info contains the unittests and you can replace the default
for example the unittester of tango looks like this
import all modules to be tested
import tango.io.Stdout;
import tango.core.Runtime;
import tango.core.stacktrace.TraceExceptions;
bool tangoUnitTester()
{
uint countFailed = 0;
uint countTotal = 1;
Stdout ("NOTE: This is still fairly rudimentary, and will only
report the").newline;
Stdout (" first error per module.").newline;
foreach ( m; ModuleInfo ) // _moduleinfo_array )
{
if ( m.unitTest) {
Stdout.format ("{}. Executing unittests in '{}' ",
countTotal, m.name).flush;
countTotal++;
try {
m.unitTest();
}
catch (Exception e) {
countFailed++;
Stdout(" - Unittest failed.").newline;
e.writeOut(delegate void(char[]s){ Stdout(s); });
continue;
}
Stdout(" - Success.").newline;
}
}
Stdout.format ("{} out of {} tests failed.", countFailed,
countTotal - 1).newline;
return true;
}
static this() {
Runtime.moduleUnitTester( &tangoUnitTester );
}
void main() {}
one can do something fancier if he wants.
To really have all test one would need to have an array (or iterator)
in the module information instead of a single global unittest
function. Alternatively one could pass some flags to the unittest
function to control its execution.
Unit testing has to continue when tests fail. All code must be
testable, compile-time code too. You need a way to assert that
things go wrong too, like exceptions, asserts, compile-time asserts,
etc when they are designed to. It's good to have a way to give a
name to tests. And unit test systems enjoy some reflection to
organize themselves, to attach tests to code automatically. During
development you want to test only parts of the code, not the whole
program. Unit testing OOP code has other needs, because in a test
you may need to break data hiding of classes and structs. If you
unit test hundred of classes you soon find the necessity of
something to help creation of fake testing objects. You need some
tools for creating mock test objects (objects that simulate external
resources). You need a help to perform performance tests, to print
reports of the testing. You need layers of testing, slow tests and
quick tests that you can run every few minutes or seconds of
programming.!
Generally the more the unit test system does automatically the
better it is, because you want to write and use unit tests in the
most fast way possible. Those things are useful, but putting most of
those things inside a compiler is not a good idea.
I think that what you want is beyond normal requests, executing all
tests, tests of one module, a single test, yes that should be
relatively simple.
More complex test series/combination are probably better served by a
specialized regression tester.
Actually I use a specialized tester that is parallel, and whose basic
testing building block is a testing function in which the arguments
for it are generated automatically (derived types have to implement
generating functions).
This is a somewhat different way to look at tests that the usual one
(inspired from haskell's QuickCheck), but one that I prefer.
In the end the power is the same, instead of fixtures to prepare a
test environment you can define a derived type whose generating
function do the fixtures, and then have the tests as function having
that type as argument.
Test suites I normally organize like the package structure.
What I have is something like that would also be useful in tango/
phobos is a pre written main like function, so that one can easily
create test suites for pieces of code.
I have for example
int mainTestFun(char[][] argStr,SingleRTest testSuite)
which can be used to create a unittester that recognizes flags to
initialize it, perform subtests,...
Fawzi