Re: DUnit: Advanced unit testing toolkit.

2013-10-07 Thread Gary Willoughby

On Sunday, 6 October 2013 at 14:24:56 UTC, ilya-stromberg wrote:

OK, maybe you are right.
Do you have any plans to integrate your code into Phobos?


Nah, a unit testing framework is not really standard lib material.


Re: DUnit: Advanced unit testing toolkit.

2013-10-06 Thread ilya-stromberg
On Saturday, 21 September 2013 at 00:40:47 UTC, Gary Willoughby 
wrote:

DUnit: Advanced unit testing toolkit.

I've needed this for a project i've been working on so i 
created a toolkit that i'm using and happy with. I must thank 
the community here for helping me with a few issues along the 
way (mostly due to poor documentation). It uses a lot of 
compile time reflection to generate the mocks, which has been 
very interesting to learn/write (to say the least).


Guys, we have at least 5 (!) different unit test projects!
Can you cooperate your efforts to create one, but wonderful?


Re: DUnit: Advanced unit testing toolkit.

2013-10-06 Thread Gary Willoughby

On Sunday, 6 October 2013 at 07:32:09 UTC, ilya-stromberg wrote:
On Saturday, 21 September 2013 at 00:40:47 UTC, Gary Willoughby 
wrote:

DUnit: Advanced unit testing toolkit.

I've needed this for a project i've been working on so i 
created a toolkit that i'm using and happy with. I must thank 
the community here for helping me with a few issues along the 
way (mostly due to poor documentation). It uses a lot of 
compile time reflection to generate the mocks, which has been 
very interesting to learn/write (to say the least).


Guys, we have at least 5 (!) different unit test projects!
Can you cooperate your efforts to create one, but wonderful?


I think it's good to have many as we are all approaching the same 
problem from different angles. I'm sure in the long run one or 
two will grow to be dominant but in the meantime many will ensure 
healthy competition on features and completeness.


Re: DUnit: Advanced unit testing toolkit.

2013-10-06 Thread ilya-stromberg

On Sunday, 6 October 2013 at 10:36:08 UTC, Gary Willoughby wrote:

On Sunday, 6 October 2013 at 07:32:09 UTC, ilya-stromberg wrote:

Guys, we have at least 5 (!) different unit test projects!
Can you cooperate your efforts to create one, but wonderful?


I think it's good to have many as we are all approaching the 
same problem from different angles. I'm sure in the long run 
one or two will grow to be dominant but in the meantime many 
will ensure healthy competition on features and completeness.


OK, maybe you are right.
Do you have any plans to integrate your code into Phobos?


Re: DUnit: Advanced unit testing toolkit.

2013-09-29 Thread Gary Willoughby
Updated DUnit with some refactored code and started to implement 
a results module.


Re: DUnit: Advanced unit testing toolkit.

2013-09-28 Thread Jacob Carlborg

On 2013-09-27 18:23, Gary Willoughby wrote:


I've just tested it on Mac OS 10.8 and it fails.

Are you sure you commented out the line of code that you need to test?
In the snippet of code above the working deprecated version is
uncommented. Comment that line out and try using the property.


Hmm, ok, you were right. The following commit made it public:

d5fda766c248b5b3671b1b498c0a7dba8bee7442

I guess we'll have to wait to the next release of DMD or use git HEAD.

--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-27 Thread Jacob Carlborg

On 2013-09-26 22:39, Gary Willoughby wrote:


I knowm, it's weird, i've been trying to find out why it's private,
everything looks like it's public. Try running the code i posted see if
you can get it to work.


It works fine using DMD 2.063.2 on Mac OS X.

--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-27 Thread Gary Willoughby
On Friday, 27 September 2013 at 06:28:54 UTC, Jacob Carlborg 
wrote:

On 2013-09-26 22:39, Gary Willoughby wrote:

I knowm, it's weird, i've been trying to find out why it's 
private,
everything looks like it's public. Try running the code i 
posted see if

you can get it to work.


It works fine using DMD 2.063.2 on Mac OS X.


I've just tested it on Mac OS 10.8 and it fails.

Are you sure you commented out the line of code that you need to 
test? In the snippet of code above the working deprecated version 
is uncommented. Comment that line out and try using the property.


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Jacob Carlborg

On 2013-09-25 21:55, Dicebot wrote:


UDAs + recent trait to get all unit-tests during compile-time really
favors instead having lot of small independent annotated unit-test blocks.


If you have more than one test per unit test block you always need to 
run them together, and in the declared order. Example:


unittest
{
@test(foo)
{
assert(1 == 1);
}

@test(bar)
{
assert(1 == 2);
}
}

You cannot run foo separated from bar. They all will always run 
together. You also cannot run bar before running foo. Running tests 
like this makes it very easy to introduce order dependencies between the 
tests.


--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Dicebot
On Thursday, 26 September 2013 at 06:07:50 UTC, Jacob Carlborg 
wrote:

On 2013-09-25 21:55, Dicebot wrote:

UDAs + recent trait to get all unit-tests during compile-time 
really
favors instead having lot of small independent annotated 
unit-test blocks.


If you have more than one test per unit test block you always 
need to run them together, and in the declared order. Example:


unittest
{
@test(foo)
{
assert(1 == 1);
}

@test(bar)
{
assert(1 == 2);
}
}

You cannot run foo separated from bar. They all will always 
run together. You also cannot run bar before running foo. 
Running tests like this makes it very easy to introduce order 
dependencies between the tests.


I was saying that if you want to have some tests independent, it 
makes much more sense to do it this way:


```
@test(foo) unittest
{
assert(1 == 1);
}

@test(bar) unittest
{
assert(1 == 2);
}
```

..and let tests within one block terminate on first failure. That 
should integrate better with existing tooling when no external 
testing library/framework is connected.


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Gary Willoughby
On Wednesday, 25 September 2013 at 21:00:00 UTC, Jacob Carlborg 
wrote:
There are property functions at line 374 and below to set the 
assert handler.


Only the deprecated version works as expected.

import core.exception;
import std.stdio;

alias void function(string file, size_t line, string msg) 
AssertHandler;


AssertHandler handler = function(string file, size_t line, string 
msg)

{
writefln(File: %s, file);
writefln(Line: %s, line);
writefln(Message: %s, msg);
};

void main(string[] args)
{
// assertHandler = handler; // --- Private!
setAssertHandler(handler); // --- Works but deprecated

assert(false, Test message.);
}


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Jacob Carlborg

On 2013-09-26 20:18, Gary Willoughby wrote:


Only the deprecated version works as expected.

import core.exception;
import std.stdio;

alias void function(string file, size_t line, string msg) AssertHandler;

AssertHandler handler = function(string file, size_t line, string msg)
{
 writefln(File: %s, file);
 writefln(Line: %s, line);
 writefln(Message: %s, msg);
};

void main(string[] args)
{
 // assertHandler = handler; // --- Private!
 setAssertHandler(handler); // --- Works but deprecated

 assert(false, Test message.);
}


I don't know which version of DMD you're using or when this part of 
druntime was update, but it's clearly not private according to the 
source code:


https://github.com/D-Programming-Language/druntime/blob/master/src/core/exception.d#L380

--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Gary Willoughby
On Thursday, 26 September 2013 at 19:25:46 UTC, Jacob Carlborg 
wrote:
I don't know which version of DMD you're using or when this 
part of druntime was update, but it's clearly not private 
according to the source code:


https://github.com/D-Programming-Language/druntime/blob/master/src/core/exception.d#L380


I knowm, it's weird, i've been trying to find out why it's 
private, everything looks like it's public. Try running the code 
i posted see if you can get it to work.


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Atila Neves

On Sunday, 22 September 2013 at 13:13:29 UTC, linkrope wrote:

Have a look at https://github.com/linkrope/dunit, especially at
the Related Projects.


Thanks for the link to 
https://github.com/atilaneves/unit-threaded!


Re: DUnit: Advanced unit testing toolkit.

2013-09-26 Thread Atila Neves
My library, unit-threaded, does just that. All unittest blocks 
from a module show up as one test case in the output.


On Thursday, 26 September 2013 at 14:56:49 UTC, Dicebot wrote:
On Thursday, 26 September 2013 at 14:51:16 UTC, Jacob Carlborg 
wrote:
Exactly. I guess I misunderstood you. Although I would 
consider having lot of small independent annotated unit-test 
blocks be basically what you're showing above.


Beg my pardon, bad wording from my side. I was referring to 
approach when testing framework defines some sort of own 
testing DSL and uses it for test case decoupling within one 
unit-test block - as far as I understand it is what some of 
currently existing D testing frameworks do.


Key point here is that good testing library should augment 
built-in facility, not replace it. Fallback to built-ins can 
still be useful.




Re: DUnit: Advanced unit testing toolkit.

2013-09-25 Thread Jacob Carlborg

On 2013-09-23 18:40, jostly wrote:


I think it's great to see the D unit testing ecosystem growing. Since
it's still relatively small, I think we have a good chance here to
create interoperability between the different frameworks.

As I see it, we have:

1. Running unit tests

This is where D shines with the builting facility for unit tests.
However, it suffers a bit from the fact that, if we use assert, it will
stop on the first assertion failure, and there is (as far as I've been
able to tell) no reliable way to run specific code before or after all
the unit tests. If I'm wrong on that assumption, please correct me, that
would simplify the spec running for specd.

In specd, the actual code inside the unittest { } sections only collect
results, and the reporting is called from a main() supplied by compiling
with version specrunner set. I haven't checked to see if your dunit do
something similar.


Agree. I only see the unittest blocks as a place to but the asserts, 
since it's not possible to put them at module level.


It's possible to implement you're own unit test handler. See:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/runtime.d#L290

It's also possibly to set the assert handler:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/exception.d#L368

But you most likely want to throw some kind of exception anyway. Because 
if an assertion is triggered in a unit test block you most likely want 
to end that unit test block, immediately.


--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-25 Thread jostly
On Wednesday, 25 September 2013 at 06:45:12 UTC, Jacob Carlborg 
wrote:

It's possible to implement you're own unit test handler. See:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/runtime.d#L290


This changes everything! Thanks for that pointer, it really was 
the missing link between what I want to do and what I thought was 
possible.


But you most likely want to throw some kind of exception 
anyway. Because if an assertion is triggered in a unit test 
block you most likely want to end that unit test block, 
immediately.


Maybe... in my mind it depends on whether there are more than one 
test per unit test block, because I really like each failing test 
to be enumerated as well. But I can see the benefit - that would 
allow me to have tests directly in the unit test block, instead 
of wrapped inside delegates like it's done in specd. Ideally, a 
generic runner would support both immediate and deferred tests.





Re: DUnit: Advanced unit testing toolkit.

2013-09-25 Thread Dicebot

On Wednesday, 25 September 2013 at 19:50:47 UTC, jostly wrote:
Maybe... in my mind it depends on whether there are more than 
one test per unit test block, because I really like each 
failing test to be enumerated as well. But I can see the 
benefit - that would allow me to have tests directly in the 
unit test block, instead of wrapped inside delegates like it's 
done in specd. Ideally, a generic runner would support both 
immediate and deferred tests.


UDAs + recent trait to get all unit-tests during compile-time 
really favors instead having lot of small independent annotated 
unit-test blocks.


Re: DUnit: Advanced unit testing toolkit.

2013-09-25 Thread Gary Willoughby
On Wednesday, 25 September 2013 at 06:45:12 UTC, Jacob Carlborg 
wrote:

It's possible to implement you're own unit test handler. See:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/runtime.d#L290

It's also possibly to set the assert handler:

https://github.com/D-Programming-Language/druntime/blob/master/src/core/exception.d#L368

But you most likely want to throw some kind of exception 
anyway. Because if an assertion is triggered in a unit test 
block you most likely want to end that unit test block, 
immediately.


That looks interesting but the unittester handler seems to run 
instead of the unittest blocks and the assert handler property is 
private.


Re: DUnit: Advanced unit testing toolkit.

2013-09-25 Thread Jacob Carlborg
On Wednesday, 25 September 2013 at 20:18:57 UTC, Gary Willoughby 
wrote:


That looks interesting but the unittester handler seems to run 
instead of the unittest blocks


I'm not exactly sure what you mean. But this is how it works. DMD 
turns each unit test block into a function. Then DMD creates a 
single function for each module, which will call all these unit 
test functions. If you access the unitTest [1] property of a 
ModuleInfo you will get the unit test runner and not the 
individual unit test functions.


To access the individual unit test functions you can use the 
getUnitTests trait, available in git HEAD:


https://github.com/D-Programming-Language/dlang.org/pull/366


and the assert handler property is private.


There are property functions at line 374 and below to set the 
assert handler.


[1] 
https://github.com/D-Programming-Language/druntime/blob/master/src/object.di#L284


--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-24 Thread linkrope

On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
I think it's great to see the D unit testing ecosystem growing. 
Since it's still relatively small, I think we have a good 
chance here to create interoperability between the different 
frameworks.


As I see it, we have:

1. Running unit tests

This is where D shines with the builting facility for unit 
tests. However, it suffers a bit from the fact that, if we use 
assert, it will stop on the first assertion failure, and there 
is (as far as I've been able to tell) no reliable way to run 
specific code before or after all the unit tests. If I'm wrong 
on that assumption, please correct me, that would simplify the 
spec running for specd.


In specd, the actual code inside the unittest { } sections only 
collect results, and the reporting is called from a main() 
supplied by compiling with version specrunner set. I haven't 
checked to see if your dunit do something similar.


In my understanding, D's built-in support for unittests is best 
suited for test cases that can be expressed as one-liners. When 
it gets more complicated, we usually use classes. That's what 
JUnit and TestNG do and that's what dunit does (this one: 
https://github.com/linkrope/dunit).


For our software, we even separate the 'src' tree from the 
'unittest' tree to not distort the coverage results.



2. Asserting results

Varies from the builtin assert() to xUnit-like assertEquals() 
to the more verbose x.must.equal(y) used in specd.


This could easily be standardized by letting all custom asserts 
throw an AssertError, though I would prefer to use another 
exception that encapsulates the expected and actual result, to 
help with bridging to reporting.


It's too easy to use 'assertEquals' wrong. JUnit defines 
'assertEquals(expected, actual)' while DUnit defines it the other 
way around. For JUnit, I've seen too many wrong uses: 
'assertEquals(answer, 42)' giving misleading messages expected 
... but got 42.


Even with UFCS, why shouldn't you write '42.assertEqual(actual)'? 
That's where the more verbose 'must' matchers shine: 
'42.must.equal(actual)' is obviously the wrong way around.


When you have violated contracts, you get 'AssertError' 
exceptions from deep within your code under test. To fix these 
errors you may wish for a stack trace. On the other hand, the 
pretty messages you get for failed test assertions should be 
enough to fix these failures. In this case, the stack trace would 
only show the test runner calling the test case.


So: 'must' matchers are better than 'assert...'; and 
'AssertError' should not be thrown for failures!



3. Reporting results

If we have moved beyond basic assert() and use some kind of 
unit test runner, then we have the ability to report a summary 
of run tests, and which (and how many) failed.


This is one area where IDE integration would be very nice, and 
I would very much prefer it if the different unit test 
frameworks agreed on one standard unit test runner interface, 
so that the IDE integration problem becomes one of adapting 
each IDE to one runner interface, instead of adapting each 
framework to each IDE.


In my experience from the Java and Scala world, the last point 
is the biggest. Users expect to be able to run unit tests and 
see the report in whatever standard way their IDE has. In 
practice this most often means that various libraries pretend 
to be JUnit when it comes to running tests, because JUnit is 
supported by all IDEs.


A few days ago, I added such a reporting to dunit. An XML test 
report is now available that uses the JUnitReport format. We use 
Jenkins (formerly known as Hudson) for continuous integration so 
that we can browse our test results and track failures. Nice!


Let's not end up in that situation, but rather work out a 
common API to run unit tests, and the D unit test community can 
be the envy of every other unit tester. :)


Agreed!


Re: DUnit: Advanced unit testing toolkit.

2013-09-24 Thread linkrope

Same hint as for specd: have a look at 'assertOp'!
http://d.puremagic.com/issues/show_bug.cgi?id=4653

alias assertOp! assertGreaterThan;
alias assertOp!= assertGreaterThanOrEqual;
alias assertOp! assertLessThan;
alias assertOp!= assertLessThanOrEqual;

avoids duplicate code.

Maybe, you can do the same for 'assertStartsWith' and 
'assertEndsWith'?


Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread Dicebot

On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
In specd, the actual code inside the unittest { } sections only 
collect results, and the reporting is called from a main() 
supplied by compiling with version specrunner set. I haven't 
checked to see if your dunit do something similar.


I think more D-way would have been to simply separate tests in 
logically separated unittest blocks and run those using static 
reflection, catching assert error. That way you will get first 
failure in a set and then continue to other sets. Does that make 
sense?


Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread jostly

On Sunday, 22 September 2013 at 13:13:29 UTC, linkrope wrote:

Have a look at https://github.com/linkrope/dunit, especially at
the Related Projects.

Until now, my preferred tool for (large-scale) unit testing in D
would be the combination of my dunit framework (of course),
DMocks-revived for mocks, and the 'must' matchers of specd.


I think it's great to see the D unit testing ecosystem growing. 
Since it's still relatively small, I think we have a good chance 
here to create interoperability between the different frameworks.


As I see it, we have:

1. Running unit tests

This is where D shines with the builting facility for unit tests. 
However, it suffers a bit from the fact that, if we use assert, 
it will stop on the first assertion failure, and there is (as far 
as I've been able to tell) no reliable way to run specific code 
before or after all the unit tests. If I'm wrong on that 
assumption, please correct me, that would simplify the spec 
running for specd.


In specd, the actual code inside the unittest { } sections only 
collect results, and the reporting is called from a main() 
supplied by compiling with version specrunner set. I haven't 
checked to see if your dunit do something similar.


2. Asserting results

Varies from the builtin assert() to xUnit-like assertEquals() to 
the more verbose x.must.equal(y) used in specd.


This could easily be standardized by letting all custom asserts 
throw an AssertError, though I would prefer to use another 
exception that encapsulates the expected and actual result, to 
help with bridging to reporting.


3. Reporting results

If we have moved beyond basic assert() and use some kind of unit 
test runner, then we have the ability to report a summary of run 
tests, and which (and how many) failed.


This is one area where IDE integration would be very nice, and I 
would very much prefer it if the different unit test frameworks 
agreed on one standard unit test runner interface, so that the 
IDE integration problem becomes one of adapting each IDE to one 
runner interface, instead of adapting each framework to each IDE.


In my experience from the Java and Scala world, the last point is 
the biggest. Users expect to be able to run unit tests and see 
the report in whatever standard way their IDE has. In practice 
this most often means that various libraries pretend to be JUnit 
when it comes to running tests, because JUnit is supported by all 
IDEs.


Let's not end up in that situation, but rather work out a common 
API to run unit tests, and the D unit test community can be the 
envy of every other unit tester. :)


Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread Russel Winder
On Mon, 2013-09-23 at 18:40 +0200, jostly wrote:
[…]
 I think it's great to see the D unit testing ecosystem growing. 
 Since it's still relatively small, I think we have a good chance 
 here to create interoperability between the different frameworks.

There is also integration and system testing of course, not just unit
testing. The same testing framework can generally be used for all forms.

In the Java sphere, JUnit gave way to TestNG exactly because TestNG
supports all forms of testing not just unit testing. Now though TestNG
is giving way to Spock enabling elements of BDD as well as TDD.

D has some unit testing capability built in, which i good, but it is
also good to have an external testing framework that can do unit,
integration and system testing supporting TDD and BDD. 

 As I see it, we have:
 
 1. Running unit tests
 
 This is where D shines with the builting facility for unit tests. 
 However, it suffers a bit from the fact that, if we use assert, 
 it will stop on the first assertion failure, and there is (as far 
 as I've been able to tell) no reliable way to run specific code 
 before or after all the unit tests. If I'm wrong on that 
 assumption, please correct me, that would simplify the spec 
 running for specd.

So the built-in is not entirely up to the task of real unit testing?

 In specd, the actual code inside the unittest { } sections only 
 collect results, and the reporting is called from a main() 
 supplied by compiling with version specrunner set. I haven't 
 checked to see if your dunit do something similar.
 
 2. Asserting results
 
 Varies from the builtin assert() to xUnit-like assertEquals() to 
 the more verbose x.must.equal(y) used in specd.

In the Scala variant of the JVM arena, ScalaTest mingles really nicely
all the classic TDD asserts styles, along with Hamcrest matchers, but
also supports the more BDD style test specifications. Spock also does
this. Corollary, D must do this.

NB Go is going through all this just now. The built in unit test
capability is minimalist and for testing the Go implementation. GoCheck
is classic TDD assert style, and there are some candidate BDD styles on
the horizon. Will Go beat D to having the capability that the JVM
languages already enjoy?

Note that Groovy and the py.test Python test framework dispense with the
need for assertEquals and that family of JUnit thingies, in favour of
using the built-in assert and catching the AssertionError exception,
doing detailed stack analysis and provided very detailed information
about the evaluated expression: power asserts. Why should D follow 1990s
thinking when there is 2010s thinking that is much better?

 This could easily be standardized by letting all custom asserts 
 throw an AssertError, though I would prefer to use another 
 exception that encapsulates the expected and actual result, to 
 help with bridging to reporting.

See above :-)

 3. Reporting results
 
 If we have moved beyond basic assert() and use some kind of unit 
 test runner, then we have the ability to report a summary of run 
 tests, and which (and how many) failed.
 
 This is one area where IDE integration would be very nice, and I 
 would very much prefer it if the different unit test frameworks 
 agreed on one standard unit test runner interface, so that the 
 IDE integration problem becomes one of adapting each IDE to one 
 runner interface, instead of adapting each framework to each IDE.
 
 In my experience from the Java and Scala world, the last point is 
 the biggest. Users expect to be able to run unit tests and see 
 the report in whatever standard way their IDE has. In practice 
 this most often means that various libraries pretend to be JUnit 
 when it comes to running tests, because JUnit is supported by all 
 IDEs.
 
 Let's not end up in that situation, but rather work out a common 
 API to run unit tests, and the D unit test community can be the 
 envy of every other unit tester. :)

Currently, and very sadly, this generally means writing an XML file
using the JUnit schema. On the other hand if D did this Eclipse, IDEA,
NetBeans, etc. would immediately render excellent data displays.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


signature.asc
Description: This is a digitally signed message part


Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread Gary Willoughby

On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
Let's not end up in that situation, but rather work out a 
common API to run unit tests, and the D unit test community can 
be the envy of every other unit tester. :)


You've raised some nice ideas and got me thinking. However, i do 
think we are missing some way of knowing when unit tests start 
and stop. I like the built in unittest blocks but it would be 
nice to have something like:


beforetests
{
...
}

aftertests
{
...
}

To apply code before and after the unit tests have run. These 
could be used to setup and the execute the reporting environment? 
I can't think of a way to do this automatically without these 
constructs.


Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread Rory McGuire
Can you not just change to going through each module running the hosts
manually? Then you can do what you like before and after each.

If your lib had  a host that started a thread which received messages about
a unittest registering itself in other user modules you could run the
unittests as  you feel like and you could stick with the -unittest dmd arg.
Reason for the thread is you can't control order of module initialization.
On 23 Sep 2013 21:30, Gary Willoughby d...@nomad.so wrote:

 On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:

 Let's not end up in that situation, but rather work out a common API to
 run unit tests, and the D unit test community can be the envy of every
 other unit tester. :)


 You've raised some nice ideas and got me thinking. However, i do think we
 are missing some way of knowing when unit tests start and stop. I like the
 built in unittest blocks but it would be nice to have something like:

 beforetests
 {
 ...
 }

 aftertests
 {
 ...
 }

 To apply code before and after the unit tests have run. These could be
 used to setup and the execute the reporting environment? I can't think of a
 way to do this automatically without these constructs.



Re: DUnit: Advanced unit testing toolkit.

2013-09-23 Thread Rory McGuire
Sorry in my last mail host should be unittest (swype auto correct errors
:))


Re: DUnit: Advanced unit testing toolkit.

2013-09-22 Thread linkrope

Have a look at https://github.com/linkrope/dunit, especially at
the Related Projects.

Until now, my preferred tool for (large-scale) unit testing in D
would be the combination of my dunit framework (of course),
DMocks-revived for mocks, and the 'must' matchers of specd.

How does your toolkit fit in?


Re: DUnit: Advanced unit testing toolkit.

2013-09-22 Thread Gary Willoughby

On Sunday, 22 September 2013 at 13:13:29 UTC, linkrope wrote:

Have a look at https://github.com/linkrope/dunit, especially at
the Related Projects.

Until now, my preferred tool for (large-scale) unit testing in D
would be the combination of my dunit framework (of course),
DMocks-revived for mocks, and the 'must' matchers of specd.

How does your toolkit fit in?


I looked at DMocks and Specd before i started work on DUnit to 
see what was out there. I'm not saying that DUnit does anything 
really different than those two combined but i'm trying to make 
it simpler to use and more intuitive.


For example DMocks uses an intermediary object to handle the 
mocks. This is thought was a bit strange as this behaviour should 
be in the mock to begin with. So my first objective was to 
provide a way of very simply creating a mock object and to 
interact with that mock object directly. This also fulfilled the 
secondary objective of moving 'setup' code out of the unit test 
and making them more easy to read. Also DUnit solved the problem 
that Dmocks doesn't address of correctly handling Object base 
class methods properly. All methods can fall-through to parent 
implementations or be replaced at runtime.


Specd is a nice approach to defining constraints but again it 
seems overkill for something that should be simple. I don't do 
anything different, i just do it in a different way.


specd: 1.must.be.greater_than(0);
dunit: 1.assertGreaterThan(0);

The reason i've gone with just providing more specific assert 
methods is that i can create nice helpful error message when 
things go wrong. For example this line:


1.assertEquals(0);

Creates this error:

+
| Failed asserting equal
+
| File: example.d
| Line: 85
+
| ✓ Expected int: 1
| ✗ Actual int: 2

Making debugging what went wrong loads easier. These messages 
give you so much useful info that you will never go back to only 
using assert() again.


Re: DUnit: Advanced unit testing toolkit.

2013-09-22 Thread Gary Willoughby
On Sunday, 22 September 2013 at 15:54:39 UTC, Gary Willoughby 
wrote:
The reason i've gone with just providing more specific assert 
methods is that i can create nice helpful error message when 
things go wrong. For example this line:


1.assertEquals(0);

Creates this error:


+

| Failed asserting equal

+

| File: example.d
| Line: 85

+

| ✓ Expected int: 1
| ✗ Actual int: 2

Making debugging what went wrong loads easier. These messages 
give you so much useful info that you will never go back to 
only using assert() again.


Actually that should read:

+
| Failed asserting equal
+
| File: example.d
| Line: 85
+
| ✓ Expected int: 0
| ✗ Actual int: 1

But you get the idea. ;)


Re: DUnit: Advanced unit testing toolkit.

2013-09-22 Thread Jacob Carlborg

On 2013-09-21 02:40, Gary Willoughby wrote:

DUnit: Advanced unit testing toolkit.

I've needed this for a project i've been working on so i created a
toolkit that i'm using and happy with. I must thank the community here
for helping me with a few issues along the way (mostly due to poor
documentation). It uses a lot of compile time reflection to generate the
mocks, which has been very interesting to learn/write (to say the least).

I think it's useful enough now to release and it would be nice to
perhaps receive some guidance as to where it should improve or fails
spectacularly.

Wikipedia: http://en.wikipedia.org/wiki/Unit_testing

DUnit: https://github.com/kalekold/dunit

See examples and documentation for usage.

Have fun.


You might want to use alternatively you could use version(unittest) 
instead of debug for the mocks. Don't know which is better.


--
/Jacob Carlborg


Re: DUnit: Advanced unit testing toolkit.

2013-09-21 Thread Gary Willoughby
DUnit has now been added to the DUB registry at 
http://code.dlang.org/.