On 2015-03-30 23:52, Andrei Alexandrescu wrote:
We're having a strong need for named unittests at Facebook for multiple
reasons.
1. We have sophisticated tooling that verifies whether unittests are
flaky. The automated monitor (for e.g. C++) figures whether a given
unittest fails several times across several commits. Unittests are
identified by name; relying on file/line is impossible because the line
of a failure is not stable across changes.
2. Again for efficient automated testing and flakiness detection, one
should be able to run only a subset of unittests by mentioning them by
line in the command line. Note that this implies there's no
interdependency between distinct unittests, which is fine because the
new ability is opt-on; I'd say is pure style anyway.
3. Mentioning unittest names in failure messages helps human
communication (e.g. "AddPeer is failing after your change"). This is
impossible with file and line numbers.
I'd like to make a DIP for named unittests. Who can help me with that?
I completely agree. I always thought that the built-in unit test support
wasn't sufficient. All of the above should be possible to implement in
library code without needing to change the language.
Now, it depends on how far you want to go. We could do something simple
as adding a UDA, which has already been suggested by others:
@name("this is a test") unittest
{
assert(false);
}
Personally I would like a complete testing framework like RSpec. I have
a very simple implementation [1] of this:
unittest
{
describe("std.uni", {
describe("toUpper", {
it("converts a string to uppercase", {
"foo".toUpper.should.eq("FOO")
});
});
describe("toLower", {
it("converts a string to lowercase", {
"Foo".toLower.should.eq("foo")
});
});
});
}
With RSpec, which supports multiple formatters, it can look like this:
---
Randomized with seed 28149
std.uni
toUpper
converts a string to uppercase
toLower
converts a string to lowercase
Finished in 0.00078 seconds (files took 0.0931 seconds to load)
2 examples, 0 failures
Randomized with seed 28149
---
This shows a failing test:
---
Randomized with seed 57730
std.uni
toLower
converts a string to lowercase
toUpper
converts a string to uppercase (FAILED - 1)
Failures:
1) std.uni toUpper converts a string to uppercase
Failure/Error: 'foo'.should == 'FOO'
expected: "FOO"
got: "foo" (using ==)
# ./spec/foo_spec.rb:6:in `block (3 levels) in <top (required)>'
Finished in 0.00298 seconds (files took 0.08553 seconds to load)
2 examples, 1 failure
Failed examples:
rspec ./spec/foo_spec.rb:5 # std.uni toUpper converts a string to uppercase
Randomized with seed 57730
---
Or with the TextMate formatter [2]. It shows both passing and failing
test. When a test fails to get a link pointing back to the editor and a
syntax highlighted snippet of the failing source code.
With RSpec it's also possible to specify the --line-number flag which
will only run the tests matching a given line number. Or the --example
flag which will run all examples (tests) matching the given string.
Is any of this interesting to have in Phobos? Otherwise I'll continue
working on my own framework.
[1] https://github.com/jacob-carlborg/dspec
[2] http://thejqr.com/2009/02/06/textmate-rspec-and-dot-spec-party.html
--
/Jacob Carlborg
