>-----Original Message----- >From: hoegsb...@gmail.com [mailto:hoegsb...@gmail.com] On Behalf Of >Kristian Høgsberg >Sent: Friday, March 02, 2012 5:17 PM > >Yeah, a few good points came up in this thread: process separation so >if one test corrupts memory it doesn't affect others, being able to >run all test even if some of them hangs/crashes, good summary of >pass/fail. And it turns out that looping through test cases and >forking for each test case isn't a lot of code. So I wrote that and >converted Arties wl_array tests to this new test runner and added a >test case for wl_map. We can just use assert to log errors and >segfaults are logged as errors automatically: Output of make check >looks like this (with an assert(0) in the array_copy test and a NULL >pointer dererence added to the array_add test: > >running "array_copy"... array-test: array-test.c:121: array_copy: Assertion >`0'\ > failed. >signal 6 >running "array_add"... signal 11 >running "array_init"... exit status 0 >3 tests, 1 pass, 2 fail >FAIL: array-test >running "map_insert_new"... exit status 0 >1 tests, 1 pass, 0 fail >PASS: map-test >=========================================================== >================ >1 of 2 tests failed >Please report to >https://bugs.freedesktop.org/enter_bug.cgi?product=wayland >=========================================================== >================ > >As it is, the assert is printed in the middle of the "running ..." but >I'll fix that so that it's printed on its own line, which means you >can step through test failures like compile errors (at least in >emacs). And if you want to debug a specific case, run it as > > $ gdb --args array-test array_add > >to run just that one case without forking. There's only a couple of >things that I think we might to add to this: 1) timeout for tests, but >lets do that if we hit that case and 2) running multiple tests in >parallel, but automake can help with that and as for 1), let's do that >if we need it. > >The code for this is in tests/test-runner.c and it's all of 93 lines, >including 22 lines of copyright header. To write a test case do: > >TEST(name_of_test) >{ > assert(5 != 7); >} > >link with test-runner.c, add the binary to TESTS and you're done. > >Kristian
Kristian, I'm actually quite pleased that you found a solution to avoid having to add/register your tests to a suite and/or test runner manually. It happens automatically... all you have to do is define your test; awesome! That will definitely reduce the chance of human error (i.e. writing a test and then forgetting to add it to a suite/runner). Let alone, it simplifies test writing and maintenance, so all the developer has to do is focus on the test case itself. That was one of the top features I wanted in a test framework. I primarily have a C++ background and it was easy to find ways to do that in that language. When I realized that I couldn't figure out how to do it in C, I was a little disappointed and just assumed it was a limitation of C and the compiler. Albeit, I didn't have a lot of time to investigate. Kudos. It's definitely a good start :) I'm curious, however, as to why you've used relative file includes in the test source files. Why not just add the path to CFLAGS? Also, any plans to add the 'tests' directory to the top-level makefile SUBDIRS? Artie _______________________________________________ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
