I'm been using Boost Test, and I've written a module that lets you add
the executable and the tests (where possible, individual named tests)
with a single command - see my github repositories for projects that
include this module. http://github.com/rpavlik
On 7/9/10 9:04 PM, Alok Govil wrote:
While I was suspecting something like this about CTest, your
explanation helps a lot.
I have integrated UnitTest++ into my flow, and it is working quite
well. It does also return a 0 if all tests pass, so one could
integrate with CTest/CDash if desirable.
Thanks
------------------------------------------------------------------------
Date: Fri, 9 Jul 2010 14:54:46 -0700
From: chillery-cm...@lambda.nu
To: b...@askmonty.org
CC: cmake@cmake.org
Subject: Re: [CMake] CTest examples
For CTest, running a test means running an executable and seeing
whether the result of that process is 0 (success) or not 0 (failure).
It is also possible to scan the output of the executable for certain
regular expressions to determine success or failure.
It's a very simple and somewhat limited mechanism. It does depend on
you writing your own test driver, which makes sense because CMake
can't possibly know how to test your code.
As an example, Zorba is an XQuery (xml query language) processor. The
majority of our tests are actual XQueries stored in text files with
the extension .xq, along with corresponding expected results in .xml
files. We have a testdriver program which executes the query, checks
the results, and returns 0 or 1.
Our CMakeLists looks vaguely like this:
add_executable(testdriver ${TESTDRIVER_SRCS})
file(GLOB_RECURSE TESTFILES "${CMAKE_CURRENT_SOURCE_DIR}" "*.xq")
foreach(TESTFILE ${TESTFILES})
add_test(${TESTFILE} testdriver ${TESTFILE})
endforeach()
This creates one test for each .xq file. The name of the test is the
name of the .xq file, and that same name is passed as an argument to
testdriver.
This works well, and lets us use ctest's functionality like -R to run
certain tests by name. It also results in nice easy-to-read listings
on CDash because there's one line for each test, making it easy to
examine failures. (Although given that we have over 20,000 tests, some
pages on CDash take a looooong time to load...) The main downside is
speed, since every single test has to start up a whole Zorba process.
When you get to unit testing, however, IMHO things don't work as well.
CMake does offer the create_test_sourcelist() command, and you should
look into it. We use that in a couple places to create a single test
driver from several separate small unit test drivers. However, you
still need to write a loop in CMakeLists to add multiple tests using
that test driver.
In another project I work on, I've been using CxxTest for a unit
testing framework. This is a really nice setup as it makes it very
trivial to write many unit tests. However, the integration with ctest
is poor. There's a reasonably good FindCxxTest script shipped with
CMake that makes it easy to get CxxTest suites running; in a nutshell:
find_package(CxxTest)
cxx_add_test(my_unit_test my_unit_test.cpp
"${CMAKE_CURRENT_SOURCE_DIR}/my_unit_test.h")
Unit tests in CxxTest are written as classes in a .h file;
cxx_add_test() runs CxxTest to generate the named .cpp file from that,
then builds it into the named test driver and adds a CTest test for
it. So far so good. However, the main benefit of CxxTest is grouping
similar test cases into a single .h file, probably with similar setup
and so forth. Unfortunately cxx_add_test() only adds one test to
CTest, and that one test runs all the test cases in the .h file. So
you lose the ability to run certain tests with ctest -R, and the
reports on CDash will just show whether ALL the tests in my_unit_test
succeeded or not.
The fundamental problem is that ctest requires a test case to be a
program execution, and has no facilities to allow a program to
represent multiple tests. So either you pay the cost in speed, or you
lose some of the flexibility ctest and cdash offer. (To be fair, even
if this were fixed in ctest, CxxTest would also need some
modifications as it currently doesn't let you run subsets of tests
either.)
Ceej
aka Chris Hillery
On Fri, Jul 9, 2010 at 4:37 AM, Bo Thorsen <b...@askmonty.org
<mailto:b...@askmonty.org>> wrote:
Hi people,
I have converted a set of applications to cmake and cpack, and now
have my eyes set on ctest.
I'd like to hear if someone here has some good advice, or links to
good advice, on how to structure tests. I'm searching for help on
how to put different tests into what executables. On how to handle
multiple tests on each classes, on how to best structure the test
of the static libraries (all of those are part of the source tree)
that are linked in to the application. And on how to test classes
from the main application itself.
I have read the ctest FAQ, documentation etc. and still don't know
anything that help me write the actual test code.
From the looks of it, ctest only provides the framework to run a
test, no help is given to write the code of the tests themselves,
is this right? I have previously been using cppunit, and it looks
like this will still be useful.
To sum it up, I'm looking for real life advice on what you guys
have done with ctest. This information seem almost completely
missing on the net, where all searches on ctest leads to useless
presentation on ctest features.
Cheers,
Bo Thorsen.
Monty Program AB.
--
MariaDB: MySQL replacement
Community developed. Feature enhanced. Backward compatible.
_______________________________________________
Powered by www.kitware.com <http://www.kitware.com>
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
_______________________________________________ Powered by
www.kitware.com Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html Please keep messages
on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ Follow this link to
subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
--
Ryan Pavlik
Human-Computer Interaction Graduate Student
Virtual Reality Applications Center
Iowa State University
rpav...@iastate.edu
http://academic.cleardefinition.com/
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
