We've run into this exact chicken and egg problem with both gtest and
Catch at my work too. In both cases, we've worked round the problem by
writing fairly fragile CMake code which parses the C++ files comprising
a test executable and ultimately calling add_test for each discovered
test case.
gtest is harder to handle than Catch in that it provides macros for
handling value-parameterised and type-parameterised tests. It would be
great if CTest were able to handle these popular test libraries
automatically. I'm really short of time of the moment, but if there are
any pointers from experts on how this might be best handled by CTest, I
could perhaps start to have a poke about if there are no other takers?
In the meantime, here's a link to our Catch-handling code:
https://github.com/Fraser999/MaidSafe/blob/846ce37c23bd86ee261e439919dfd7000a8f9372/cmake_modules/add_catch_tests.cmake
The file's commented at the top. It probably doesn't cover all Catch
macros, and I'd be amazed if it handled all C++ coding styles properly,
but it worked fairly reliably for us. We don't use Catch any more, so
I'm not even sure if it works with the current version of Catch. For
comparison (of the complexity) here's our gtest equivalent:
https://github.com/Fraser999/MaidSafe/blob/72af3a0def2f03af7320696f5ea3d241c5af9bdc/cmake_modules/add_gtests.cmake
(it's old code and could be made more efficient I'm sure - but again it
works fine for us).
Cheers,
Fraser.
On 27/01/2015 17:14, David Cole via CMake wrote:
The best thing to do would be to make CTest aware of unit test
frameworks like this, and have it be automatic.
Support for cppunit, gtest and Catch (and any other popular ones)
would all be appreciated by many, I'm sure.
Got time to dig into ctest a little bit?
D
On Tue, Jan 27, 2015 at 11:18 AM, Robert Dailey
<rcdailey.li...@gmail.com> wrote:
I wouldn't say this is an artificial problem. let me go into more detail.
A single test can define any number of test cases. These are unknown
to CMake scripts, as they are written and defined in code. The ideal
solution is to have the test program output a list of all test cases
and based on that list, automatically generate the add_test commands.
That way I don't have to worry about hard-coding test case names in
the build scripts. When a new test case is added to the test code (no
new files would be added), "magically" a new unit test appears to
CMake.
But none of this can be done unless the unit test executables are
built. Thus the chicken and egg problem. To do it as you suggest would
require me to hard-code the test cases. Example:
In MyTest.cpp:
#include <catch.hpp>
TEST_CASE("test1") {}
TEST_CASE("test2") {}
In CMakeLists.txt:
add_executable( MyTest MyTest.cpp )
add_test( NAME MyTest_test1 COMMAND test1 )
add_test( NAME MyTest_test2 COMMAND test2 )
Am I correctly understanding what you have in mind? If so, can you
think of a way to automate this a little better?
On Tue, Jan 27, 2015 at 7:18 AM, David Cole <dlrd...@aol.com> wrote:
It's only a chicken and egg problem if you constrain yourself artificially.
When you define a new unit test source file, there are two things you
need to do:
(1) add it to its executable
(2) write an add_test line that invokes that specific unit test
You could possibly encapsulate those actions into an add_unit_test
macro so that it only seems like you have one thing to do, but
certainly, it's possible.
You must be doing #1 already, so you just have to make sure #2 always
happens whenever #1 happens.
Both eggs.
D
On Mon, Jan 26, 2015 at 4:05 PM, Robert Dailey <rcdailey.li...@gmail.com> wrote:
I believe so, but then you run into the chicken and egg problem:
I need to fully build all unit tests in order to know all the test
cases, but I need to define the tests at the generation phase (before
build).
I'm just not sure how to handle this. The best i can think of is to
generate 1 test executable per CPP file that we write in our unit
tests (generally 1 CPP file per class that is unit tested). Any ideas?
On Mon, Jan 26, 2015 at 10:05 AM, David Cole <dlrd...@aol.com> wrote:
Can you run a command that executes one test case at a time, or do you
have to run all 50 when you execute the tests for a library?
ctest just runs whatever you give it with add_test -- if you want to
filter a collection of unit tests to run only a single unit test, then
the unit test framework you're using would have to support that. Does
Catch allow you to run the tests individually?
On Mon, Jan 26, 2015 at 12:30 AM, Robert Dailey
<rcdailey.li...@gmail.com> wrote:
Hi,
I'm using Catch as my unit test framework:
https://github.com/philsquared/Catch
Is it possible for CTest to report each TEST_CASE block as an
individual test? My understanding is that CTest will only treat each
executable as a test. However, my test structure is as follows:
1 library
1 test project
Each test project has 1 test CPP file per each class in the library.
This way I implement tests for classes in a corresponding CPP file.
Each CPP file contains multiple test cases (defined by TEST_CASE macro).
The resulting output of `ctest -T Test` shows only 1 test, even though
that may be around 50 test cases. I'd like CMake to show the pass/fail
status of each one. Is this possible?
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
