2013/1/8 BILODEAU Andre <andre.bilod...@safran-engineering.com>: > I just began to make ctest replace autotools for the program I develop, but > I still have some problems while running ctest. > > Computer: > Intel(R) Core(TM)2 Duo CPU > E7500 @ 2.93GHz > RAM 3,37 GO > > System: > Microsoft Windows XP > Version 2002 > Service Pack 3 > > Environment: > Cygwin 1.7.17(0.262/5/3) > > cmake version: > 2.8.9 > > Context: > Application made using cmake. Testing by ctest. > Tests are run either by the command: > make test > or > ctest > Example: > ------------------------------------------------------------------------ > $ ctest > Test project /cygdrive/d/F053022/Programs/middle-1.4/build > Start 1: t_proto_child > 1/30 Test #1: t_proto_child .................... Passed 0.09 sec > Start 2: t_CsvReader > ... > > 18/30 Test #18: test_05 .......................... Passed 0.28 sec > Start 19: test_06 > 19/30 Test #19: test_06 .......................... Passed 0.23 sec > Start 20: test_07 > 20/30 Test #20: test_07 ..........................***Not Run 0.25 sec > Start 21: test_08 > 21/30 Test #21: test_08 ..........................***Not Run 0.27 sec > Start 22: test_09 > 22/30 Test #22: test_09 .......................... Passed 0.33 sec > Start 23: test_10 > ... > > 25/30 Test #25: test_12 .......................... Passed 0.33 sec > Start 26: test_13 > 26/30 Test #26: test_13 .......................... Passed 0.31 sec > Start 27: test_14 > 27/30 Test #27: test_14 ..........................***Not Run 0.37 sec > Start 28: test_15 > 28/30 Test #28: test_15 .......................... Passed 0.44 sec > Start 29: test_16 > 29/30 Test #29: test_16 .......................... Passed 0.53 sec > Start 30: test_17 > 30/30 Test #30: test_17 ..........................***Not Run 0.39 sec > 87% tests passed, 4 tests failed out of 30 > Total Test time (real) = 7.86 sec > The following tests FAILED: > 20 - test_07 (BAD_COMMAND) > 21 - test_08 (BAD_COMMAND) > 27 - test_14 (BAD_COMMAND) > 30 - test_17 (BAD_COMMAND) > Errors while running CTest > ------------------------------------------------------------------------ > > The tests that fail make it randomly. That is, for instance, if I run, > for the test test_07 that failed: > > ctest -R test_07 > > the test may succeed. Or if I run make test again, the set of tests that > fail may be different. > > All test_nn test are built on the same model. for instance: > > add_test (test_07 ${PROJECT_SOURCE_DIR}/test/test_driver > -N 07 -S 0 -P test_parser) > > As the tests were run originally from automake, an environment variable > is set, at end of CMakeLists.txt file by the command: > > set_tests_properties ( > test_01 test_02 test_03 test_04 test_05 > test_06 test_07 test_08 test_09 test_10 > test_11 test_12 test_13 test_14 test_15 > test_16 test_17 > PROPERTIES ENVIRONMENT srcdir=${PROJECT_SOURCE_DIR}/test) > > A random failure of the tested program is discarded, because the tested > program is activated through a shell, "test_driver". > > This shell outputs a trace upon entry and exit. > The final trace on exit is produced by the following script lines (korn > shell). If > the trace produced contains "PASS", I don't see how the exit status could > differ from 0:
Unless the run of the test timed-out just before exiting ? Did you try setting PASS_REGULAR_EXPRESSION or FAIL_REGULAR_EXPRESSION (with set_tests_properties) in order to use output of the test to evaluate FAIL/PASS status instead of returned exit code? > > if [[ ${errors} -gt 0 ]] ; then > MSG=FAIL > else > MSG=PASS > fi > echo "${MSG}" >> trc_${NUM}.txt > exit ${errors} > > All tests mentioned as "Not Run" are actually run, since a corresponding > trace output file is found for each test. > > In all cases, the trace output is normal, indicating correct entry and > a normal status on exit, just before the exit command, whith the 0 > parameter that would normally correspond to success. Did you try to run ctest -VV i.e. in extra verbose mode and try to track down the difference between failing and non-failing case? Does this happen in alternate environment, i.e. under Linux? > > Thus, the problem seems to come from ctest itself. > ------------------------------------------------------------------------ > On the internet, typing "cmake random BAD_COMMAND error" gets some stories, > but with no interesting answer. > > Is this king of behavior already known ? > Does that correspond to a bug ? > Is there a known workaround ? > > andre > > > > > -- > > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Follow this link to subscribe/unsubscribe: > http://www.cmake.org/mailman/listinfo/cmake -- Erk Le gouvernement représentatif n'est pas la démocratie -- http://www.le-message.org -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake