Yeah, you are right. If ctest itself fails it should return non-zero.
We use qtestlib [1] from Qt5 and add every testsuite with ADD_TEST and dump
a logfile that will be parsed by xunit plugin [2] of jenkins.
ADD_TEST(${TESTNAME} ${TESTNAME} -platform offscreen -v2 -o
${CMAKE_CURRENT_BINARY_DIR}/results.${TESTNAME}.log.xml,xml)
The problem is that the test binaries of qtestlib will return non-zero if
some testcase is broken. So ctest will also return non-zero because a
testsuite "failed".
Of course, this is the correct behaviour if something goes wrong. But if
ctest returns non-zero, jenkins will abort the whole build instead of let
xunit plugin mark the build as "unstable" because testcase XY failed.
Best regards
André
[1] http://doc.qt.io/qt-5/qtest-overview.html
[2] https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
2015-08-14 16:01 GMT+02:00 Brad King <brad.k...@kitware.com>:
> On 08/13/2015 04:09 PM, A. Klitzing wrote:
> > We use ctest on Jenkins in a shell build step. Jenkins will FAIL
> > the build if the last command (ctest) will return a non-zero
> > return code even the configured UNSTABLE/FAIL threshold is not
> > reached.
> >
> > This won't be necessary if ctest could ignore the return codes.
>
> If something goes wrong with the administration of running the
> dashboard client, like "git fetch" fails, then CTest should report
> failure via a non-zero return code. Some cleanup of this was done
> recently to tolerate failed tests and such without making the process
> exit code non-zero. If you've found remaining cases where normal
> testing failures produce a non-zero exit from ctest then we should
> fix those first.
>
> Can you explain your case in more detail, please?
>
> Thanks,
> -Brad
>
>
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
