Re: [CMake] using ctest launchers leads to empty warning/error messages on the dashboard
On Friday 24 July 2015 10:23:48 Matthias Kretz wrote:
> set(ENV{CTEST_USE_LAUNCHERS_DEFAULT} "ON")
>
> I don't get any output on the CDash dashboard anymore. AFAICT the Build
> directory in Testing/tag/ is removed but never recreated. I.e. if I call
Maybe the better lead:
I made the UseLaunchers setting behave just as BuildDirectory, in terms of
requiring its presence. And I get:
Cannot find UseLaunchers key in the DartConfiguration.tcl
But OTOH:
grep Launch DartConfiguration.tcl
UseLaunchers: ON
Cheers,
Matthias
signature.asc
Description: This is a digitally signed message part.
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] using ctest launchers leads to empty warning/error messages on the dashboard
OK, this patch is bogus. Because
cmCTestBuildHandler::LaunchHelper::LaunchHelper should call MakeDirectory
already if this->Handler->UseCTestLaunch is true.
Makes my test results even more strange then.
On Friday 24 July 2015 11:28:13 Matthias Kretz wrote:
> On Friday 24 July 2015 10:23:48 Matthias Kretz wrote:
> > set(ENV{CTEST_USE_LAUNCHERS_DEFAULT} "ON")
> >
> > I don't get any output on the CDash dashboard anymore. AFAICT the Build
> > directory in Testing/tag/ is removed but never recreated.
>
> With the following patch I get output back to the dashboard:
> --- a/Source/CTest/cmCTestLaunch.cxx
> +++ b/Source/CTest/cmCTestLaunch.cxx
> @@ -207,6 +207,7 @@ void cmCTestLaunch::ComputeFileNames()
>this->LogDir = d;
>cmSystemTools::ConvertToUnixSlashes(this->LogDir);
>this->LogDir += "/";
> + cmSystemTools::MakeDirectory(this->LogDir);
>
>// We hash the input command working dir and command line to obtain
>// a repeatable and (probably) unique name for log files.
>
> But the dashboard still shows the conflated output of the parallel build
> instead of the correct context per source/object file.
>
> Cheers,
> Matthias
signature.asc
Description: This is a digitally signed message part.
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] using ctest launchers leads to empty warning/error messages on the dashboard
On Friday 24 July 2015 10:23:48 Matthias Kretz wrote:
> set(ENV{CTEST_USE_LAUNCHERS_DEFAULT} "ON")
>
> I don't get any output on the CDash dashboard anymore. AFAICT the Build
> directory in Testing/tag/ is removed but never recreated.
With the following patch I get output back to the dashboard:
--- a/Source/CTest/cmCTestLaunch.cxx
+++ b/Source/CTest/cmCTestLaunch.cxx
@@ -207,6 +207,7 @@ void cmCTestLaunch::ComputeFileNames()
this->LogDir = d;
cmSystemTools::ConvertToUnixSlashes(this->LogDir);
this->LogDir += "/";
+ cmSystemTools::MakeDirectory(this->LogDir);
// We hash the input command working dir and command line to obtain
// a repeatable and (probably) unique name for log files.
But the dashboard still shows the conflated output of the parallel build
instead of the correct context per source/object file.
Cheers,
Matthias
signature.asc
Description: This is a digitally signed message part.
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
[CMake] using ctest launchers leads to empty warning/error messages on the dashboard
Hi,
using
set(ENV{CTEST_USE_LAUNCHERS_DEFAULT} "ON")
I don't get any output on the CDash dashboard anymore. AFAICT the Build
directory in Testing/tag/ is removed but never recreated. I.e. if I call
CTEST_LAUNCH_LOGS=./not-existing-directory make
the launcher output is not stored to disk. With
CTEST_LAUNCH_LOGS=./ make
I get launch-* and warning-* files in the cwd.
Is this a bug in ctest or am I doing something wrong? I was testing with cmake
version 3.3.0-rc3.
Cheers,
Matthias
--
──────
Matthias Kretz
SIMD easy and portable: https://github.com/VcDevel/Vc
──
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 20:09:09 Alexander Neundorf wrote: > On Thursday 25 April 2013, Matthias Kretz wrote: > > On Thursday 25 April 2013 09:46:03 Bill Hoffman wrote: > > > On 4/25/2013 9:15 AM, Matthias Kretz wrote: > > > > I have several checks in my project that make cmake error out with > > > > FATAL_ERROR. This is submitted as error to the dashboard. In reality > > > > it > > > > would be an error if this error would not appear - and it is the > > > > expected result if cmake errors out. Is there a possiblity to invert > > > > the result of the configure step for certain system setups? > > > > > > > > Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script > > > > and > > > > just submit Notes that elaborate on the abort. But that doesn't show > > > > up > > > > on the dashboard at all. > > > > > > > > Any ideas? > > > > > > Is this for an individual test? If so, you can use the regex matching > > > instead of return code to determine pass fail. Would that work? > > > > I'm not sure we're talking about the same thing. I'm talking about > > ctest_configure - i.e. before build or tests. IIUC you're talking about > > regex matching at ctest_test? > > so the good result is if the configure step of your project fails ? Exactly. And running some cmake code in the test phase is certainly possible, but not what I'm looking for. It would require the configure step of my project to pass, disable all targets, and enable a test that doesn't do anything useful anymore, because the test already happened at configure time. So I could also simply skip that and just let configure pass + disable all targets. -- Matthias Kretz (Germany)<>< http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 10:30:14 Bill Hoffman wrote: > On 4/25/2013 10:05 AM, Matthias Kretz wrote: > > I'm not sure we're talking about the same thing. I'm talking about > > ctest_configure - i.e. before build or tests. IIUC you're talking about > > regex matching at ctest_test? > > But, if you have a FATAL_ERROR in the configure step, no build files > will be created and you won't even get a build of the software to test. Which is the correct behavior. And I'd like my dashboard to tell me that it works. Without building or "testing". The test in this case consists of making sure that my cmake script aborts with the right error message. >I thought you might have test cases that were complete projects. I > don't think you can change this. It is not just the error in the > output, it is that cmake returns an error code when run. Basically the > configure command fails. Yes. And if it wouldn't fail it would be an error. That's what I mean: I need to invert the logic of this part. Well, maybe it's better that those builds don't show on the dashboard at all... Cheers, Matthias -- Matthias Kretz (Germany)<>< http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
On Thursday 25 April 2013 09:46:03 Bill Hoffman wrote: > On 4/25/2013 9:15 AM, Matthias Kretz wrote: > > I have several checks in my project that make cmake error out with > > FATAL_ERROR. This is submitted as error to the dashboard. In reality it > > would be an error if this error would not appear - and it is the expected > > result if cmake errors out. Is there a possiblity to invert the result of > > the configure step for certain system setups? > > > > Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and > > just submit Notes that elaborate on the abort. But that doesn't show up > > on the dashboard at all. > > > > Any ideas? > > Is this for an individual test? If so, you can use the regex matching > instead of return code to determine pass fail. Would that work? I'm not sure we're talking about the same thing. I'm talking about ctest_configure - i.e. before build or tests. IIUC you're talking about regex matching at ctest_test? Also note that CTEST_CUSTOM_ERROR_EXCEPTION has no effect (and it's documented to be used for ctest_build only). Cheers, Matthias -- Matthias Kretz (Germany)<>< http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] turning inverting dashboard interpretation of FATAL_ERRORs at cmake
Hi, I have several checks in my project that make cmake error out with FATAL_ERROR. This is submitted as error to the dashboard. In reality it would be an error if this error would not appear - and it is the expected result if cmake errors out. Is there a possiblity to invert the result of the configure step for certain system setups? Alternatively I'd skip the CTEST_CONFIGURE call in the ctest script and just submit Notes that elaborate on the abort. But that doesn't show up on the dashboard at all. Any ideas? Cheers, Matthias -- ____ Matthias Kretz (Germany)<>< http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] How to ignore configure warnings in a ctest run?
Thanks for that workaround. Will do that...
Here's an idea how the dashboard could be improved:
When I open the configure warnings page on the dashboard there could be a
button to mark the warning as expected. From then on, for every build with the
same "build name" and "site" the warning would be hidden. If the warning goes
away a warning appears, showing what warning message was expected.
Regards,
Matthias
On Tuesday 29 May 2012 14:39:54 David Cole wrote:
> CDash "configure" warnings are computed dynamically by CDash based only on
> the configure output itself with php code like this:
>
> $pos1 = strpos($log, 'CMake Warning', $position);
> $pos2 = strpos($log, 'Warning:', $position);
>
> (see CDash/models/buildconfigure.php for more details)
>
> So the only way to avoid them showing up on CDash is to avoid emitting the
> warnings in the first place.
>
>
> You can use code like this in your CMakeLists file if you wish to show the
> warning only for non-dashboard builds:
>
> if("$ENV{DASHBOARD_TEST_FROM_CTEST}" STREQUAL "")
> # developer (non-dashboard) build
> message(WARNING "show warning to real user")
> else()
> # dashboard build
> message(STATUS "avoiding warning on dashboard")
> endif()
>
> That environment variable is set by ctest only when it is running a
> dashboard via the -D command line options, or via a -S script.
>
>
> HTH,
> David
>
> On Fri, May 25, 2012 at 5:38 AM, Matthias Kretz wrote:
> > Hi,
> >
> > in my project I added a few warnings to tell users of the library about
> > compiler issues or such. But when I run a nightly build with such a
> > compiler I
> > really don't care to see that warning (well, rather I'd like to see a
> > warning
> > if the warning is missing) on the dashboard. But I didn't succeed in
> > ignoring
> > the warning yet. It appears that CTEST_CUSTOM_WARNING_EXCEPTION is not
> > used
> > for configure output, is it?
> > Do you know of any way to do it, or would it be possible to have some way
> > to
> > do it in a future version?
> >
> > Regards,
> >
> >Matthias
> >
> > --
> >
> > Matthias Kretz (Germany)<><
> > http://kretzfamily.de/
> >
> > --
> >
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
--
Matthias Kretz (Germany)<><
http://kretzfamily.de/
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
[CMake] How to ignore configure warnings in a ctest run?
Hi, in my project I added a few warnings to tell users of the library about compiler issues or such. But when I run a nightly build with such a compiler I really don't care to see that warning (well, rather I'd like to see a warning if the warning is missing) on the dashboard. But I didn't succeed in ignoring the warning yet. It appears that CTEST_CUSTOM_WARNING_EXCEPTION is not used for configure output, is it? Do you know of any way to do it, or would it be possible to have some way to do it in a future version? Regards, Matthias -- ____ Matthias Kretz (Germany)<>< http://kretzfamily.de/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] many false positives in ctest error/warning reporting
Hi,
On Saturday 21 May 2011 12:38:58 Alexander Neundorf wrote:
> On Thursday 19 May 2011, Matthias Kretz wrote:
> > On Tuesday 10 May 2011 21:47:41 Alexander Neundorf wrote:
> > > Are you using cmake >= 2.8 ?
> > > If so, there the switch CTEST_USE_LAUNCHERS. If this is set in your
> > > ctest- script, the output parsing works better.
> >
> > After I set CTEST_USE_LAUNCHERS (I didn't change anything else at the
> > same time), all warnings and errors are suppressed. I.e. the dashboard
> > sees 0 warnings and 0 errors on every build now. Which is clearly wrong.
>
> No idea, I never had such problems myself.
> Can you post the ctest script and the ctest settings file you are using ?
I attached the four files, which are the most relevant (You can browse the
whole repo at http://code.compeng.uni-frankfurt.de/projects/vc/repository). I
start nightly runs with
% ./Test_vc.sh Nightly
The shell script gathers meta information and exports it via env vars to the
test.cmake script.
(One day I'd like to get rid of the wrapping shell script and do it all in the
test.cmake script.)
Regards,
Matthias
--
Matthias Kretz (Germany)<><
http://kretzfamily.de/
test.cmake
Description: Binary data
Test_vc.sh
Description: Binary data
CTestCustom.cmake
Description: Binary data
CTestConfig.cmake
Description: Binary data
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] many false positives in ctest error/warning reporting
Hi, On Tuesday 10 May 2011 21:47:41 Alexander Neundorf wrote: > Are you using cmake >= 2.8 ? > If so, there the switch CTEST_USE_LAUNCHERS. If this is set in your ctest- > script, the output parsing works better. After I set CTEST_USE_LAUNCHERS (I didn't change anything else at the same time), all warnings and errors are suppressed. I.e. the dashboard sees 0 warnings and 0 errors on every build now. Which is clearly wrong. Can you give me a hint how to debug this? You can find the ctest debug output of a run with lots of warnings and errors at http://compeng.uni- frankfurt.de/~kretz/ctest-debug.log Cheers, Matthias -- ____ Matthias Kretz (Germany)<>< http://kretzfamily.de/ ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] cdash build name
On Tuesday 17 May 2011 14:28:30 Yngve Inntjore Levinsen wrote:
> On Tuesday 17 May 2011 14.01.00 Rolf Eike Beer wrote:
> > Yngve Inntjore Levinsen wrote:
> > > set(CTEST_BUILD_NAME "${osname}-${cpu}-prod")
> > > message("CTEST name: ${CTEST_BUILD_NAME}")
> >
> > You must set the build name before including the CTest modules.
>
> Aha, did not know that. Tried and now it works very well (using BUILDNAME
> and not CTEST_BUILD_NAME), thanks a bunch!
>
> > The compiler version is always unknown as this information is not
> > transferred (properly) in the XML to CDash. See
> > http://www.cmake.org/Bug/view.php?id=7867
>
> Oh well then we just have to wait for the bug to be fixed I guess...
I use the ctest build name to put all information about the host system and
compiler in there. E.g. it looks like this: "master Description: openSUSE 11.3
(x86_64) x86_64 g++ (GCC) 4.6.1 20110429 (prerelease)"
So basically "".
Cheers,
Matthias
--
Matthias Kretz (Germany)<><
http://kretzfamily.de/
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] many false positives in ctest error/warning reporting
Hi Alex! On Tuesday 10 May 2011 21:47:41 Alexander Neundorf wrote: > On Tuesday 10 May 2011, Matthias Kretz wrote: > Are you using cmake >= 2.8 ? > If so, there the switch CTEST_USE_LAUNCHERS. If this is set in your ctest- > script, the output parsing works better. Are you already using this ? Yes, that looks like a very useful switch. I've never seen it before - but that's probably because I wrote the ctest stuff in cmake 2.6 times. Isn't cmake able to figure out by itself that it's running in ctest submission mode? Then I'd consider to enable launchers per default for this. I can't see a reason why you don't want to have it. > > AFAICS the incorrect reports mostly come from this regex: > > ([^ :]+):([0-9]+): ([^ \t])) > > In my opinion this regex is way too general and I'd like to disable it. > > But I certainly don't want to put this regex into > > CTEST_CUSTOM_ERROR_EXCEPTION. > > Why not ? Becaue cmake first matches the error regexps and then discards all candidates that match CTEST_CUSTOM_ERROR_EXCEPTION. Since almost all error messages match this regexp, it would suppress everything I want to have reported. Or did I misunderstand this? -- Matthias Kretz (Germany)<>< http://kretzfamily.de/ ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] many false positives in ctest error/warning reporting
Hi, the regular expressions for errors and warnings in Source/CTest/cmCTestBuildHandler.cxx appear to be too general. I have a dashboard where all of the nightly builds are done with parallel builds (-j24 for the machine with the most cores). Often this results in messed up compiler output since the output streams of the different make threads are not synchronized. So one can easily get something like: "from /.../Vc- Continuous7/include/Vc/Vc:22/.../Vc- Continuous7/include/Vc/global.h:139:8:/.../Vc- Continuous7/include/Vc/global.h:139:8: ," which is then matched as an error. (See http://code.compeng.uni- frankfurt.de/dashboard/viewBuildError.php?buildid=36336 ). In reality the output that got pushed to the next line showed that it's a warning, and even one that I put into the code there myself and thus rather would warn about if it's missing. Since that feature is not there, I just suppress all the warnings that I expect. So, in fact, this build should have turned out clean, except for the "mandel.cpp:115: warning: 'int min(int, int)' defined but not used" warning. I got one error and 3 warnings. AFAICS the incorrect reports mostly come from this regex: ([^ :]+):([0-9]+): ([^ \t])) In my opinion this regex is way too general and I'd like to disable it. But I certainly don't want to put this regex into CTEST_CUSTOM_ERROR_EXCEPTION. Do you have any idea how to do this without patching cmake itself? Cheers, Matthias -- ________ Matthias Kretz (Germany)<>< http://kretzfamily.de/ ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
