Re: [CMake] Parallel build with ctest 2.8.2
On Wednesday 21 July 2010, Ryan Pavlik wrote: My CreateDashboardScripts.cmake and DashboardScript.cmake.in located in this repository might be a useful example - I also had a hard time figuring out how to do the new-style script. http://github.com/rpavlik/wiimote-head-tracker-gui/tree/master/cmake/ For use by KDE4 I wrote KDECTestNightly.cmake, which you can find here: http://websvn.kde.org/trunk/KDE/kdesdk/cmake/nightly-support/ , complete with documentation at the top. It does not yet support git. It is actually completely KDE independent and can be used also for non-KDE non-Qt projects. The kde_ prefix is just used to point out that these varibales/macros do not come from the ctest as it comes with cmake, but from the extra module. It's a cmake module which offers some functions/macros and variables for use in new-style ctest scripts, to make it easier to write them. Here is an example ctest-script which drives the nightly build of the kdelibs project: http://websvn.kde.org/trunk/KDE/kdesdk/cmake/nightly-support/KDE/KDELibsNightly.cmake?revision=1117002view=markup Alex ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
I'd just like to confirm that I've seen exactly the same bug since upgrading from 2.8.0 to 2.8.1. I've submitted a bug at: http://public.kitware.com/Bug/view.php?id=11026 On 21/07/10 16:06, Bill Hoffman wrote: On 7/21/2010 10:41 AM, Emmanuel Christophe wrote: Hi Dave, Thanks for the information. I was not aware of the new ctest script style: I got this from Luis when I set up the ITK nightly a while ago. That's also what seems to appear on: http://www.cmake.org/Wiki/CMake_Scripting_Of_CTest Do you have any pointer for the new-style ctest? Currently no. I would like to figure this out: On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i This is just a configured file at the end of the day. And the .in file has this: MakeCommand: @MAKECOMMAND@ So, if the cache has: MAKECOMMAND:STRING=/usr/bin/make -i -j8 Then it should work... Might be a double configure thing going on or something -Bill Thanks, Emmanuel On Wed, Jul 21, 2010 at 19:50, David Coledavid.c...@kitware.com wrote: I am guessing that this commit from Dec. 2009: 0b38bb4c535ae972d7f973e3e69945a6d0c14d75 to fix this bug: http://public.kitware.com/Bug/view.php?id=2336 is related to your problem... I would recommend updating your script to a new-style (since ctest 2.4 at least) script that uses the ctest_build() (and related) functions rather than executing -D Nightly as a CTEST_COMMAND. But... that's just a way to make it work with this newer ctest. There should still be some way to pass -j flags to an old-style script, or for use with a -D command line. If anybody figures out why this is happening, please reply to this thread with a recommended fix to the code. I am not surprised that this bug fix introduced a new issue like this. It was a tricky one to try to figure out a completely backwards compatible solution for. Thanks, David On Wed, Jul 21, 2010 at 3:28 AM, Emmanuel Christophe emmanuel.christo...@gmail.com wrote: On Tue, Jul 20, 2010 at 21:23, Bill Hoffmanbill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
On Tue, Jul 20, 2010 at 21:23, Bill Hoffman bill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
I am guessing that this commit from Dec. 2009: 0b38bb4c535ae972d7f973e3e69945a6d0c14d75 to fix this bug: http://public.kitware.com/Bug/view.php?id=2336 is related to your problem... I would recommend updating your script to a new-style (since ctest 2.4 at least) script that uses the ctest_build() (and related) functions rather than executing -D Nightly as a CTEST_COMMAND. But... that's just a way to make it work with this newer ctest. There should still be some way to pass -j flags to an old-style script, or for use with a -D command line. If anybody figures out why this is happening, please reply to this thread with a recommended fix to the code. I am not surprised that this bug fix introduced a new issue like this. It was a tricky one to try to figure out a completely backwards compatible solution for. Thanks, David On Wed, Jul 21, 2010 at 3:28 AM, Emmanuel Christophe emmanuel.christo...@gmail.com wrote: On Tue, Jul 20, 2010 at 21:23, Bill Hoffman bill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
Hi Dave, Thanks for the information. I was not aware of the new ctest script style: I got this from Luis when I set up the ITK nightly a while ago. That's also what seems to appear on: http://www.cmake.org/Wiki/CMake_Scripting_Of_CTest Do you have any pointer for the new-style ctest? Thanks, Emmanuel On Wed, Jul 21, 2010 at 19:50, David Cole david.c...@kitware.com wrote: I am guessing that this commit from Dec. 2009: 0b38bb4c535ae972d7f973e3e69945a6d0c14d75 to fix this bug: http://public.kitware.com/Bug/view.php?id=2336 is related to your problem... I would recommend updating your script to a new-style (since ctest 2.4 at least) script that uses the ctest_build() (and related) functions rather than executing -D Nightly as a CTEST_COMMAND. But... that's just a way to make it work with this newer ctest. There should still be some way to pass -j flags to an old-style script, or for use with a -D command line. If anybody figures out why this is happening, please reply to this thread with a recommended fix to the code. I am not surprised that this bug fix introduced a new issue like this. It was a tricky one to try to figure out a completely backwards compatible solution for. Thanks, David On Wed, Jul 21, 2010 at 3:28 AM, Emmanuel Christophe emmanuel.christo...@gmail.com wrote: On Tue, Jul 20, 2010 at 21:23, Bill Hoffman bill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
My CreateDashboardScripts.cmake and DashboardScript.cmake.in located in this repository might be a useful example - I also had a hard time figuring out how to do the new-style script. http://github.com/rpavlik/wiimote-head-tracker-gui/tree/master/cmake/ Ryan On 7/21/2010 9:41 AM, Emmanuel Christophe wrote: Hi Dave, Thanks for the information. I was not aware of the new ctest script style: I got this from Luis when I set up the ITK nightly a while ago. That's also what seems to appear on: http://www.cmake.org/Wiki/CMake_Scripting_Of_CTest Do you have any pointer for the new-style ctest? Thanks, Emmanuel On Wed, Jul 21, 2010 at 19:50, David Coledavid.c...@kitware.com wrote: I am guessing that this commit from Dec. 2009: 0b38bb4c535ae972d7f973e3e69945a6d0c14d75 to fix this bug: http://public.kitware.com/Bug/view.php?id=2336 is related to your problem... I would recommend updating your script to a new-style (since ctest 2.4 at least) script that uses the ctest_build() (and related) functions rather than executing -D Nightly as a CTEST_COMMAND. But... that's just a way to make it work with this newer ctest. There should still be some way to pass -j flags to an old-style script, or for use with a -D command line. If anybody figures out why this is happening, please reply to this thread with a recommended fix to the code. I am not surprised that this bug fix introduced a new issue like this. It was a tricky one to try to figure out a completely backwards compatible solution for. Thanks, David On Wed, Jul 21, 2010 at 3:28 AM, Emmanuel Christophe emmanuel.christo...@gmail.com wrote: On Tue, Jul 20, 2010 at 21:23, Bill Hoffmanbill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake -- Ryan Pavlik HCI Graduate Student Virtual Reality Applications Center Iowa State University rpav...@iastate.edu http://academic.cleardefinition.com Internal VRAC/HCI Site: http://tinyurl.com/rpavlik ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
On 7/21/2010 10:41 AM, Emmanuel Christophe wrote: Hi Dave, Thanks for the information. I was not aware of the new ctest script style: I got this from Luis when I set up the ITK nightly a while ago. That's also what seems to appear on: http://www.cmake.org/Wiki/CMake_Scripting_Of_CTest Do you have any pointer for the new-style ctest? Currently no. I would like to figure this out: On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i This is just a configured file at the end of the day. And the .in file has this: MakeCommand: @MAKECOMMAND@ So, if the cache has: MAKECOMMAND:STRING=/usr/bin/make -i -j8 Then it should work... Might be a double configure thing going on or something -Bill Thanks, Emmanuel On Wed, Jul 21, 2010 at 19:50, David Coledavid.c...@kitware.com wrote: I am guessing that this commit from Dec. 2009: 0b38bb4c535ae972d7f973e3e69945a6d0c14d75 to fix this bug: http://public.kitware.com/Bug/view.php?id=2336 is related to your problem... I would recommend updating your script to a new-style (since ctest 2.4 at least) script that uses the ctest_build() (and related) functions rather than executing -D Nightly as a CTEST_COMMAND. But... that's just a way to make it work with this newer ctest. There should still be some way to pass -j flags to an old-style script, or for use with a -D command line. If anybody figures out why this is happening, please reply to this thread with a recommended fix to the code. I am not surprised that this bug fix introduced a new issue like this. It was a tricky one to try to figure out a completely backwards compatible solution for. Thanks, David On Wed, Jul 21, 2010 at 3:28 AM, Emmanuel Christophe emmanuel.christo...@gmail.com wrote: On Tue, Jul 20, 2010 at 21:23, Bill Hoffmanbill.hoff...@kitware.com wrote: On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. On the CMakeCache.txt: //Command used to build entire project from the command line. MAKECOMMAND:STRING=/usr/bin/make -i -j8 On the DartConfiguration.tcl: MakeCommand: /usr/bin/make -i There is a mismatch here... I would have to see the cmake script you are using. The script is attached with the submissions: http://www.cdash.org/CDash/viewNotes.php?buildid=669361 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake -- Bill Hoffman Kitware, Inc. 28 Corporate Drive Clifton Park, NY 12065 bill.hoff...@kitware.com http://www.kitware.com 518 881-4905 (Direct) 518 371-3971 x105 Fax (518) 371-4573 ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Parallel build with ctest 2.8.2
On 7/19/2010 11:10 PM, Emmanuel Christophe wrote: Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Build command: /usr/bin/make -i Start Time: 2010-07-19T11:51:09 EDT End Time: 2010-07-19T13:56:33 EDT So, it is not using make -j. You should look at the CMakeCache.txt and the DartConfiguration.tcl file and what values they have. I would have to see the cmake script you are using. -Bill ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] Parallel build with ctest 2.8.2
Hi all, I used to do parallel build for my nightly with ctest by putting MAKECOMMAND:STRING=/usr/bin/make -i -j8 in my .cmake script. Since a few month ago, it does not seem to work anymore. This is clearly visible on my submission to the ITK dashboard: http://www.cdash.org/CDash/buildSummary.php?buildid=669361 Displaying the build graph shows that the build time went from 50 to 150 min around April 5-6th. As I'm using a regularly updated Debian testing, that could be linked to the update to cmake 2.8.1. I'm not sure if there is a way to check the exact data when a package was updated for Debian. I'm now using cmake 2.8.2. Anyway, I also have a cmake 2.6 that I compiled from the source and in this case, the MAKECOMMAND is taken into account and the build is parallel, that shows that the issue is with cmake/ctest. I tried what was suggested in this thread: http://www.cmake.org/pipermail/cmake/2009-December/033617.html but no success. Has anybody noticed something similar? I could find another debian build that show a compilation time increase around april on the ITK dashboad http://www.cdash.org/CDash/buildSummary.php?buildid=669205 On the VTK dashboard, at least one debian build shows significant compilation time increase (x2-3) also in April http://www.cdash.org/CDash/buildSummary.php?buildid=668868 Emmanuel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
