[CMake] parallel ctest spawning too many threads
Fellow CMake users:
I'm having a problem with one of my projects when I run 'ctest -j'. The
problem is that after 70-90 tests, ctest decides to submit all of the
remaining tests at once (~200 tests worth). For example (actual test names
replaced to protect the innocent :-)
% ctest -j16
Test project /var/tmp/foo
Start 192: testA_2
Start 191: testB_4
Start 190: testC_3
Start 189: testD_2
Start 188: testE_4
1/279 Test #189: testD_2 . Passed
1.81 sec
Start 187: testF_3
2/279 Test #192: testA_2 . Passed
2.98 sec
3/279 Test #191: testB_4 . Passed
2.81 sec
Start 186: testG_2
Start 185: testH_4
4/279 Test #188: testE_4 . Passed
2.81 sec
5/279 Test #190: testC_3 . Passed
2.92 sec
Start 184: testI_3
Start 183: testJ_2
6/279 Test #187: testF_3 . Passed
2.02 sec
Start 182: testK_4
7/279 Test #186: testG_2 . Passed
2.44 sec
...
82/279 Test #249: testZA_2 Passed
8.46 sec
83/279 Test #245: testZB_2 Passed
3.12 sec
Start 278: testZC_4
84/279 Test #278: testZC_4 Passed
4.27 sec
Start 266: testZD_4
Start 263: testZE_4
Start 275: testZF_4
Start 270: testZG_4
Start 269: testZH_4
# ...about 190 tests started all at the same time.
Start 89: testZZA_1
Start 88: testZZB_1
^C
These tests are running under mpirun (trailing number indicates number of
cores needed for the test) and so the total load on my machine jumps to
about 400!
My tests are registered using commands similar to this:
add_test(
NAMEtestA_4
COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 4
${MPIEXEC_POSTFLAGS}
$
)
set_tests_properties( testA_4
PROPERTIES PROCESSORS "4" )
Has anyone seen this 'many threads spawning' behavior for ctest (I couldn't
find anything in the bug tracker). Is there a known resolution?
Thank you,
-kt
-
Kelly (KT) Thompson
k...@lanl.gov
505.665.8090
Los Alamos National Laboratory
CCS-2, MS D409, TA-3/B-422/R-101
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] parallel ctest spawning too many threads
This is the first I've ever heard of it...
Cool bug!
(that was just a wee bit of sarcasm accompanied by a wry smile...)
What version of ctest?
Is this a project that is publicly available, so that I can try to reproduce
it here and help these "innocent" coders...?
Thanks,
David
On Wed, Apr 20, 2011 at 6:35 PM, Kelly Thompson wrote:
> Fellow CMake users:
>
> I'm having a problem with one of my projects when I run 'ctest -j'. The
> problem is that after 70-90 tests, ctest decides to submit all of the
> remaining tests at once (~200 tests worth). For example (actual test names
> replaced to protect the innocent :-)
>
> % ctest -j16
> Test project /var/tmp/foo
>Start 192: testA_2
>Start 191: testB_4
>Start 190: testC_3
>Start 189: testD_2
>Start 188: testE_4
> 1/279 Test #189: testD_2 . Passed
> 1.81 sec
>Start 187: testF_3
> 2/279 Test #192: testA_2 . Passed
> 2.98 sec
> 3/279 Test #191: testB_4 . Passed
> 2.81 sec
>Start 186: testG_2
>Start 185: testH_4
> 4/279 Test #188: testE_4 . Passed
> 2.81 sec
> 5/279 Test #190: testC_3 . Passed
> 2.92 sec
>Start 184: testI_3
>Start 183: testJ_2
> 6/279 Test #187: testF_3 . Passed
> 2.02 sec
>Start 182: testK_4
> 7/279 Test #186: testG_2 . Passed
> 2.44 sec
>
> ...
>
> 82/279 Test #249: testZA_2 Passed
> 8.46 sec
> 83/279 Test #245: testZB_2 Passed
> 3.12 sec
>Start 278: testZC_4
> 84/279 Test #278: testZC_4 Passed
> 4.27 sec
>Start 266: testZD_4
>Start 263: testZE_4
>Start 275: testZF_4
>Start 270: testZG_4
>Start 269: testZH_4
>
> # ...about 190 tests started all at the same time.
>
>Start 89: testZZA_1
>Start 88: testZZB_1
> ^C
>
> These tests are running under mpirun (trailing number indicates number of
> cores needed for the test) and so the total load on my machine jumps to
> about 400!
>
> My tests are registered using commands similar to this:
>
>add_test(
> NAMEtestA_4
> COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 4
> ${MPIEXEC_POSTFLAGS}
> $
> )
>set_tests_properties( testA_4
> PROPERTIES PROCESSORS "4" )
>
> Has anyone seen this 'many threads spawning' behavior for ctest (I couldn't
> find anything in the bug tracker). Is there a known resolution?
>
> Thank you,
>
> -kt
> -
> Kelly (KT) Thompson
> k...@lanl.gov
> 505.665.8090
>
> Los Alamos National Laboratory
> CCS-2, MS D409, TA-3/B-422/R-101
>
>
> ___
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] parallel ctest spawning too many threads
David,
The project is not publicly available (sorry - nothing I can do about that).
This is also the reason I made fictitious test names. I'm trying to
reproduce the issue by creating a standalone project that I can
redistribute. Once that is done, I will post it.
% ctest --version
ctest version 2.8.4
-kt
From: David Cole [mailto:david.c...@kitware.com]
Sent: Thursday, April 21, 2011 6:34 AM
To: k...@lanl.gov
Cc: cmake@cmake.org
Subject: Re: [CMake] parallel ctest spawning too many threads
This is the first I've ever heard of it...
Cool bug!
(that was just a wee bit of sarcasm accompanied by a wry smile...)
What version of ctest?
Is this a project that is publicly available, so that I can try to reproduce
it here and help these "innocent" coders...?
Thanks,
David
On Wed, Apr 20, 2011 at 6:35 PM, Kelly Thompson wrote:
Fellow CMake users:
I'm having a problem with one of my projects when I run 'ctest -j'. The
problem is that after 70-90 tests, ctest decides to submit all of the
remaining tests at once (~200 tests worth). For example (actual test names
replaced to protect the innocent :-)
% ctest -j16
Test project /var/tmp/foo
Start 192: testA_2
Start 191: testB_4
Start 190: testC_3
Start 189: testD_2
Start 188: testE_4
1/279 Test #189: testD_2 . Passed
1.81 sec
Start 187: testF_3
2/279 Test #192: testA_2 . Passed
2.98 sec
3/279 Test #191: testB_4 . Passed
2.81 sec
Start 186: testG_2
Start 185: testH_4
4/279 Test #188: testE_4 . Passed
2.81 sec
5/279 Test #190: testC_3 . Passed
2.92 sec
Start 184: testI_3
Start 183: testJ_2
6/279 Test #187: testF_3 . Passed
2.02 sec
Start 182: testK_4
7/279 Test #186: testG_2 . Passed
2.44 sec
...
82/279 Test #249: testZA_2 Passed
8.46 sec
83/279 Test #245: testZB_2 Passed
3.12 sec
Start 278: testZC_4
84/279 Test #278: testZC_4 Passed
4.27 sec
Start 266: testZD_4
Start 263: testZE_4
Start 275: testZF_4
Start 270: testZG_4
Start 269: testZH_4
# ...about 190 tests started all at the same time.
Start 89: testZZA_1
Start 88: testZZB_1
^C
These tests are running under mpirun (trailing number indicates number of
cores needed for the test) and so the total load on my machine jumps to
about 400!
My tests are registered using commands similar to this:
add_test(
NAMEtestA_4
COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 4
${MPIEXEC_POSTFLAGS}
$
)
set_tests_properties( testA_4
PROPERTIES PROCESSORS "4" )
Has anyone seen this 'many threads spawning' behavior for ctest (I couldn't
find anything in the bug tracker). Is there a known resolution?
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] parallel ctest spawning too many threads
The CMake test suite itself has ~200 tests in it.
Does the same behavior occur when building and testing CMake itself?
VTK and ITK have even more tests, if you are able to build either of those
projects...
What system is this on? (OS version, compiler version...)
Thx,
David
On Thu, Apr 21, 2011 at 11:46 AM, Kelly Thompson wrote:
> David,
>
>
>
> The project is not publicly available (sorry – nothing I can do about
> that). This is also the reason I made fictitious test names. I’m trying to
> reproduce the issue by creating a standalone project that I can
> redistribute. Once that is done, I will post it.
>
>
>
> % ctest --version
>
> ctest version 2.8.4
>
>
>
> -kt**
>
>
>
> *From:* David Cole [mailto:david.c...@kitware.com]
> *Sent:* Thursday, April 21, 2011 6:34 AM
> *To:* k...@lanl.gov
> *Cc:* cmake@cmake.org
> *Subject:* Re: [CMake] parallel ctest spawning too many threads
>
>
>
> This is the first I've ever heard of it...
>
> Cool bug!
> (that was just a wee bit of sarcasm accompanied by a wry smile...)
>
> What version of ctest?
>
> Is this a project that is publicly available, so that I can try to
> reproduce it here and help these "innocent" coders...?
>
>
> Thanks,
> David
>
> On Wed, Apr 20, 2011 at 6:35 PM, Kelly Thompson wrote:
>
> Fellow CMake users:
>
> I'm having a problem with one of my projects when I run 'ctest -j'. The
> problem is that after 70-90 tests, ctest decides to submit all of the
> remaining tests at once (~200 tests worth). For example (actual test names
> replaced to protect the innocent :-)
>
> % ctest -j16
> Test project /var/tmp/foo
>Start 192: testA_2
>Start 191: testB_4
>Start 190: testC_3
>Start 189: testD_2
>Start 188: testE_4
> 1/279 Test #189: testD_2 . Passed
> 1.81 sec
>Start 187: testF_3
> 2/279 Test #192: testA_2 . Passed
> 2.98 sec
> 3/279 Test #191: testB_4 . Passed
> 2.81 sec
>Start 186: testG_2
>Start 185: testH_4
> 4/279 Test #188: testE_4 . Passed
> 2.81 sec
> 5/279 Test #190: testC_3 . Passed
> 2.92 sec
>Start 184: testI_3
>Start 183: testJ_2
> 6/279 Test #187: testF_3 . Passed
> 2.02 sec
>Start 182: testK_4
> 7/279 Test #186: testG_2 . Passed
> 2.44 sec
>
> ...
>
> 82/279 Test #249: testZA_2 Passed
> 8.46 sec
> 83/279 Test #245: testZB_2 Passed
> 3.12 sec
>Start 278: testZC_4
> 84/279 Test #278: testZC_4 Passed
> 4.27 sec
>Start 266: testZD_4
>Start 263: testZE_4
>Start 275: testZF_4
>Start 270: testZG_4
>Start 269: testZH_4
>
> # ...about 190 tests started all at the same time.
>
>Start 89: testZZA_1
>Start 88: testZZB_1
> ^C
>
> These tests are running under mpirun (trailing number indicates number of
> cores needed for the test) and so the total load on my machine jumps to
> about 400!
>
> My tests are registered using commands similar to this:
>
>add_test(
> NAMEtestA_4
> COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} 4
> ${MPIEXEC_POSTFLAGS}
> $
> )
>set_tests_properties( testA_4
> PROPERTIES PROCESSORS "4" )
>
> Has anyone seen this 'many threads spawning' behavior for ctest (I couldn't
> find anything in the bug tracker). Is there a known resolution?
>
>
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
