Hi all, I am using CMake again since a couple of weeks for a project involving the build of recognition system. This system is made of hundred sub recognition system that must be all trained with about ~10k files as input.
I ended up writing something like that: ====== BEGIN # A project with hundred targets depending on one command with 10k dependencies. cmake_minimum_required(VERSION 2.8) project(HundredTargetsDepOnOneCommandWithManyDeps) include(CTest) # Define LARGE_LIST with 10000 input files. include(large_list.cmake) foreach(i ${LARGE_LIST}) file(WRITE ${i} "input") endforeach(i) add_custom_command( OUTPUT bottle_neck COMMAND ${CMAKE_COMMAND} -E touch bottle_neck DEPENDS ${LARGE_LIST} COMMENT "Executing command with large number of dependencies..." ) set(OUTPUTS ) foreach(i RANGE 100) set(_output output_${i}) add_custom_target(${_output}-target ALL DEPENDS bottle_neck ) list(APPEND OUTPUTS ${_output}) endforeach(i) add_executable(AlwaysTrue AlwaysTrue.c) ====== END Note that the hundred targets have only one dependency. Running CMake on this project is slow. The configuration is fast but the generation is very slow. Here some examples: With Unix Makefiles $ time make rebuild_cache Running CMake to regenerate build system... -- Configuration done (this is long) -- Generation done (this is long) -- Build files have been written to: ... make rebuild_cache 120.54s user 6.08s system 97% cpu 2:10.07 total Memory used is normal (htop indicates 82488 VIRT) and CPU is at 100%. $ time make make 129.46s user 3.76s system 99% cpu 2:14.28 total With Ninja: $ time ninja -v rebuild_cache ninja rebuild_cache 98.89s user 0.10s system 99% cpu 1:39.02 total $ time ninja ninja 0.01s user 0.04s system 86% cpu 0.055 total So here my two questions: 1/ Do you think it is possible to improve the performance? (I have not profile it yet). 2/ Is there a way to set a timeout to ctest so that my test fail. I have that in Tests/CMakeLists.txt but it does not work: set(_test_name HundredTargetsDepOnOneCommandWithManyDeps) add_test(${_test_name} ${CMAKE_CTEST_COMMAND} --build-and-test "${CMake_SOURCE_DIR}/Tests/${_test_name}" "${CMake_BINARY_DIR}/Tests/${_test_name}" --build-two-config ${build_generator_args} --build-project ${_test_name} --timeout 10 --test-command AlwaysTrue) list(APPEND TEST_BUILD_DIRS "${CMake_BINARY_DIR}/Tests/${_test_name}") unset(_test_name) I have pushed my work so far on my github clone of CMake. https://github.com/nicolasdespres/CMake/tree/topic/large-deps-perf I am willing to do the work on this topic but I wanted to have your insight first. Thanks, -- Nicolas Desprès
-- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers