[CMake] Deferred CPU count variable in CMake for Makefile generator?

2011-09-09 Thread Clifford Yapp
Per the customizing Makefile email, I'd like to ask if a mechanism exists or
could be implemented to allow CMakeLists.txt files to define commands that
pass a CPU count to subcommands - e.g. something that allows the generators
to take a syntax along the lines of:

ADD_CUSTOM_TARGET(subbuild
 COMMAND ${CMAKE_COMMAND} --build ${TARGET_DIR} -- -j${PARENT_CPUS}
)

and generate makefiles that will take make JOBS=3 subbuild and have cmake
in turn launch make -j3 in the target directory - in other words,
${PARENT_CPUS} would be a cue for the backend generator to add logic to pass
down parallel make settings to the subbuild command.  (Ideally -j3 could be
passed down, but it looks like make itself precludes that so some other
variable would have to be used at the top level.)

Any insight is appreciated - this one has got me pretty stumped as to the
right way to achieve this effect.

Thanks,
CY
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

Re: [CMake] Deferred CPU count variable in CMake for Makefile generator?

2011-09-09 Thread David Cole
On Fri, Sep 9, 2011 at 10:49 AM, Clifford Yapp cliffy...@gmail.com wrote:
 Per the customizing Makefile email, I'd like to ask if a mechanism exists or
 could be implemented to allow CMakeLists.txt files to define commands that
 pass a CPU count to subcommands - e.g. something that allows the generators
 to take a syntax along the lines of:

 ADD_CUSTOM_TARGET(subbuild
  COMMAND ${CMAKE_COMMAND} --build ${TARGET_DIR} -- -j${PARENT_CPUS}
 )

 and generate makefiles that will take make JOBS=3 subbuild and have cmake
 in turn launch make -j3 in the target directory - in other words,
 ${PARENT_CPUS} would be a cue for the backend generator to add logic to pass
 down parallel make settings to the subbuild command.  (Ideally -j3 could be
 passed down, but it looks like make itself precludes that so some other
 variable would have to be used at the top level.)

 Any insight is appreciated - this one has got me pretty stumped as to the
 right way to achieve this effect.

 Thanks,
 CY


 ___
 Powered by www.kitware.com

 Visit other Kitware open-source projects at
 http://www.kitware.com/opensource/opensource.html

 Please keep messages on-topic and check the CMake FAQ at:
 http://www.cmake.org/Wiki/CMake_FAQ

 Follow this link to subscribe/unsubscribe:
 http://www.cmake.org/mailman/listinfo/cmake


But, -- -j${PARENT_CPUS} is make-specific anyhow, right? So the
add_custom_target you propose only works with the Makefile generators
where the make in question actually supports -j.

In the ExternalProject module, we use the following logic to have
custom targets build where the child processes use the same parallel
job controller as the parent make process, but only for make
systems. To do this we use the literal $(MAKE) as the command to do
the actual build. And then, for non-make systems, we fall back to
cmake --build

# Full code for this snippet starts at line 658 of the
Modules/ExternalProject.cmake file presently in CMake 'master' ...
#
# 
http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;h=a37771b9;hb=3ae6060f#l658
#
if(cfg_cmd_id STREQUAL cmake)
  # CMake project.  Select build command based on generator.
  get_target_property(cmake_generator ${name} _EP_CMAKE_GENERATOR)
  if(${CMAKE_GENERATOR} MATCHES Make AND
 (${cmake_generator} MATCHES Make OR NOT cmake_generator))
# The project uses the same Makefile generator.  Use recursive make.
set(cmd $(MAKE))
... code elided for brevity ...
  else()
# Drive the project with cmake --build.
get_target_property(cmake_command ${name} _EP_CMAKE_COMMAND)
if(cmake_command)
  set(cmd ${cmake_command})
else()
  set(cmd ${CMAKE_COMMAND})
endif()
set(args --build ${binary_dir} --config ${CMAKE_CFG_INTDIR})
... code elided for brevity ...
  endif()
else() # if(cfg_cmd_id STREQUAL configure)
  # Non-CMake project.  Guess make and make install and make test.
  # But use $(MAKE) to get recursive parallel make.
  set(cmd $(MAKE))
... code elided for brevity ...
endif()

With this logic, we are able to use make -j8 at the
parent/top-level/super- project level, and then all the child makes
that occur via $(MAKE) share the 8 slots for running parallel
commands as the make progresses.

Would that be a sufficient solution, or do you really need to pass
flags down for some of your custom targets based on some of the makes
that you have to deal with?

Let us know.


Thanks,
David
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake


Re: [CMake] Deferred CPU count variable in CMake for Makefile generator?

2011-09-09 Thread Clifford Yapp
Looks like $(MAKE) will do it, at least in testing so far - awesome!  Thanks
David.

Cliff.

On Fri, Sep 9, 2011 at 11:15 AM, David Cole david.c...@kitware.com wrote:


  if(${CMAKE_GENERATOR} MATCHES Make AND
 (${cmake_generator} MATCHES Make OR NOT cmake_generator))
# The project uses the same Makefile generator.  Use recursive make.
set(cmd $(MAKE))

 Would that be a sufficient solution, or do you really need to pass
 flags down for some of your custom targets based on some of the makes
 that you have to deal with?

 Let us know.


 Thanks,
 David

___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake