Re: [CMake] CMake Project Generation Speedup
The best advice I had for myself: profile the code and tweak it. The last I looked, and that was a while ago, the generators could use a threaded write queue so that the file output would overlap with file generation. Those probably could be memory mapped files, too, to further minimize the overhead: writing to mapped pages that are dumped to disk via DMA (that’s how page flushes work) is cheaper than copying all the data in little chunks into a small buffer in the file output code, and then once again into the page that ends up flushed to disk. Never mind that the write latencies are on the hot path, where they don’t belong. If generators could be const-correct (not writing anywhere into the Makefile data structures), they could all be parallelized as well – but I’m not sure whether that’s a minor hack or a major rework. Cheers, Kuba > 20 mars 2019 kl. 16:53 skrev J. Caleb Wherry : > > Did anything ever come of this? > > I am in a similar boat: we have >800 targets on our full build (native C++, > Managed C++, C#, Java, CUDA, etc) and the majority of the time for the > configure/generate steps takes place in the generate step (>70%). > > I understand there is a lot of IO since the generate step has to write the > project files and filters for each C++ project (the majority of our projects) > for VS generators (what we use). I'm just looking to see if there is anything > to look at or potentially speedup up the generate step besides "get a faster > drive". > > Thanks! > -Caleb > >> On Thu, Nov 17, 2016 at 11:15 AM Damian wrote: >> Hi all, >> >> We are still in the process of switching our large Make-based build to >> CMake. One of the issues we're running into is the time it takes to reparse >> and regenerate the CMake project (whether ninja, VS, or make) after >> touching any CMake file. To give you an idea, we have about 1000 targets and >> that takes a good 2 min for CMake to rerun. >> >> Are there any plans to speed this up? Maybe parallelize it in some way or do >> a better job regenerating only what needs regenerating? Is there anything we >> can do on our side to reduce our regeneration times? >> >> For example, if using a VS generator, each directory in the source that has >> a CMakeLists.txt gets a .vcproj and .sln generated. Ideally, if I touch one >> of those CMakeLists.txt, only that .sln/.vcproj would get regenerated. >> >> Thanks for any help. >> -- >> >> Powered by www.kitware.com >> >> Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> >> Kitware offers various services to support the CMake community. For more >> information on each offering, please visit: >> >> CMake Support: http://cmake.org/cmake/help/support.html >> CMake Consulting: http://cmake.org/cmake/help/consulting.html >> CMake Training Courses: http://cmake.org/cmake/help/training.html >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/cmake > > > -- > J. Caleb Wherry > Scientific Software Engineer > > http://www.calebwherry.com > +1 (615) 708-5651 > calebwhe...@gmail.com > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
[CMake] Is there an idiomatic way of setting platform specific compiler options in a generic way?
I am developing platform support for an embedded target (Z8 Encore!), and it
mostly works – but I’d like to make it easier to set the compiler and linker
flags, as there’s a ton of them: and they are nothing like GNU flags, so they
only apply when building for that particular target.
Thus a question: is there a preferred way for the user to set platform-specific
flags without having to know them exactly? I’m thinking of having some sort of
platform-agnostic means of selecting compile and link flags, and then
translating those for the platform the target is generated for.
Specifically, I want to avoid having the following in CMakeLists for the
project:
if("${COMPILER_ID}" strequal "ez8cc")
target_compile_options(target1 PRIVATE -const:RAM)
elseif("${COMPILER_ID}" strequal "gcc" and "${CMAKE_SYSTEM_PROCESSOR}" strequal
"somecpu")
target_compile_definitions(target1 PRIVATE ram rom far near) # those shall be
empty
endif()
I’d like to do something like this instead:
set_target_properties(target1 PROPERTIES OPT_CONST RAM OPT_PACK TIGHT …)
Then, a process_target_opts() function called at the end of CMakeLists would
iterate the targets, and for each target call process_${COMPILER_ID}_opts() (if
such function would be present or issue a warning otherwise). That function
then converts the OPT_ options to platform-specific flags, based on cpu and/or
compiler.
Is it something that would be more-or-less idiomatic, or is there another
preferred way of doing it? I really don’t want to list the options manually for
each target/platform combo, as they’d differ quite a bit, and I have lots of
targets (hundreds).
I’d appreciate any hints.
Cheers, Kuba Ober
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Remove folders created by install
Use something to read the manifest, extract paths from it, remove the duplicates, then iterate them, and remove the ones that are empty. It’d be a bad idea probably to remove ones that are not empty. Cheers, Kuba > 16 feb. 2019 kl. 09:47 skrev Felix Crazzolara : > > Hi everyone > > For my smaller projects I'd like to have 'uninstall' functionality. To remove > installed files I can call: > > xargs rm < build/install_manifest.txt > > Unfortunately this won't delete any folders generated by the installation. Is > there a different file that keeps track of the created directories, or what > is the recommended way to implement such functionality? > > Example: > Suppose that I install _some_header.hpp in > /include// using the command install(TARGETS > EXPORT -targets ARCHIVE DESTINATION lib > PUBLIC_HEADER DESTINATION include/) then I want not only to > remove /include//_some_header.hpp, but > also the directory /include//. > > Cheers, > > Felix Crazzolara > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
[CMake] Generator expressions and rule variables
Hi, I'm trying to integrate cmake with a set of build tools whose command
lines look nothing like standard gnu tools.
Unfortunately, both the generator expressions and the rule variables seem
very limited: I have to do some custom massaging of the arguments that get
passed to the compiler/linker. I've resorted to using cmake in script mode
as a proxy to the compiler and linker, passing all the rule variables
verbatim, and then having the script call the compiler or linker.
This suffers from problems with escaping of spaces, at the very least. It
seems like those compilation commands would need nested lists (a
hypothetical construct thus far): the outer list would be the list of
commands - as it is now, the inner list would be the list of arguments, and
they'd be properly escaped etc. Some of it could be solved if
COMMAND_EXPAND_LISTS could be somehow applied to normal targets, and thus
to the underlying CMAKE__COMPILE_OBJECT, etc.
Simplifying, thus far I set CMAKE_C_COMPILE_OBJECT to ${CMAKE_COMMAND}
-DV1= ... -DVn= -P
${CMAKE_CURRENT_LIST_DIR}/helper.cmake. The helper does the rest. It's
brittle because of poor escaping support. I could also use
C_COMPILER_LAUNCHER with same effect but it couldn't be a cmake script,
since cmake doesn't understand "cmake -D... -P script.cmake -- arbitrary
arguments" (it could without any trouble - it's a small patch; would it be
something worthwhile to add?).
The problem is that I don't really need the overhead of invoking cmake or
another script/process for every compiled/linked output. For both make and
ninja generators, all I'd need is to generate a custom target command that
bypasses the result of the built-in generator. But I also want my
CMakeLists to look familiar and be compatible with additional generators,
thus having a custom "add_my_executable" function wouldn't work, and
overriding "add_executable" to modify some targets, and pass others
directly to "add_executable_" (the undocumented handle to the overridden
function) seems like a big hack.
So, what I'd want to do is to have a function that can be invoked for every
target, before the target is passed to the generator, and that could
inspect the target and modify its properties and override variables in its
scope. In my case, I'd write this function so that each target would get
its own "CMAKE_C_COMPILE_OBJECT" or "CMAKE_C_LINK_EXECUTABLE", fully
expanded, without any generator expressions nor rule variables.
Does cmake already offer any hook like it? A cursory examination of the
source code doesn't seem to indicate so (btw: the code is clear and easy to
follow, so I don't have high hopes...). And if not, would it be a
worthwhile addition? It'd obviate the need for both generator expressions
and rule variables - personally I think that they are half-baked hacks,
with rule variables being even more limited and really only meshing with
gnu-like tools. I imagine that to speed things up, a pre-filter could be
applied - say that "my_function" should be invoked for all executable
targets, for C language and foocc compiler:
"CMAKE_ADD_FILTER(add_executable, my_function, LANGUAGE=c
COMPILER_ID=foocc)". For every invocation of `add_executable`, CMAKE would
check if the filters match, and if so it'd invoke the function, which then
could modify target properties and variables visible to the generator (i.e.
it could regenerate CMAKE_C_COMPILE_OBJECT).
Another approach could be to have said function act like COMPILER_LAUNCHER,
and we'd then have something like:
"CMAKE_ADD_COMPILER_LAUNCHER(my_function, LANGUAGE=c, COMPILER_ID=foocc,
TYPE=executable)"
Cheers, Kuba Ober
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
