Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On 2019-10-13 17:15-0400 Paul Smith wrote: A unity source file can lump together N real source files from the same target (library/executable) as long as those files don't have extra source-specific flags, because all other files in a given target have the same flags. I agree that it is not at all clear whether this is the cause of the issue. So I will stop speculating about that cause even though that is a fun activity. :-) Nevertheless, it does sound like your complex test case does expose some issue with EITHER how you have implemented the build system for your complex case OR how CMake currently rejects certain builds from being included in Unity builds. So to figure out which it is, I think your next step should be to attempt to implement the simplest possible self-contained test case that illustrates the Unity exclusion issue you have found. I believe that course has already been suggested in this thread so I am seconding that motion. My own experiences with such implementations is I often cannot demonstrate the issue in a simple way, i.e., the "CMake bug" often tends to be due to an issue with my implementation of the build system for the complex case. However, when I have demonstrated with such a simple test case there really is a CMake bug, that test case normally helps CMake developers in a big way to find and fix the issue. Alan __ Alan W. Irwin Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
Hi Paul, As another reference point, I verified that -DCMAKE_UNITY_BUILD=ON works with the VTK-m ( https://gitlab.kitware.com/vtk/vtk-m ) project. I only verified using a clean CMake 3.16 build directory. On Thu, Oct 10, 2019 at 6:43 PM Paul Smith wrote: > > On Thu, 2019-10-10 at 14:57 -0400, Robert Maynard via CMake wrote: > > * The "UNITY_BUILD" target property was added to tell generators to > > batch include source files for faster compilation times. > > Are there any instructions on how to make this work? I tried this: > > cmake -G 'Unix Makefiles' -DCMAKE_UNITY_BUILD=ON . > > Then ran "make". The output showed I had just as many output .o files > as input .cpp files and that make ran one compile command per .cpp > file. > > Is there something else I need to do to enable unity builds in my cmake > files, than just give the above option? The docs imply that the above > is all that's needed. > > Cheers! > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On Sat, 2019-10-12 at 19:25 -0700, Alan W. Irwin wrote: > > Virtually all my properties are set on a per-target basis, so I assumed > > they wouldn't be an issue here. > > > > Am I misunderstanding that? > > I think the current documentation is ambiguous about this. But > certain target and directory properties can affect the flags used to > compile the source code just as much as source code properties. So I > am virtually positive the implementation has to pay attention to all > these sources of properties to decide what can/cannot be lumped into a > Unity source file. If Kyle confirms this guess, then the > documentation should be changed accordingly to remove the ambiguity > about this. I know no more than you but I'm quite sure that it's not the case that adding library/executable-level options can cause unity to be disabled. I've never heard of _ANY_ such target that didn't have at least some properties defined for it. A unity source file can lump together N real source files from the same target (library/executable) as long as those files don't have extra source-specific flags, because all other files in a given target have the same flags. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On 2019-10-12 15:40-0400 Paul Smith wrote: On Fri, 2019-10-11 at 10:17 -0400, Kyle Edwards wrote: On Fri, Oct 11, 2019 at 1:36 AM Alan W. Irwin < alan.w.irwin1...@gmail.com wrote: The source files that have COMPILE_OPTIONS, COMPILE_DEFINITIONS, COMPILE_FLAGS, or INCLUDE_DIRECTORIES will also be skipped." This is by far the most likely reason. We added this restriction because we don't want files that have different COMPILE_FLAGS etc. to be lumped together in a unity file. We decided that this was good enough as a first past, but the way forward is to intelligently group together files that have the same COMPILE_OPTIONS, COMPILE_DEFINITIONS, COMPILE_FLAGS, and INCLUDE_DIRECTORIES. I saw this in the manual, and I interpreted it to mean that if we used something like set_source_files_properties() or set_property(SOURCE) to set these properties on specific source files, then those files which are impacted by this won't be part of unity builds. That seems quite sensible to me. However, I don't do that hardly anywhere at all; maybe one or two files have an extra INCLUDE_DIRECTORIES setting or similar. Virtually all my properties are set on a per-target basis, so I assumed they wouldn't be an issue here. Am I misunderstanding that? I think the current documentation is ambiguous about this. But certain target and directory properties can affect the flags used to compile the source code just as much as source code properties. So I am virtually positive the implementation has to pay attention to all these sources of properties to decide what can/cannot be lumped into a Unity source file. If Kyle confirms this guess, then the documentation should be changed accordingly to remove the ambiguity about this. Alan __ Alan W. Irwin Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On Fri, 2019-10-11 at 10:17 -0400, Kyle Edwards wrote: > On Fri, Oct 11, 2019 at 1:36 AM Alan W. Irwin > < > alan.w.irwin1...@gmail.com > > wrote: > > The source files that have COMPILE_OPTIONS, COMPILE_DEFINITIONS, > > COMPILE_FLAGS, or INCLUDE_DIRECTORIES will also be skipped." > > This is by far the most likely reason. We added this restriction > because we don't want files that have different COMPILE_FLAGS etc. to > be lumped together in a unity file. We decided that this was good > enough as a first past, but the way forward is to intelligently group > together files that have the same COMPILE_OPTIONS, > COMPILE_DEFINITIONS, COMPILE_FLAGS, and INCLUDE_DIRECTORIES. I saw this in the manual, and I interpreted it to mean that if we used something like set_source_files_properties() or set_property(SOURCE) to set these properties on specific source files, then those files which are impacted by this won't be part of unity builds. That seems quite sensible to me. However, I don't do that hardly anywhere at all; maybe one or two files have an extra INCLUDE_DIRECTORIES setting or similar. Virtually all my properties are set on a per-target basis, so I assumed they wouldn't be an issue here. Am I misunderstanding that? It's also possible that a previous attempt to introduce unity builds to our setup via cotire is interfering with the new, standardized unity builds. I am not too familiar with the details of that but I can try to look into it. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On Fri, Oct 11, 2019 at 1:36 AM Alan W. Irwin wrote: > The source files that have COMPILE_OPTIONS, COMPILE_DEFINITIONS, > COMPILE_FLAGS, or INCLUDE_DIRECTORIES will also be skipped." This is by far the most likely reason. We added this restriction because we don't want files that have different COMPILE_FLAGS etc. to be lumped together in a unity file. We decided that this was good enough as a first past, but the way forward is to intelligently group together files that have the same COMPILE_OPTIONS, COMPILE_DEFINITIONS, COMPILE_FLAGS, and INCLUDE_DIRECTORIES. Kyle -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On 2019-10-10 18:21-0400 Paul Smith wrote: On Thu, 2019-10-10 at 14:57 -0400, Robert Maynard via CMake wrote: * The "UNITY_BUILD" target property was added to tell generators to batch include source files for faster compilation times. Are there any instructions on how to make this work? I tried this: cmake -G 'Unix Makefiles' -DCMAKE_UNITY_BUILD=ON . Then ran "make". The output showed I had just as many output .o files as input .cpp files and that make ran one compile command per .cpp file. Is there something else I need to do to enable unity builds in my cmake files, than just give the above option? The docs imply that the above is all that's needed. Hi Paul: I have never tried unity builds, but your question did inspire me to look at [the unity build documentation](https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html) which if you follow the links to other related documentation does imply what you have done above is correct and should by default group 8 source-code files into a lump to be compiled together. Since you are not getting that behaviour, I am wondering if one or more of the documented reasons for skipping unity builds is making a difference in your case. Those reasons are the following: "The source files marked by GENERATED will be skipped from unity build. This applies also for the source files marked with SKIP_UNITY_BUILD_INCLUSION. The source files that have COMPILE_OPTIONS, COMPILE_DEFINITIONS, COMPILE_FLAGS, or INCLUDE_DIRECTORIES will also be skipped." Good luck, and I hope you will keep the list informed of any further progress you make with your unity build experiments. Alan __ Alan W. Irwin Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __ Linux-powered Science __ -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
[CMake] Unity builds (was: Re: [ANNOUNCE] CMake 3.16.0-rc1 is ready for testing)
On Thu, 2019-10-10 at 14:57 -0400, Robert Maynard via CMake wrote: > * The "UNITY_BUILD" target property was added to tell generators to > batch include source files for faster compilation times. Are there any instructions on how to make this work? I tried this: cmake -G 'Unix Makefiles' -DCMAKE_UNITY_BUILD=ON . Then ran "make". The output showed I had just as many output .o files as input .cpp files and that make ran one compile command per .cpp file. Is there something else I need to do to enable unity builds in my cmake files, than just give the above option? The docs imply that the above is all that's needed. Cheers! -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
