Re: [cmake-developers] Using a custom DSL compiler in CMake
On Tue, Jun 21, 2016 at 16:05:47 -0400, Brad King wrote: > On 06/21/2016 02:18 PM, Chris Bieneman wrote: > > Our DSL compiler can generate .d files, but hooking that up to > > CMake is a harder problem. > > With some work it may be possible to teach our Ninja generator how > to consume .d files in custom commands. Perhaps such support could > even be extended to the Makefile generators instead of using our > approximate scanning implementation. Here's the relevant issue for Ninja: https://gitlab.kitware.com/cmake/cmake/issues/15479 --Ben -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Using a custom DSL compiler in CMake
Brad, Thanks for the detailed response. This all makes a lot of sense. I’ll have to stew on this a bit and see where it leads me. In general it sounds to me like there is no easy solution here. Thanks, -Chris > On Jun 21, 2016, at 1:05 PM, Brad King wrote: > > On 06/21/2016 02:18 PM, Chris Bieneman wrote: >> Our current solution to work with this tool in CMake > > For reference, a quick look at LLVM's `cmake/modules/TableGen.cmake` > indicates that the current solution is based on add_custom_command. > >> The harder part is that our DSL supports C-like includes. > > The add_custom_command signature has an IMPLICIT_DEPENDS option > for this, but it only works for Makefile generators. > >> Our DSL compiler can generate .d files, but hooking that up to >> CMake is a harder problem. > > With some work it may be possible to teach our Ninja generator how > to consume .d files in custom commands. Perhaps such support could > even be extended to the Makefile generators instead of using our > approximate scanning implementation. > > In general implicit dependencies cannot be cleanly supported by > the Visual Studio or Xcode generators. Those environments provide > no hooks for custom build-time dependency scanning. At best one > can have the custom command produce a file as a side effect that > can be consumed by CMake on the next run. This would work for > all generators but causes lots of CMake re-runs. > >> My thought was to try and treat TableGen as a language. >> There are some complications with that because we don’t >> actually have a compiler for it at configuration time. > > I don't think that is a good approach both because we won't > be able to enable the language without the compiler already > existing and because custom languages do not work well in > the IDE generators (and will have the same dependency scanning > problems as above). See above approaches instead. > > -Brad > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Using a custom DSL compiler in CMake
On 06/21/2016 02:18 PM, Chris Bieneman wrote: > Our current solution to work with this tool in CMake For reference, a quick look at LLVM's `cmake/modules/TableGen.cmake` indicates that the current solution is based on add_custom_command. > The harder part is that our DSL supports C-like includes. The add_custom_command signature has an IMPLICIT_DEPENDS option for this, but it only works for Makefile generators. > Our DSL compiler can generate .d files, but hooking that up to > CMake is a harder problem. With some work it may be possible to teach our Ninja generator how to consume .d files in custom commands. Perhaps such support could even be extended to the Makefile generators instead of using our approximate scanning implementation. In general implicit dependencies cannot be cleanly supported by the Visual Studio or Xcode generators. Those environments provide no hooks for custom build-time dependency scanning. At best one can have the custom command produce a file as a side effect that can be consumed by CMake on the next run. This would work for all generators but causes lots of CMake re-runs. > My thought was to try and treat TableGen as a language. > There are some complications with that because we don’t > actually have a compiler for it at configuration time. I don't think that is a good approach both because we won't be able to enable the language without the compiler already existing and because custom languages do not work well in the IDE generators (and will have the same dependency scanning problems as above). See above approaches instead. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] Using a custom DSL compiler in CMake
On 21/06/2016 18:18, Chris Bieneman wrote: Hello cmake-developers, I’m trying to find a solution to a long running problem in our build. In LLVM we have a domain specific language named TableGen that we use to generate header files used throughout the project. Our current solution to work with this tool in CMake is pretty terrible for a number of reasons. The biggest problem is that we don’t have good dependency handling around the inputs and outputs from the DSL. Some of this is caused by our CMake targets not being granular (which I know how to fix). The harder part is that our DSL supports C-like includes. Our DSL compiler can generate .d files, but hooking that up to CMake is a harder problem. My thought was to try and treat TableGen as a language. There are some complications with that because we don’t actually have a compiler for it at configuration time. Any thoughts and guidance would be greatly appreciated. I feel as if I’m about to fall into a deep and scary rabbit hole. In one of the projects I maintain at work, we generate a lot of different C++ sources and headers using a custom generator tool. It consumes an XML schema, and then processes it to turn that formal data model into a set of C++ classes, enums and serialisation functions, using a series of templates. I needed to make this work seamlessly with CMake, and so I wrote a set of functions to introspect the dependencies and then generate all the necessary targets with appropriate dependencies. While I can't claim this is going to be the best use of CMake in the world, it's quite functional and does its job well. https://github.com/openmicroscopy/bioformats/blob/develop/cpp/cmake/XsdFu.cmake This contains helper macros to do the introspection. We basically run the generator twice for each "target" using --print-generated and --print-depends options. These cause the corresponding files to be dumped to stdout. We then use these file lists as the OUTPUT and DEPENDS lists for add_custom_command. NB. This tool outputs multiple source files at once which all go into the OUTPUT. You can then see those macros in use here: https://github.com/openmicroscopy/bioformats/blob/develop/cpp/lib/ome/xml/CMakeLists.txt#L42 where we then add those outputs to a regular library target. While I think this is too custom to be of direct use, it might be useful for inspiration. The essential point is to introspect the dependencies and outputs as you run cmake (which can emit the includes you mentioned as dependencies), and then use that information to generate the appropriate custom commands. Everything will then be built at build time, in parallel. How you do the introspection is entirely up to you. I added support to the generator itself since it already had the relevant information (it's a python tool) and was quick and easy to add. You could also parse the files directly with cmake, or with a custom script in any language you like. Hope that's at least provided some ideas! Regards, Roger -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
[cmake-developers] Using a custom DSL compiler in CMake
Hello cmake-developers, I’m trying to find a solution to a long running problem in our build. In LLVM we have a domain specific language named TableGen that we use to generate header files used throughout the project. Our current solution to work with this tool in CMake is pretty terrible for a number of reasons. The biggest problem is that we don’t have good dependency handling around the inputs and outputs from the DSL. Some of this is caused by our CMake targets not being granular (which I know how to fix). The harder part is that our DSL supports C-like includes. Our DSL compiler can generate .d files, but hooking that up to CMake is a harder problem. My thought was to try and treat TableGen as a language. There are some complications with that because we don’t actually have a compiler for it at configuration time. Any thoughts and guidance would be greatly appreciated. I feel as if I’m about to fall into a deep and scary rabbit hole. Thanks, -Chris -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
