[CMake] Example of using Visual Studio for Nightly Dashboard
Can anyone point my to a project, including scripts etc, that shows how to use Visual Studio as the build mechanism for a Visual Fortran project? My Problem is the following: I am generating a Visual Studio with Intel Fortran compiler (v16.x) from CMake but when I load the resulting sln file into Visual Studio and compile the only thing that gets compiled are a few *.c files in the project and NOT the 100 actual *.f90 files. If I use NMake from the command line then I can actually invoke "nmake" and have my project properly compile. I am not sure what is going on or where the issue is. I also tried to use NMake from the command line but I am have no idea which environment and/or cmake variables I need to set in order for my batch file and .cmake file combination to actually get the Intel Fortran compiler set properly. I tried using "call ... " with the proper path to the Intel Fortran .bat file that lays out all the variables but it seems to just prematurely exit the command prompt and nothing gets configured or compiled. I can successfully configure a .sln file for my project during a "nightly" build but then the .sln does not actually compile any of the fortran source files. If anyone has any insight into these issues I would greatly appreciate the help. Thanks -- Michael A. Jackson BlueQuartz Software, LLC [e]: mike.jack...@bluequartz.net -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Private dependencies of static libraries exported as targets
On 2016-08-17 at 05:19 -0500, Nicholas Braden wrote: > Huh, this is weird. It seems to be an issue with the export/import > mechanism. If you make a project where everything is in the same > CMakeLists.txt and there is no install step, there is no include > directory pollution. See my attached project for example. On my > system > I run this to build: > > mkdir build && cd build && cmake -G "MinGW Makefiles" .. && cmake > --build . -- VERBOSE=1 > > When it builds main: > > C:\MinGW\bin\g++.exe@CMakeFiles/main.dir/includes_CXX.rsp > -std=gnu++14 -o CMakeFiles\main.dir\main.cpp.obj -c > C:\Users\LB\Code\cmake-private-static-dependencies\main.cpp > > The entirety of the includes_CXX.rsp file: > > -IC:/Users/LB/Code/cmake-private-static-dependencies/c > > The project you provided definitely has the include directory > pollution problem - the client includes_CXX.rsp file contains an > -isystem directive for bar. It looks like the generated > FooTargets-noconfig.cmake contains this line (setting properties on > foo): > > IMPORTED_LINK_INTERFACE_LIBRARIES_NOCONFIG "bar" > > I'm not fully sure but I think this is wrong because it is saying > that > bar is a public dependency. This seems to be an oversight in CMake, > as > there is no 'private' alternative of the above variable for imported > targets - that is, there is no IMPORTED_LINK_LIBRARIES_NOCONFIG, thus > there is no way for CMake to provide special treatment as in my > example. It seems CMake literally converts private dependencies of > static libraries to public dependencies only when using the export > functionality. > > ...and that's as much as I can figure right now. Can you confirm that > my attached example doesn't exhibit the problem? If it doesn't have > the include directory pollution problem, I'd be inclined to say it's > a > bug with the export/import functionality. Indeed, your sample project does not exhibit include path bloat. So this really looks like a limitation of CMake import/export mechanism (I say limitation, not bug, because there are simply no target properties detailed enough to allow this behavior). There is a property IMPORTED_LINK_DEPENDENT_LIBRARIES_ which is _almost_ what we want, but just for shared libraries. -- Ivan Shapovalov / intelfx / signature.asc Description: This is a digitally signed message part -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] dependencies between external projects (not DEPENDS)
Usually there would be variables like A_ROOT which allow the FindA.cmake module being used by B to correctly find everything. If there isn't, you'd have to write your own FindA.cmake module in either case. If you can't modify B, you could use the patch step of the external project to replace the broken FindA.cmake with your modern FindA.cmake with minimal effort while you wait for B to accept your pull request. The takeaway is, if B doesn't have a way for you to say "Look for A here first/only", that's a bug in B's FindA module. As for recursive dependencies, there is the CMakeFindDependencyMacro module, but you have to be careful about the can of worms you are opening - you'd want to only find versions which are binary compatible and also deal with cases of the wrong version of the package having already been found. Sometimes it's easier to just tell the client to find the dependencies for you, rather than trying to do it for them and potentially getting it wrong. I honestly don't know what the best approach is here. On Mon, Aug 15, 2016 at 10:56 AM, Neil Carlson wrote: > I'm struggling with how to handle dependencies between external projects in > a > superbuild. The issue is different than simply ensuring that one gets built > before > another using the DEPENDS keyword -- that's trivial. > > Suppose I have two external libraries A and B, where B depends on A. It is > not > always possible to correctly define variables like A_LIBRARIES that feed > into the > externalproject_add for B that anticipate what will come out of the A > configure/build. > There are sometimes additional link libraries that only come to light after > A is > actually configured. > > How do people handle these situations? I'm thinking of a multi-pass > approach > where one runs cmake/make twice. The first installs A skipping the > dependent B. > The second builds B (skipping a found A) using correctly set variables from > a > FindA module. > > Incidentally, is it kosher to have a Find module invoke a find_package? > There are > examples in of this in standard Find modules. This creates problems in the > superbuild context, where say A is found, but rejected by the superbuild > because > of missing features, for example, but FindB finds A and uses it. > > Thanks for your advice. > > -Neil > > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] [cmake-developers] Need ideas/opinions on third party library management
On 17-Aug-16 04:29, Florent Castelli wrote: The Boost source download is cached outside of the build directory in a unique folder. So it’s effectively only done once for all platforms and then reused. This is true for local machines and for custom build servers like your personal Jenkins. For Travis/AppVeyor you have to create root folder with 3rd parties from scratch for each build (at least for free public accounts). Yes. If you're using a free shared service, that's not something you can count on. If you host your CI, you can do neat tricks like this, use ccache or other similar techs. What if you don't need "tricks" and make everything plain and simple? You’ll also have symbols and the sources available for debugging properly, and they’re not always available with a binary distribution. Just to clarify: with Hunter you're creating binaries from sources, so everything you install with `cmake --build _builds --target install` (or `./b2 install` for Boost) is available. So you build each dependency separately, install them, and then use them in your top level dependency. No, in shared root folder. That works, but you have the extra overhead of the instrumentation for the whole build. Even a no-op build will have to recurse in the build folder for each dependency and run again, which is slow. This is why I prefer Ninja to the Makefile generator: it has a global view of everything and you get a quick no-op build. Not sure I understand that. What is the overhead? Of course building from source is not an option for such monsters like Qt or OpenCV. Ready-to-use binaries is something critical for real life applications. There is no way to test everything on Travis/AppVeyor without this feature. Well, you don’t have to use Travis or AppVeyor. It's the most powerful and easy to setup services I know. If you know any better solutions please share. Well, you don't have a full control on the environment, so I wouldn't say it's the most powerful. Please share your solution. I have worked with Jenkins before and will not say that it's something easy customizable. Shareable folder - yes, good, but other stuff is a complete pain. You can setup Travis in a seconds, add AppVeyor and you got Linux, OSX, Windows testing. How much time will take to start and tune Jenkins master and connect several slaves with different OSes on them? Then add bunch of projects and tune them, then create dev branches for testing and so on. Convenient for sure. It probably fits smaller structures very well. Bigger companies have the resources to host their own service most of the time and requirements that force them to do so. Those are probably the ones that will have the manpower to handle a super-build type build system. Why do not have both? Hunter can share downloads/root/cache on local folder for such CIs like Jenkins/custom so you don't need to rebuild everything. At the same time binaries from server can be downloaded for "build-from-scratch" environment like Travis. Anyway what about users? So you think it's okay to spend several hours of building from sources to just to run simple example of your product? Spotify isn’t at the same scale as most projects hosted there and we have different requirements and resources. Admittedly, Spotify doesn’t use Qt anymore, so this isn’t a problem for us. It's not about Qt, it's about expandability. Use 20 of smaller libraries and you will have quite the same issues. As I said before, if I have build scripts for 20 small libraries and I want to update a build flag affecting the ABI, I don't have to do anything but just change the flag once. In your case, you'll have to tweak the 20 build scripts for each library to reflect it. The dependencies are only intermediate products you use for the final one. I don't want to deal with them constantly. It's not true, I don't need to tweak 20 scripts, I just need to tweak one toolchain file. Note that by integrating everything in the same project, you also have proper dependencies and you will only build what you need. You may save some time by doing that. And caching is important, when done in the right way! With Hunter you're installing only what you need too, it does respect options like FOO_WITH_OPENSSL or FOO_WITH_QT, it download only binaries for toolchain you're currently working on, etc. Don't want to make a discussion too broad. You said that it's hard to manage binaries for a lot of configuration, I'm saying that it's possible and is very handy. I'm not saying it's impossible. I'm saying the overhead of managing binaries is just a burden we shouldn't have to accept in the C/C++ world. If you can build build everything from source all the time in a super-build, why wouldn't you do it? Because it's not practical. I have such experience with Gentoo, I prefer do something useful instead of watching on "emerge world" progress. Super-build doesn't scale, what if
Re: [CMake] [cmake-developers] Need ideas/opinions on third party library management
On 17-Aug-16 08:36, Elizabeth A. Fischer wrote: > > I don't think CMake is the best place to do it, > Can you provide any details? I personally think that CMake is a > natural and the only place where it should be done. The most important reason here is because there are combinatorially many versions of a package you COULD install, depending on what versions of its dependencies you link with, and CMake provides nothing to address that issue. CMake provides an abstraction. "Slots" that you need to fill: if(FOO_WITH_TESTS) # need to have GTest installed find_package(GTest) endif() if(FOO_WITH_OPENSSL) # need to have OpenSSL installed find_package(OpenSSL) endif() And it should drive package manager. At least I find that approach natural and convenient, see no problems with it. You can have as much combinations of versions/options/dependencies as you need: * https://docs.hunter.sh/en/latest/overview/customization/hunter-id.html * https://docs.hunter.sh/en/latest/overview/customization/config-id.html See here for an overview of how Spack addresses the combinatorial versioning issue (which no other auto-builder does, to the best of my knowledge): http://hpcugent.github.io/easybuild/files/SC14_BoF_Spack.pdf That's what I was talking about. I think that there is no need to introduce new funky syntax like "spack install mpileaks@1.1.2 %gcc@4.7.3 +debug". We already have CMAKE_CXX_COMPILER and CMAKE_BUILD_TYPE/CMAKE_CONFIGURATION_TYPES. Version can be set by CMake options too. Effectively you can do: option(FOO_STABLE_BUILD "Stable build" ON) option(FOO_EXPERIMENTAL_BUILD "Experimental build" OFF) if(APPLE AND IOS AND FOO_STABLE_BUILD) hunter_config(BooPackage VERSION "1.0") endif() if(WIN32 AND FOO_EXPERIMENTAL_BUILD) hunter_config(BooPackage VERSION "2.0-beta" CMAKE_ARGS BOO_NEW_STUFF=YES) endif() Once you've built something, it's nice to be able to re-use it. If I have a top-level CMake project that automatically builds three dependencies, will other projects be able to make use of those dependencies I've built? Yes, libraries should be installed to the shared root, not to local folder: * https://docs.hunter.sh/en/latest/overview/shareable.html Or do they become private? No. Though you can make it private by setting CMake variable. It will use separate directory just like virtualenv do. If libraries cannot be shared between applications, you will get a LOT of library bloat, especially among the low-level libraries that get repeated numerous times. Admittedly, this may not be such an issue in some environments where people are really only focused on building one thing. If you make a project, you might see it as a "top-level" project. But someone else might want to build something bigger on top of your work. You can never assume that "this package is top-level and no one will ever depend on it." No issue here, see notes above. Another obvious problem with using CMake for everything is that not all packages come with CMake builds; most do not, in fact. Even if we CAN re-write all the buils into CMake, that is a lot of extra effort. As Florent has discovered, upstream authors do not always see a CMake build in a favorable light, and these re-worked builds are not always as functional as the original. Moreover... writing a Spack recipe is an order of magnitude easier than writing a CMake build. Usually, it's just a matter of calling `configure` or `cmake` with the right options. Again, converting to CMake is a best option, but not the only possible one. E.g. OpenSSL, Boost, autotool-based package like X11 can be used as is: * https://github.com/ruslo/hunter/blob/b4c370e32798cc3da74c37e4156c3bfc77add379/cmake/projects/Boost/hunter.cmake#L21 * https://github.com/ruslo/hunter/blob/b4c370e32798cc3da74c37e4156c3bfc77add379/cmake/projects/OpenSSL/hunter.cmake#L17 * https://github.com/ruslo/hunter/blob/b4c370e32798cc3da74c37e4156c3bfc77add379/cmake/projects/x11/hunter.cmake#L20 Although we can maybe imagine a world in which everyone eventually abandons Autotools for CMake, it is still not realistic to expect that Python, Haskell or Java programs will ever come with CMake builds. This would be OK if each language exited in its own silo. But they don't. Python packages (built with setuptools) routinely depend on C-based packages (built with Autotools or CMake). By being agnostic to the build system, auto-builders (like Spack, Macports, HomeBrew, etc) are able to make packages work together, regardless of the build system chosen for each one. That's exactly what Hunter do, but using CMake as a driver. In the sense that CMake is a Turing-complete language, there's no fundamental reason you CAN'T write a meta-builder in CMake. But gosh... the language sure is arcane (but still better than Autotools by a long shot). I like to imagine that if CMake were starting off today, it would be written in Python. Language is a
Re: [CMake] Private dependencies of static libraries exported as targets
On 2016-08-16 at 20:14 -0500, Nicholas Braden wrote: > Ah, I misunderstood what you were asking about. It would be pretty > weird if CMake didn't know that static libraries always need all > their > dependencies linked regardless of privacy, but I agree it should at > least be mentioned somewhere. My bad. > > As for include path bloat, I cannot replicate this in my test project > - CMake will link all the dependencies as required but will NOT > violate "PRIVATE" for other things like include directories. Could > you > give an example where you are seeing a static library's private > dependency's include path being added when linking the static > library? Please take a look at https://github.com/intelfx/cmake-demo. This is a sample dependency chain of two libraries and one executable (client -> foo -> bar), both libraries are found and linked as IMPORTED targets. The client is built with VERBOSE=1 -- notice the compiler command lines. On my system it says (pretty self-explanatory): /usr/bin/c++-isystem /home/intelfx/devel/__auxiliary/experiments/cmake-static-transitive/prefix/foo/include -isystem /home/intelfx/devel/__auxiliary/experiments/cmake-static-transitive/prefix/bar/include -o CMakeFiles/client.dir/src/ main.cpp.o -c /home/intelfx/devel/__auxiliary/experiments/cmake-static-transitive/client/src/main.cpp -- Ivan Shapovalov / intelfx / signature.asc Description: This is a digitally signed message part -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
