Re: [CMake] Fake dependencies of executables to static libs
Hello Francis, On Thu, Apr 4, 2019 at 6:49 PM Francis Giraldeau < francis.girald...@gmail.com> wrote: > Maybe the problem is related to Cotire? I see that it is used in > mredis. I tried to reproduce the issue, and it behaves correctly. You > might want to try to reproduce using the following minimal repo: Alas, I can safely rule out cotire. I only included it a few months ago and the issue was already ages old by then. Also, it is optional and some use it, some don't. What comes to mind now as a possible explanation, as I just a minute ago noticed is what button to press. Perhaps it is a Visual Studio thing. When I, for example, work on the mredis lib I do that test driven using the various testing tools. Now in order to do this, I generally set the testing suite as "StartUp Project", as VC calls the currently active target. Then I press 'build target' (as opposed to 'build solution') or simply just "Debug", the green play button to run a test. When the source to the testing suite is changed, both operations cause build and link of the tool and then run it as they should. When I change a source in the lib they do not. However, when I press 'build solution', all dependent binaries do seem to be relinked. I just tested this. It's simply that I normally don't do this for low level testing builds because I don't need to build the whole solution just for one test. Interesting. I never noticed that. May this shed some light on the issue? Cheers, Stephan -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Fake dependencies of executables to static libs
Maybe the problem is related to Cotire? I see that it is used in mredis. I tried to reproduce the issue, and it behaves correctly. You might want to try to reproduce using the following minimal repo: https://gitlab.com/fgiraldeau/cotire-demo Francis Le jeu. 4 avr. 2019 à 03:17, Stephan Menzel a écrit : > > Hello Paul and Shoaib, > > I'm answering to the both of you. > > On Thu, Apr 4, 2019 at 7:13 AM Paul Smith wrote: >> >> Now although I don't understand the exact reasons behind that, over the >> years came to accept that depedencies between the lib and the resulting >> artifcats are not working. Like, when I change a source file to mylib, the >> lib will get rebuilt but the resulting test tool won't be relinked. >> >> >> This is definitely not true, as stated. I feel like I must be missing >> something about your stated environment because this is really basic, >> fundamental CMake behavior, which has always worked. > > > First of all, thanks for the information. I was not aware that this is > supposed to work. As I said, I've been struggling with this issue for years > now and I recall posting this to this list too but the only explanation I > ever found was something along the lines of "dependencies would exist to > objects inside the lib, not the lib as it is a static archive". But I'm > pulling this out of my 'memory' now, nothing to rely on. All I remembered is > that this is not supposed to work. But now you say the opposite, which gives > me a little hope. > >> >> If in your environment you are not seeing executables rebuilt when libraries >> they depend on are updated, then there is something more complex about it >> than the straightforward case you provide above. > > > I have seen this in two major projects I have been working on in the last > years. Both were Linux and Windows, MSVC 14 and 15 on Windows. And I do > believe the issue is on Windows. I am not sure which CMake version I have > seen this first on. Let's make it 3.10.x cause this was when I posted this to > the list last time but by then I was already having the problem for a longer > time. > > Today I am using CMake version 3.13.2 but I have been using many different > versions up to this one. > > Both projects were large but not huge, having about 10 - 15 targets plus > additional 50 unit tests plus another 5 test tools. Unfortunately I cannot > publish the projects themselves but one that I currently see the issue with > is mredis (https://github.com/MrMoose/mredis) which, among others, is pulled > in as a submodule. C++ only, some protobuf code generation is involved but > not in the targets that I'm talking about. > > Since the issue has been persisting for so long now, the projects and > CMakeLists have substantially changed many times, so it's really hard to come > up with something I would consider 'strange' or 'different' as a possible > explanation. > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Fake dependencies of executables to static libs
Hello Paul and Shoaib, I'm answering to the both of you. On Thu, Apr 4, 2019 at 7:13 AM Paul Smith wrote: > Now although I don't understand the exact reasons behind that, over the > years came to accept that depedencies between the lib and the resulting > artifcats are not working. Like, when I change a source file to mylib, the > lib will get rebuilt but the resulting test tool won't be relinked. > > > This is definitely not true, as stated. I feel like I must be missing > something about your stated environment because this is really basic, > fundamental CMake behavior, which has always worked. > First of all, thanks for the information. I was not aware that this is supposed to work. As I said, I've been struggling with this issue for years now and I recall posting this to this list too but the only explanation I ever found was something along the lines of "dependencies would exist to objects inside the lib, not the lib as it is a static archive". But I'm pulling this out of my 'memory' now, nothing to rely on. All I remembered is that this is not supposed to work. But now you say the opposite, which gives me a little hope. > If in your environment you are not seeing executables rebuilt when > libraries they depend on are updated, then there is something more complex > about it than the straightforward case you provide above. > I have seen this in two major projects I have been working on in the last years. Both were Linux and Windows, MSVC 14 and 15 on Windows. And I do believe the issue is on Windows. I am not sure which CMake version I have seen this first on. Let's make it 3.10.x cause this was when I posted this to the list last time but by then I was already having the problem for a longer time. Today I am using CMake version 3.13.2 but I have been using many different versions up to this one. Both projects were large but not huge, having about 10 - 15 targets plus additional 50 unit tests plus another 5 test tools. Unfortunately I cannot publish the projects themselves but one that I currently see the issue with is mredis (https://github.com/MrMoose/mredis) which, among others, is pulled in as a submodule. C++ only, some protobuf code generation is involved but not in the targets that I'm talking about. Since the issue has been persisting for so long now, the projects and CMakeLists have substantially changed many times, so it's really hard to come up with something I would consider 'strange' or 'different' as a possible explanation. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Fake dependencies of executables to static libs
On Thu, 2019-04-04 at 06:56 +0100, Stephan Menzel wrote: > add_library(mylib STATIC mysources) > add_excecutable(mytest mytest.cpp) > target_link_libraries(mytest mylib) > > Now although I don't understand the exact reasons behind that, over > the years came to accept that depedencies between the lib and the > resulting artifcats are not working. Like, when I change a source > file to mylib, the lib will get rebuilt but the resulting test tool > won't be relinked. This is definitely not true, as stated. I feel like I must be missing something about your stated environment because this is really basic, fundamental CMake behavior, which has always worked. If in your environment you are not seeing executables rebuilt when libraries they depend on are updated, then there is something more complex about it than the straightforward case you provide above. Rather than try to find a workaround it would be better to find the root cause. If you can provide a sample setup that behaves as you describe perhaps we could explain what's happening. Also please let us know which version of CMake you're using. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Fake dependencies of executables to static libs
Could you share more details about your CMake version, the platform you're on,
and the generator you're using?
I haven't run into this issue (and it's pretty counter to how I'd expect CMake
to behave). The only similar setting I'm aware of is
https://cmake.org/cmake/help/latest/prop_tgt/LINK_DEPENDS_NO_SHARED.html, but
that's for shared libraries, not static libraries.
As an experiment, I used the following CMakeLists.txt:
cmake_minimum_required(VERSION 3.4.3)
project(cmtest C)
add_library(mylib STATIC source.c)
add_executable(mytest test.c)
target_link_libraries(mytest mylib)
I'm running CMake 3.12.3 on macOS 10.14.1, and I tried both the Makefile and
Ninja generators. With both, any modification to source.c causes mytest to be
rebuilt, as I'd expect (and unlike what you're seeing).
If you're able to provide a small self-contained CMakeLists.txt that reproduces
the issue, that'd also be helpful for debugging.
Thanks,
Shoaib Meenai
From: CMake on behalf of Stephan Menzel
Date: Wednesday, April 3, 2019 at 10:56 PM
To: cmake
Subject: Re: [CMake] Fake dependencies of executables to static libs
Hello again,
no ideas so far? Perhaps I can elaborate. I have tried multiple approaches over
the years. Most recently adding the source files of the lib explicitly as
dependencies to the created executable.
Like this:
set (SRC
MySource1.cpp
MySource2.cpp
)
set (HDR
MySource1.hpp
MySource2.hpp
)
add_library(mylib STATIC ${SRC} ${HDR})
add_executable(myexecutable main.cpp)
target_link_libraries(myexecutable
mylib
)
add_dependencies(myexecutable ${SRC})
This leads to errors though:
CMake Error at CMakeLists.txt:79 (add_dependencies):
The dependency target "MySource1.cpp" of target "myexecutable" does not exist.
Obviously, I cannot simply state a filename and treat it as a target. I have
however seen something similar been done when it comes to code generation. I
have also seen explicit subtargets for files being created and then depend on
those subtargets. But I don't really know how to do this the right way.
Alternatively I would also appreciate input about what exactly the problem with
depending on static libs is. The explanations I remember from my last efforts
to tackle this weren't really satisfying. If I understood in more detail why
this isn't possible I may be able to better craft a workaround on my own.
Really, anything helps. I am losing time and nerve every day because of this
and have been paying this price for so many years that I think it really is
time to come up with a solution.
Cheers,
Stephan
On Tue, Apr 2, 2019 at 5:54 AM Stephan Menzel
mailto:stephan.men...@gmail.com>> wrote:
Hello List,
In our projects we are using static linkage on Windows (MSVC) and Linux. A
number of static libs are created and then linked into executables, such as
unit tests, tools and the actual production artifacts. Very normal stuff.
add_library(mylib STATIC mysources)
add_excecutable(mytest mytest.cpp)
target_link_libraries(mytest mylib)
Now although I don't understand the exact reasons behind that, over the years
came to accept that depedencies between the lib and the resulting artifcats are
not working. Like, when I change a source file to mylib, the lib will get
rebuilt but the resulting test tool won't be relinked.
For test driven development or busy change cycles that can put quite a strain
on effort, especially on windows. My normal workflow is that I always click in
mytest.cpp, add and remove a space to mark it dirty as quickly as possible and
thereby cause the rebuild. But sometimes I forget this and this caused me quite
a bit of extra work.
My question is, can I fake that dependency somehow? Perhaps by some hack or
using some workaround?
What I want to achieve is that when I change the lib, the unit test (or any
other test tool using it) that depends on it will be relinked, even if it has
not changed.
Is that possible somehow?
Cheers,
Stephan
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Fake dependencies of executables to static libs
Hello again,
no ideas so far? Perhaps I can elaborate. I have tried multiple approaches
over the years. Most recently adding the source files of the lib explicitly
as dependencies to the created executable.
Like this:
set (SRC
MySource1.cpp
MySource2.cpp
)
set (HDR
MySource1.hpp
MySource2.hpp
)
add_library(mylib STATIC ${SRC} ${HDR})
add_executable(myexecutable main.cpp)
target_link_libraries(myexecutable
mylib
)
add_dependencies(myexecutable ${SRC})
This leads to errors though:
CMake Error at CMakeLists.txt:79 (add_dependencies):
The dependency target "MySource1.cpp" of target "myexecutable" does not
exist.
Obviously, I cannot simply state a filename and treat it as a target. I
have however seen something similar been done when it comes to code
generation. I have also seen explicit subtargets for files being created
and then depend on those subtargets. But I don't really know how to do this
the right way.
Alternatively I would also appreciate input about what exactly the problem
with depending on static libs is. The explanations I remember from my last
efforts to tackle this weren't really satisfying. If I understood in more
detail why this isn't possible I may be able to better craft a workaround
on my own.
Really, anything helps. I am losing time and nerve every day because of
this and have been paying this price for so many years that I think it
really is time to come up with a solution.
Cheers,
Stephan
On Tue, Apr 2, 2019 at 5:54 AM Stephan Menzel
wrote:
> Hello List,
>
> In our projects we are using static linkage on Windows (MSVC) and Linux. A
> number of static libs are created and then linked into executables, such as
> unit tests, tools and the actual production artifacts. Very normal stuff.
>
> add_library(mylib STATIC mysources)
> add_excecutable(mytest mytest.cpp)
> target_link_libraries(mytest mylib)
>
> Now although I don't understand the exact reasons behind that, over the
> years came to accept that depedencies between the lib and the resulting
> artifcats are not working. Like, when I change a source file to mylib, the
> lib will get rebuilt but the resulting test tool won't be relinked.
> For test driven development or busy change cycles that can put quite a
> strain on effort, especially on windows. My normal workflow is that I
> always click in mytest.cpp, add and remove a space to mark it dirty as
> quickly as possible and thereby cause the rebuild. But sometimes I forget
> this and this caused me quite a bit of extra work.
> My question is, can I fake that dependency somehow? Perhaps by some hack
> or using some workaround?
>
> What I want to achieve is that when I change the lib, the unit test (or
> any other test tool using it) that depends on it will be relinked, even if
> it has not changed.
>
> Is that possible somehow?
>
> Cheers,
> Stephan
>
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
[CMake] Fake dependencies of executables to static libs
Hello List, In our projects we are using static linkage on Windows (MSVC) and Linux. A number of static libs are created and then linked into executables, such as unit tests, tools and the actual production artifacts. Very normal stuff. add_library(mylib STATIC mysources) add_excecutable(mytest mytest.cpp) target_link_libraries(mytest mylib) Now although I don't understand the exact reasons behind that, over the years came to accept that depedencies between the lib and the resulting artifcats are not working. Like, when I change a source file to mylib, the lib will get rebuilt but the resulting test tool won't be relinked. For test driven development or busy change cycles that can put quite a strain on effort, especially on windows. My normal workflow is that I always click in mytest.cpp, add and remove a space to mark it dirty as quickly as possible and thereby cause the rebuild. But sometimes I forget this and this caused me quite a bit of extra work. My question is, can I fake that dependency somehow? Perhaps by some hack or using some workaround? What I want to achieve is that when I change the lib, the unit test (or any other test tool using it) that depends on it will be relinked, even if it has not changed. Is that possible somehow? Cheers, Stephan -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
