Re: [CMake] Newbie question: Static linking
On 23 May 2011 00:53, Hendrik Sattler p...@hendrik-sattler.de wrote:
Am Montag, 23. Mai 2011, 01:36:14 schrieb Sanatan Rai:
After cmake, and make all, the libraries build as static archives, ie I get
liblib1.a, liblib2.a, libhelper1.a, libhelper2.a and executable myProj in
the appropriate locations. However, the executable myProj does not appear
to have linked statically to libhelper1.a and libhelper2.a. These
libraries contain global initialisers for certain objects: which doesn't
happen when I run the executable.
Most likely, the linker just drops those global initialisers when linking
statically. See the linker options in man ld to prevent that or give the
library an initialisation method.
I don't understand: so why does it work at all? I guess what I
don't understand is that
I don't see symbols from helper1 and helper2 in the final executable,
when I build them
as separate libraries, but I do when I express the dependencies thus:
add_executable(myTarget target.cpp helpers/helper1.cpp helpers/helper2.cpp)
The `global initialisation' stuff is just the following pattern:
namespace {
helper1 *helper1Creator()
{
return (new helper1());
}
const bool helper1Registered = factory::instance().registerhelper
(helper1, helper1Creator);
}
So when I put the helpers in a separate library, these lines are not
called: which
suggests that the library was not loaded in memory. On the other hand,
when I build
them with the executable, the lines are called and everything works as expected.
I am not sure what else I can tell the linker. The libraries are
static so should be linked
statically. As far as I can tell, somehow the two helper libraries are
not being linked
at all.
Sorry, but I am quite confused.
--Sanatan
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
Zitat von Sanatan Rai sana...@gmail.com:
The `global initialisation' stuff is just the following pattern:
namespace {
helper1 *helper1Creator()
{
return (new helper1());
}
const bool helper1Registered = factory::instance().registerhelper
(helper1, helper1Creator);
}
So when I put the helpers in a separate library, these lines are not
called.
Then you don't understand the implications of static libraries, yet.
Use the following from man ld:
--whole-archive
For each archive mentioned on the command line after the
--whole-archive option, include every object file in the archive in
the link, rather than searching the archive for the required object
files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting
shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know
about this option, so you have to use -Wl,-whole-archive. Second,
don't forget to use -Wl,-no-whole-archive after your list of
archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
HS
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 10:23 AM, Hendrik Sattler wrote:
Zitat von Sanatan Rai sana...@gmail.com:
The `global initialisation' stuff is just the following pattern:
namespace {
helper1 *helper1Creator()
{
return (new helper1());
}
const bool helper1Registered = factory::instance().registerhelper
(helper1, helper1Creator);
}
So when I put the helpers in a separate library, these lines are not
called.
Then you don't understand the implications of static libraries, yet.
Use the following from man ld:
--whole-archive
Foreach archive mentioned on the command line after
the
--whole-archive option, include every object file in the archive in
the link, rather than searching the archive for the required object
files. This is normally used to turn an archive file into a shared
library, forcing every object to be included inthe
resulting
shared library. This option may be used more than once.
Twonotes when using this option from gcc: First, gcc doesn't
know
about this option, so you have to use -Wl,-whole-archive. Second,
don't forget to use -Wl,-no-whole-archive after your
list of
archives, because gcc will add its own listof archives to
your
link and you may not want this flag to affect those as well.
HS
But this will only work when using GNU ld (or compatible). AFAIK the
linker on APPLE uses a different option, and MSVC doesn't have an
equivalent at all (you can only specify individual symbols that should
be linked forcibly).
The real solution is to ditch this whole idea of global, static
initialization objects. It's usually a very bad solution, leading to all
kinds of non-deterministic behavior. You'll be much better off with an
explicit registration scheme.
My 2c.
Michael
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 23 May 2011 10:18, Michael Wild them...@gmail.com wrote:
On 05/23/2011 10:23 AM, Hendrik Sattler wrote:
Zitat von Sanatan Rai sana...@gmail.com:
The `global initialisation' stuff is just the following pattern:
namespace {
helper1 *helper1Creator()
{
return (new helper1());
}
const bool helper1Registered = factory::instance().registerhelper
(helper1, helper1Creator);
}
So when I put the helpers in a separate library, these lines are not
called.
Then you don't understand the implications of static libraries, yet.
Use the following from man ld:
--whole-archive
For each archive mentioned on the command line after the
--whole-archive option, include every object file in the archive in
the link, rather than searching the archive for the required object
files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the
snipped
Thanks Hendrik, that works! For the record, I have added the following:
target_link_libraries(myTarget
-Wl,-whole-archive -L./helpers -lhelper1 -Wl,-no-whole-archive
-Wl,-whole-archive -L./helpers -lhelper2 -Wl,-no-whole-archive)
I am not happy about having to provide the search path explicitely, but hey.
But this will only work when using GNU ld (or compatible). AFAIK the
linker on APPLE uses a different option, and MSVC doesn't have an
equivalent at all (you can only specify individual symbols that should
be linked forcibly).
The real solution is to ditch this whole idea of global, static
initialization objects. It's usually a very bad solution, leading to all
kinds of non-deterministic behavior. You'll be much better off with an
explicit registration scheme.
My 2c.
Michael
I couldn't agree more. However, there does not appear to be an alternative
way of doing this registration. Briefly, here is the problem I am
trying to solve.
* I have a base class (say) myBase.
* I expect other users of my library to derive objects from myBase.
* The application `creates' the derived objects as per the pattern above,
based on which ones the user wants as per a config file.
* So: if the user specifies an object that is not built, then the app
is going to
throw an exception and abort.
* The whole point of doing this is that the application is a
`framework' for the
user `to host his derived class'.
This approach permits me to keep the application and the `framework' code
agnostic to the user's derived classes.
I'd be more than happy to find a better way of achieving the
framework's agnosticism
to the derived classes, while providing the user to derive whatever
objects he likes.
When I solved this problem on .NET I was forced to run through all the
assemblies
to find the relevant objects and then use reflection to instantiate
them. That was even
uglier.
Thanks all!
--Sanatan
--
Sanatan Rai
3, Admirals Court,
30, Horselydown Lane,
London, SE1 2LJ.
+44-20-7403-2479.
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
You might want to take a look at the Factory design pattern.
-
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jack...@bluequartz.net
BlueQuartz Software Dayton, Ohio
Sent from my mobile device.
On May 23, 2011, at 5:51, Sanatan Rai sana...@gmail.com wrote:
On 23 May 2011 10:18, Michael Wild them...@gmail.com wrote:
On 05/23/2011 10:23 AM, Hendrik Sattler wrote:
Zitat von Sanatan Rai sana...@gmail.com:
The `global initialisation' stuff is just the following pattern:
namespace {
helper1 *helper1Creator()
{
return (new helper1());
}
const bool helper1Registered = factory::instance().registerhelper
(helper1, helper1Creator);
}
So when I put the helpers in a separate library, these lines are not
called.
Then you don't understand the implications of static libraries, yet.
Use the following from man ld:
--whole-archive
Foreach archive mentioned on the command line after the
--whole-archive option, include every object file in the archive in
the link, rather than searching the archive for the required object
files. This is normally used to turn an archive file into a shared
library, forcing every object to be included inthe
snipped
Thanks Hendrik, that works! For the record, I have added the following:
target_link_libraries(myTarget
-Wl,-whole-archive -L./helpers -lhelper1 -Wl,-no-whole-archive
-Wl,-whole-archive -L./helpers -lhelper2 -Wl,-no-whole-archive)
I am not happy about having to provide the search path explicitely, but hey.
But this will only work when using GNU ld (or compatible). AFAIK the
linker on APPLE uses a different option, and MSVC doesn't have an
equivalent at all (you can only specify individual symbols that should
be linked forcibly).
The real solution is to ditch this whole idea of global, static
initialization objects. It's usually a very bad solution, leading to all
kinds of non-deterministic behavior. You'll be much better off with an
explicit registration scheme.
My 2c.
Michael
I couldn't agree more. However, there does not appear to be an alternative
way of doing this registration. Briefly, here is the problem I am
trying to solve.
* I have a base class (say) myBase.
* I expect other users of my library to derive objects from myBase.
* The application `creates' the derived objects as per the pattern above,
based on which ones the user wants as per a config file.
* So: if the user specifies an object that is not built, then the app
is going to
throw an exception and abort.
* The whole point of doing this is that the application is a
`framework' for the
user `to host his derived class'.
This approach permits me to keep the application and the `framework' code
agnostic to the user's derived classes.
I'd be more than happy to find a better way of achieving the
framework's agnosticism
to the derived classes, while providing the user to derive whatever
objects he likes.
When I solved this problem on .NET I was forced to run through all the
assemblies
to find the relevant objects and then use reflection to instantiate
them. That was even
uglier.
Thanks all!
--Sanatan
--
Sanatan Rai
3, Admirals Court,
30, Horselydown Lane,
London, SE1 2LJ.
+44-20-7403-2479.
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. Michael ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 23 May 2011 13:38, Michael Wild them...@gmail.com wrote: On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. I guess this is getting to be off topic, but indeed the anonymous namespace trick is supposed to do exactly that. I am not trying to be difficult here---just that it is not clear to me that the solution to this problem is that straightforward. When all the code files are linked in one monolithic bloc, everything works correctly. It is when one starts dividing them into individual libraries that this problem occurs. I haven't seen a solution to this problem either in books or via google. --Sanatan ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 03:25 PM, Sanatan Rai wrote: On 23 May 2011 13:38, Michael Wild them...@gmail.com wrote: On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. I guess this is getting to be off topic, but indeed the anonymous namespace trick is supposed to do exactly that. I am not trying to be difficult here---just that it is not clear to me that the solution to this problem is that straightforward. When all the code files are linked in one monolithic bloc, everything works correctly. It is when one starts dividing them into individual libraries that this problem occurs. I haven't seen a solution to this problem either in books or via google. --Sanatan The problem is, that when you link a static library to another binary (be it shared library or executable) only the *required* symbols are used, all others get discarded. Since nothing in your code actually references those global instances in the anonymous namespace (the linker doesn't care about that, BTW), they are ignored. Four solutions: 1. Only do monolithic builds. 2. Use shared libraries/DLLs 3. Use --whole-archive or similar and hack your way through MSVC (I did it once. It was ugly. Very ugly. See https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef791054a30c. Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I replaced it with an explicit registration scheme now...) 4. Use an explicit registration scheme. For sanity's sake, go with 4. Michael ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On May 23, 2011, at 10:11 AM, Michael Wild wrote: On 05/23/2011 03:25 PM, Sanatan Rai wrote: On 23 May 2011 13:38, Michael Wild them...@gmail.com wrote: On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. I guess this is getting to be off topic, but indeed the anonymous namespace trick is supposed to do exactly that. I am not trying to be difficult here---just that it is not clear to me that the solution to this problem is that straightforward. When all the code files are linked in one monolithic bloc, everything works correctly. It is when one starts dividing them into individual libraries that this problem occurs. I haven't seen a solution to this problem either in books or via google. --Sanatan The problem is, that when you link a static library to another binary (be it shared library or executable) only the *required* symbols are used, all others get discarded. Since nothing in your code actually references those global instances in the anonymous namespace (the linker doesn't care about that, BTW), they are ignored. Four solutions: 1. Only do monolithic builds. 2. Use shared libraries/DLLs 3. Use --whole-archive or similar and hack your way through MSVC (I did it once. It was ugly. Very ugly. See https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef791054a30c. Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I replaced it with an explicit registration scheme now...) 4. Use an explicit registration scheme. For sanity's sake, go with 4. Michael I use 4 in my own code and everything just works and I have the same type of setup as the original poster. I have a few of my own concrete classes and the user can create new ones. They just have to register their own new classes in addition to calling the RegisterKnowFactories() method first. This ensures everything links correctly and is not that much to ask your programmers to do. I think VTK/ITK/ParaView may also use these types of design patterns. ___ Mike Jackson www.bluequartz.net Principal Software Engineer mike.jack...@bluequartz.net BlueQuartz Software Dayton, Ohio ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 04:40 PM, aaron.mead...@thomsonreuters.com wrote: -Original Message- From: cmake-boun...@cmake.org [mailto:cmake-boun...@cmake.org] On Behalf Of Michael Wild Sent: Monday, May 23, 2011 9:12 AM To: cmake@cmake.org Subject: Re: [CMake] Newbie question: Static linking On 05/23/2011 03:25 PM, Sanatan Rai wrote: On 23 May 2011 13:38, Michael Wild them...@gmail.com wrote: On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. I guess this is getting to be off topic, but indeed the anonymous namespace trick is supposed to do exactly that. I am not trying to be difficult here---just that it is not clear to me that the solution to this problem is that straightforward. When all the code files are linked in one monolithic bloc, everything works correctly. It is when one starts dividing them into individual libraries that this problem occurs. I haven't seen a solution to this problem either in books or via google. --Sanatan The problem is, that when you link a static library to another binary (be it shared library or executable) only the *required* symbols are used, all others get discarded. Since nothing in your code actually references those global instances in the anonymous namespace (the linker doesn't care about that, BTW), they are ignored. Four solutions: 1. Only do monolithic builds. 2. Use shared libraries/DLLs 3. Use --whole-archive or similar and hack your way through MSVC (I did it once. It was ugly. Very ugly. See https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef 791054a30c. Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I replaced it with an explicit registration scheme now...) 4. Use an explicit registration scheme. For sanity's sake, go with 4. Michael Couldn't you do: 5) Add references to the global instances inside something within the same copilational unit which you know will be imported. (Such as a static reference to the global instance inside a function you know will be called.) That, effectively, is an explicit registration scheme. Michael ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
-Original Message- From: cmake-boun...@cmake.org [mailto:cmake-boun...@cmake.org] On Behalf Of Michael Wild Sent: Monday, May 23, 2011 9:12 AM To: cmake@cmake.org Subject: Re: [CMake] Newbie question: Static linking On 05/23/2011 03:25 PM, Sanatan Rai wrote: On 23 May 2011 13:38, Michael Wild them...@gmail.com wrote: On 05/23/2011 02:20 PM, Sanatan Rai wrote: On 23 May 2011 12:54, Michael Jackson mike.jack...@bluequartz.net wrote: You might want to take a look at the Factory design pattern. That's exactly what I use... --Sanatan Yes, but you are registering the concrete factories implicitly instead of explicitly, which is causing you the trouble you experience. Better have your user provide a function registering his/her classes explicitly. I guess this is getting to be off topic, but indeed the anonymous namespace trick is supposed to do exactly that. I am not trying to be difficult here---just that it is not clear to me that the solution to this problem is that straightforward. When all the code files are linked in one monolithic bloc, everything works correctly. It is when one starts dividing them into individual libraries that this problem occurs. I haven't seen a solution to this problem either in books or via google. --Sanatan The problem is, that when you link a static library to another binary (be it shared library or executable) only the *required* symbols are used, all others get discarded. Since nothing in your code actually references those global instances in the anonymous namespace (the linker doesn't care about that, BTW), they are ignored. Four solutions: 1. Only do monolithic builds. 2. Use shared libraries/DLLs 3. Use --whole-archive or similar and hack your way through MSVC (I did it once. It was ugly. Very ugly. See https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef 791054a30c. Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I replaced it with an explicit registration scheme now...) 4. Use an explicit registration scheme. For sanity's sake, go with 4. Michael Couldn't you do: 5) Add references to the global instances inside something within the same copilational unit which you know will be imported. (Such as a static reference to the global instance inside a function you know will be called.) Aaron This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters. ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 23 May 2011 15:11, Michael Wild them...@gmail.com wrote:
Yes, but you are registering the concrete factories implicitly instead
of explicitly, which is causing you the trouble you experience.
Better have your user provide a function registering his/her classes
explicitly.
I guess this is getting to be off topic, but indeed the
anonymous namespace trick is supposed to do exactly that.
snipped
libraries that this problem occurs. I haven't seen a solution to this
problem either in books or via google.
--Sanatan
The problem is, that when you link a static library to another binary
(be it shared library or executable) only the *required* symbols are
used, all others get discarded. Since nothing in your code actually
references those global instances in the anonymous namespace (the linker
doesn't care about that, BTW), they are ignored.
Four solutions:
1. Only do monolithic builds.
2. Use shared libraries/DLLs
3. Use --whole-archive or similar and hack your way through MSVC (I did
it once. It was ugly. Very ugly. See
https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef791054a30c.
Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I
replaced it with an explicit registration scheme now...)
4. Use an explicit registration scheme.
For sanity's sake, go with 4.
Ok: you've lost me here. Are you saying a trick like:
namespace {
helper1 *createHelper1() { return (new Helper1());}
const bool isRegistered =
factory::instance().registerHelper(helper1, createHelper1);
}
isn't explicit?
I could do the following: create a global object in the cpp, that
belongs to a type that would
do the registration. Would you regard this also as implicit?
The problem with either of these approaches is the same: if the code
lives in a separate library
that is loaded only when needed, then these registrations don't take
place. So the framework doesn't
know that the objects can be available.
Apologies for seeming obtuse but I don't follow what you mean by
`explicit registration' here.
--Sanatan
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 04:51 PM, Sanatan Rai wrote:
On 23 May 2011 15:11, Michael Wild them...@gmail.com wrote:
Yes, but you are registering the concrete factories implicitly instead
of explicitly, which is causing you the trouble you experience.
Better have your user provide a function registering his/her classes
explicitly.
I guess this is getting to be off topic, but indeed the
anonymous namespace trick is supposed to do exactly that.
snipped
libraries that this problem occurs. I haven't seen a solution to this
problem either in books or via google.
--Sanatan
The problem is, that when you link a static library to another binary
(be it shared library or executable) only the *required* symbols are
used, all others get discarded. Since nothing in your code actually
references those global instances in the anonymous namespace (the linker
doesn't care about that, BTW), they are ignored.
Four solutions:
1. Only do monolithic builds.
2. Use shared libraries/DLLs
3. Use --whole-archive or similar and hack your way through MSVC (I did
it once. It was ugly. Very ugly. See
https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef791054a30c.
Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I
replaced it with an explicit registration scheme now...)
4. Use an explicit registration scheme.
For sanity's sake, go with 4.
Ok: you've lost me here. Are you saying a trick like:
namespace {
helper1 *createHelper1() { return (new Helper1());}
const bool isRegistered =
factory::instance().registerHelper(helper1, createHelper1);
}
isn't explicit?
I could do the following: create a global object in the cpp, that
belongs to a type that would
do the registration. Would you regard this also as implicit?
The problem with either of these approaches is the same: if the code
lives in a separate library
that is loaded only when needed, then these registrations don't take
place. So the framework doesn't
know that the objects can be available.
Apologies for seeming obtuse but I don't follow what you mean by
`explicit registration' here.
--Sanatan
Everything that relies on static/global initialization to register
factories is an implicit scheme. An explicit scheme is where the
dependent code (e.g. the main() function) calls a function to do the
registration.
Michael
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 23 May 2011 16:00, Michael Wild them...@gmail.com wrote: Everything that relies on static/global initialization to register factories is an implicit scheme. An explicit scheme is where the dependent code (e.g. the main() function) calls a function to do the registration. Ok, got you. However, would this not imply a monolithic build? How is main to know that an object of a type belonging to a base class of interest exists in linked library? As I mentioned earlier, I did this a little more explicitly in a .NET project in the following manner: * I had a factory object, whose job it was to hold creator functions for objects of classes derived from a base class of interest. * The factory was a singleton, and had a static method that could be called. The method loaded all linked assemblies, and picked out classes that were derived from the base class. * Then it explicitly registered the class with the factory. This seems to me to be a hybrid between implicit and explicit registration. The actual mechanics as one might imagine relied heavily on .NET reflection calls. One of the ugliest bits of code I have ever written. --Sanatan ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On May 23, 2011, at 11:09 AM, Sanatan Rai wrote: On 23 May 2011 16:00, Michael Wild them...@gmail.com wrote: Everything that relies on static/global initialization to register factories is an implicit scheme. An explicit scheme is where the dependent code (e.g. the main() function) calls a function to do the registration. Ok, got you. However, would this not imply a monolithic build? How is main to know that an object of a type belonging to a base class of interest exists in linked library? As I mentioned earlier, I did this a little more explicitly in a .NET project in the following manner: * I had a factory object, whose job it was to hold creator functions for objects of classes derived from a base class of interest. * The factory was a singleton, and had a static method that could be called. The method loaded all linked assemblies, and picked out classes that were derived from the base class. * Then it explicitly registered the class with the factory. This seems to me to be a hybrid between implicit and explicit registration. The actual mechanics as one might imagine relied heavily on .NET reflection calls. One of the ugliest bits of code I have ever written. --Sanatan Take a look at http://scm.bluequartz.net/mxa/mxadatamodel/trees/master/Code/MXA/DataImport for an example of how I did this in one of my older projects. The code that ties this together is then in http://scm.bluequartz.net/mxa/mxadatamodel/blobs/master/Examples/DataImport/main.cpp This should short enough for you to follow along. I am not saying mine is the utopian example but it does work. Mike Jackson ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On Mon, May 23, 2011 at 10:51 AM, Sanatan Rai sana...@gmail.com wrote:
On 23 May 2011 15:11, Michael Wild them...@gmail.com wrote:
Yes, but you are registering the concrete factories implicitly instead
of explicitly, which is causing you the trouble you experience.
Better have your user provide a function registering his/her classes
explicitly.
I guess this is getting to be off topic, but indeed the
anonymous namespace trick is supposed to do exactly that.
snipped
libraries that this problem occurs. I haven't seen a solution to this
problem either in books or via google.
--Sanatan
The problem is, that when you link a static library to another binary
(be it shared library or executable) only the *required* symbols are
used, all others get discarded. Since nothing in your code actually
references those global instances in the anonymous namespace (the linker
doesn't care about that, BTW), they are ignored.
Four solutions:
1. Only do monolithic builds.
2. Use shared libraries/DLLs
3. Use --whole-archive or similar and hack your way through MSVC (I did
it once. It was ugly. Very ugly. See
https://github.com/themiwi/cppcheck/tree/227378f763d50b005b7dd2167e2cef791054a30c
.
Especially lib/CMakeLists.txt and lib/generateStaticLinkFlags.cmake. I
replaced it with an explicit registration scheme now...)
4. Use an explicit registration scheme.
For sanity's sake, go with 4.
Ok: you've lost me here. Are you saying a trick like:
namespace {
helper1 *createHelper1() { return (new Helper1());}
const bool isRegistered =
factory::instance().registerHelper(helper1, createHelper1);
}
isn't explicit?
That's correct. It's not explicit. Because if nothing references the bool
variable isRegistered then the linker is free to (possibly) throw away
it's initialization because it's not referenced. This code may work with
some compilers; but it is not guaranteed to work.
I could do the following: create a global object in the cpp, that
belongs to a type that would
do the registration. Would you regard this also as implicit?
The problem with either of these approaches is the same: if the code
lives in a separate library
that is loaded only when needed, then these registrations don't take
place. So the framework doesn't
know that the objects can be available.
Apologies for seeming obtuse but I don't follow what you mean by
`explicit registration' here.
Explicit registration is having a method as mentioned earlier, something
like RegisterKnownFactories and then asking clients to make sure they call
that method first before calling anything that requires a factory to create
something.
HTH,
David
--Sanatan
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 05:09 PM, Sanatan Rai wrote: On 23 May 2011 16:00, Michael Wild them...@gmail.com wrote: Everything that relies on static/global initialization to register factories is an implicit scheme. An explicit scheme is where the dependent code (e.g. the main() function) calls a function to do the registration. Ok, got you. However, would this not imply a monolithic build? How is main to know that an object of a type belonging to a base class of interest exists in linked library? As I mentioned earlier, I did this a little more explicitly in a .NET project in the following manner: * I had a factory object, whose job it was to hold creator functions for objects of classes derived from a base class of interest. * The factory was a singleton, and had a static method that could be called. The method loaded all linked assemblies, and picked out classes that were derived from the base class. * Then it explicitly registered the class with the factory. This seems to me to be a hybrid between implicit and explicit registration. The actual mechanics as one might imagine relied heavily on .NET reflection calls. One of the ugliest bits of code I have ever written. --Sanatan If it is acceptable to have just one hook, just require your user to define a specific function, e.g. registerUserFactories which you then call. Otherwise it becomes more tricky. Michael ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 05:09 PM, Sanatan Rai wrote:
On 23 May 2011 16:00, Michael Wild them...@gmail.com wrote:
Everything that relies on static/global initialization to register
factories is an implicit scheme. An explicit scheme is where the
dependent code (e.g. the main() function) calls a function to do the
registration.
Ok, got you. However, would this not imply a monolithic build? How is
main to know that an object of a type belonging to a base class of interest
exists in linked library?
In order to further clarify things, look at the following project:
# CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(GLOBALS CXX)
SET(CMAKE_VERBOSE_MAKEFILE ON)
ADD_LIBRARY(helper STATIC helper.cxx)
ADD_EXECUTABLE(main main.cxx)
TARGET_LINK_LIBRARIES(main helper)
// main.cxx:
int main(void){return 0;}
// helper.cxx:
#include iostream
class helper {};
namespace {
bool registerhelper(const char *ident, helper *(*creator)())
{
std::cout ident = (void *)creator std::endl;
}
helper *helperCreator()
{
return (new helper());
}
const bool helperRegistered = registerhelper(helper, helperCreator);
}
If I'm not mistaken, this is roughly what you do in your project.
Although the main target is linked against libhelper.a, the object file
helper.cxx.o is dropped because none of its entities is referred to by
main.cxx, or in other words: Which reason the linker should have to
include helper.cxx.o in the final binary? To make helper.cxx.o be
included, a single reference from main.cxx usually suffices:
// helper.cxx:
#include iostream
int h; // -- For external reference.
class helper {};
namespace {
bool registerhelper(const char *ident, helper *(*creator)())
{
std::cout ident = (void *)creator std::endl;
}
helper *helperCreator()
{
return (new helper());
}
const bool helperRegistered = registerhelper(helper, helperCreator);
}
// main.cxx:
extern int h;
int h0 = h;
int main(void){return 0;}
Now, the entities from helper.cxx.o are included in the final binary,
i.e. you'll see the ident=... message. Of course, the same happens
when you include helper.cxx.o immediately in the main executable by
mentioning helper.cxx among main's source files, i.e. a monolithic
build. However, be aware that advanced linkers and, in particular,
optimising compiler back-ends may feel free to remove entities that
seem to be unnecessary for the program to run, as David has remarked
in the meantime.
Michael's advice to explicitly register your factory classes means that
there will be an external reference to the concerned object files, so
they'll be included and the registration will be done via the boolean
constants' initialisation, and that's what I'd advise, too. ATM, you
try to have some actions performed in your program without referring
to these actions by any means, and this simply does not work.
Besides, when using --[no-]whole-archive in order to force the linker
to include all of a static library's object files - be aware of the
already mentioned limitations and the fact that you will get *all*
object files, not just the ones you need - you might intersperse
these flags immediately in TARGET_LINK_LIBRARIES() without -L/-l:
target_link_libraries(myTarget
-Wl,--whole-archive helper1 helper2 -Wl,--no-whole-archive)
As I mentioned earlier, I did this a little more explicitly in a
.NET project in the following
manner:
* I had a factory object, whose job it was to hold creator functions
for objects of classes
derived from a base class of interest.
* The factory was a singleton, and had a static method that could be
called. The method loaded
all linked assemblies, and picked out classes that were derived
from the base class.
* Then it explicitly registered the class with the factory.
This seems to me to be a hybrid between implicit and explicit
registration. The actual mechanics
as one might imagine relied heavily on .NET reflection calls. One of
the ugliest bits of code I have
ever written.
If the object file holding the factory is placed in a static library,
and if there's no reference to any of this object file's entities from
anywhere, the above-noted approach would also fail on *nix, but you say
...a static method that could be called.: Calling this method *is* a
reference to the factory's object file from outside, so it would be
included in the final binary, and everything works fine.
In summary, this whole issue is not related to C++ or even to CMake,
but to the manner static libraries are handled: The linker - at least
the GNU one - picks out entire object files, or drops them if they are
not referred to. This is something one must keep in mind, particularly
when dealing with global objects for initialisation purposes. BTW, also
keep in mind that the order of initialisations, i.e. constructor calls,
among global objects is unspecified; this might become important when
such global objects refer to each
Re: [CMake] Newbie question: Static linking
On 23 May 2011 17:46, Michael Hertling mhertl...@online.de wrote:
On 05/23/2011 05:09 PM, Sanatan Rai wrote:
On 23 May 2011 16:00, Michael Wild them...@gmail.com wrote:
Everything that relies on static/global initialization to register
factories is an implicit scheme. An explicit scheme is where the
dependent code (e.g. the main() function) calls a function to do the
registration.
snipped
# CMakeLists.txt:
CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
PROJECT(GLOBALS CXX)
SET(CMAKE_VERBOSE_MAKEFILE ON)
ADD_LIBRARY(helper STATIC helper.cxx)
ADD_EXECUTABLE(main main.cxx)
snipped
helper *helperCreator()
{
return (new helper());
}
const bool helperRegistered = registerhelper(helper, helperCreator);
}
If I'm not mistaken, this is roughly what you do in your project.
Indeed.
snipped
// helper.cxx:
#include iostream
int h; // -- For external reference.
class helper {};
namespace {
bool registerhelper(const char *ident, helper *(*creator)())
{
std::cout ident = (void *)creator std::endl;
}
helper *helperCreator()
{
return (new helper());
}
const bool helperRegistered = registerhelper(helper, helperCreator);
}
// main.cxx:
extern int h;
int h0 = h;
int main(void){return 0;}
Now, the entities from helper.cxx.o are included in the final binary,
i.e. you'll see the ident=... message. Of course, the same happens
when you include helper.cxx.o immediately in the main executable by
mentioning helper.cxx among main's source files, i.e. a monolithic
build. However, be aware that advanced linkers and, in particular,
optimising compiler back-ends may feel free to remove entities that
seem to be unnecessary for the program to run, as David has remarked
in the meantime.
Absolutely.
Michael's advice to explicitly register your factory classes means that
there will be an external reference to the concerned object files, so
they'll be included and the registration will be done via the boolean
constants' initialisation, and that's what I'd advise, too. ATM, you
try to have some actions performed in your program without referring
to these actions by any means, and this simply does not work.
Besides, when using --[no-]whole-archive in order to force the linker
to include all of a static library's object files - be aware of the
already mentioned limitations and the fact that you will get *all*
object files, not just the ones you need - you might intersperse
these flags immediately in TARGET_LINK_LIBRARIES() without -L/-l:
target_link_libraries(myTarget
-Wl,--whole-archive helper1 helper2 -Wl,--no-whole-archive)
Yup, interspersing these flags is the approach I am taking at the moment.
As I mentioned earlier, I did this a little more explicitly in a
.NET project in the following
manner:
* I had a factory object, whose job it was to hold creator functions
for objects of classes
derived from a base class of interest.
snipped
If the object file holding the factory is placed in a static library,
and if there's no reference to any of this object file's entities from
anywhere, the above-noted approach would also fail on *nix, but you say
...a static method that could be called.: Calling this method *is* a
reference to the factory's object file from outside, so it would be
included in the final binary, and everything works fine.
Quite.
In summary, this whole issue is not related to C++ or even to CMake,
but to the manner static libraries are handled: The linker - at least
the GNU one - picks out entire object files, or drops them if they are
not referred to. This is something one must keep in mind, particularly
when dealing with global objects for initialisation purposes. BTW, also
keep in mind that the order of initialisations, i.e. constructor calls,
among global objects is unspecified; this might become important when
such global objects refer to each other.
I do realise that it was off-topic, but people very kindly offered suggestions!
So one had to go on!
Many thanks to all for a careful and useful discussion.
Unfortunately, I am stuck with the paradigm of having to kludge loading an
entire library. For various reasons the one may not have a reference to
the hosted object in main, which must remain agnostic. Indeed, in my particular
line of business (finance), this happens to be a `standard' pattern.
Whether good or bad is a discussion that'd be too off topic...(though am happy
to continue the discussion off/on list if people are so inclined!).
--Sanatan
___
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
Sanatan Rai wrote: Unfortunately, I am stuck with the paradigm of having to kludge loading an entire library. For various reasons the one may not have a reference to the hosted object in main, which must remain agnostic. Indeed, in my particular line of business (finance), this happens to be a `standard' pattern. Whether good or bad is a discussion that'd be too off topic...(though am happy to continue the discussion off/on list if people are so inclined!). What about your application calling an initStaticLibs() function which is defined in an header. That header is written by CMake using FILE(WRITE ...) because CMake knows which libs are there and by this which init functions need to be called. They just have to be somehow related to the library name. Eike signature.asc Description: This is a digitally signed message part. ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
On 05/23/2011 08:42 PM, Sanatan Rai wrote: On 23 May 2011 17:46, Michael Hertling mhertl...@online.de wrote: In summary, this whole issue is not related to C++ or even to CMake, but to the manner static libraries are handled: The linker - at least the GNU one - picks out entire object files, or drops them if they are not referred to. This is something one must keep in mind, particularly when dealing with global objects for initialisation purposes. BTW, also keep in mind that the order of initialisations, i.e. constructor calls, among global objects is unspecified; this might become important when such global objects refer to each other. I do realise that it was off-topic, but people very kindly offered suggestions! So one had to go on! Many thanks to all for a careful and useful discussion. Please don't get me wrong on this point: Far be it from me to criticise your concern as off-topic; on a build system's mailing list, it rather isn't, IMO. Instead, I just wanted to point out that this issue is not immediately related to CMake or C++ but to the implications of static libraries. Of course, we can frankly discuss it here, especially how to address it with the means of CMake. Unfortunately, I am stuck with the paradigm of having to kludge loading an entire library. For various reasons the one may not have a reference to the hosted object in main, which must remain agnostic. Indeed, in my particular line of business (finance), this happens to be a `standard' pattern. Whether good or bad is a discussion that'd be too off topic...(though am happy to continue the discussion off/on list if people are so inclined!). In this thread, you have spoken about loading a library, library loaded when needed and library loaded in memory, e.g., but static libraries aren't loaded in this sense; they're examined at link time, and that's it. May it be possible that shared libraries are what you actually want, i.e. is there any reason why the libraries in question must be static? Note that shared libraries don't have the limitations of static ones; say, they are not cherry-picked w.r.t. object files. If you really want to have actions performed at the program's startup without any explicit ado, you might take a look at GCC's constructor/ destructor attributes, but be aware that this is highly dependent on the underlying binary format and the corresponding development tools. Regards, Michael ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Newbie question: Static linking
Am Montag, 23. Mai 2011, 01:36:14 schrieb Sanatan Rai: After cmake, and make all, the libraries build as static archives, ie I get liblib1.a, liblib2.a, libhelper1.a, libhelper2.a and executable myProj in the appropriate locations. However, the executable myProj does not appear to have linked statically to libhelper1.a and libhelper2.a. These libraries contain global initialisers for certain objects: which doesn't happen when I run the executable. Most likely, the linker just drops those global initialisers when linking statically. See the linker options in man ld to prevent that or give the library an initialisation method. HS ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
