Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Ben Walton bwal...@opencsw.org wrote: Excerpts from Yann Rouillard's message of Sat Aug 25 11:10:57 -0400 2012: Hi Yann, For 2., I don't know yet. I would like some input from Dago on the best way to add distribution wide LDFLAGS while still allowing to override them. I was thinking about adding a COMMON_LDFLAGS mgar variable that could easily be overriden in the Makefile. What about: bwalton @ unstable10s : ~/opencsw/.buildsys/v2 $ svn diff Index: gar.conf.mk === --- gar.conf.mk (revision 19080) +++ gar.conf.mk (working copy) @@ -706,7 +706,7 @@ CFLAGS ?= $(strip $($(GARCOMPILER)_CC_FLAGS) $(_CATEGORY_CFLAGS) $(EXTRA_CFLAGS)) CXXFLAGS ?= $(strip $($(GARCOMPILER)_CXX_FLAGS) $(_CATEGORY_CXXFLAGS) $(EXTRA_CXXFLAGS)) CPPFLAGS ?= $(strip $($(GARCOMPILER)_CPP_FLAGS) $(_CATEGORY_CPPFLAGS) $(EXTRA_CPPFLAGS) $(INCLUDE_FLAGS)) -LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) $(EXTRA_LDFLAGS) $(LINKER_FLAGS)) +LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) I hope, you only use LDFLAGS in case of a known defective upstream build system. Note that using LDFLAGS may cause a sane build system to missbehave. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Hi Jörg, Am 27.08.2012 um 12:29 schrieb joerg.schill...@fokus.fraunhofer.de (Joerg Schilling): Ben Walton bwal...@opencsw.org wrote: Excerpts from Yann Rouillard's message of Sat Aug 25 11:10:57 -0400 2012: For 2., I don't know yet. I would like some input from Dago on the best way to add distribution wide LDFLAGS while still allowing to override them. I was thinking about adding a COMMON_LDFLAGS mgar variable that could easily be overriden in the Makefile. What about: bwalton @ unstable10s : ~/opencsw/.buildsys/v2 $ svn diff Index: gar.conf.mk === --- gar.conf.mk (revision 19080) +++ gar.conf.mk (working copy) @@ -706,7 +706,7 @@ CFLAGS ?= $(strip $($(GARCOMPILER)_CC_FLAGS) $(_CATEGORY_CFLAGS) $(EXTRA_CFLAGS)) CXXFLAGS ?= $(strip $($(GARCOMPILER)_CXX_FLAGS) $(_CATEGORY_CXXFLAGS) $(EXTRA_CXXFLAGS)) CPPFLAGS ?= $(strip $($(GARCOMPILER)_CPP_FLAGS) $(_CATEGORY_CPPFLAGS) $(EXTRA_CPPFLAGS) $(INCLUDE_FLAGS)) -LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) $(EXTRA_LDFLAGS) $(LINKER_FLAGS)) +LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) I hope, you only use LDFLAGS in case of a known defective upstream build system. Note that using LDFLAGS may cause a sane build system to misbehave. I think you mean LD_OPTIONS to misbehave and LDFLAGS is ok? Best regards -- Dago -- You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process. - xkcd #896 smime.p7s Description: S/MIME cryptographic signature ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Dagobert Michelsen d...@opencsw.org wrote: I hope, you only use LDFLAGS in case of a known defective upstream build system. Note that using LDFLAGS may cause a sane build system to misbehave. I think you mean LD_OPTIONS to misbehave and LDFLAGS is ok? You are right - sorry for my confusion. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Yann Rouillard y...@pleiades.fr.eu.org writes: But if everybody is ok with it, here is how we could try to enable it to gradually: 1. write a checkpkg test to test if direct binding if properly enabled in a package, 2. enable Direct Binding manually for a reduced set of packages (at least my packages :) ) (we just have to pass -Bdirect to SUN ld) 3. wait a bit to see if something unexpected happens :) 3. if it works, enable it by globally adding the option to LINKER_FLAGS 4. enable the checkpkg direct binding test by default so we can catch even packages that don't use LDFLAGS Fully agree with the sole condition that there must be a mechanism to inhibit this behavior; by consequence, the check can be also overridden. -- Peter ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Hi Ben, 1. write a checkpkg test to test if direct binding if properly enabled in a package, How do you envision this check being implemented? As a positive or negative check? Well I think negative check is a better way to make sure direct binding is enabled in packages ;) Besides it's the way checkpkg works, doesn't it ? Of course, maintainers will be able to override the check (like other checkpkg tests), but at least they will need to do it on purpose. 2. enable Direct Binding manually for a reduced set of packages (at least my packages :) ) (we just have to pass -Bdirect to SUN ld) I see you're doing this already! +1 Want to join me on this so we have a wider packages set ? :) 3. wait a bit to see if something unexpected happens :) 3. if it works, enable it by globally adding the option to LINKER_FLAGS 4. enable the checkpkg direct binding test by default so we can catch even packages that don't use LDFLAGS +1. I think it's a sound plan. Thank you for your feedback ! Yann ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Excerpts from Yann Rouillard's message of Sat Aug 25 10:55:28 -0400 2012: Hi Yann, How do you envision this check being implemented? As a positive or negative check? Well I think negative check is a better way to make sure direct binding is enabled in packages ;) Besides it's the way checkpkg works, doesn't it ? Yes, I'm just thinking about a transition phase where it will be really noisy (as you mentioned). If it's the way we're going to go though, the negative check is proper and to be overridden by maintainers if the package doesn't comply. Of course, maintainers will be able to override the check (like other checkpkg tests), but at least they will need to do it on purpose. 2. enable Direct Binding manually for a reduced set of packages (at least my packages :) ) (we just have to pass -Bdirect to SUN ld) I see you're doing this already! +1 Want to join me on this so we have a wider packages set ? :) Sure. I'll toggle it on for a few packages as I rebuild them in the next while. Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Excerpts from Yann Rouillard's message of Sat Aug 25 11:10:57 -0400 2012: Hi Yann, For 2., I don't know yet. I would like some input from Dago on the best way to add distribution wide LDFLAGS while still allowing to override them. I was thinking about adding a COMMON_LDFLAGS mgar variable that could easily be overriden in the Makefile. What about: bwalton @ unstable10s : ~/opencsw/.buildsys/v2 $ svn diff Index: gar.conf.mk === --- gar.conf.mk (revision 19080) +++ gar.conf.mk (working copy) @@ -706,7 +706,7 @@ CFLAGS ?= $(strip $($(GARCOMPILER)_CC_FLAGS) $(_CATEGORY_CFLAGS) $(EXTRA_CFLAGS)) CXXFLAGS ?= $(strip $($(GARCOMPILER)_CXX_FLAGS) $(_CATEGORY_CXXFLAGS) $(EXTRA_CXXFLAGS)) CPPFLAGS ?= $(strip $($(GARCOMPILER)_CPP_FLAGS) $(_CATEGORY_CPPFLAGS) $(EXTRA_CPPFLAGS) $(INCLUDE_FLAGS)) -LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) $(EXTRA_LDFLAGS) $(LINKER_FLAGS)) +LDFLAGS ?= $(strip $($(GARCOMPILER)_LD_FLAGS) $(_CATEGORY_LDFLAGS) $(EXTRA_LDFLAGS) $(LINKER_FLAGS) $(ifeq $(NO_LD_DIRECT),,-Bdirect)) ASFLAGS ?= $(strip $($(GARCOMPILER)_AS_FLAGS) $(_CATEGORY_ASFLAGS) $(EXTRA_ASFLAGS)) OPTFLAGS ?= $(strip $($(GARCOMPILER)_CC_FLAGS) $(_CATEGORY_OPTFLAGS) $(EXTRA_OPTFLAGS)) FFLAGS ?= $(strip $($(GARCOMPILER)_FFLAGS) $(_CATEGORY_FFLAGS) $(EXTRA_FFLAGS)) That would allow a maintainer to set NO_LD_DIRECT in the recipe to skip that flag but for everything else it would be set. Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Hi Yann, Am 25.08.2012 um 18:03 schrieb Yann Rouillard y...@pleiades.fr.eu.org: How is LD_OPTIONS handled in the build process ? I do know very much this environment variable. I meant I do not know. LD_OPTIONS is passed as environment variable directly to the linker and processed before any other option on the command line. Most of the time this works for runtime paths and it is especially important to pass $ISALIST in RPATH as '$' is sensitive to evaluation when passed with LDFLAGS - depending on the build system the '$' needs to be quoted two or even three times. The downside is that prepending /opt/csw/lib can brake the test suite because the installed library is preferred over the newly built one: http://wiki.opencsw.org/porting-faq#toc10 For the discussed variables like direct or ignore it would be perfect as it is not order relevant. Best regards -- Dago -- You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process. - xkcd #896 smime.p7s Description: S/MIME cryptographic signature ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Excerpts from Yann Rouillard's message of Tue Aug 07 16:43:08 -0400 2012: Hi Yann, Sorry it took me so long to get back to this...It's an interesting topic and I think you've outlined a good solution. Direct Binding changes the behaviour of the linker at runtime, when a program or a library has a direct binding against a library, the linker will now link a symbol against the exact library that provided the symbol at compile time. That exactly solves the problem we have here because openssl 0.9.8 symbols will be linked against libssl0.9.8 and not libssl1.0.0 (and vice versaà. I think that for 99% of our use cases this is definitely a good thing to turn on. In my understanding, the only place that having it on globally (-B direct) as opposed to toggling on and off per shared object (-z direct) is if something was relying on interposition (using the first symbol found in the default search algorithm - normal runtime binding). The only case I could think of where that would be desirable behaviour is in something that dlopen()'s multiple objects for a plugin system where one plugin could redefine an interface and a second plugin could re-redefine it and so on. It's like that there are other cases where it _might_ be desirable behaviour but I have to think it's by far the minority. So to avoid futures difficulties of the same kind, I am proposing to enable direct binding by default for all opencsw software !) I think this is a good change to add. I don't see a lot of cons, maybe these ones: - sometimes, it seems some programs do need the original linker behaviour but that could be fixed by some other ld options, Yes, in those cases, either turning off direct linking or somehow inserting the on/off toggles to only use direct linking for some objects. - we never enabled it so maybe we will uncover some problems, - it works only with Sun ld (we don't use GNU ld somewhere do we - ?) I'd almost be surprised if there wasn't some package that used gnu ld but I'm not currently aware of any. 1. write a checkpkg test to test if direct binding if properly enabled in a package, How do you envision this check being implemented? As a positive or negative check? 2. enable Direct Binding manually for a reduced set of packages (at least my packages :) ) (we just have to pass -Bdirect to SUN ld) I see you're doing this already! +1 3. wait a bit to see if something unexpected happens :) 3. if it works, enable it by globally adding the option to LINKER_FLAGS 4. enable the checkpkg direct binding test by default so we can catch even packages that don't use LDFLAGS +1. I think it's a sound plan. Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Excerpts from Ben Walton's message of Fri Aug 24 15:04:16 -0400 2012: runtime binding). The only case I could think of where that would be desirable behaviour is in something that dlopen()'s multiple objects for a plugin system where one plugin could redefine an interface and a second plugin could re-redefine it and so on. It's like that there are other cases where it _might_ be desirable behaviour but I have to think it's by far the minority. This isn't even valid. I'm not sure where I was going with that. dlopen()'d things wouldn't be directly applicable here. I can't think of a case where something would prefer the current behaviour if given direct linking as an option. There is likely something that does care but as long as we can disable in that case, I think direct linking is definitely the way to go. Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
Re: [csw-maintainers] Proposal: direct binding for all the opencsw software stack
Excerpts from Yann Rouillard's message of Tue Aug 07 16:43:08 -0400 2012: Hi Yann, Linux uses symbol versioning to solve this problem but in the Solaris world, even if symbol versioning does exist, that is not the solution to this problem and it turns that a solaris linker feature called direct binding is a right approach (see the thread http://lists.opencsw.org/pipermail/maintainers/2012-July/017064.html and the explanation in oracle manual: http://docs.oracle.com/cd/E19963-01/html/819-0690/aehzq.html ). I need to read about this, but it sounds like it might be a good approach. Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 ___ maintainers mailing list maintainers@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
