Re: Mac OS X: OpenSSL again
On Wed, Jun 06, 2007, Anders F Björklund wrote:
Ralf S. Engelschall wrote:
So appending -Wl,-search_paths_first to the LDFLAGS should work too.
Cool, that's it! If we now override the cc, gcc and ld commands under
Mac OS X and enfore this option, the various linking problems you have
observed should be gone. What about the following (untested) patch?
The patch works, though we might want to filter out the extra warning
issued when using gcc/g++ for just compiling and not doing any linking:
-search_paths_first: linker input file unused because linking not done
Otherwise you get one of those for each gcc -c, since the override
can't tell whether gcc is going to be used for compiling or linking...
Oh, I see. Ok, I've now improved the wrapper scripts. Please manually
remove prefix/libexec/openpkg/override/{cc,gcc} and upgrade to the
latest bootstrap version as of today. Then if -c or -E is present
on the compiler command line the -Wl,-search_paths_first should be no
longer used and this way the warning be gone.
And need to provide fake binutils/gcc packages, with /usr symlinks.
(so that dependencies and such work, otherwise it'll try to install)
/openpkg/bin/cc - /usr/bin/cc
/openpkg/bin/gcc - /usr/bin/gcc
This can be provided by openpkg-import.
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
__
OpenPKG http://openpkg.org
Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
Ralf S. Engelschall wrote: And need to provide fake binutils/gcc packages, with /usr symlinks. (so that dependencies and such work, otherwise it'll try to install) /openpkg/bin/cc - /usr/bin/cc /openpkg/bin/gcc - /usr/bin/gcc This can be provided by openpkg-import. Will do that, then: use openpkg-import to replace binutils and gcc. One little problem is that the binutils and cctools differ in content, so the following programs are missing: addr2line, objcopy, objdump, readelf --anders PS. I think odcctools in OpenKPG is mostly working, http://www.algonet.se/~afb/openpkg/odcctools.spec (but it is very experimental at this point in time) The Apple GCC compiler didn't build yet, though... __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
This can be provided by openpkg-import. Will do that, then: use openpkg-import to replace binutils and gcc. One little problem is that the binutils and cctools differ in content, so the following programs are missing: addr2line, objcopy, objdump, readelf I fixed the missing symlink target with ignoring the return code: http://www.algonet.se/~afb/openpkg/openpkg-import.diff However, this still did not work because OpenPKG will fail to locate the real compiler e.g. i686-apple-darwin8-gcc-4.0.1 (from /usr/bin), but only find the gcc-4.0 driver front-end... (symlinked from cc) So either I need to bring some more symlinks in, or do it differently ? --anders __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
Ralf S. Engelschall wrote: However, this still did not work because OpenPKG will fail to locate the real compiler e.g. i686-apple-darwin8-gcc-4.0.1 (from /usr/bin), but only find the gcc-4.0 driver front-end... (symlinked from cc) So either I need to bring some more symlinks in, or do it differently ? This I do not understand: what _IS_ /usr/bin/gcc under MacOS? Is is a strange wrapper script which is unable to find its backends when called via a symlink? Ok, let's try a different approach: in the latest openpkg-import I've now replaced the symlinks with wrapper scripts which directly _exec_ the /usr/bin/gcc. Perhaps this works better. It's a strange wrapper alright, although a program not a script. If we start from the beginning: /usr/bin/cc and /usr/bin/gcc are just symlinks to the currently selected GCC compiler, as chosen by gcc_select and as default for the Mac OS X version. e.g. /usr/bin/cc - gcc-4.0 /usr/bin/gcc - gcc-4.0 Anyway, for Mac OS X 10.1 - 10.3 is pretty straight forward - it's one of the binaries gcc2 or gcc3 (3.1) or gcc-3.3. However, for Mac OS X 10.4 things get more complicated since it can build Universal Binaries for the two architectures... So gcc-4.0 will call upon either i686-apple-darwin8-gcc-4.0.1 or powerpc-apple-darwin-gcc-4.0.1, to compile for either arch. If you use universal flags like -arch ppc -arch i386 it will do one compile for ppc and one compile for i386, and lipo(1) together. This front-end compiler driver is part of the Apple GCC compiler. (see gcc/config/darwin-driver.c in the gcc-5363 Darwin sources) Each architecture is multi-lib, so there are four potential arch targets: ppc, i386, ppc64 and x86_64 - all in the same binary... HTH (will try the exec) --anders PS. The default is of course to compile for the local -arch only. __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
Ralf S. Engelschall wrote:
Oh, I see. Ok, I've now improved the wrapper scripts. Please manually
remove prefix/libexec/openpkg/override/{cc,gcc} and upgrade to the
latest bootstrap version as of today. Then if -c or -E is present
on the compiler command line the -Wl,-search_paths_first should be no
longer used and this way the warning be gone.
While you're modifying the ld wrapper script, you might want to
consider filtering out --hash-style=gnu which later versions of
Fedora and also CentOS 5 seems to be adding to GCC automagically:
See http://www.algonet.se/~afb/openpkg/openpkg-centos5-gcc.txt
Error is /openpkg/bin/ld: unrecognized option '--hash-style=gnu'
i.e. OpenPKG binutils and the system GCC don't play well together
--anders
PS.
I'm sure there is a better place to report this bug, but I'm tired.
(i.e. I searched for bug on the front page, didn't find anything)
__
OpenPKG http://openpkg.org
Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
Ralf S. Engelschall wrote: However, the *reason* for the workaround is the fact that since OpenPKG does not provide a shared OpenSSL library, Darwin ld(1) will try to use the /usr/lib variant instead... So this will affect everything else using OpenSSL as well. The real solution would be to _FORCE_ Mac OS X's ld(1) into prefering the libs provided under -L _first_. It is absolutely bogus to always prefer /usr/lib libraries over -L. I even consider this broken, although I can image what problems the Apple people wanted to solve. So, is there a way to _FORCE_ ld(1) to do the correct thing, i.e., search for the paths under -L options first and /usr/lib last? Perhaps via an option or even a small _WRAPPER_ around ld(1) which tricks it (via some LD_PRELOAD or whatever tricks) into doing the right thing. Well, as I was considering a custom version of cctools for OpenPKG anyway - I guess it could be patched to search for static libraries the first time, instead of looking the entire path for dynamic first. I don't know of any linker option to have it look for static libs (other than explicitly listing them by name), but either a wrapper to the binary or even patching the source of the binary is doable. But OpenSSL should probably continue to have its Makefiles patched because it's a) rather trivial and b) is used in the bootstrapping BTW, we do not have _any_ MacOS X boxes available at OpenPKG. So I cannot do very much on my own on this issue, of course. If someone is able to grant me remote access (via an unprivileged rse account and SSH key authentication is fully fine) to a Mac OS X 10.4 box I can investigate myself for us and try to find a ld(1) tricking solution. What I am investigating is to package the Darwin GCC cross-compilers for OpenPKG use, from originals at http://ranger.befunk.com/fink/darwin-cross/ Then you should be able to build Darwin packages (*not* Mac OS X, but...) as well, from your Linux or Solaris build machines using cross-compilation ? Unlike Apple's built-in cross-compiler, this one did use i386 arch for Darwin 8 instead of i686 but it shouldn't really matter much if at all. The version of the tools and headers are a bit old, so they probably need some updating to Darwin 8.9 source that was only rather recently released. And I haven't used Darwin in a while, so I'm not sure if it still works... OpenDarwin shut down a while ago so the stand-alone Darwin OS is dead now. Please contact me if someone of you can (temporarily) grant me access to such a box. I might be able to set up a 10.3/PPC box, but no 10.4/x86 available - sorry. I'll see if can get one sponsored somewhere, either existing or a Mac Mini. But at the moment I only have our desktop machines on the local network... Source code for ld and the other Mac OS X tools are browsable online at http://www.opensource.apple.com/darwinsource/DevToolsAug2006/cctools -622.3/ but you need to use odcctools to get a version that actually compiles. The Apple code is released under the Apple Public Source License v 2.0, whose text is available online at http://www.opensource.apple.com/apsl/ --anders __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
I don't know of any linker option to have it look for static libs (other than explicitly listing them by name), but either a wrapper to the binary or even patching the source of the binary is doable. Reading through the source code I realize I didn't look *that* hard: -search_paths_first By default when the -dynamic flag is in effect, the -lx and -weak-lx options first search for a file of the form `libx.dylib' in each directory in the library search path, then a file of the form `libx.a' is searched for in the library search paths. This option changes it so that in each path `libx.dylib' is searched for then `libx.a' before the next path in the library search path is searched. So appending -Wl,-search_paths_first to the LDFLAGS should work too. --anders __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
I did approach 1) for OpenSSL and approach 3) for Apache, and then the pitch succeeded* in running apache+lynx. :-) Hmm, almost forgot to mention that I did have to change the default apache configuration before it would start up: OpenPKG: start: apache:FAILED openpkg:rc:WARNING: /openpkg:apache:%start: failed with return code 1 openpkg:rc:NOTICE: output from stdout/stderr is following: +-- | Syntax error on line 201 of /openpkg/etc/apache/apache.base: | SSLMutex: Semaphores not available on this platform +-- where Line #201 said: SSLMutexsem --anders __ OpenPKG http://openpkg.org Developer Communication List openpkg-dev@openpkg.org
Re: Mac OS X: OpenSSL again
On Sun, Jun 03, 2007, Anders F Björklund wrote:
I don't know of any linker option to have it look for static libs
(other than explicitly listing them by name), but either a wrapper
to the binary or even patching the source of the binary is doable.
Reading through the source code I realize I didn't look *that* hard:
-search_paths_first
By default when the -dynamic flag is in effect, the -lx and
-weak-lx options first search for a file of the form
`libx.dylib' in each directory in the library search path, then
a file of the form `libx.a' is searched for in the library
search paths. This option changes it so that in each path
`libx.dylib' is searched for then `libx.a' before the next path
in the library search path is searched.
So appending -Wl,-search_paths_first to the LDFLAGS should work too.
Cool, that's it! If we now override the cc, gcc and ld commands under
Mac OS X and enfore this option, the various linking problems you have
observed should be gone. What about the following (untested) patch?
Index: openpkg.spec
===
RCS file: /v/openpkg/cvs/openpkg-src/openpkg/openpkg.spec,v
retrieving revision 1.589
diff -u -d -u -d -u -d -r1.589 openpkg.spec
--- openpkg.spec3 Jun 2007 09:47:55 - 1.589
+++ openpkg.spec3 Jun 2007 21:02:49 -
@@ -2437,6 +2437,29 @@
chmod 775 %{l_prefix}/lib/openpkg/override/install-info
fi
;;
+*-*-macos10.* | *-*-darwin* )
+gcc=`%{l_prefix}/lib/openpkg/shtool path gcc`
+cc=`%{l_prefix}/lib/openpkg/shtool path cc`
+ld=`%{l_prefix}/lib/openpkg/shtool path ld`
+if [ .$gcc != . -a ! -f %{l_prefix}/lib/openpkg/override/gcc ];
then
+( echo #!/bin/sh
+ echo $gcc -Wl,-search_paths_first \[EMAIL PROTECTED]
+) %{l_prefix}/lib/openpkg/override/gcc
+chmod 775 %{l_prefix}/lib/openpkg/override/gcc
+fi
+if [ .$cc != . -a ! -f %{l_prefix}/lib/openpkg/override/cc ];
then
+( echo #!/bin/sh
+ echo $cc -Wl,-search_paths_first \[EMAIL PROTECTED]
+) %{l_prefix}/lib/openpkg/override/cc
+chmod 775 %{l_prefix}/lib/openpkg/override/cc
+fi
+if [ .$ld != . -a ! -f %{l_prefix}/lib/openpkg/override/ld ];
then
+( echo #!/bin/sh
+ echo $ld -search_paths_first \[EMAIL PROTECTED]
+) %{l_prefix}/lib/openpkg/override/ld
+chmod 775 %{l_prefix}/lib/openpkg/override/ld
+fi
+;;
esac
# FIXME: hack to workaround problems in environments with too few
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
__
OpenPKG http://openpkg.org
Developer Communication List openpkg-dev@openpkg.org
