On 8/4/2010 2:15 PM, Ralf Wildenhues wrote:
> * Charles Wilson wrote on Tue, Aug 03, 2010 at 07:22:46AM CEST:
>> Now, since I've used --prefix=/mingw and NOT
>> --prefix=/usr/i686-pc-mingw32/sys-root/mingw, I can "cheat" even on
>> win32. I can take the binary runtime components -- even the -devel
>> components! -- and install them in
>> C: (or D: or E:)
>> and they'll end up under
>> C:/mingw
>> D:/mingw
>> or whathaveyou. Thanks to a quirk in window's path resolution, any
>> unixish absolute path is interpreted as if it were a path from the drive
>> root of the CWD. So, if CWD is on C:, then /mingw/libexec is
>> interpreted as C:/mingw/libexec. Since ALL the unixish paths will start
>> with /mingw, this will Do The Right Thing.
>
> Sorry, but I don't think that is viable. Say, C:/mingw is owned by the
> administrator, but D:/mingw is domain of user A. Starting your app on
> D: as user B makes B susceptible to files which A owns. In general, you
> cannot leave away the drive letter when specifying the --prefix.
Then you are stuck with building twice. Once for your sysroot
installation, and once for the "real" installation on $host.
However...that would STILL break:
libfoo.la
a) compiled for sysroot using the cross compiler, and installed there,
with --prefix=/something and --with-sysroot
b) compiled for $host using the cross compiler, and installed THERE,
with --prefix=E:/mingw or whatever.
Then, you compile libbar.la which depends on libfoo:
a) compiled for sysroot using the cross compiler, and installed there,
with --prefix=/something and --with-sysroot. libbar.la(a)'s
dependency_libs will look like
-L=/something =/something/lib/libfoo.la
This is obviously useful for future compilation of libbaz.la.
b) compiled for $host using the cross compiler, and eventually
installed there, with --prefix=E:/mingw or whatever. However...
libbar.la(b)'s dependency_libs will look like
-L=/something =/something/lib/libfoo.la
too! But that's clearly wrong, if you want to copy libbar.la(b)
directly over to $host and use it there, under E:/mingw.
Now, this is cross-compiler 101, for all architectures. You really need
to use the same "prefix" underneath your sysroot that you use on the
actual $host. Unfortunately, there is no such thing, on unix, as a
prefix named "E:/somedir" (note the missing initial '/').
But, that is untenable: on the one hand you want to assert that "thou
shalt include X: in thy prefixes when configuring for $host=mingw" even
when cross-compiling for any purpose, but then there's this impossible
requirement that the "prefix" underneath the sysroot be identical to the
"prefix" on $host -- otherwise this whole cross compile thing falls
apart, once you start chaining dependencies.
Now, if "/something" in the (a) cases above were "/mingw" (e.g.
"matched" the drive-letter-stripped version from the (b) cases above),
then it would actually work correctly -- but, you'd be open to exploits
from D:/mingw.
And this problem is not limited to libtool's .la files; if any path
containing $prefix is compiled into a binary, or recorded anywhere,
you'll have this problem.
The only real solution, as I've said, is:
1) don't cross compile for $host=mingw. Ever. Always compile
natively, and use --prefix=X:/stuff WITH the drive letter.
2) Fix *every* package that records ANY directory information
so that it is completely relocatable. Each binary or script, or
tool that parses a text file containing directory information,
should autodetect its installation location, and deduce relative
paths to its resources from that.
#2 is a gigantic job, and it's not libtool's task to do that for the
universe of software packages. Now, MAYBE, we might take a look at
relocation support for .la files -- but I fear that would open us up to
more security holes than you could shake a stick at. Resolving all
paths into relative paths from whatever is recorded in libdir=''? Oh, boy...
#1 is clearly unacceptable as well.
Hence...I think the consensus is:
I) if you're compiling natively for mingw, use --prefix=X:/foo,
because it avoids the possible security issues. And, it's just
objectively the correct thing to do.
II) if you're cross-compiling -- whether for installation into
the $build's sysroot, or for actual deployment onto $host -- use
--prefix=/unixish/path, and make sure it matches in both cases.
You'll exploit a quirk -- AND open your users up to a possible
security exploit -- but there currently is no other viable way
to do cross-compiles for $host=mingw with chained dependencies,
such that they would actually WORK.
> Sure. I don't disagree with leaving the sysroot part out of --prefix.
>
>> However, items built for case (2) -- direct deployment on the $host (or
>> compiled "natively" ON the mingw host, that is MSYS/MinGW) -- are by
>> convention configured as:
>> configure --prefix=`cd /mingw && pwd -W`
>> which ends up as
>> configure --prefix=C:/MinGW
>> and the installation is usually preformed as
>> make install prefix=/tmp/something/MinGW
>
> I guess what I don't see is why the former case doesn't also use
> --prefix=C:/MinGW
Because if you then try to reconstruct the actual location on the disk,
on $build under sysroot, you get:
/usr/i686-pc-mingw32/sysrootC:/MinGW/
and now, the word "sysrootC:", besides being a sibling, rather than a
child, of the ACTUAL sysroot returned by the compiler via
-print-sysroot, is unsplitable: how can libtool extract the "C:/MinGW"
part when it is geared towards splitting paths at path-delimiter characters?
Because "C:/MinGW" doesn't start with "/", $sysroot$prefix is not a
reversible operation. And the reversibility is a KEY requirement for
any sane sysroot handling -- by libtool or pkg-config or anybody else.
> [...]
>> Anyway, this workflow ensures that the actual, DOS-ish paths are
>> embedded anywhere such things are hardcoded, which is both good and bad.
>> But it also ensures that there IS no concatenation of paths that might
>> start with X:, since we replace the ENTIRE prefix with a unixish one
>> during make install. This technique works, even today on cross setups,
>> if you build one package at a time for deployment on $host.
>
> Ahh.
>
>> But you
>> can't easily do a "chain" of such builds on a cross system, without
>> effort. You sorta have to do twice the builds: one of type (1) and one
>> of type (2), for each library in the chain:
>>
>> a) zlib class (1) installed on $build in sysroot under $prefix
>> b) zlib class (2) installed on $host under $prefix
>> c) libpng class (1) installed on $build in sysroot under $prefix
>> this will link against the libz.dll.a from (a). If you have
>> support on $build for running $host binaries (wine?) then at
>> runtime it would use the libz-1.dll from (a) as well.
>> d) libpng class (2) installed on $host under $prefix
>> this will link against libz.dll.a from (a)!!! However, once
>> installed on $host, it will RUN against zlib-1.dll from (b).
>> and so on, and so on.
>>
>> Using method (2) alone would not yet be appropriate for a case (1)
>> installation (e.g. into a cross $build system's sysroot), because then
>> you WOULD be concatenating a unixish sysroot with a dosish "prefix".
>> (e.g. you can't really use (b) and install it on $build in sysroot, and
>> pretend that's a suitable replacement for (a); ditto (d) vs. (c)).
>
> Is this still relevant in light of my above reply?
Actually, I think it's even MORE relevant. Short version:
1) cross-compiler-101: --prefix under $sysroot must == prefix on $host.
Otherwise lots of stuff breaks.
2) for $host=mingw, the prefered prefix formula is a dos path, X:/foo.
3) but you can't use that formula under $sysroot on $build, because
$sysroot$prefix where prefix=X:/foo is not a reversible operation
4) so we are stuck with a workaround for $host=mingw cross compiles,
where we exploit a flaw in Windows path resolution: "unix-style"
absolute paths (that start with /) are always interpreted with
an implied X: drive spec, where 'X' is derived from CWD. While
this is a possible security hole, it is also the only current
mechanism to actual build chains of dependent projects using a
cross compiler for this $host.
>>> If yes, then
>>> very cool, and I'd very much like this to be exposed in the testsuite
>>> (only on w32 systems, of course, and the C:/... path should be something
>>> unlikely to clash with any actual user files, in case make install does
>>> write there after all).
>>
>> I'm not entirely sure what you want to test here -- or that it would be
>> something that *libtool's* testsuite should cover (wouldn't it be more
>> appropriate as part of automake's?)
>
> Please allow me to give a sort-of simple-minded answer, hope I can make
> the gist of it clear:
>
> If Libtool gains the --with-sysroot feature, then that should be tested
> as part of the Libtool testsuite.
OK.
> Currently these tests all skip if the
> compiler found is not GCC or has not been configured with a sysroot.
Right.
> These two side-conditions mean that when I run the testsuite on all the
> systems and in all the configurations that I have been testing
> prereleases in the past, *none* of them will actually cover any of the
> new sysroot functionality, because even the cross compilers that I test
> do not use a sysroot.
Ah -- but gcc supports --sysroot even if it wasn't compiled with one (my
cygwin native 4.3.4 doesn't have a -print-sysroot, tho; so I wonder if
the --sysroot=/test/dir actually works for that compiler!)
However, my linux native compiler supports both (and, of course, wasn't
actually compiled with a sysroot of its own):
$ gcc --sysroot=/bob -print-sysroot
/bob
So, you could easily create a test that uses the native system compiler
to "fake" a cross-toolchain "sysroot" test. But to make that work you'd
still have to create a temporary fake sysroot underneath /bob as a
symlink farm using /usr/lib and /usr/include -- which would quickly
become painful, since most systems have LOTS of stuff under those two
directories...
And then there's the fact that the toolchain ITSELF has bits installed
under /usr/lib/gcc/*/...
> Given the experience that new Libtool tests need on the average about
> three or more iterations to get right portably, this makes me a bit
> nervous. We should really, really try to get test coverage high for
> this feature on as many setups as possible.
Which definitely argues AGAINST making --with-sysroot the default, even
if you ARE using a cross compiler, right away. The key thing is, we
want to ensure that
1) native compilers don't break -- that's covered by the rest of the
test suite
2) cross-compiler setups don't break, when libtool is used in its
default --without-sysroot (--with-sysroot=) mode. I think this
is covered by "the rest of the test suite" when you build and
test libtool using a cross toolchain that has no sysroot.
Last -- and frankly least important in comparison to keeping EXISTING
functionality working -- is the
3) test the new --with-sysroot support in conjunction with cross
toolchains that actually have a sysroot. This is covered by the
three new tests -- but requires toolchains with sysroot support.
The only answer for (3) is "Get Ralf Access To More Cross Toolchains".
FYI, the Mandriva 2010 mingw32-gcc package is built with a sysroot;
there's a VMware image here:
http://www.vmware.com/appliances/directory/va/434563
but you'd probably have to then do 'urpmi --fuzzy mingw' from inside it
as root to install the cross compiler. If VirtualBox is more your
style, the newest pre-packaged image I found was Mandriva 2009, and I
don't know how its mingw cross compiler was configured. So you'd
probably be better off converting the vmware image of 2010 to vdi format
using this procedure;
http://linuxpoison.blogspot.com/2010/06/how-to-convert-vmware-image-vmdk-to.html
> If there is anything in Automake that isn't working for this kind of
> test, well, then let's not use automake for this part of the Libtool
> testsuite. We've been avoiding it in most of the new testsuite already
> anyway.
Well, my confusion was you seemed to be concerned that 'make install
DESTDIR=...' doesn't work with $host=mingw (or, at least, when
--prefix=D:/Dos/Abs/Path is used). But DESTDIR is a creature of
automake. libtool has some support for it, but the "breakage" is
fundamental to the design of the DESTDIR thing: all install paths are
immediately prepended by $(DESTDIR).
That's just broken, unless $prefix begins with a directory separator.
(Even if you cheat and ensure that DESTDIR itself *ends* with a dirsep,
the fact is that
/my/destdir/C:/mingw
is an invalid path on win32 ANYWAY (e.g. if $build=win32, since $build
is where 'make install' happens), because ':' is not a legal character
for a filename!
But these issues are not something either libtool or automake should
worry about, unless they want to redesign the entire concept of DESTDIR.
(And good luck with that, when --prefix=/entirely/new/tmp/prefix already
works!)
--
Chuck
