In the general case, does this actually change the API/ABI that your
application uses, or does it just change the ABI that your application
_claims_ to expect? That is, does your memcpy.c emit code that actually
intends to call memcpy@GLIBC_2.2.5, or does it emit code that intends to
call memcpy@@GLIBC_2.14 but lies about it because of .symver?
In the particular case of memcpy, I think this is fine because the ABI
change from 2.2.5 to 2.14 is forwards-compatible, if I'm reading the
manpage and glibc commit 0354e355 right. glibc used to have a memcpy that
was safe to call with overlapping regions; the spec says memcpy requires
the regions not to overlap, and you should use memmove if they overlap. In
glibc 2.13, they optimized it assuming the regions didn't overlap, which
broke older programs. So in glibc 2.14, they aliased memcpy@GLIBC_2.2.5 to
memmove, and added the symbol version GLIBC_2.14 to the new, optimized
memcpy.
For an application calling memcpy correctly, I think this means _either_
memcpy@GLIBC_2.2.5 or memcpy@@GLIBC_2.14 is fine to call, and if you have
the option of either, memcpy@@GLIBC_2.14 is going to be faster (which is
why newer toolchains default to it), but the two have the same calling
convention and everything.
This means that, if the only incompatibility is just memcpy, this approach
should work -- but also you can probably define a weak symbol named
memcpy@@GLIBC_2.14 that just relocates to memcpy@GLIBC_2.2.5 (and perhaps
auditwheel can stuff this symbol into your ELF objects, without needing to
change the compilation process). If the final system's libc provides
memcpy@@GLIBC_2.14, then you'll still get the faster version.
Is this the only incompatible symbol worth worrying about? If there are
others that actually changed ABI in a backwards-incompatible way (that is,
you can't call a program compiled with the new symbol against the old
symbol, and glibc provides two disjoint versioned implementations) then I
suspect this is unsafe.
--
Geoffrey Thomas
https://ldpreload.com
[email protected]
On Fri, 7 Jul 2017, Robert T. McGibbon wrote:
Hey all,
I think I may have figured out a new way to build manylinux1 on non-CentOS 5
machines with newer toolchains, at least in relatively
simple cases. The thing that prevents most libraries from being
manylinux1-compatible is that they link against too-recent versioned
symbols in glibc. This suggests, then, that we might be able to fix the problem
during compilation by forcing the linker to link the
libraries against older (manylinux1 compatible) symbols. It seems like there
are some assembly + linker tricks (also this) that work to
force just that.
In order to test this out, I took a look at the symbols that were causing
manylinux1 incompatibility in a project of mine when compiled
on CentOS 7. In this case, it was just memcpy@@GLIBC_2.14.
So, I dropped this file into my project:
```
$ cat memcpy.c
#include <string.h>
asm (".symver memcpy, memcpy@GLIBC_2.2.5");
void *__wrap_memcpy(void *dest, const void *src, size_t n) {
return memcpy(dest, src, n);
}
```
And then modified my setup.py to
````
+def manylinux1(extensions):
+ for ext in extensions:
+ ext.sources.append('memcpy.c')
+ ext.extra_link_args.append('-Wl,--wrap=memcpy')
+ return extensions
+
setup(name='project_name
author='Robert McGibbon',
author_email='[email protected]',',
- ext_modules=extensions,
+ ext_modules=manylinux1(extensions),
```
Lo and behold, it actually works! Obviously one would have to wrap more symbols
for other projects that make heavier use of glibc and
there's nothing that this can do about the fact for wheels that link against
external, precompiled libraries that auditwheel grafts into
the manylinux wheel, since it requires changes to the compile, but it's still
cool.
Has anyone tried this kind of thing before?
--
-Robert
_______________________________________________
Wheel-builders mailing list
[email protected]
https://mail.python.org/mailman/listinfo/wheel-builders
