Re: Amd64 relocation R_X86_64_32S in a static lib
On Wed, 6 Nov 2013, Torbjorn Granlund wrote: Exact decision for the change? I'm not sure what you mean by 'decision' there. If you're wondering about the _reason_ for the change (why we did it), the answer is so that ASLR is applied not just to the code in shared libraries but also the code in executables. If you're wondering what was done to implement the change, the part that seems to be annoying you is that in 5.3 the default for gcc is to pass -fpie to the C compiler and -pie to the linker. I am not annoyed. I am trying to understand the ABI change. Once I have done that, I will carefully adhere to it. The change makes be quite worried, since it *seems* to be done without proper understanding of the issues involved. Furthermore, the ABI change has not been properly announced, and the new ABI seems to be undocumented. That in turn makes me expect further ABI changes in future OpenBSD releases, which will likely break our code again. When GMP works incorrectly, then people tend to care very little whether it is due to a GMP bug, or due to platform problems. Yes, you can. If you look at libc.a, for example, you'll see it contains R_X86_64_GOTPCREL and R_X86_64_PLT32 relocations. Those work both when building a PIE executable and when building a fully static non-PIE executable. In the latter case they are resolved by the linker when generating the executable. Are you telling me that the most future-proof code would be all-PIC in the *static* GMP build? Should we perhaps stay away from R_X86_64_64 (also in a static lib)? Yes, static libs should be PIC (or a pic variant made available). Consider somebody wanting to link gmp statically into a shared library for example - or as in this case, linking gmp statically into a PIE executable. In the absense of documentation, could you tell which of the relocs on page 71 of the official AMD64 ELF ABI are to be supported on OpenBSD from now and on? It's very likely exactly the same as those that are allowed for PIE executables on all AMD64 ELF systems. I understand that on OpenBSD -fpie is now simply the default (for SUSE we build selected security relevant apps with -fpie, not all apps due to the fear of runtime overhead). Richard. -- Richard Biener rguent...@suse.de SUSE / SUSE Labs SUSE LINUX Products GmbH - Nuernberg - AG Nuernberg - HRB 16746 GF: Jeff Hawn, Jennifer Guild, Felix Imend ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
On Wed, 6 Nov 2013, Torbjorn Granlund wrote: ... The change makes be quite worried, since it *seems* to be done without proper understanding of the issues involved. Appearances can be deceiving. Furthermore, the ABI change has not been properly announced, I guess the release announcement wasn't enough? and the new ABI seems to be undocumented. Please file that bug upstream with GNU binutils, as they are the ones that have set the behavior of the -pie option. That in turn makes me expect further ABI changes in future OpenBSD releases, which will likely break our code again. Looking at the total ABI, including such things as types definitions and kernel syscalls, I can _guarantee_ that it'll change. Indeed, for 5.5 the type of time_t, clock_t, and ino_t has been changed to be long long on all archs, changing the size of many structures and rolling new syscall numbers. But I don't think that that's the level you're talking about. The low-level ABI of register use and accepted relocations changes much more rarely. We'll probably get TLS support hammered out in a way we're happy with at some point, which will affect the %fs register of course, but that's about the only thing at that level I can think of in the pipeline, and it's doesn't affect existing code anyway. Yes, you can. If you look at libc.a, for example, you'll see it contains R_X86_64_GOTPCREL and R_X86_64_PLT32 relocations. Those work both when building a PIE executable and when building a fully static non-PIE executable. In the latter case they are resolved by the linker when generating the executable. Are you telling me that the most future-proof code would be all-PIC in the *static* GMP build? Should we perhaps stay away from R_X86_64_64 (also in a static lib)? Yes, and yes. In the absense of documentation, could you tell which of the relocs on page 71 of the official AMD64 ELF ABI are to be supported on OpenBSD from now and on? I would love to see that information from the binutils people as well. Philip ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
On 2013/11/07 10:43, Richard Biener wrote: I understand that on OpenBSD -fpie is now simply the default (for SUSE we build selected security relevant apps with -fpie, not all apps due to the fear of runtime overhead). That is correct, -fpie is the default, and we explicitly disable it in a very small number of cases (basically just emacs and a couple of compilers). ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
Exact decision for the change? I'm not sure what you mean by 'decision' there. If you're wondering about the _reason_ for the change (why we did it), the answer is so that ASLR is applied not just to the code in shared libraries but also the code in executables. If you're wondering what was done to implement the change, the part that seems to be annoying you is that in 5.3 the default for gcc is to pass -fpie to the C compiler and -pie to the linker. I am not annoyed. I am trying to understand the ABI change. Once I have done that, I will carefully adhere to it. The change makes be quite worried, since it *seems* to be done without proper understanding of the issues involved. Furthermore, the ABI change has not been properly announced, and the new ABI seems to be undocumented. That in turn makes me expect further ABI changes in future OpenBSD releases, which will likely break our code again. When GMP works incorrectly, then people tend to care very little whether it is due to a GMP bug, or due to platform problems. Yes, you can. If you look at libc.a, for example, you'll see it contains R_X86_64_GOTPCREL and R_X86_64_PLT32 relocations. Those work both when building a PIE executable and when building a fully static non-PIE executable. In the latter case they are resolved by the linker when generating the executable. Are you telling me that the most future-proof code would be all-PIC in the *static* GMP build? Should we perhaps stay away from R_X86_64_64 (also in a static lib)? In the absense of documentation, could you tell which of the relocs on page 71 of the official AMD64 ELF ABI are to be supported on OpenBSD from now and on? -- Torbjörn ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
Philip Guenther guent...@gmail.com writes: Ah, but you are, sorta. In OpenBSD 5.3, platforms where the compiler and toolchain support were for robust for it were switched to build PIE objects and executables by default. So yes, that object _is_ expected to be position independent. c.f. the gcc-local(1) manpage. At least on OpenBSD, the processor will define __PIC__ and __pic__ when building both PIC _and_ PIE objects, so you can test those to determine whether the ASM should use position-independent code sequences. I am aware of that people have started using PIE as a novel term for PIC, sometimes with a subtle difference in meaning, sometimes with the exact same meaning. It is however evident that you're making some much larger distinction. Which one? R_X86_64_64 is safe for use in PIC/PIE code, though I would expect %rip relative addressing to be more efficient when it's applicable. Now I get really confused. You disallow R_X86_64_32S which is of the type S + A but allow R_X86_64_64 which is also of the type S + A. That simply makes no sense at all. If you allow R_X86_64_64 in a relocatable executable, that you will need load-time patching, which precludes any sharing. That's bad. But then why not allow R_X86_64_32S too? Now we have (at least) two OpenBSD ABIs for AMD64, pre 5.3 and now 5.3, 5.4. To make sense of things, I would not be surprised to see R_X86_64_64 banned from 5.5 and on, creating a 3rd OpenBSD AMD64 ABI. What was the exact decision for the change in 5.3. Is it spelled out somewhere for me to read? As I wrote, the seems to work approach isn't good for me, I prefer to do software engineering in a slightly more analytical manner. Is there some code which will work for all OpenBSD AMD64 releases? E.g., can I count on a PLT and GOT always? I really don't want to support many ABIs for one OS, such support tend to be fragile. Torbjörn ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
Torbjorn Granlund t...@gmplib.org writes: Now we have (at least) two OpenBSD ABIs for AMD64, pre 5.3 and now 5.3, 5.4. To make sense of things, I would not be surprised to see R_X86_64_64 banned from 5.5 and on, creating a 3rd OpenBSD AMD64 ABI. I don't understand the fine details of which reloc types make sense in pic code, but if I understand Philip correctly, the main problem is not a ABI change, but changed compiler default. And then you get link errors when linking together pic objects created by the compiler, with non-pic objects created from the gmp assembly files. I think you'd get similar problems as if a user configures gmp with, e.g., --disable-shared CFLAGS=-fpic. Do you agree with this analysis? To figure out if an invocation $(CC) $(CFLAGS) generates pic code or not, one needs a configure test like AC_TRY_COMPILE( ... #if __PIC__ #error bla bla #endif ... ) Using the result of that test when deciding whether or not assembly files should use pic mode should give the correct behavior, both with compilers doing pic by default, and users enabling it explicitly. /Niels -- Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
On Tue, 5 Nov 2013, Torbjorn Granlund wrote: Philip Guenther guent...@gmail.com writes: Ah, but you are, sorta. In OpenBSD 5.3, platforms where the compiler and toolchain support were for robust for it were switched to build PIE objects and executables by default. So yes, that object _is_ expected to be position independent. c.f. the gcc-local(1) manpage. At least on OpenBSD, the processor will define __PIC__ and __pic__ when building both PIC _and_ PIE objects, so you can test those to determine whether the ASM should use position-independent code sequences. I am aware of that people have started using PIE as a novel term for PIC, sometimes with a subtle difference in meaning, sometimes with the exact same meaning. It is however evident that you're making some much larger distinction. Which one? We give it the same meaning as gcc and its documentation. For example: `-fpie' `-fPIE' These options are similar to `-fpic' and `-fPIC', but generated position independent code can be only linked into executables. Usually these options are used when `-pie' GCC option will be used during linking. R_X86_64_64 is safe for use in PIC/PIE code, though I would expect %rip relative addressing to be more efficient when it's applicable. Now I get really confused. You disallow R_X86_64_32S which is of the type S + A but allow R_X86_64_64 which is also of the type S + A. That simply makes no sense at all. If you allow R_X86_64_64 in a relocatable executable, that you will need load-time patching, which precludes any sharing. Yeah, I spoke too broadly: R_X86_64_64 is only okay for relocations in writable segments. The OpenBSD libc has some of those in its .data segment, for exmaple. You're correct that R_X86_64_64 relocations against the text segment prevent sharing. That's bad. But then why not allow R_X86_64_32S too? Because there's no guarantee that the target is within 2GB, as the reference may resolve to the symbol in a different shared object which is too far away. What was the exact decision for the change in 5.3. Is it spelled out somewhere for me to read? Exact decision for the change? I'm not sure what you mean by 'decision' there. If you're wondering about the _reason_ for the change (why we did it), the answer is so that ASLR is applied not just to the code in shared libraries but also the code in executables. If you're wondering what was done to implement the change, the part that seems to be annoying you is that in 5.3 the default for gcc is to pass -fpie to the C compiler and -pie to the linker. Is there some code which will work for all OpenBSD AMD64 releases? E.g., can I count on a PLT and GOT always? I really don't want to support many ABIs for one OS, such support tend to be fragile. Yes, you can. If you look at libc.a, for example, you'll see it contains R_X86_64_GOTPCREL and R_X86_64_PLT32 relocations. Those work both when building a PIE executable and when building a fully static non-PIE executable. In the latter case they are resolved by the linker when generating the executable. Philip Guenther ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
Torbjorn Granlund t...@gmplib.org writes: If one tries to assemble and link the former code on a 5.3/5.4 system, one gets again the error relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC. What other objects were involved in the link? I guess something like crt.o was compiled as non-pic on pre-5.3 systems, but as pic-code on 5.3? I'd like to know if the problem is that the linker input files are an somehow incompatible mix of pic and non-pic objects, or if the problem is is that by default the linker insists on creating a position independent executable (PIE). I'd call this an ABI change since code valid under the AMD64 ELF ABI as well as on older OpenBSD releases is no longer supported. If you move an *executable* from pre-5.3 to 5.3, it might still work fine, if non-position-independent-executables are still supported, and compatible crt.o and the like are included in the executable. It's very clear that the ABI-spec defines which executables and shared libraries are valid, but I imagine one could have a long flamewa^H^H^H^H^H^Hdebate on whether or not the ABI really specifies which object files and static libraries are valid when used with the default compiler settings. But in any case, it's going to be inconvenient for users if static libraries can't be copied over to a later OS release. (There are pros and cons with feature tests, though. I consider the free software OSes out there to be primary targets for GMP, and I'd like to avoid creating GMP objects that cannot be moved between FooBSD x.y and FooBSD x.y+k. I think it would be a good test to compile gmp on pre-5.3, and use that library when compiling and linking some simple program on openbsd-5.3. Trying both with a static and a shared gmp library. And maybe trying also with --disable-assembly. If any of that fails, that's clearly demonstrates some type of ABI incompatibility. I'd also like to see the motivation for the discontinued support for movl$17, %edi call*jumptab(,%rax,8) Which seems to be a very sensible thing to use in pic code. Offsets between text and data segment are usually known at link time. Does openbsd use some flavor of address space randomization which adds a random load-time offset between .text and .data? Regards, /Niels -- Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Re: Amd64 relocation R_X86_64_32S in a static lib
Philip Guenther guent...@gmail.com writes: That's bad. But then why not allow R_X86_64_32S too? Because there's no guarantee that the target is within 2GB, as the reference may resolve to the symbol in a different shared object which is too far away. Can you use ELF visibility to provide the needed guarantee that it will never resolve to a symbol in a different shared object? I think references *within* a shared object is an important and common case, worthy of some optimization. Regards, /Niels -- Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
Amd64 relocation R_X86_64_32S in a static lib
I am working on getting the GMP bignum library to work better on OpenBSD. With current GMP sources (GMP 5.0.x, 5.1.x, and development head) a 'fat' build will not work on amd64 under OpenBSD 5.3 and 5.4. With older version of OpenBSD (I've tested 4.9, 5.0, 5.2) things work as expected. The problem is related to relocs, in particlar R_X86_64_32S which we use in a GMP assembly file for 'fat' binaries. On the problem OpenBSD releases, a fat GMP build end with this error: libtool: link: gcc -std=gnu99 -O2 -pedantic -fomit-frame-pointer -m64 -o t-bswap t-bswap.o ./.libs/libtests.a /var/tmp/gmp-obj/hannahobsd64v53-stat-fat/.libs/libgmp.a ../.libs/libgmp.a /usr/bin/ld: /var/tmp/gmp-obj/hannahobsd64v53-stat-fat/.libs/libgmp.a(fat_entry.o): relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC /var/tmp/gmp-obj/hannahobsd64v53-stat-fat/.libs/libgmp.a: could not read symbols: Bad value The error message is strange, considering that we certainly are not making a shared object. We are linking a plain object with a static library. I have not been able to find any information about what OpenBSD is trying to do. But I can discern some dislike for R_X86_64_32S in OpenBSD 5.3 5.4; the bundled gcc typically avoids constructs which lead to its use, and instead uses GOT relocs. In older OpenBSD releases, the construct worked as expected, and the bundled gcc did not avoid generating R_X86_64_32S (except for PIC code of course). I consider this a major deviation from the established AMD64 ELF ABI. It's your decision, and I will not try to make you change your minds. But I need to understand the decision and how you intend to proceed. Do you have a document describing the AMD64 OpenBSD ABI with which developers could comply? If you don't have such a document, then please help with the present problem. * Which subset of the AMD64 relocs are to be supported by OpenBSD henceforth? * It seems R_X86_64_64 works in non-PIC code such as in the GMP example, and while slower than R_X86_64_32S, GMP could use it as a workaround. But do you intend to keep that working, or do you intend to completely de-support any non-GOT data references? The last question is motivated by the fact that you seem to have made gcc use just GOT data references. That something seems to work is not good enough for me. -- Torbjörn ___ gmp-devel mailing list gmp-devel@gmplib.org http://gmplib.org/mailman/listinfo/gmp-devel
