Re: x86_64-gnu port -- help needed
On 9 Mar 2023, at 19:44, Sergey Bugaev wrote: > > On Thu, Mar 9, 2023 at 8:26 PM Andreas Schwab wrote: >> Similarily, something is pulling in strtoul.os because it references a >> symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c. >> >> elf/librtld.mapT should tell you where the references come from. > > Thank you!!! > > Apparently I'm misunderstanding how static linking works. I have > always assumed that linking a static library works just like linking > in all of its object files -- i.e. conceptually *all* object files are > linked in, but the linker could optimize some away if they are not > referenced by anything else. But what you're saying seems to indicate > that object files are only pulled in by something referencing a symbol > that they define, i.e. this is done lazily, not eagerly. Object files listed explicitly on the command line are eagerly added. Object files from libraries are lazily added. With --gc-sections some parts of the resulting binary may also later be deleted, but that is opt-in and limited in scope (and there are ways to override that). Jess > In my mental > model, it is fully expected that you do get all the errors like > duplicate symbols and undefined symbol references from the static > library, even if you don't actually use much of it, because > conceptually you're linking all of it in. That's why I was so > surprised to see this work for i686-gnu. > > That also explains why it's seemingly linking all of glibc into ld.so: > it's only supposed to pull in some (hopefully very few) objects that > it needs, and not all of it. > > And by inspecting elf/librtld.mapT, I see that the reason it's pulling > _exit.os in is that ld.so needs __libc_tls_initialized in a bunch of > places, and that starts a chain of dependencies that ultimately leads > to _exit.os. And that also explains that difference between i686-gnu > and x86_64-gnu: i686-gnu just kind of looks at %gs to figure out if > TLS has been initialized, but on x86_64 I had to add > __libc_tls_initialized as an explicit flag. > > I recall that ldsodefs.h/GL is the way to share some piece of data > between ld.so and glibc proper? Should I put __libc_tls_initialized in > there? How do I access GL() values from assembler, do I need to add it > to some xxx-offsets.sym file? In fact, I see that there is already a > sysdeps/x86_64/rtld-offsets.sym, that contains offsetof (struct > rtld_global, _dl_tls_generation), so likely yes. > > Alternatively, what if I made a separate weak declaration of > __libc_tls_initialized in dl-sysdep.c, would that work & get > automatically upgraded to the "real" one once glibc is loaded? Do I > understand it correctly that the downside of this approach is that > this will generate another GLIBC_PRIVATE symbol reference (or will > it?), and that's not nice, whereas rtld_global is one symbol > multiplexing many useful pieces of data? > > Sergey >
Re: x86_64-gnu port -- help needed
On Thu, Mar 9, 2023 at 8:26 PM Andreas Schwab wrote: > Similarily, something is pulling in strtoul.os because it references a > symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c. > > elf/librtld.mapT should tell you where the references come from. Thank you!!! Apparently I'm misunderstanding how static linking works. I have always assumed that linking a static library works just like linking in all of its object files -- i.e. conceptually *all* object files are linked in, but the linker could optimize some away if they are not referenced by anything else. But what you're saying seems to indicate that object files are only pulled in by something referencing a symbol that they define, i.e. this is done lazily, not eagerly. In my mental model, it is fully expected that you do get all the errors like duplicate symbols and undefined symbol references from the static library, even if you don't actually use much of it, because conceptually you're linking all of it in. That's why I was so surprised to see this work for i686-gnu. That also explains why it's seemingly linking all of glibc into ld.so: it's only supposed to pull in some (hopefully very few) objects that it needs, and not all of it. And by inspecting elf/librtld.mapT, I see that the reason it's pulling _exit.os in is that ld.so needs __libc_tls_initialized in a bunch of places, and that starts a chain of dependencies that ultimately leads to _exit.os. And that also explains that difference between i686-gnu and x86_64-gnu: i686-gnu just kind of looks at %gs to figure out if TLS has been initialized, but on x86_64 I had to add __libc_tls_initialized as an explicit flag. I recall that ldsodefs.h/GL is the way to share some piece of data between ld.so and glibc proper? Should I put __libc_tls_initialized in there? How do I access GL() values from assembler, do I need to add it to some xxx-offsets.sym file? In fact, I see that there is already a sysdeps/x86_64/rtld-offsets.sym, that contains offsetof (struct rtld_global, _dl_tls_generation), so likely yes. Alternatively, what if I made a separate weak declaration of __libc_tls_initialized in dl-sysdep.c, would that work & get automatically upgraded to the "real" one once glibc is loaded? Do I understand it correctly that the downside of this approach is that this will generate another GLIBC_PRIVATE symbol reference (or will it?), and that's not nice, whereas rtld_global is one symbol multiplexing many useful pieces of data? Sergey
Re: x86_64-gnu port -- help needed
On Mär 09 2023, Sergey Bugaev wrote: > x86_64-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o > elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a > mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)' > -Wl,-Map,elf/librtld.mapT > x86_64-gnu/bin/ld: libc_pic.a(_exit.os): in function `__GI__exit': > posix/../sysdeps/mach/hurd/_exit.c:50: multiple definition of `__GI__exit'; > elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:728: first defined > here The only symbol provided by _exit.os that is not defined by ../sysdeps/mach/hurd/dl-sysdep.c is _Exit. Does anything call this function? > x86_64-gnu/bin/ld: libc_pic.a(strtoul.os): in function > `__GI___strtoul_internal': > stdlib/../sysdeps/wordsize-64/strtoul.c:108: multiple definition of > `__GI___strtoul_internal'; > elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:713: first defined > here Similarily, something is pulling in strtoul.os because it references a symbol from there not defined by ../sysdeps/mach/hurd/dl-sysdep.c. elf/librtld.mapT should tell you where the references come from. > 1. What is this about, what's it even trying to do? If this is linking the > dynamic linker (ld.so / rtld — these are the same thing, right?), then why > does it need the dl-sysdep version if the real version is available? The comments in ../sysdeps/mach/hurd/dl-sysdep.c indicate that the dynamic linker wants to use some stripped down implementation of these functions. > 3. How come this works on i686-gnu, the duplicated symbols are clearly > present there too! Pure luck, perhaps. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
x86_64-gnu port -- help needed
Hello! So continuing with the x86_64-gnu port, I wrote and tweaked some things and got it to compile further, and now I'm facing linking issues. I've been scratching my head (figuratively...) about this one for > 24 hours now, so it's only appropriate that I ask for some help. First, the error message: x86_64-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)' -Wl,-Map,elf/librtld.mapT x86_64-gnu/bin/ld: libc_pic.a(_exit.os): in function `__GI__exit': posix/../sysdeps/mach/hurd/_exit.c:50: multiple definition of `__GI__exit'; elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:728: first defined here x86_64-gnu/bin/ld: libc_pic.a(strtoul.os): in function `__GI___strtoul_internal': stdlib/../sysdeps/wordsize-64/strtoul.c:108: multiple definition of `__GI___strtoul_internal'; elf/dl-allobjs.os:elf/../sysdeps/mach/hurd/dl-sysdep.c:713: first defined here (I've trimmed down the paths to be less unwieldy; in reality they're all absolute, e.g. /home/sergey/dev/crosshurd64/src/glibc/build/elf/librtld.map.o) My understanding is that __GI__ symbols are created by the hidden_proto/hidden_def macros from include/libc-symbols.h, and used for avoiding PLT when using glibc symbols from inside glibc itself (how it's different to the __-prefixed symbols, I have not yet figured out). There is indeed the __GI__exit strong .text symbol in both libc_pic.a(_exit.os) and elf/dl-allobjs.os: $ nm -A libc_pic.a 2>/dev/null | rg __GI__exit libc_pic.a:version.os: U __GI__exit libc_pic.a:abort.os: U __GI__exit libc_pic.a:exit.os: U __GI__exit libc_pic.a:_exit.os:0460 T __GI__exit libc_pic.a:daemon.os: U __GI__exit libc_pic.a:openchild.os: U __GI__exit libc_pic.a:forkpty.os: U __GI__exit $ nm -A elf/dl-allobjs.os | rg _exit elf/dl-allobjs.os:000192b0 T __check__exit_no_hidden elf/dl-allobjs.os:000192b0 W _exit elf/dl-allobjs.os:000192b0 T __GI__exit elf/dl-allobjs.os: U __proc_mark_exit but this same thing is true of my i686-gnu build, which somehow works! $ nm -A libc_pic.a 2>/dev/null | rg __GI__exit libc_pic.a:version.os: U __GI__exit libc_pic.a:abort.os: U __GI__exit libc_pic.a:exit.os: U __GI__exit libc_pic.a:_exit.os:04a0 T __GI__exit libc_pic.a:daemon.os: U __GI__exit libc_pic.a:openchild.os: U __GI__exit libc_pic.a:forkpty.os: U __GI__exit $ nm -A elf/dl-allobjs.os | rg _exit elf/dl-allobjs.os:000190a0 T __check__exit_no_hidden elf/dl-allobjs.os:000190a0 W _exit elf/dl-allobjs.os:000190a0 T __GI__exit elf/dl-allobjs.os: U __proc_mark_exit Here, it just says i686-gnu-gcc -nostdlib -nostartfiles -r -o elf/librtld.map.o elf/librtld.mapT.o '-Wl,-(' elf/dl-allobjs.os libc_pic.a mach/libmachuser_pic.a hurd/libhurduser_pic.a -lgcc '-Wl,-)' -Wl,-Map,elf/librtld.mapT i686-gnu/bin/ld: warning: elf/dl-allobjs.os: requires executable stack (because the .note.GNU-stack section is executable) (Again, I've trimmed the paths.) Questions: 1. What is this about, what's it even trying to do? If this is linking the dynamic linker (ld.so / rtld — these are the same thing, right?), then why does it need the dl-sysdep version if the real version is available? If this is something else, such as maybe testing that the dynamic linker and glibc proper don't have symbol conflicts, then clearly they do! 2. Why does dl-sysdep define a strong __GI__exit and not a weak one? Isn't the point to upgrade to the full version once it's available? (But would that even work, considering __GI__ function calls do not go through PLT?) 3. How come this works on i686-gnu, the duplicated symbols are clearly present there too! 4. Whatever the answer to #2 is, why doesn't it work on x86_64-gnu? Sergey