Bug#793641: glibc: too few static TLS slots
Control: tag -1 fixed-upstream Hi Carlos, On 17.08.2015 21:04, Carlos O'Donell wrote: On Sun, Aug 16, 2015 at 12:48 PM, Andreas Cadhalpun andreas.cadhal...@googlemail.com wrote: until there's a better tested and working way to transition to ffmpeg? This really doesn't have that much to do with the transition to ffmpeg. Any other library that (indirectly) links against sufficiently many STATIC_TLS using ones and is used in some plugin is also affected. Note that there are currently 14 STATIC_TLS slots and gst-libav uses 6, while it used 4 previously. It is a mere coincidence that this increase causes totem to hit the arbitrary limit in glibc. The math can't be done that easily, it's actually a heuristic, but pretend it works this way for the sake of this discussion. Apologies for the inaccuracies in my explanation. I'm not that familiar with glibc internals. ;) The only immediate solution is for debian's glibc to increase DTV_SURPLUS to 32 or higher. This is exactly what I did for Fedora. The other alternative is to backport Alex's fixes for Sourceware bugs BZ #17090, BZ #17620, BZ #17621, BZ #17628, which corrects the heuristics behind DTV_SURPLUS which prevent the loading of STATIC_TLS. Without Alex's fixes the implementation is limited, by a heuristic, to a limited number of libraries with STATIC_TLS, and after Alex's fixes (available in glibc 2.22) the limit is You may load as many STATIC_TLS libraries as long as you have static image space for all of them (which is a much higher arbitrary limit). So this is already fixed upstream. Great! In that case I don't mind working around that in ffmpeg until glibc 2.22 reaches sid. Best regards, Andreas
Bug#793641: glibc: too few static TLS slots
Hi, On 16.08.2015 17:12, Andreas Henriksson wrote: Thanks for your interest in this issue. You're welcome. On Sun, Jul 26, 2015 at 09:35:16PM +0200, Andreas Cadhalpun wrote: Control: severity 793641 important [...] I'd like to start out with questioning this gstreamer1.0-libav is completely unuable now, breaking very popular programs like totem. Severity should in my opinion be grave. The severity had been normal originally so I increased it. I didn't chose grave, because this bug doesn't make glibc unusable. You can adjust the severity, if you think that is appropriate, but arguing about bug severity is usually not helpful. [...] The link [1] provided there is quite helpful. [...] 1: https://bugzilla.redhat.com/show_bug.cgi?id=1124987 It indeed is, and I'd like to point out comment #22: For the record it is not a glibc bug. It is a defect in the loaded libraries, they should not require TLS. Fundamentally it's a defect that we allowed developers to build shared libraries with static TLS in the first place. https://bugzilla.redhat.com/show_bug.cgi?id=1124987#c22 Then let me also point out form comment #15: 108 /lib64/libgomp.so.1 - OK. May use STATIC_TLS, it's part of the implementation. Language runtime support for gomp shared across all programs. 0/lib64/libcrypt.so.1 0 /lib64/libm.so.6 0 /lib64/libnsl.so.1 0 /lib64/libnss_files.so.2 0 /lib64/libpthread.so.0 0 /lib64/libresolv.so.2 0 /lib64/librt.so.1 0 /lib64/libutil.so.1 - OK, all part of the implementation (glibc). So it's not a bug in libgomp and libresolv and hence can only be fixed in glibc. No matter if this is a bug in glibc or not, ffmpeg maintainers should take any 'temporary' measures at hand to ensure usability and not expect changes to be done on a whim to critical system components like glibc. I had hoped that one of the glibc maintainers would comment on this bug, before we resort to workarounds. This breakage has persisted way too long now. However, surprisingly few people complained about it... If there are no other option, As I wrote the other option would be to work around this bug by not linking ffmpeg against some external libraries. Can you confirm that removing --enable-libsoxr from ffmpeg's debian/rules avoids this problem? If not, also removing --enable-libssh should work. maybe it's time to start planning transitioning back to libav No. until there's a better tested and working way to transition to ffmpeg? This really doesn't have that much to do with the transition to ffmpeg. Any other library that (indirectly) links against sufficiently many STATIC_TLS using ones and is used in some plugin is also affected. Note that there are currently 14 STATIC_TLS slots and gst-libav uses 6, while it used 4 previously. It is a mere coincidence that this increase causes totem to hit the arbitrary limit in glibc. Speaking to #debian-multimedia there doesn't seem to be much interest in dealing with the breakage caused by the ffmpeg transition right now. Since the bug is in glibc, only the glibc maintainers can fix it. Best regards, Andreas
Bug#793641: glibc: too few static TLS slots
Control: severity 793641 important Control: reassign 793689 libc6 2.19-19 Control: reassign 793641 libc6 2.19-19 Control: forcemerge 793641 793689 Control: retitle 793641 glibc: too few static TLS slots Control: tags 793641 patch Hi, On 26.07.2015 17:42, Tom Rauchenwald wrote: I can't play mp4 files using totem anymore, only a black screen is displayed. This looks similar to bug #793641 for vlc thus I'm merging the two. The link [1] provided there is quite helpful. The problem is that ffmpeg links against more libraries than libav and some of those use static TLS. Using the method from the redhat bug to determine which libraries use TLS reveals: $ for lib in $(ldd /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstlibav.so | grep = | sed 's/.*= \([/a-z0-9\._+-]*\) .*/\1/g' | sort); do echo readelf -d -W $lib; readelf -d -W $lib; done | egrep 'readelf|STATIC_TLS' | grep -B1 TLS readelf -d -W /lib/x86_64-linux-gnu/libc.so.6 0x001e (FLAGS) STATIC_TLS -- readelf -d -W /lib/x86_64-linux-gnu/libm.so.6 0x001e (FLAGS) STATIC_TLS -- readelf -d -W /lib/x86_64-linux-gnu/libpthread.so.0 0x001e (FLAGS) STATIC_TLS readelf -d -W /lib/x86_64-linux-gnu/libresolv.so.2 0x001e (FLAGS) STATIC_TLS readelf -d -W /lib/x86_64-linux-gnu/librt.so.1 0x001e (FLAGS) STATIC_TLS -- readelf -d -W /usr/lib/x86_64-linux-gnu/libgomp.so.1 0x001e (FLAGS) STATIC_TLS Previously that was only: readelf -d -W /lib/x86_64-linux-gnu/libc.so.6 0x001e (FLAGS) STATIC_TLS -- readelf -d -W /lib/x86_64-linux-gnu/libm.so.6 0x001e (FLAGS) STATIC_TLS -- readelf -d -W /lib/x86_64-linux-gnu/libpthread.so.0 0x001e (FLAGS) STATIC_TLS readelf -d -W /lib/x86_64-linux-gnu/librt.so.1 0x001e (FLAGS) STATIC_TLS Thus it can happen that when totem/vlc tries to load the plugin not enough free static TLS slots remain. I was told on #debian-multimedia to report the bug against ffmpeg. The only thing ffmpeg could do is not to link against some libraries, losing some features. I think this would be wrong. Thus I think the only solution is to increase the DTV_SURPLUS limit in glibc. Therefore I'm reassigning this bug there. I get the following message on the console: (totem:7753): GStreamer-WARNING **: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstlibav.so': dlopen: cannot load any more object with static TLS ** Message: Missing plugin: gstreamer|1.0|totem|H.264 decoder|decoder-video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)high, parsed=(boolean)true (H.264 decoder) ** Message: Automatic missing codec installation not supported (helper script missing) The file is present on my system: -rw-r--r-- 1 root root 236016 Jul 20 20:29 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstlibav.so Any help would be appreciated! This works again for me, after installing a libc6 from glibc rebuilt with the following patch: ---8--- --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -393,7 +393,7 @@ struct rtld_global #define TLS_SLOTINFO_SURPLUS (62) /* Number of additional slots in the dtv allocated. */ -#define DTV_SURPLUS(14) +#define DTV_SURPLUS(64) /* Initial dtv of the main thread, not allocated with normal malloc. */ EXTERN void *_dl_initial_dtv; ---8--- Best regards, Andreas 1: https://bugzilla.redhat.com/show_bug.cgi?id=1124987 -- To UNSUBSCRIBE, email to debian-glibc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55b53674.7010...@googlemail.com
Bug#792921: [sparc64] linking against libx264 crashes runtime linker
Hi Carlos, Thanks for looking into this. ;) On 20.07.2015 17:18, Carlos O'Donell wrote: On Mon, Jul 20, 2015 at 3:35 AM, Andreas Cadhalpun andreas.cadhal...@googlemail.com wrote: Program received signal SIGBUS, Bus error. 0xf801cda4 in elf_dynamic_do_Rela (skip_ifunc=optimized out, lazy=0, nrelative=optimized out, relsize=optimized out, reladdr=optimized out, map=0xf80100023570) at do-rel.h:111 Usually a corrupted library. Check md5sums. I don't think this problem is caused by a corrupted libx264. First, it's also happening on the sparc64 buildds, see e.g. [1]. Second, I rebuilt the current x264 locally and it shows the same problem. Best regards, Andreas 1: http://buildd.debian-ports.org/status/fetch.php?pkg=ffmpegarch=sparc64ver=7%3A2.7.1-1stamp=1435386635 -- To UNSUBSCRIBE, email to debian-glibc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55ae9448.6080...@googlemail.com
Bug#792921: [sparc64] linking against libx264 crashes runtime linker
On 21.07.2015 21:44, Carlos O'Donell wrote: Does the problem always reproduce or just sometimes? Always. If it's just sometimes then it's much much harder to figure out what's wrong. If it were just sometimes, I wouldn't have been able to trace it to libx264... You'll need a dedicated person to track down exactly what is the concurrency issue and why it's failing. What I don't understand is why it only fails for libx264, but e.g. libx265 is fine. Also, I don't see how the code, where the crash happens, can possibly crash: From do-rel.h [1]: 85: const ElfW(Rel) *relative = r; 86: r += nrelative; [...] 111:for (; relative r; ++relative) 112: DO_ELF_MACHINE_REL_RELATIVE (map, l_addr, relative); gdb claims it crashes at line 111. Best regards, Andreas 1: https://sources.debian.net/src/glibc/2.19-19/elf/do-rel.h/?hl=111#L111 -- To UNSUBSCRIBE, email to debian-glibc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/55aead25.5000...@googlemail.com
Bug#792921: [sparc64] linking against libx264 crashes runtime linker
Package: x264,libc6
Version: x264/2:0.142.2431+gita5831aa-1
Version: libc6/2.19-19
Severity: important
X-Debbugs-Cc: debian-sp...@lists.debian.org
Dear Maintainers,
The x264 package on sparc64 is unusable, because linking against
the libx264 library causes the runtime linker to crash:
$ cat main.c
int main () {
return 0;
}
$ rm ./main
rm: cannot remove './main': No such file or directory
$ cc -g -lx264 main.c -o main
$ ./main; echo $?
138
$ gdb --batch -ex r -ex bt -ex q --args ./main
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
Program received signal SIGBUS, Bus error.
0xf801cda4 in elf_dynamic_do_Rela (skip_ifunc=optimized out, lazy=0,
nrelative=optimized out, relsize=optimized out, reladdr=optimized out,
map=0xf80100023570) at do-rel.h:111
111 do-rel.h: No such file or directory.
#0 0xf801cda4 in elf_dynamic_do_Rela (skip_ifunc=optimized out,
lazy=0, nrelative=optimized out, relsize=optimized out, reladdr=optimized
out, map=0xf80100023570) at do-rel.h:111
#1 _dl_relocate_object (scope=optimized out, reloc_mode=optimized out,
consider_profiling=optimized out, consider_profiling@entry=0) at
dl-reloc.c:264
#2 0xf8013f2c in dl_main (phdr=optimized out, phnum=optimized out,
user_entry=optimized out, auxv=optimized out) at rtld.c:2205
#3 0xf80100018be4 in _dl_sysdep_start
(start_argptr=start_argptr@entry=0x7fefdc0, dl_main=0xf8011e00
dl_main) at ../elf/dl-sysdep.c:249
#4 0xf8015630 in _dl_start_final (arg=0x7fefdc0) at rtld.c:331
#5 _dl_start (arg=0x7fefdc0) at rtld.c:557
#6 0xf80116ec in _start () from /lib64/ld-linux.so.2
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I'm not sure what the cause for this crash is, so I'm filing this bug
against both packages. Please reassign as appropriate.
Best regards,
Andreas
--
To UNSUBSCRIBE, email to debian-glibc-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/55aca4db.8080...@googlemail.com
Bug#752237: libc0.3: send() asked to transmit 0 chars still triggers recv() on Hurd
Hi Richard, On 28.06.2014 10:48, Richard Braun wrote: Thanks for the report. There are actually two sides of the problem. First, I agree that there seems to be a bug, but let's take a closer look at the spec. The return value for recv() is defined as : Upon successful completion, recv() shall return the length of the message in bytes. If no messages are available to be received and the peer has performed an orderly shutdown, recv() shall return 0. This doesn't creates a strict equivalence between orderly shutdown and shall return 0. But in practice, this seems to be the actual assumption, so let's say that there is indeed a Hurd bug here. By the way, although the send() and recv() functions themselves (in glibc) may benefit from additions to filter out empty messages (at least send()), the server functions are those found in pflocal concerning AF_UNIX sockets. But there is also a bug in the client code, IMO. Here is how send() is specified : The length of the message to be sent is specified by the length argument. [...] If the socket argument refers to a socket and the flags argument is 0, the send() function is equivalent to write(). And here is write() : If nbyte is zero and the file is not a regular file, the results are unspecified. We might also want to change this though, since the behaviour observed on other systems seems more appropriate. Thanks for looking into this. Indeed, according to the specification, the behavior is unspecified. (Which IMHO is a bug in the specification.) But the specification also says: Before any action described below is taken, and if nbyte is zero and the file is a regular file, the write() function may detect and return errors as described below. In the absence of errors, or if error detection is not performed, the write() function shall return zero and have no other results. This holds only for regular files, but the principle of least surprise suggests that it should also hold for special files, if that is possible. On 28.06.2014 11:51, Richard Braun wrote: I've committed a fix upstream [1], please see if it solves your issues as well. I tried this patch and with it applied both the example program and clamd/clamdscan work as they should. But I can't comment on Samuel's objections to the patch. Best regards, Andreas -- To UNSUBSCRIBE, email to debian-glibc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53aef770.6060...@googlemail.com
Bug#752237: libc0.3: send() asked to transmit 0 chars still triggers recv() on Hurd
Package: libc0.3
Version: 2.19-3
Severity: normal
X-Debbugs-CC: debian-h...@lists.debian.org
Dear Maintainer,
it seems send() on Hurd doesn't work like it does everywhere else.
Attached is a simple test case.
To reproduce the problem, execute make in a folder with the attached
client.c, server.c and Makefile.
Then run ./server, open another terminal and run ./client.
The usual output is:
$ ./server
Message: 'TEST'
$ ./client
Socket works.
Answer: 'ANSWER'
But on Hurd one gets:
$ ./server
Message: ''
$ ./client
Socket works.
Answer: 'ANSWER'
This is because the client is calling:
send(sockfd, , 0, 0)
Normally this doesn't trigger recv() in the server and thus can be used
to test, whether the socket is working.
But on Hurd it actually sends an empty message, so that the real
message, which is sent later, is not received.
As a workaround, one can comment out this test, which is done in the
server, so that the ANSWER reaches the client.
This should be handled on Hurd in the same way as on other platforms.
I noticed this bug, because it breaks the communication between clamd
and clamdscan on Hurd, thus leading to test failures and ultimately a FTBFS.
Best regards,
Andreas
-- System Information:
Debian Release: jessie/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: hurd-i386 (i686-AT386)
Kernel: GNU-Mach 1.4-486/Hurd-0.5
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Versions of packages libc0.3 depends on:
ii hurd-libs0.3 1:0.5.git20140526-2
ii libgcc1 1:4.9.0-7
Versions of packages libc0.3 recommends:
pn libc0.3-i686 none
Versions of packages libc0.3 suggests:
ii debconf [debconf-2.0] 1.5.53
pn glibc-doc none
#include errno.h
#include stdio.h
#include sys/socket.h
#include sys/un.h
#include sys/unistd.h
char sock_name[] = TEST.ctl;
int connect_socket()
{
struct sockaddr_un server;
int sockfd;
memset(server, 0, sizeof(server));
server.sun_family = AF_UNIX;
strncpy(server.sun_path, sock_name, sizeof(server.sun_path));
server.sun_path[sizeof(server.sun_path) - 1] = '\0';
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd 0) {
printf(Can't create socket %s: %s\n, sock_name, strerror(errno));
return -1;
}
if (connect(sockfd, (struct sockaddr *) server, sizeof(struct sockaddr_un)) 0) {
printf(Can't connect to socket %s: %s\n, sock_name, strerror(errno));
close(sockfd);
return -2;
}
return sockfd;
}
int main()
{
char buff[20];
int sockfd = connect_socket();
if (sockfd 0) {
return sockfd;
}
if (send(sockfd, , 0, 0) 0) {
printf(Socket not working: %s\n, strerror(errno));
} else {
printf(Socket works.\n);
}
if (send(sockfd, TEST, 4, 0) 0) {
printf(Could not write to socket: %s\n, strerror(errno));
close(sockfd);
return 1;
}
memset(buff, 0, sizeof (buff));
if (recv(sockfd, buff, sizeof (buff), 0) 0) {
printf(Error receiving answer: %s\n, strerror(errno));
close(sockfd);
return 2;
}
printf(Answer: '%s'\n, buff);
close(sockfd);
return 0;
}
#! /usr/bin/make -f
CC ?= gcc
CFLAGS ?= -g -pedantic -Wall -Wextra
all: server client
%:
$(CC) $(CFLAGS) $@.c -o $@
#include errno.h
#include stdio.h
#include sys/socket.h
#include sys/un.h
#include sys/unistd.h
char sock_name[] = TEST.ctl;
int create_socket()
{
struct sockaddr_un server;
int sockfd;
memset(server, 0, sizeof(server));
server.sun_family = AF_UNIX;
strncpy(server.sun_path, sock_name, sizeof(server.sun_path));
server.sun_path[sizeof(server.sun_path) - 1] = '\0';
sockfd = socket(AF_UNIX, SOCK_STREAM, 0);
if (sockfd 0) {
printf(Can't create socket %s: %s\n, sock_name, strerror(errno));
return -1;
}
if (bind(sockfd, (struct sockaddr *) server, sizeof(struct sockaddr_un)) 0) {
printf(Socket file %s could not be bound: %s\n, server.sun_path, strerror(errno));
close(sockfd);
return -2;
}
if (listen(sockfd, 20) 0) {
printf(listen() error: %s\n, strerror(errno));
close(sockfd);
return -3;
}
return sockfd;
}
int main()
{
int ret, con;
char buff[20];
int sockfd = create_socket();
if (sockfd 0) {
return sockfd;
}
con = accept(sockfd, 0, 0);
memset(buff, 0, sizeof(buff));
if (recv(con, buff, sizeof (buff), 0) = 0) {
printf(Message: '%s'\n, buff);
}
/*
if (write(con, , 0) 0) {
printf(Socket not working: %s\n, strerror(errno));
return 1;
} else {
printf(Socket works.\n);
}
*/
if (write(con, ANSWER, 6) 0) {
printf(Could not write to socket: %s\n, strerror(errno));
ret = 2;
}
close(con);
if (unlink(sock_name) == -1) {
printf(Socket file %s could not be removed: %s\n, sock_name, strerror(errno));
ret = 3;
}
close(sockfd);
return ret;
}
