control: reassign -1 librrd8
control: retitle -1 librrd8: crashes with SIGBUS in rrd_write
Hi,
On 2023-07-11 15:53, Tim McConnell wrote:
>
>
> On Tue, 2023-07-11 at 22:34 +0200, Aurelien Jarno wrote:
> > Hi,
> >
> > On 2023-07-11 15:28, Tim McConnell wrote:
> > >
> > >
> > > On Tue, 2023-07-11 at 21:11 +0200, Aurelien Jarno wrote:
> > > > Hi,
> > > >
> > > > On 2023-07-11 11:21, Tim McConnell wrote:
> > > > >
> > > > >
> > > > > On Mon, 2023-07-10 at 23:17 +0200, Aurelien Jarno wrote:
> > > > > > You might want
> > > > > > to upgrade to version 2.37-5 to check if it solves your issue
> > > > > Okay that's done and it's still doing it. The entry from
> > > > > Journalctl
> > > > > shows module libudev1 if that's of any use.
> > > > >
> > > > > Started systemd-coredump@1785-616863-0.service - Process Core
> > > > > Dump
> > > > > (PID
> > > > > 616863/UID 0).
> > > > > Jul 11 10:52:06 DebianTim systemd-coredump[616865]: Process
> > > > > 616847
> > > > > (collectd) of user 0 dumped core.
> > > > >
> > > > > Module
> > > > > libudev.so.1
> > > > > from deb systemd-252.11-1.amd64
> > > > > Stack trace
> > > > > of
> > > > > thread 616848:
> > > > > #0
> > > > > 0x7fce1335e9f2 __memmove_ssse3 (libc.so.6 + 0x16d9f2)
> > > > > #1
> > > > > 0x7fce131156d9 rrd_write (librrd.so.8 + 0x346d9)
> > > > > #2
> > > > > 0x7fce13120acd n/a (librrd.so.8 + 0x3facd)
> > > > > #3
> > > > > 0x7fce13122962 n/a (librrd.so.8 + 0x41962)
> > > > > #4
> > > > > 0x7fce1317c370 n/a (rrdtool.so + 0x3370)
> > > > > #5
> > > > > 0x7fce132793ec start_thread (libc.so.6 + 0x883ec)
> > > > > #6
> > > > > 0x7fce132f9a1c __clone3 (libc.so.6 + 0x108a1c)
> > > > >
> > > >
> > > > Thanks for the details. This shows that the binary crashing
> > > > regularly
> > > > is
> > > > collectd. This is very unlikely that the issue is linked to the
> > > > locales,
> > > > and your test is confirming that.
> > > >
> > > > It's also not clear that it's a glibc issue, it's more likely an
> > > > issue
> > > > in collectd or librrd8. It appears that systemd-coredump saved a
> > > > coredump when the process crashed. You should be able do use
> > > > "coredumpctl" to get the list of cores. You can select one
> > > > coredump
> > > > and
> > > > examine it with gdb using "coredumpctl debug ". Then when
> > > > under
> > > > gdb
> > > > you should be able to run "thread apply all bt" to get the
> > > > backtrace.
> > > > That should allows to better understand the issue.
> > > >
> > > > Regards
> > > > Aurelien
> > > >
> > > I'm unsure how helpful this is ( I am not a programmer) but:
> > > thread apply all bt
> >
> > Thanks, that's already much more useful. However I forgot to tell you
> > to
> > install the libc6-dbg package before getting the backtrace, sorry
> > about
> > that. Could you please install it and follow the same procedure
> > again?
> >
> > Thanks
> > Aurelien
> >
> It's already there, any others?
I was expecting to see the arguments of the glibc functions, but anyway
I have been able to understand the code path.
collectd uses the librrd8 library to handle data, which at some point
calls the glibc memmove function. The crashes (SIGBUS) happens in the
memmove function when trying to destination buffer:
| .p2align 4,, 4
| L(copy_0_15):
| cmpl$4, %edx
| jb L(copy_0_3)
| cmpl$8, %edx
| jb L(copy_4_7)
| movq0(%rsi), %rcx
| movq-8(%rsi, %rdx), %rsi
=> movq%rcx, 0(%rdi)
| movq%rsi, -8(%rdi, %rdx)
| ret
memmove is called in
the rrd_write function, and more precisely in the memmove call:
| ssize_t rrd_write(
| rrd_file_t *rrd_file,
| const void *buf,
| size_t count)
| {
[...]
| #ifdef HAVE_MMAP
| size_told_size = rrd_file->file_len;
|
| if (count == 0)
| return 0;
| if (buf == NULL)
| return -1; /* EINVAL */
|
| if ((rrd_file->pos + count) > old_size) {
| rrd_set_error
| ("attempting to write beyond end of file (%ld + %ld > %ld)",
| rrd_file->pos, count, old_size);
| return -1;
| }
| /* can't use memcpy since the areas overlap when tuning */
=>memmove(rrd_simple_file->file_start + rrd_file->pos, buf, count);
| rrd_file->pos += count;
| return count; /* mimic write() semantics */
librrd8 uses mma
