[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
bubblewrap (0.4.0-1ubuntu3) focal; urgency=medium * d/p/update-output-patterns-libcap-2.29.patch: cherry-pick fix proposed fix to capability drop-related tests, which broke with newer libcap2. -- Łukasz 'sil2100' Zemczak Wed, 26 Feb 2020 21:39:11 +0100 ** No longer affects: chrony (Ubuntu) ** Also affects: bubblewrap (Ubuntu) Importance: Undecided Status: New ** Changed in: bubblewrap (Ubuntu) Importance: Undecided => High ** Changed in: bubblewrap (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bubblewrap/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
** Changed in: bubblewrap (Debian) Status: Unknown => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
Both migrated, issue resolved. libcap2 | 1:2.32-1| focal | source, amd64, arm64, armhf, i386, ppc64el, s390x bubblewrap | 0.4.0-1ubuntu3 | focal | source, amd64, arm64, armhf, i386, ppc64el, s390x ** Changed in: libcap2 (Ubuntu) Status: New => Fix Released ** Changed in: chrony (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
** Bug watch added: Debian Bug tracker #951577 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951577 ** Also affects: bubblewrap (Debian) via https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951577 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
FYI - libcap2 2.32 is blocking on bubblewrap (bug 1863733) that needs the tests updated (https://github.com/containers/bubblewrap/issues/353) to pass proposed migration. FYI No current chrony tests in excuses that are blocking anyone (and if so testing vs libcap2 from proposed makes it work) - we can wait on bubblewrap a few days (it is a test-only bugfix, that isn't feature freeze related). ** Bug watch added: github.com/containers/bubblewrap/issues #353 https://github.com/containers/bubblewrap/issues/353 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
** Changed in: libcap2 (Debian) Status: Unknown => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
FYI: 2.32 was uploaded to Debian, should resolve via autosync -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
Since it is a sync I filed a Debian bug for it, but due to our FF it depends how they respond. => https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951492 ** Bug watch added: Debian Bug tracker #951492 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951492 ** Also affects: libcap2 (Debian) via https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951492 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
** Changed in: chrony (Ubuntu) Status: New => Confirmed ** Changed in: libcap2 (Ubuntu) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
libcap2 breaks it, libcap2 has a fix. I'd prefer that we get libcap2 2.32 to fix it. References: - https://sites.google.com/site/fullycapable/release-notes-for-libcap - https://bugzilla.kernel.org/show_bug.cgi?id=206539 ** Also affects: libcap2 (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
$ gdb ../../chronyd (gdb) set environment LD_PRELOAD /home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac//clknetsim.so (gdb) set environment CLKNETSIM_NODE 1 (gdb) set environment CLKNETSIM_SOCKET tmp/sock (gdb) run -d -f tmp/conf.1 But then we found it: 1. avoidance for now via -DCLKNETSIM_DISABLE_SYSCALL 2. long term libcap2 2.32 fixes the issue ** Bug watch added: Linux Kernel Bug Tracker #206539 https://bugzilla.kernel.org/show_bug.cgi?id=206539 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
I added some read -p "Press enter to continue" into the test to see if the chronyd processes exist in the real test before reaching some stage. It starts/hangs/crashes inside this stage: run_test || test_fail Debugging into this function leads to start_client from clknetsim It does all kind of stuff including LD_PRELOAD LD_PRELOAD=/home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac//clknetsim.so CLKNETSIM_NODE=1 CLKNETSIM_SOCKET=tmp/sock chronyd -d -f tmp/conf.1 cho LD_PRELOAD=/home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac//clknetsim.so CLKNETSIM_NODE=2 CLKNETSIM_SOCKET=tmp/sock chronyd -d -f tmp/conf.2 Here I finally hit the early init fail: $ export PATH="../../:$PATH" LD_PRELOAD=/home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac//clknetsim.so CLKNETSIM_NODE=1 CLKNETSIM_SOCKET=tmp/sock chronyd -d -f tmp/conf.1 chronyd: client.c:2162: syscall: Assertion `0' failed. Aborted (core dumped) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
Formerly broken system with libcap2 1:2.31-1 -> Reverting to former libcap2 1:2.27-1 $ v=1:2.27-1; sudo apt install libcap-dev=$v libcap2=$v libcap2-bin=$v (without rebuilding chrony or the test which was built against 1:2.31-1) => Working now Formerly working system with libcap2 1:2.27-1 -> upgrading to new libcap2 1:2.31-1 $ sudo apt install libcap-dev/focal-proposed libcap2/focal-proposed libcap2-bin/focal-proposed (without rebuilding chrony or the test which was built against 1:2.31-1) => Now hanging TL;DR: it does not depend which version clknetsim or chrony was built against, it just matters which version of libcap2 is installed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
** Attachment added: "verbose test log running into the hang" https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+attachment/5328922/+files/verbose-test-log-into-hang.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
Test generates a config like: $ cat tmp/conf.1 pidfile tmp/pidfile.1 allow cmdallow bindcmdaddress 0.0.0.0 local stratum 1 and $ cat tmp/conf.2 pidfile tmp/pidfile.2 allow cmdallow bindcmdaddress 0.0.0.0 server 192.168.123.1 And then runs the daemons: $ chronyd -d -f tmp/conf.1 $ chronyd -d -f tmp/conf.2 reproducing that both initialize. And finally the test $ /home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac//clknetsim -o tmp/log.offset -f tmp/log.freq -p tmp/log.packets -R 1 -r 210 -l 1 -s tmp/sock tmp/conf 2 Waiting for 2 clients... Not saying/doing anything at this stage - but where to debug ...? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
Installed two test environments with proposed pinned so that only what I want from proposed comes from there. Then I was installing dependencies and the test as d/t/control and the test script would run it. Therefore the difference between good/bad environment is just: $ sudo apt install libcap-dev/focal-proposed libcap2/focal-proposed libcap2-bin/focal-proposed The test rebuilds chrony and it does so with the new libcap2, so the issue might be on that side and not on clknetsim which doesn't even link to libcap2. I build it the same (after having the new libcap2 installed) and ran the test manually. $ export CLKNETSIM_PATH="/home/ubuntu/clknetsim-156b8e4f63fd8174104e3dd05bda19166b5ab0ac/" $ ./run -i 1 -m 0 001-defaults Good case: $ ./run -i 1 -m 0 001-defaults 001-defaults Testing default test settings: network with 1*1 servers and 1 clients: non-default settings: starting node 1:OK starting node 2:OK running simulation: OK checking chronyd exit: node 1: OK node 2: OK checking source selection: node 2: OK checking mean/min incoming/outgoing packet interval: node 1: 2.06e+02 2.06e+02 6.41e+01 6.41e+01 OK node 2: 2.06e+02 2.06e+02 6.41e+01 6.41e+01 OK checking clock sync time, max/rms time/freq error: node 2: 131 7.06e-05 1.61e-06 3.36e-05 1.89e-07 OK PASS SUMMARY: TOTAL 1 PASSED 1 FAILED 0() () SKIPPED 0 () Bad-Case: $ ./run -i 1 -m 0 001-defaults 001-defaults Testing default test settings: network with 1*1 servers and 1 clients: non-default settings: starting node 1:OK starting node 2:OK running simulation: -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1863590] Re: Chrony test hangs with libpcap2 1:2.31-1
The clknetsim hangs here: (gdb) bt #0 0x7f5061169307 in __libc_accept (fd=fd@entry=6, addr=addr@entry=..., len=len@entry=0x0) at ../sysdeps/unix/sysv/linux/accept.c:26 #1 0x564411e91dad in Network::prepare_clients (this=0x564413923eb0) at network.cc:131 #2 0x564411e8ceff in main (argc=, argv=0x7fff72e0eb08) at server.cc:206 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1863590 Title: Chrony test hangs with libpcap2 1:2.31-1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/chrony/+bug/1863590/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
