** Description changed: [IMPACT] - The version of collectd shipped with Ubuntu 18.04 (Bionic) provides a ceph plugin that is incompatible with the version of Ceph shipped in the same distribution. + A subset of Ceph metrics no-longer get logged with Ceph luminous (v12 and late) + + The version of collectd shipped with Ubuntu 18.04 (Bionic) provides a + ceph plugin that is incompatible with the version of Ceph shipped in the + same distribution. The version of collectd is 5.7.2-2ubuntu1 The version of ceph is 12.2.4-0ubuntu1.1 This patch for collectd is required for correct interoperation with Ceph 12+: https://github.com/collectd/collectd/pull/2464 The first version of collectd to contain this patch is 5.8.0. Without this patch, errors of the following form will be logged by collectd, and many ceph-specific metrics will not be collected: May 28 09:31:56 stor-a collectd[2141]: ceph plugin: cconn_handle_event(name=osd.2,i=0,st=4): error 1 May 28 09:31:56 stor-a collectd[2141]: ceph plugin: ds Bluestore.kvFlushLat.avgtime was not properly initialized. May 28 09:31:56 stor-a collectd[2141]: ceph plugin: JSON handler failed with status -1. + [TEST CASE] * Install Bionic and have a Luminous and/or late Ceph environment. * Install collectd ** Enable Ceph plugin in /etc/collectd/collectd.conf ** Configure Ceph plugin daemon in /etc/collectd/collectd.conf * Restart collectd ** systemctl stop collectd.service ** systemctl start collectd.service * Force collection ** collectd -C /etc/collectd/collectd.conf * Monitor /var/log/syslog ** tail -f /var/log/syslog | grep -i collectd ** tail -f /var/log/syslog | grep -i "ceph plugin" [POTENTIAL REGRESSION] * Bionic's oldest Ceph version supported is Luminous, so the backward incompatibility with previous ceph versions is not a problem here. * Upstream faced a segfault situation in the Ceph plugin with Mimic version via issue: https://github.com/collectd/collectd/issues/2572, this problem is also addressed in the current SRU (d/p/ceph-plugin- Fix-2572.patch) * The new Ceph support is already part of debian and Ubuntu Cosmic and Disco. * Right now it seems like the Ceph plugin is not working at all anyway. It couldn't go worse after this SRU than the current situation. - * A test package with the fixes has been made available pre-sru to + * A test package with the fixes has been made available pre-sru to impacted user(s) and the feedback were positive. (See comment #5) [OTHER INFORMATION] # Upstream commits: 647ac31b Add support for ceph version luminous: https://github.com/collectd/collectd/commit/647ac31bf9db60b1685d6d8d25be65375ba85891 de05fb53 ceph plugin: Fix #2572: https://github.com/collectd/collectd/commit/de05fb53fad6bc998f585b704ca0caeadc14a035 $ git describe --contains 647ac31b collectd-5.8.0~29^2~9 $ git describe --contains de05fb53 collectd-5.8.1~38 # rmadison ==> collectd | 5.7.2-2ubuntu1 | bionic/universe collectd | 5.8.0-5.2 | cosmic/universe collectd | 5.8.1-1.2 | disco/universe [ORIGINAL DESCRIPTION] The version of collectd shipped with Ubuntu 18.04 (Bionic) provides a ceph plugin that is incompatible with the version of Ceph shipped in the same distribution. The version of collectd is 5.7.2-2ubuntu1 The version of ceph is 12.2.4-0ubuntu1.1 This patch for collectd is required for correct interoperation with Ceph 12+: https://github.com/collectd/collectd/pull/2464 The first version of collectd to contain this patch is 5.8.0. Without this patch, errors of the following form will be logged by collectd, and many ceph-specific metrics will not be collected: May 28 09:31:56 stor-a collectd[2141]: ceph plugin: cconn_handle_event(name=osd.2,i=0,st=4): error 1 May 28 09:31:56 stor-a collectd[2141]: ceph plugin: ds Bluestore.kvFlushLat.avgtime was not properly initialized. May 28 09:31:56 stor-a collectd[2141]: ceph plugin: JSON handler failed with status -1.
** Description changed: [IMPACT] A subset of Ceph metrics no-longer get logged with Ceph luminous (v12 and late) The version of collectd shipped with Ubuntu 18.04 (Bionic) provides a ceph plugin that is incompatible with the version of Ceph shipped in the same distribution. The version of collectd is 5.7.2-2ubuntu1 The version of ceph is 12.2.4-0ubuntu1.1 This patch for collectd is required for correct interoperation with Ceph 12+: https://github.com/collectd/collectd/pull/2464 The first version of collectd to contain this patch is 5.8.0. Without this patch, errors of the following form will be logged by collectd, and many ceph-specific metrics will not be collected: May 28 09:31:56 stor-a collectd[2141]: ceph plugin: cconn_handle_event(name=osd.2,i=0,st=4): error 1 May 28 09:31:56 stor-a collectd[2141]: ceph plugin: ds Bluestore.kvFlushLat.avgtime was not properly initialized. May 28 09:31:56 stor-a collectd[2141]: ceph plugin: JSON handler failed with status -1. - [TEST CASE] * Install Bionic and have a Luminous and/or late Ceph environment. * Install collectd ** Enable Ceph plugin in /etc/collectd/collectd.conf ** Configure Ceph plugin daemon in /etc/collectd/collectd.conf * Restart collectd ** systemctl stop collectd.service ** systemctl start collectd.service * Force collection ** collectd -C /etc/collectd/collectd.conf * Monitor /var/log/syslog ** tail -f /var/log/syslog | grep -i collectd ** tail -f /var/log/syslog | grep -i "ceph plugin" + + * Check Ceph plugin metrics [POTENTIAL REGRESSION] * Bionic's oldest Ceph version supported is Luminous, so the backward incompatibility with previous ceph versions is not a problem here. * Upstream faced a segfault situation in the Ceph plugin with Mimic version via issue: https://github.com/collectd/collectd/issues/2572, this problem is also addressed in the current SRU (d/p/ceph-plugin- Fix-2572.patch) * The new Ceph support is already part of debian and Ubuntu Cosmic and Disco. * Right now it seems like the Ceph plugin is not working at all anyway. It couldn't go worse after this SRU than the current situation. * A test package with the fixes has been made available pre-sru to impacted user(s) and the feedback were positive. (See comment #5) [OTHER INFORMATION] # Upstream commits: 647ac31b Add support for ceph version luminous: https://github.com/collectd/collectd/commit/647ac31bf9db60b1685d6d8d25be65375ba85891 de05fb53 ceph plugin: Fix #2572: https://github.com/collectd/collectd/commit/de05fb53fad6bc998f585b704ca0caeadc14a035 $ git describe --contains 647ac31b collectd-5.8.0~29^2~9 $ git describe --contains de05fb53 collectd-5.8.1~38 # rmadison ==> collectd | 5.7.2-2ubuntu1 | bionic/universe collectd | 5.8.0-5.2 | cosmic/universe collectd | 5.8.1-1.2 | disco/universe [ORIGINAL DESCRIPTION] The version of collectd shipped with Ubuntu 18.04 (Bionic) provides a ceph plugin that is incompatible with the version of Ceph shipped in the same distribution. The version of collectd is 5.7.2-2ubuntu1 The version of ceph is 12.2.4-0ubuntu1.1 This patch for collectd is required for correct interoperation with Ceph 12+: https://github.com/collectd/collectd/pull/2464 The first version of collectd to contain this patch is 5.8.0. Without this patch, errors of the following form will be logged by collectd, and many ceph-specific metrics will not be collected: May 28 09:31:56 stor-a collectd[2141]: ceph plugin: cconn_handle_event(name=osd.2,i=0,st=4): error 1 May 28 09:31:56 stor-a collectd[2141]: ceph plugin: ds Bluestore.kvFlushLat.avgtime was not properly initialized. May 28 09:31:56 stor-a collectd[2141]: ceph plugin: JSON handler failed with status -1. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1774032 Title: Bionic: CollectD ceph plugin is incompatible with Ceph 12+ (Luminous) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/collectd/+bug/1774032/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs