[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
Actually, upgrading my kernel to 4.13 fixed the issue for me. root@myserver:~$ uname -r 4.13.0-32-generic root@myserver:~$ rbd ls testimage root@myserver:~$ rbd map testimage /dev/rbd0 root@myserver:~$ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728739/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
Same issue for me: root@myserver:~# uname -r 4.4.0-112-generic root@myserver:~# ceph version ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable) root@myserver:~# rbd create testimage --size 10G root@myserver:~# rbd ls -l NAMESIZE PARENT FMT PROT LOCK testimage 10240M 2 root@myserver:~# rbd map testimage rbd: sysfs write failed In some cases useful info is found in syslog - try "dmesg | tail". rbd: map failed: (110) Connection timed out root@myserver:~# dmesg | tail [ 1392.958472] libceph: mon0 51.254.215.26:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 [ 1392.958721] libceph: mon0 51.254.215.26:6789 missing required protocol features ... same last two messages repeat -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
I am facing this issue in kernel version: "4.4.0-104" root@cc-swi01-tky1:~# uname -r 4.4.0-104-generic root@cc-swi01-tky1:~# ceph --version ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable) root@cc-swi01-tky1:~# sudo rbd create image01 --size 1024 --pool testpool root@cc-swi01-tky1:~# rbd ls -p testpool image01 root@cc-swi01-tky1:~# sudo rbd map image01 --pool testpool --name client.admin rbd: sysfs write failed In some cases useful info is found in syslog - try "dmesg | tail". rbd: map failed: (110) Connection timed out # dmesg | tail [516822.815270] libceph: mon0 10.X.8.201:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 [516822.842737] libceph: mon0 10.X.8.201:6789 missing required protocol features -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
FYI, this patch seems to have introduced a regression, see https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1737033 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Released Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728739/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
This bug was fixed in the package linux - 4.4.0-103.126 --- linux (4.4.0-103.126) xenial; urgency=low * linux: 4.4.0-103.126 -proposed tracker (LP: #1736181) * CVE-2017-1000405 - mm, thp: Do not make page table dirty unconditionally in touch_p[mu]d() * CVE-2017-16939 - netlink: add a start callback for starting a netlink dump - ipsec: Fix aborted xfrm policy dump crash linux (4.4.0-102.125) xenial; urgency=low * linux: 4.4.0-102.125 -proposed tracker (LP: #1733541) * tar -x sometimes fails on overlayfs (LP: #1728489) - ovl: check if all layers are on the same fs - ovl: persistent inode number for directories * NVMe timeout is too short (LP: #1729119) - nvme: update timeout module parameter type * Set PANIC_TIMEOUT=10 on Power Systems (LP: #1730660) - [Config]: Set PANIC_TIMEOUT=10 on ppc64el * Cannot pair BLE remote devices when using combo BT SoC (LP: #1731467) - Bluetooth: increase timeout for le auto connections * CIFS errors on 4.4.0-98, but not on 4.4.0-97 with same config (LP: #1729337) - SMB3: Validate negotiate request must always be signed * Plantronics P610 does not support sample rate reading (LP: #1719853) - ALSA: usb-audio: Add sample rate quirk for Plantronics P610 * Invalid btree pointer causes the kernel NULL pointer dereference (LP: #1729256) - xfs: reinit btree pointer on attr tree inactivation walk * Samba mount/umount in docker container triggers kernel Oops (LP: #1729637) - ipv6: only call ip6_route_dev_notify() once for NETDEV_UNREGISTER - ipv6: fix NULL dereference in ip6_route_dev_notify() * [kernel] tty/hvc: Use opal irqchip interface if available (LP: #1728098) - tty/hvc: Use opal irqchip interface if available * Device hotplugging with MPT SAS cannot work for VMWare ESXi (LP: #1730852) - scsi: mptsas: Fixup device hotplug for VMWare ESXi * NMI watchdog: BUG: soft lockup on Guest upon boot (KVM) (LP: #1727331) - KVM: PPC: Book3S: Treat VTB as a per-subcore register, not per-thread * Attempt to map rbd image from ceph jewel/luminous hangs (LP: #1728739) - crush: ensure bucket id is valid before indexing buckets array - crush: ensure take bucket value is valid - crush: add chooseleaf_stable tunable - crush: decode and initialize chooseleaf_stable - libceph: advertise support for TUNABLES5 - libceph: MOSDOpReply v7 encoding * Xenial update to 4.4.98 stable release (LP: #1732698) - adv7604: Initialize drive strength to default when using DT - video: fbdev: pmag-ba-fb: Remove bad `__init' annotation - PCI: mvebu: Handle changes to the bridge windows while enabled - xen/netback: set default upper limit of tx/rx queues to 8 - drm: drm_minor_register(): Clean up debugfs on failure - KVM: PPC: Book 3S: XICS: correct the real mode ICP rejecting counter - iommu/arm-smmu-v3: Clear prior settings when updating STEs - powerpc/corenet: explicitly disable the SDHC controller on kmcoge4 - ARM: omap2plus_defconfig: Fix probe errors on UARTs 5 and 6 - crypto: vmx - disable preemption to enable vsx in aes_ctr.c - iio: trigger: free trigger resource correctly - phy: increase size of MII_BUS_ID_SIZE and bus_id - serial: sh-sci: Fix register offsets for the IRDA serial port - usb: hcd: initialize hcd->flags to 0 when rm hcd - netfilter: nft_meta: deal with PACKET_LOOPBACK in netdev family - IPsec: do not ignore crypto err in ah4 input - Input: mpr121 - handle multiple bits change of status register - Input: mpr121 - set missing event capability - IB/ipoib: Change list_del to list_del_init in the tx object - s390/qeth: issue STARTLAN as first IPA command - (config) Add NET_DSA=n - net: dsa: select NET_SWITCHDEV - platform/x86: hp-wmi: Fix detection for dock and tablet mode - cdc_ncm: Set NTB format again after altsetting switch for Huawei devices - KEYS: trusted: sanitize all key material - KEYS: trusted: fix writing past end of buffer in trusted_read() - platform/x86: hp-wmi: Fix error value for hp_wmi_tablet_state - platform/x86: hp-wmi: Do not shadow error values - x86/uaccess, sched/preempt: Verify access_ok() context - workqueue: Fix NULL pointer dereference - crypto: x86/sha1-mb - fix panic due to unaligned access - KEYS: fix NULL pointer dereference during ASN.1 parsing [ver #2] - ARM: 8720/1: ensure dump_instr() checks addr_limit - ALSA: seq: Fix OSS sysex delivery in OSS emulation - ALSA: seq: Avoid invalid lockdep class warning - MIPS: microMIPS: Fix incorrect mask in insn_table_MM - MIPS: Fix CM region target definitions - MIPS: SMP: Use a completion event to signal CPU up - MIPS: Fix race on setting and getting cpu_online_mask - MIPS: SMP: Fix deadlock & online race - test: firmware_class: report errors properly on failure - selftests: firmware: add empty string and async t
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
Successfully verified this with kernel 4.4.0-102-generic in the xenial- proposed repository. I verified mapping rbd devices against Ceph versions: 0.94.10 Hammer release (from trusty-kilo cloud-archive) 10.2.7 Jewel release (from xenial-updates) 12.2.0 Luminous release (from xenial-pike cloud-archive) All versions had optimal tunables set, which previously prevented rbd mapping from completing for Jewel and Luminous releases due to missing TUNABLES5 support. All tests successfully mapped, formatted, and mounted. Relevant kernel log snippets: Nov 29 18:59:29 juju-49feb5-ceph-6 kernel: [0.00] Linux version 4.4.0-102-generic (buildd@lgw01-amd64-055) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5) ) #125-Ubuntu SMP Tue Nov 21 15:15:11 UTC 2017 (Ubuntu 4.4.0-102.125-generic 4.4.98) RBD Mapping to Ceph Hammer Cluster v0.94.10 (trusty-kilo cloud-archive) Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.314991] Key type ceph registered Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.317382] libceph: loaded (mon/osd proto 15/24) Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.323276] rbd: loaded (major 251) Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.343492] libceph: client44139 fsid 6547bd3e-1397-11e2-82e5-53567c8d32dc Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.346357] libceph: mon2 10.5.0.3:6789 session established Nov 29 19:01:45 juju-49feb5-ceph-6 kernel: [ 148.415717] rbd: rbd0: added with size 0x28000 Nov 29 19:02:04 juju-49feb5-ceph-6 kernel: [ 167.519366] EXT4-fs (rbd0): mounted filesystem with ordered data mode. Opts: (null) RBD Mapping to Ceph Jewel Cluster v10.2.7 (xenial/trusty-mitaka cloud-archive) Nov 29 19:17:48 juju-49feb5-ceph-6 kernel: [ .874853] libceph: client64310 fsid 6547bd3e-1397-11e2-82e5-53567c8d32dc Nov 29 19:17:48 juju-49feb5-ceph-6 kernel: [ .876730] libceph: mon1 10.5.0.14:6789 session established Nov 29 19:17:48 juju-49feb5-ceph-6 kernel: [ .924995] rbd: rbd0: added with size 0x28000 Nov 29 19:17:54 juju-49feb5-ceph-6 kernel: [ 1117.704119] EXT4-fs (rbd0): mounted filesystem with ordered data mode. Opts: (null) RBD Mapping to Ceph Luminous Cluster v12.2.0 (xenial-ocata cloud-archive) Nov 29 19:41:42 juju-49feb5-ceph-6 kernel: [ 2545.672870] libceph: client4216 fsid 6547bd3e-1397-11e2-82e5-53567c8d32dc Nov 29 19:41:42 juju-49feb5-ceph-6 kernel: [ 2545.679350] libceph: mon1 10.5.0.12:6789 session established Nov 29 19:41:42 juju-49feb5-ceph-6 kernel: [ 2545.717272] rbd: rbd0: added with size 0x28000 Nov 29 19:42:04 juju-49feb5-ceph-6 kernel: [ 2567.088206] EXT4-fs (rbd0): mounted filesystem with ordered data mode. Opts: (null) ** Tags removed: verification-needed-xenial ** Tags added: verification-done-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Committed Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed- xenial'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Committed Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage n
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
** Changed in: linux (Ubuntu Xenial) Status: Triaged => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Fix Committed Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728739/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
** Changed in: linux (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Fix Released Status in linux source package in Xenial: Triaged Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728739/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
** Also affects: linux (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Status: Confirmed => Triaged ** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Xenial) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Xenial) Status: New => Triaged -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Triaged Status in linux source package in Xenial: Triaged Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Oct 31 01:23 seq crw-rw 1 root audio 116, 33 Oct 31 01:23 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 16.04 Ec2AMI: ami-0001 Ec2AMIManifest: FIXME Ec2AvailabilityZone: nova Ec2InstanceType: m1.small Ec2Kernel: unavailable Ec2Ramdisk: unavailable IwConfig: Error: [Errno 2] No such file or directory Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub MachineType: OpenStack Foundation OpenStack Nova Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 RelatedPackageVersions: linux-restricted-modules-4.4.0-98-generic N/A linux-backports-modules-4.4.0-98-generic N/A linux-firmwareN/A RfKill: Error: [Errno 2] No such file or directory Tags: xenial ec2-images Uname: Linux 4.4.0-98-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.10.1-1ubuntu1~cloud0 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-zesty dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: dmi.product.name: OpenStack Nova dmi.product.version: 15.0.6 dmi.sys.vendor: OpenStack Foundation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1728739/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.l
[Kernel-packages] [Bug 1728739] Re: Attempt to map rbd image from ceph jewel/luminous hangs
apport information ** Tags added: apport-collected ec2-images xenial ** Description changed: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were first introduced in the 4.5 Linux kernel and are thus not available in the 16.04 LTS 4.4 Linux Kernel. Attempting to map RBD images as block devices will fail due to not being able to understand these new tunables: (from kern.log) Oct 30 21:19:05 ceph-7 kernel: [ 815.674075] Key type ceph registered Oct 30 21:19:05 ceph-7 kernel: [ 815.676862] libceph: loaded (mon/osd proto 15/24) Oct 30 21:19:05 ceph-7 kernel: [ 815.678970] rbd: loaded (major 251) Oct 30 21:19:05 ceph-7 kernel: [ 815.689556] libceph: mon0 10.5.0.19:6789 feature set mismatch, my 106b84a842a42 < server's 40106b84a842a42, missing 400 Oct 30 21:19:05 ceph-7 kernel: [ 815.692897] libceph: mon0 10.5.0.19:6789 missing required protocol features Support for the new CRUSH tunables were added in upstream kernel 4.5 in http://www.spinics.net/lists/ceph-devel/msg28421.html. [Test Case] 1. Deploy a Jewel or Luminous Ceph cluster. 2. Create rbd image suitable for the kernel client: $ rbd create --pool rbd --image-feature layering --size 1G test 3. Map the rbd image to the local server: $ rbd map --pool rbd test [Regression Potential] Minimal. Code is limited to kernel rbd driver and new code should primarily affect clients connecting to clusters with the new tunables options. [Additional Info] A workaround is to change the crush tunables configured for the Ceph cluster to a legacy version (hammer or lower) via: $ ceph osd crush tunables hammer - However, changing the tunables to hammer fails to allow the cluster to - take advantage of newer placement strategies which reduces the amount of - data movement throughout the cluster. + However, changing the tunables to hammer fails to allow the cluster to take advantage of newer placement strategies which reduces the amount of data movement throughout the cluster. + --- + AlsaDevices: + total 0 + crw-rw 1 root audio 116, 1 Oct 31 01:23 seq + crw-rw 1 root audio 116, 33 Oct 31 01:23 timer + AplayDevices: Error: [Errno 2] No such file or directory + ApportVersion: 2.20.1-0ubuntu2.10 + Architecture: amd64 + ArecordDevices: Error: [Errno 2] No such file or directory + AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: + CRDA: N/A + DistroRelease: Ubuntu 16.04 + Ec2AMI: ami-0001 + Ec2AMIManifest: FIXME + Ec2AvailabilityZone: nova + Ec2InstanceType: m1.small + Ec2Kernel: unavailable + Ec2Ramdisk: unavailable + IwConfig: Error: [Errno 2] No such file or directory + Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub + MachineType: OpenStack Foundation OpenStack Nova + Package: linux (not installed) + PciMultimedia: + + ProcEnviron: + TERM=xterm-256color + PATH=(custom, no user) + LANG=en_US.UTF-8 + SHELL=/bin/bash + ProcFB: + + ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-98-generic root=UUID=d7006b2f-ace6-464d-8b21-17180b3ed360 ro console=tty1 console=ttyS0 + ProcVersionSignature: Ubuntu 4.4.0-98.121-generic 4.4.90 + RelatedPackageVersions: + linux-restricted-modules-4.4.0-98-generic N/A + linux-backports-modules-4.4.0-98-generic N/A + linux-firmwareN/A + RfKill: Error: [Errno 2] No such file or directory + Tags: xenial ec2-images + Uname: Linux 4.4.0-98-generic x86_64 + UpgradeStatus: No upgrade log present (probably fresh install) + UserGroups: + + _MarkForUpload: True + dmi.bios.date: 04/01/2014 + dmi.bios.vendor: SeaBIOS + dmi.bios.version: 1.10.1-1ubuntu1~cloud0 + dmi.chassis.type: 1 + dmi.chassis.vendor: QEMU + dmi.chassis.version: pc-i440fx-zesty + dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.1-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr15.0.6:cvnQEMU:ct1:cvrpc-i440fx-zesty: + dmi.product.name: OpenStack Nova + dmi.product.version: 15.0.6 + dmi.sys.vendor: OpenStack Foundation ** Attachment added: "CurrentDmesg.txt" https://bugs.launchpad.net/bugs/1728739/+attachment/5000570/+files/CurrentDmesg.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1728739 Title: Attempt to map rbd image from ceph jewel/luminous hangs Status in linux package in Ubuntu: Confirmed Bug description: [Impact] Attempting to map an rbd image using the 4.4 LTS (Xenial) kernel client from a Jewel or Luminous Ceph cluster with optimal tunables fails due to feature set mismatch. The Jewel release of Ceph introduced a new set of CRUSH tunables. These tunables were firs
