It seems you did not remove existing qemu-* packages ? Also if your qemu-img shows rbd support no need to replace it - what version of qemu-img are you using, how did you install it ? Stock centos version did not (up to 1-2months ago) did not support rbd, maybe it does now after redhat asquired inktank/ceph...?
Also post results of the ./configure to see if the libvirt was compiled with rbd support.... Andrija Sent from Google Nexus 4 On Jun 20, 2014 10:33 AM, "Praveen Buravilli" <[email protected]> wrote: > Hi Andrija, > > Thanks for providing detailed instructions. > I have executed all of the steps given in http://pastebin.com/HwCZEASR > and also in http://admintweets.com/centos-kvm-and-ceph-client-side-setup/. > But still facing the same issue. Any other ideas? > > Also, when I tried to install qemu-* rpms, it reported lot of dependency > issues(attaching the output file). > > Please note that "qemu-img" was reporting support for rbd earlier also. > Probably ceph osd node was also running on the same KVM node which might > have updated "rbd" in list of supported formats. > > Error: > ==== > [root@kvm-ovs-002 src-DONT-TOUCH]# virsh pool-define /tmp/rbd.xml > error: Failed to define pool from /tmp/rbd.xml > error: internal error: missing backend for pool type 8 (rbd) > > [root@kvm-ovs-002 src-DONT-TOUCH]# qemu-img | grep rbd > Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat qcow2 > qed vhdx parallels nbd blkdebug host_cdrom host_floppy host_device file > gluster gluster gluster gluster rbd > > Thanks, > Praveen Kumar Buravilli > > > -----Original Message----- > From: Andrija Panic [mailto:[email protected]] > Sent: 20 June 2014 14:35 > To: [email protected] > Subject: Re: Adding ceph RBD storage failed in CentOS 6.5 KVM > > Been there, done that....: > > This libvirt error "error: internal error missing backend for pool type 8" > means that libvirt was not compiled with RBD backend support. > > Here are my steps to compile libvirt 1.2.3 few months ago - change > configure options if you want, I tried to use as much options as possible. > http://pastebin.com/HwCZEASR (note the "ceph-devel" must be installed as > in instructions in order to be able to compile with RBD support) > > Also note, that if you are using CentOS 6.5, the "-s" flag was removed > from qemu-img packages, meaning you will not be able to use snapshot > functionality on cloudstack (not related to libbvirt), this is ANY > snapshoting will be broken - there is workarround down :) > > > Also, beside making sure libvirt can talk to RBD/CEPH, you MUST be sure > your qemu-img and qemu-kvm was compiled with RBD support, check like this: > qemu-img | grep "Supported formats" > should get somwething like this - note the "rbd" on the end of output: > > *Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat qcow2 > qed parallels nbd blkdebug host_cdrom host_floppy host_device file rbd* > > > If both qemu and libvirt are fine, you will be able to add CEPH to ACS 4.2 > or newer. > > If you have stock (CentOS) versions of qemu-img and qemu-kvm - they are > NOT compiled with RBD support, so you will not be able to use CEPH. > You need to install Intank's version of RPM packages, that are based on > official RedHat stock code of those packages, but are pathced for RBD/CEPH > support. > > Refer to http://admintweets.com/centos-kvm-and-ceph-client-side-setup/ in > order to download Intanks's RPMs - note that the latest RPMs you will find > are probably also based on RHEL 6.5 version, that is missing "-s" flag, so > you will still NOT be able to use disk snapshotting in ACS... > > I solved this by installing a little bit older RPMs from Intanks (qemu-img > and qemu-kvm that are based on RHEL 6.2 that still has that famous "-s" > flag preseent), please let me know if you need this provided, since they > are NOT present on the Intanks download page at the moment exact versions > of packages I installed and which are working fine for me: > qemu-kvm-tools-0.12.1.2-2.415.el6.3ceph.x86_64 > qemu-img-0.12.1.2-2.355.el6.2.cuttlefish.x86_64 > qemu-kvm-0.12.1.2-2.415.el6.3ceph.x86_64 > > This whole setup is working fine for me... > > Hope that helps, was going through the same pain, as you do now... :) > > Best, > Andrija Panic > > > > > On 20 June 2014 04:38, Praveen Buravilli <[email protected]> > wrote: > > > Hi, > > > > > > > > I am facing an issue in adding ceph RBD storage to CloudStack. It is > > failing with “Failed to add datasource” error. > > > > I have followed all the available instructions related to CloudStack, > > KVM and ceph storage integration. > > > > > > > > CenOS 6.5 KVM is used as KVM node here. I have read in some blog that > > we need to compile LibVirt in CenOS KVM nodes to make Ceph storage to > > work with in CloudStack. > > > > Hence I have got git cloned LibVirt package from its source and > > upgraded LibVirt and Qemu versions. > > > > (Commands used à git cone #####, ./autogen.sh, make, make install). > > > > > > > > It seems CentOS 6.5 KVM needs to be enabled for RBD (driver) support > > which needs to be specified as a parameter while compiling LibVirt. > > > > > > > > Can anyone throw some pointers on how to rectify this problem? > > > > > > > > *Management Server Exception:* > > > > 2014-06-20 09:58:03,757 DEBUG [agent.transport.Request] > > (catalina-exec-6:null) Seq 1-1602164611: Received: { Ans: , MgmtId: > > 52234925782, via: 1, Ver: v1, Flags: 10, { Answer } } > > > > 2014-06-20 09:58:03,757 DEBUG [agent.manager.AgentManagerImpl] > > (catalina-exec-6:null) Details from executing class > > com.cloud.agent.api.ModifyStoragePoolCommand: > > java.lang.NullPointerException > > > > at > > com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePo > > ol(LibvirtStorageAdaptor.java:531) > > > > at > > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePo > > ol(KVMStoragePoolManager.java:185) > > > > at > > com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePo > > ol(KVMStoragePoolManager.java:177) > > > > > > > > I even tried defining a pool using virsh command even that is failing > > with “internal error missing backend for pool type 8”. > > > > This indicates my KVM LibVirt is not supported for RBD. > > > > > > > > *Virsh exception on manual pool definition* > > > > <pool type='rbd'> > > > > <name>c574980a-19fc-37e9-b6e3-788a7439575d</name> > > > > <uuid>c574980a-19fc-37e9-b6e3-788a7439575d</uuid> > > > > <source> > > > > <host name='192.168.153.25' port='6789'/> > > > > <name>cloudstack</name> > > > > <auth username='cloudstack' type='ceph'> > > > > <secret uuid='c574980a-19fc-37e9-b6e3-788a7439575d'/> > > > > </auth> > > > > </source> > > > > </pool> > > > > > > > > [root@kvm-ovs-002 agent]# virsh pool-define /tmp/rbd.xml > > > > error: Failed to define pool from /tmp/rbd.xml > > > > error: internal error missing backend for pool type 8 > > > > > > > > The Ceph storage is working fine and confirmed with following > > statistics of it. > > > > > > > > *Ceph output* > > > > [root@kvm-ovs-002 ~]# ceph auth list > > > > installed auth entries: > > > > > > > > osd.0 > > > > key: AQCwTKFTSOudGhAAsWAMRFuCqHjvTQKEV0zjvw== > > > > caps: [mon] allow profile osd > > > > caps: [osd] allow * > > > > client.admin > > > > key: AQBRQqFTWOjBKhAA2s7KnL1z3h7PuKeqXMd7SA== > > > > caps: [mds] allow > > > > caps: [mon] allow * > > > > caps: [osd] allow * > > > > client.bootstrap-mds > > > > key: AQBSQqFTYKm6CRAAjjZotpN68yJaOjS2QTKzKg== > > > > caps: [mon] allow profile bootstrap-mds > > > > client.bootstrap-osd > > > > key: AQBRQqFT6GzXNxAA4ZTmVX6LIu0k4Sk7bh2Ifg== > > > > caps: [mon] allow profile bootstrap-osd > > > > client.cloudstack > > > > key: AQBNTaFTeCuwFRAA0NE7CCm9rwuq3ngLcGEysQ== > > > > caps: [mon] allow r > > > > caps: [osd] allow rwx pool=cloudstack > > > > > > > > [root@ceph ~]# ceph status > > > > cluster 9c1be0b6-f600-45d7-ae0f-df7bcd3a82cd > > > > health HEALTH_WARN 292 pgs degraded; 292 pgs stale; 292 pgs stuck > > stale; 292 pgs stuck unclean; 1/1 in osds are down; clock skew > > detected on > > mon.kvm-ovs-00 > > 2 > > > > monmap e1: 2 mons at {ceph= > > 192.168.153.25:6789/0,kvm-ovs-002=192.168.160.3: 6789/0}, election > > epoch 10, quorum 0,1 ceph,kvm-ovs-002 > > > > osdmap e8: 1 osds: 0 up, 1 in > > > > pgmap v577: 292 pgs, 4 pools, 0 bytes data, 0 objects > > > > 26036 MB used, 824 GB / 895 GB avail > > > > 292 stale+active+degraded > > > > > > > > [root@kvm-ovs-002 agent]# cat /etc/redhat-release > > > > CentOS release 6.5 (Final) > > > > > > > > The compiled LibVirt is showing upgraded version in virsh but still > > finding old rpm packages in KVM. > > > > Give me some hint on whether to clean-up these old RPMs? > > > > > > > > *Virsh version* > > > > [root@kvm-ovs-002 agent]# virsh version > > > > Compiled against library: libvirt 1.2.6 > > > > Using library: libvirt 1.2.6 > > > > Using API: QEMU 1.2.6 > > > > Running hypervisor: QEMU 0.12.1 > > > > > > > > [root@kvm-ovs-002 agent]# rpm -qa | grep qemu > > > > qemu-kvm-tools-0.12.1.2-2.415.el6_5.10.x86_64 > > > > qemu-kvm-0.12.1.2-2.415.el6.3ceph.x86_64 > > > > gpxe-roms-qemu-0.9.7-6.10.el6.noarch > > > > qemu-kvm-0.12.1.2-2.415.el6_5.10.x86_64 > > > > qemu-img-0.12.1.2-2.415.el6.3ceph.x86_64 > > > > qemu-img-0.12.1.2-2.415.el6_5.10.x86_64 > > > > qemu-kvm-tools-0.12.1.2-2.415.el6.3ceph.x86_64 > > > > qemu-guest-agent-0.12.1.2-2.415.el6_5.10.x86_64 > > > > You have mail in /var/spool/mail/root > > > > [root@kvm-ovs-002 agent]# rpm -qa | grep libvirt > > > > libvirt-python-0.10.2-29.el6_5.9.x86_64 > > > > libvirt-java-0.4.9-1.el6.noarch > > > > libvirt-cim-0.6.1-9.el6_5.1.x86_64 > > > > libvirt-client-0.10.2-29.el6_5.9.x86_64 > > > > libvirt-devel-0.10.2-29.el6_5.9.x86_64 > > > > fence-virtd-libvirt-0.2.3-15.el6.x86_64 > > > > libvirt-0.10.2-29.el6_5.9.x86_64 > > > > libvirt-snmp-0.0.2-4.el6.x86_64 > > > > > > > > *Attached all the log files from management and kvm servers. > > > > > > > > Thanks, > > > > *Praveen Kumar Buravilli* > > > > > > > > > > -- > > Andrija Panić > -------------------------------------- > http://admintweets.com > -------------------------------------- >
