RE: Unable to add Ceph KVM node in cloudstack
Plese enable debug loging in acs agent,restart and retry to add host and please post logs. On Sep 14, 2015 8:34 AM, "Shetty, Pradeep" wrote: > Yes, libvirt has been compiled to support rbd and using ceph's qemu rpm > packages. > > > Regards, > Pradeep > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 14 September 2015 11:52 > To: users@cloudstack.apache.org > Cc: Vadim Kimlaychuk > Subject: RE: Unable to add Ceph KVM node in cloudstack > > Did you compile libvirt and qemu with RBD support? > Centos6.5 ( libvitt and qemu) does not support RBD/ceph as storage out of > the box... > On Sep 14, 2015 7:37 AM, "Shetty, Pradeep" wrote: > > > Hi, > > > > Is Ceph 0.94 Hammer compatible/stable with CentOS 6.5 build and > > cloudstack version 4.4.2 > > > > Thanks, > > Pradeep > > > > -Original Message- > > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > > Sent: 11 September 2015 20:42 > > To: Vadim Kimlaychuk > > Cc: users@cloudstack.apache.org > > Subject: Re: Unable to add Ceph KVM node in cloudstack > > > > Dont really know that to be honest... even previously, you can patch > > qemu source code yourslef with RBD support - nothing stops you to do > > so - so this is just decision from RH or whatever - Im just guessing - > > anyway it is the way it is - ubuntu oit of the box suport, CentOS not > afaik... > > > > On 11 September 2015 at 13:56, Vadim Kimlaychuk > > wrote: > > > > > Andrija, > > > > > > If RBD libvirt driver now belongs to RH does that mean > > > Ubuntu/Debian will loose it some day as it is proprietary software? > > > Haven't follow Ceph achievements since "Emperor" > > > > > > Vadim. > > > > > > > > > On 2015-09-11 14:39, Andrija Panic wrote: > > > > > > Np, Im actually very disapointed by RH decision to have this > > > approach > > > - > > >> since now CEPH/Inktank is part of RH etc... > > >> I didnt want to play with Ubuntu at all (dont let me start about > > >> all the possible issues/bugs we found) but I dont like idea of > > >> compiling stuff on production... > > >> > > >> On 11 September 2015 at 13:34, Vadim Kimlaychuk > > >> > > >> wrote: > > >> > > >> Andrija, > > >> > > >> I don't have such broad experience with different CentOS/RHEL > > >> versions, but I am a bit surprised that these distributions does > > >> not have RBD support at all. That means they have to maintain their > > >> own "restricted" > > >> version of these libraries apart from mainstream. Now I am even > > >> more happy with my Debian/Ubuntu installations :) > > >> > > >> Thank you for information! > > >> > > >> Vadim. > > >> > > >> On 2015-09-11 14:20, Andrija Panic wrote: > > >> > > >> Vadim, > > >> afaik you are right for mgmt server. As for KVM nodes - libvirt and > > >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I > > >> have not played a lot with CentOS7, but did quick check long time > > >> ago and I thing it also doesnt support RBD - Red Hat wants you to > > >> buy RHEV to support RBD - of course, there are workarround, > > >> self-compile libvirt etc...similar for qemu/qemu-img. > > >> > > >> I know for sure that Ubuntu 14.x suports RBD out of the box - since > > >> we are running it with CEPH. > > >> > > >> Best > > >> > > >> On 11 September 2015 at 10:47, Vadim Kimlaychuk > > >> > > >> wrote: > > >> > > >> Andrija, > > >> > > >> Are you talking about CS Management server OS? I thought it does > > >> not really matter if it includes libvirt or not. All calls to > > >> storage should go through hypervisor and any KVM host goes with > > >> libvirt. The version of libvirt for old OS may not be aligned with > > >> latest Ceph release that is true. Cloud architect must check > > >> compatibility version matrix for all products Ceph/KVM/CS anyway. > > >> > > >> Vadim. > > >> > > >> On 2015-09-11 11:24, Andrija Panic wrote: > > >> > > >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps
RE: Unable to add Ceph KVM node in cloudstack
Yes, libvirt has been compiled to support rbd and using ceph's qemu rpm packages. Regards, Pradeep -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: 14 September 2015 11:52 To: users@cloudstack.apache.org Cc: Vadim Kimlaychuk Subject: RE: Unable to add Ceph KVM node in cloudstack Did you compile libvirt and qemu with RBD support? Centos6.5 ( libvitt and qemu) does not support RBD/ceph as storage out of the box... On Sep 14, 2015 7:37 AM, "Shetty, Pradeep" wrote: > Hi, > > Is Ceph 0.94 Hammer compatible/stable with CentOS 6.5 build and > cloudstack version 4.4.2 > > Thanks, > Pradeep > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 11 September 2015 20:42 > To: Vadim Kimlaychuk > Cc: users@cloudstack.apache.org > Subject: Re: Unable to add Ceph KVM node in cloudstack > > Dont really know that to be honest... even previously, you can patch > qemu source code yourslef with RBD support - nothing stops you to do > so - so this is just decision from RH or whatever - Im just guessing - > anyway it is the way it is - ubuntu oit of the box suport, CentOS not afaik... > > On 11 September 2015 at 13:56, Vadim Kimlaychuk > wrote: > > > Andrija, > > > > If RBD libvirt driver now belongs to RH does that mean > > Ubuntu/Debian will loose it some day as it is proprietary software? > > Haven't follow Ceph achievements since "Emperor" > > > > Vadim. > > > > > > On 2015-09-11 14:39, Andrija Panic wrote: > > > > Np, Im actually very disapointed by RH decision to have this > > approach > > - > >> since now CEPH/Inktank is part of RH etc... > >> I didnt want to play with Ubuntu at all (dont let me start about > >> all the possible issues/bugs we found) but I dont like idea of > >> compiling stuff on production... > >> > >> On 11 September 2015 at 13:34, Vadim Kimlaychuk > >> > >> wrote: > >> > >> Andrija, > >> > >> I don't have such broad experience with different CentOS/RHEL > >> versions, but I am a bit surprised that these distributions does > >> not have RBD support at all. That means they have to maintain their > >> own "restricted" > >> version of these libraries apart from mainstream. Now I am even > >> more happy with my Debian/Ubuntu installations :) > >> > >> Thank you for information! > >> > >> Vadim. > >> > >> On 2015-09-11 14:20, Andrija Panic wrote: > >> > >> Vadim, > >> afaik you are right for mgmt server. As for KVM nodes - libvirt and > >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I > >> have not played a lot with CentOS7, but did quick check long time > >> ago and I thing it also doesnt support RBD - Red Hat wants you to > >> buy RHEV to support RBD - of course, there are workarround, > >> self-compile libvirt etc...similar for qemu/qemu-img. > >> > >> I know for sure that Ubuntu 14.x suports RBD out of the box - since > >> we are running it with CEPH. > >> > >> Best > >> > >> On 11 September 2015 at 10:47, Vadim Kimlaychuk > >> > >> wrote: > >> > >> Andrija, > >> > >> Are you talking about CS Management server OS? I thought it does > >> not really matter if it includes libvirt or not. All calls to > >> storage should go through hypervisor and any KVM host goes with > >> libvirt. The version of libvirt for old OS may not be aligned with > >> latest Ceph release that is true. Cloud architect must check > >> compatibility version matrix for all products Ceph/KVM/CS anyway. > >> > >> Vadim. > >> > >> On 2015-09-11 11:24, Andrija Panic wrote: > >> > >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also > >> 12.x) > >> - not with CentOS6, and not sure really about CentOS7 (probably not > >> also...) > >> > >> On 11 September 2015 at 09:53, Vadim Kimlaychuk > >> > >> wrote: > >> > >> For KVM hosts RBD comes "out of the box" with libvirt. For > >> XenServer Ceph > >> (RBD) storage type is not yet implemented. I would say it is very > >> hypervisor-specific and does not depend much from Cloudstack. > >> > >> Regards, > >> > >> Vadim. > >> > >> On 2015-09-11 08:55, Shetty, Pradeep wrote: > >>
RE: Unable to add Ceph KVM node in cloudstack
Did you compile libvirt and qemu with RBD support? Centos6.5 ( libvitt and qemu) does not support RBD/ceph as storage out of the box... On Sep 14, 2015 7:37 AM, "Shetty, Pradeep" wrote: > Hi, > > Is Ceph 0.94 Hammer compatible/stable with CentOS 6.5 build and cloudstack > version 4.4.2 > > Thanks, > Pradeep > > -Original Message- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: 11 September 2015 20:42 > To: Vadim Kimlaychuk > Cc: users@cloudstack.apache.org > Subject: Re: Unable to add Ceph KVM node in cloudstack > > Dont really know that to be honest... even previously, you can patch qemu > source code yourslef with RBD support - nothing stops you to do so - so > this is just decision from RH or whatever - Im just guessing - anyway it is > the way it is - ubuntu oit of the box suport, CentOS not afaik... > > On 11 September 2015 at 13:56, Vadim Kimlaychuk > wrote: > > > Andrija, > > > > If RBD libvirt driver now belongs to RH does that mean > > Ubuntu/Debian will loose it some day as it is proprietary software? > > Haven't follow Ceph achievements since "Emperor" > > > > Vadim. > > > > > > On 2015-09-11 14:39, Andrija Panic wrote: > > > > Np, Im actually very disapointed by RH decision to have this approach > > - > >> since now CEPH/Inktank is part of RH etc... > >> I didnt want to play with Ubuntu at all (dont let me start about all > >> the possible issues/bugs we found) but I dont like idea of > >> compiling stuff on production... > >> > >> On 11 September 2015 at 13:34, Vadim Kimlaychuk > >> wrote: > >> > >> Andrija, > >> > >> I don't have such broad experience with different CentOS/RHEL > >> versions, but I am a bit surprised that these distributions does not > >> have RBD support at all. That means they have to maintain their own > >> "restricted" > >> version of these libraries apart from mainstream. Now I am even more > >> happy with my Debian/Ubuntu installations :) > >> > >> Thank you for information! > >> > >> Vadim. > >> > >> On 2015-09-11 14:20, Andrija Panic wrote: > >> > >> Vadim, > >> afaik you are right for mgmt server. As for KVM nodes - libvirt and > >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have > >> not played a lot with CentOS7, but did quick check long time ago and > >> I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to > >> support RBD - of course, there are workarround, self-compile libvirt > >> etc...similar for qemu/qemu-img. > >> > >> I know for sure that Ubuntu 14.x suports RBD out of the box - since > >> we are running it with CEPH. > >> > >> Best > >> > >> On 11 September 2015 at 10:47, Vadim Kimlaychuk > >> wrote: > >> > >> Andrija, > >> > >> Are you talking about CS Management server OS? I thought it does not > >> really matter if it includes libvirt or not. All calls to storage > >> should go through hypervisor and any KVM host goes with libvirt. The > >> version of libvirt for old OS may not be aligned with latest Ceph > >> release that is true. Cloud architect must check compatibility > >> version matrix for all products Ceph/KVM/CS anyway. > >> > >> Vadim. > >> > >> On 2015-09-11 11:24, Andrija Panic wrote: > >> > >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) > >> - not with CentOS6, and not sure really about CentOS7 (probably not > >> also...) > >> > >> On 11 September 2015 at 09:53, Vadim Kimlaychuk > >> wrote: > >> > >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer > >> Ceph > >> (RBD) storage type is not yet implemented. I would say it is very > >> hypervisor-specific and does not depend much from Cloudstack. > >> > >> Regards, > >> > >> Vadim. > >> > >> On 2015-09-11 08:55, Shetty, Pradeep wrote: > >> > >> I have one question: > >> Is ceph rbd protocol supported by default on libvirt version > >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd > >> within cloudstack. > >> > >> Regards, > >> Pradeep > >> > >> -Original Message- > >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] > >> Sent: 0
RE: Unable to add Ceph KVM node in cloudstack
Hi, Is Ceph 0.94 Hammer compatible/stable with CentOS 6.5 build and cloudstack version 4.4.2 Thanks, Pradeep -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: 11 September 2015 20:42 To: Vadim Kimlaychuk Cc: users@cloudstack.apache.org Subject: Re: Unable to add Ceph KVM node in cloudstack Dont really know that to be honest... even previously, you can patch qemu source code yourslef with RBD support - nothing stops you to do so - so this is just decision from RH or whatever - Im just guessing - anyway it is the way it is - ubuntu oit of the box suport, CentOS not afaik... On 11 September 2015 at 13:56, Vadim Kimlaychuk wrote: > Andrija, > > If RBD libvirt driver now belongs to RH does that mean > Ubuntu/Debian will loose it some day as it is proprietary software? > Haven't follow Ceph achievements since "Emperor" > > Vadim. > > > On 2015-09-11 14:39, Andrija Panic wrote: > > Np, Im actually very disapointed by RH decision to have this approach > - >> since now CEPH/Inktank is part of RH etc... >> I didnt want to play with Ubuntu at all (dont let me start about all >> the possible issues/bugs we found) but I dont like idea of >> compiling stuff on production... >> >> On 11 September 2015 at 13:34, Vadim Kimlaychuk >> wrote: >> >> Andrija, >> >> I don't have such broad experience with different CentOS/RHEL >> versions, but I am a bit surprised that these distributions does not >> have RBD support at all. That means they have to maintain their own >> "restricted" >> version of these libraries apart from mainstream. Now I am even more >> happy with my Debian/Ubuntu installations :) >> >> Thank you for information! >> >> Vadim. >> >> On 2015-09-11 14:20, Andrija Panic wrote: >> >> Vadim, >> afaik you are right for mgmt server. As for KVM nodes - libvirt and >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have >> not played a lot with CentOS7, but did quick check long time ago and >> I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to >> support RBD - of course, there are workarround, self-compile libvirt >> etc...similar for qemu/qemu-img. >> >> I know for sure that Ubuntu 14.x suports RBD out of the box - since >> we are running it with CEPH. >> >> Best >> >> On 11 September 2015 at 10:47, Vadim Kimlaychuk >> wrote: >> >> Andrija, >> >> Are you talking about CS Management server OS? I thought it does not >> really matter if it includes libvirt or not. All calls to storage >> should go through hypervisor and any KVM host goes with libvirt. The >> version of libvirt for old OS may not be aligned with latest Ceph >> release that is true. Cloud architect must check compatibility >> version matrix for all products Ceph/KVM/CS anyway. >> >> Vadim. >> >> On 2015-09-11 11:24, Andrija Panic wrote: >> >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) >> - not with CentOS6, and not sure really about CentOS7 (probably not >> also...) >> >> On 11 September 2015 at 09:53, Vadim Kimlaychuk >> wrote: >> >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer >> Ceph >> (RBD) storage type is not yet implemented. I would say it is very >> hypervisor-specific and does not depend much from Cloudstack. >> >> Regards, >> >> Vadim. >> >> On 2015-09-11 08:55, Shetty, Pradeep wrote: >> >> I have one question: >> Is ceph rbd protocol supported by default on libvirt version >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd >> within cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 09 September 2015 12:07 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> I have installed CS 4.2 with Ceph storage successfully around year ago. >> There were no problem with configuration except the fact that you >> can't configure RBD storage as primary storage during Zone set-up. >> It must be NFS. >> After Zone is set-up and running you may add another primary storage >> and there you have list of choices where RBD type is present. >> After RBD is added you may assign this as default storage for >> particular cluster assuming that each cluster has its own pr
Re: Unable to add Ceph KVM node in cloudstack
Dont really know that to be honest... even previously, you can patch qemu source code yourslef with RBD support - nothing stops you to do so - so this is just decision from RH or whatever - Im just guessing - anyway it is the way it is - ubuntu oit of the box suport, CentOS not afaik... On 11 September 2015 at 13:56, Vadim Kimlaychuk wrote: > Andrija, > > If RBD libvirt driver now belongs to RH does that mean Ubuntu/Debian > will loose it some day as it is proprietary software? Haven't follow Ceph > achievements since "Emperor" > > Vadim. > > > On 2015-09-11 14:39, Andrija Panic wrote: > > Np, Im actually very disapointed by RH decision to have this approach - >> since now CEPH/Inktank is part of RH etc... >> I didnt want to play with Ubuntu at all (dont let me start about all the >> possible issues/bugs we found) but I dont like idea of compiling stuff >> on production... >> >> On 11 September 2015 at 13:34, Vadim Kimlaychuk >> wrote: >> >> Andrija, >> >> I don't have such broad experience with different CentOS/RHEL >> versions, but I am a bit surprised that these distributions does not have >> RBD support at all. That means they have to maintain their own >> "restricted" >> version of these libraries apart from mainstream. Now I am even more happy >> with my Debian/Ubuntu installations :) >> >> Thank you for information! >> >> Vadim. >> >> On 2015-09-11 14:20, Andrija Panic wrote: >> >> Vadim, >> afaik you are right for mgmt server. As for KVM nodes - libvirt and >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have not >> played a lot with CentOS7, but did quick check long time ago and I thing >> it >> also doesnt support RBD - Red Hat wants you to buy RHEV to support RBD - >> of >> course, there are workarround, self-compile libvirt etc...similar for >> qemu/qemu-img. >> >> I know for sure that Ubuntu 14.x suports RBD out of the box - since we are >> running it with CEPH. >> >> Best >> >> On 11 September 2015 at 10:47, Vadim Kimlaychuk >> wrote: >> >> Andrija, >> >> Are you talking about CS Management server OS? I thought it does >> not really matter if it includes libvirt or not. All calls to storage >> should go through hypervisor and any KVM host goes with libvirt. The >> version of libvirt for old OS may not be aligned with latest Ceph release >> that is true. Cloud architect must check compatibility version matrix for >> all products Ceph/KVM/CS anyway. >> >> Vadim. >> >> On 2015-09-11 11:24, Andrija Panic wrote: >> >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - >> not with CentOS6, and not sure really about CentOS7 (probably not also...) >> >> On 11 September 2015 at 09:53, Vadim Kimlaychuk >> wrote: >> >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph >> (RBD) storage type is not yet implemented. I would say it is very >> hypervisor-specific and does not depend much from Cloudstack. >> >> Regards, >> >> Vadim. >> >> On 2015-09-11 08:55, Shetty, Pradeep wrote: >> >> I have one question: >> Is ceph rbd protocol supported by default on libvirt version >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd >> within >> cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 09 September 2015 12:07 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> I have installed CS 4.2 with Ceph storage successfully around year ago. >> There were no problem with configuration except the fact that you can't >> configure RBD storage as primary storage during Zone set-up. >> It must be NFS. >> After Zone is set-up and running you may add another primary storage and >> there you have list of choices where RBD type is present. >> After RBD is added you may assign this as default storage for particular >> cluster assuming that each cluster has its own primary storage. >> >> PS. I have also used Ceph object gateway to connect secondary storage. It >> works as expected. It would be nice to know which way are you going to >> integrate Ceph into CS? >> >> Vadim. >> >> On 2015-09-09 08:09, Shetty, Pradeep wrote: >> >> Please share steps for adding cep
Re: Unable to add Ceph KVM node in cloudstack
Andrija, If RBD libvirt driver now belongs to RH does that mean Ubuntu/Debian will loose it some day as it is proprietary software? Haven't follow Ceph achievements since "Emperor" Vadim. On 2015-09-11 14:39, Andrija Panic wrote: Np, Im actually very disapointed by RH decision to have this approach - since now CEPH/Inktank is part of RH etc... I didnt want to play with Ubuntu at all (dont let me start about all the possible issues/bugs we found) but I dont like idea of compiling stuff on production... On 11 September 2015 at 13:34, Vadim Kimlaychuk wrote: Andrija, I don't have such broad experience with different CentOS/RHEL versions, but I am a bit surprised that these distributions does not have RBD support at all. That means they have to maintain their own "restricted" version of these libraries apart from mainstream. Now I am even more happy with my Debian/Ubuntu installations :) Thank you for information! Vadim. On 2015-09-11 14:20, Andrija Panic wrote: Vadim, afaik you are right for mgmt server. As for KVM nodes - libvirt and qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have not played a lot with CentOS7, but did quick check long time ago and I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to support RBD - of course, there are workarround, self-compile libvirt etc...similar for qemu/qemu-img. I know for sure that Ubuntu 14.x suports RBD out of the box - since we are running it with CEPH. Best On 11 September 2015 at 10:47, Vadim Kimlaychuk wrote: Andrija, Are you talking about CS Management server OS? I thought it does not really matter if it includes libvirt or not. All calls to storage should go through hypervisor and any KVM host goes with libvirt. The version of libvirt for old OS may not be aligned with latest Ceph release that is true. Cloud architect must check compatibility version matrix for all products Ceph/KVM/CS anyway. Vadim. On 2015-09-11 11:24, Andrija Panic wrote: RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - not with CentOS6, and not sure really about CentOS7 (probably not also...) On 11 September 2015 at 09:53, Vadim Kimlaychuk wrote: For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph (RBD) storage type is not yet implemented. I would say it is very hypervisor-specific and does not depend much from Cloudstack. Regards, Vadim. On 2015-09-11 08:55, Shetty, Pradeep wrote: I have one question: Is ceph rbd protocol supported by default on libvirt version 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 09 September 2015 12:07 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [1 [1]] [1 [1]] [1 [1]] [1 [1]] m.html [1 [2]] Regards, Vadim On 2015-09-08 08:18,
Re: Unable to add Ceph KVM node in cloudstack
Np, Im actually very disapointed by RH decision to have this approach - since now CEPH/Inktank is part of RH etc... I didnt want to play with Ubuntu at all (dont let me start about all the possible issues/bugs we found) but I dont like idea of compiling stuff on production... On 11 September 2015 at 13:34, Vadim Kimlaychuk wrote: > > Andrija, > > I don't have such broad experience with different CentOS/RHEL > versions, but I am a bit surprised that these distributions does not have > RBD support at all. That means they have to maintain their own "restricted" > version of these libraries apart from mainstream. Now I am even more happy > with my Debian/Ubuntu installations :) > > Thank you for information! > > Vadim. > > > On 2015-09-11 14:20, Andrija Panic wrote: > > Vadim, >> >> afaik you are right for mgmt server. As for KVM nodes - libvirt and >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have not >> played a lot with CentOS7, but did quick check long time ago and I thing >> it >> also doesnt support RBD - Red Hat wants you to buy RHEV to support RBD - >> of >> course, there are workarround, self-compile libvirt etc...similar for >> qemu/qemu-img. >> >> I know for sure that Ubuntu 14.x suports RBD out of the box - since we are >> running it with CEPH. >> >> Best >> >> On 11 September 2015 at 10:47, Vadim Kimlaychuk >> wrote: >> >> Andrija, >> >> Are you talking about CS Management server OS? I thought it does >> not really matter if it includes libvirt or not. All calls to storage >> should go through hypervisor and any KVM host goes with libvirt. The >> version of libvirt for old OS may not be aligned with latest Ceph release >> that is true. Cloud architect must check compatibility version matrix for >> all products Ceph/KVM/CS anyway. >> >> Vadim. >> >> On 2015-09-11 11:24, Andrija Panic wrote: >> >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - >> not with CentOS6, and not sure really about CentOS7 (probably not also...) >> >> On 11 September 2015 at 09:53, Vadim Kimlaychuk >> wrote: >> >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph >> (RBD) storage type is not yet implemented. I would say it is very >> hypervisor-specific and does not depend much from Cloudstack. >> >> Regards, >> >> Vadim. >> >> On 2015-09-11 08:55, Shetty, Pradeep wrote: >> >> I have one question: >> Is ceph rbd protocol supported by default on libvirt version >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd >> within >> cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 09 September 2015 12:07 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> I have installed CS 4.2 with Ceph storage successfully around year ago. >> There were no problem with configuration except the fact that you can't >> configure RBD storage as primary storage during Zone set-up. >> It must be NFS. >> After Zone is set-up and running you may add another primary storage and >> there you have list of choices where RBD type is present. >> After RBD is added you may assign this as default storage for particular >> cluster assuming that each cluster has its own primary storage. >> >> PS. I have also used Ceph object gateway to connect secondary storage. It >> works as expected. It would be nice to know which way are you going to >> integrate Ceph into CS? >> >> Vadim. >> >> On 2015-09-09 08:09, Shetty, Pradeep wrote: >> >> Please share steps for adding ceph RBD as primary storage using KVM >> hypervisor in cloudstack. >> >> We are well aware of the best practices/recommendations for cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 08 September 2015 12:14 >> To: users@cloudstack.apache.org >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> From documentation : >> Warning -- It is NOT recommended to run services on this host not >> controlled by CloudStack. >> >> Good practice -- KVM host must be virgin-clean. >> >> Vadim. >> >> On 2015-09-08 09:14, Shetty, P
Re: Unable to add Ceph KVM node in cloudstack
Andrija, I don't have such broad experience with different CentOS/RHEL versions, but I am a bit surprised that these distributions does not have RBD support at all. That means they have to maintain their own "restricted" version of these libraries apart from mainstream. Now I am even more happy with my Debian/Ubuntu installations :) Thank you for information! Vadim. On 2015-09-11 14:20, Andrija Panic wrote: Vadim, afaik you are right for mgmt server. As for KVM nodes - libvirt and qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have not played a lot with CentOS7, but did quick check long time ago and I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to support RBD - of course, there are workarround, self-compile libvirt etc...similar for qemu/qemu-img. I know for sure that Ubuntu 14.x suports RBD out of the box - since we are running it with CEPH. Best On 11 September 2015 at 10:47, Vadim Kimlaychuk wrote: Andrija, Are you talking about CS Management server OS? I thought it does not really matter if it includes libvirt or not. All calls to storage should go through hypervisor and any KVM host goes with libvirt. The version of libvirt for old OS may not be aligned with latest Ceph release that is true. Cloud architect must check compatibility version matrix for all products Ceph/KVM/CS anyway. Vadim. On 2015-09-11 11:24, Andrija Panic wrote: RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - not with CentOS6, and not sure really about CentOS7 (probably not also...) On 11 September 2015 at 09:53, Vadim Kimlaychuk wrote: For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph (RBD) storage type is not yet implemented. I would say it is very hypervisor-specific and does not depend much from Cloudstack. Regards, Vadim. On 2015-09-11 08:55, Shetty, Pradeep wrote: I have one question: Is ceph rbd protocol supported by default on libvirt version 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 09 September 2015 12:07 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [1 [1]] [1 [1]] [1 [1]] m.html [1 [2]] Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps t
Re: Unable to add Ceph KVM node in cloudstack
Vadim, afaik you are right for mgmt server. As for KVM nodes - libvirt and qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have not played a lot with CentOS7, but did quick check long time ago and I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to support RBD - of course, there are workarround, self-compile libvirt etc...similar for qemu/qemu-img. I know for sure that Ubuntu 14.x suports RBD out of the box - since we are running it with CEPH. Best On 11 September 2015 at 10:47, Vadim Kimlaychuk wrote: > Andrija, > > Are you talking about CS Management server OS? I thought it does > not really matter if it includes libvirt or not. All calls to storage > should go through hypervisor and any KVM host goes with libvirt. The > version of libvirt for old OS may not be aligned with latest Ceph release > that is true. Cloud architect must check compatibility version matrix for > all products Ceph/KVM/CS anyway. > > Vadim. > > > > On 2015-09-11 11:24, Andrija Panic wrote: > > RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - not >> with CentOS6, and not sure really about CentOS7 (probably not also...) >> >> On 11 September 2015 at 09:53, Vadim Kimlaychuk >> wrote: >> >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph >> (RBD) storage type is not yet implemented. I would say it is very >> hypervisor-specific and does not depend much from Cloudstack. >> >> Regards, >> >> Vadim. >> >> On 2015-09-11 08:55, Shetty, Pradeep wrote: >> >> I have one question: >> Is ceph rbd protocol supported by default on libvirt version >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd >> within >> cloudstack. >> >> Regards, >> Pradeep >> >> -----Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 09 September 2015 12:07 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> I have installed CS 4.2 with Ceph storage successfully around year ago. >> There were no problem with configuration except the fact that you can't >> configure RBD storage as primary storage during Zone set-up. >> It must be NFS. >> After Zone is set-up and running you may add another primary storage and >> there you have list of choices where RBD type is present. >> After RBD is added you may assign this as default storage for particular >> cluster assuming that each cluster has its own primary storage. >> >> PS. I have also used Ceph object gateway to connect secondary storage. It >> works as expected. It would be nice to know which way are you going to >> integrate Ceph into CS? >> >> Vadim. >> >> On 2015-09-09 08:09, Shetty, Pradeep wrote: >> >> Please share steps for adding ceph RBD as primary storage using KVM >> hypervisor in cloudstack. >> >> We are well aware of the best practices/recommendations for cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 08 September 2015 12:14 >> To: users@cloudstack.apache.org >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> From documentation : >> Warning -- It is NOT recommended to run services on this host not >> controlled by CloudStack. >> >> Good practice -- KVM host must be virgin-clean. >> >> Vadim. >> >> On 2015-09-08 09:14, Shetty, Pradeep wrote: >> >> Hello Vadim, >> >> We have followed the same guide and using OVS version 2.3.2 . >> >> Ceph storage cluster is running on same KVM hypervisor and it fails >> when adding to cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 08 September 2015 11:34 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: Re: Unable to add Ceph KVM node in cloudstack >> >> Hello Shetty, >> >> It seems you agent or libvirt configuration is wrong. Please, specify >> also do you use OVS or Linux bridge? >> Did you follow this guide? >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv >> [1] >> [1 [1]] >> [1 [1]] >> m.html >> [1 [2]] >> >> Regards, >> >> Vadim >> >> On 2015-09-08 08:18, Shetty, Pradeep wrote: >
Re: Unable to add Ceph KVM node in cloudstack
Andrija, Are you talking about CS Management server OS? I thought it does not really matter if it includes libvirt or not. All calls to storage should go through hypervisor and any KVM host goes with libvirt. The version of libvirt for old OS may not be aligned with latest Ceph release that is true. Cloud architect must check compatibility version matrix for all products Ceph/KVM/CS anyway. Vadim. On 2015-09-11 11:24, Andrija Panic wrote: RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - not with CentOS6, and not sure really about CentOS7 (probably not also...) On 11 September 2015 at 09:53, Vadim Kimlaychuk wrote: For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph (RBD) storage type is not yet implemented. I would say it is very hypervisor-specific and does not depend much from Cloudstack. Regards, Vadim. On 2015-09-11 08:55, Shetty, Pradeep wrote: I have one question: Is ceph rbd protocol supported by default on libvirt version 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 09 September 2015 12:07 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [1 [1]] [1 [1]] m.html [1 [2]] Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep Links: -- [1 [1]] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] [2] [2 [2]] Links: -- [1 [1]] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [1 [1]] [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] [2] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
Re: Unable to add Ceph KVM node in cloudstack
RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) - not with CentOS6, and not sure really about CentOS7 (probably not also...) On 11 September 2015 at 09:53, Vadim Kimlaychuk wrote: > For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph > (RBD) storage type is not yet implemented. I would say it is very > hypervisor-specific and does not depend much from Cloudstack. > > Regards, > > Vadim. > > > On 2015-09-11 08:55, Shetty, Pradeep wrote: > > I have one question: >> >> Is ceph rbd protocol supported by default on libvirt version >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within >> cloudstack. >> >> Regards, >> Pradeep >> >> -Original Message- >> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >> Sent: 09 September 2015 12:07 >> To: users@cloudstack.apache.org >> Cc: Shetty, Pradeep >> Subject: RE: Unable to add Ceph KVM node in cloudstack >> >> Pradeep, >> >> I have installed CS 4.2 with Ceph storage successfully around year ago. >> There were no problem with configuration except the fact that you can't >> configure RBD storage as primary storage during Zone set-up. >> It must be NFS. >> After Zone is set-up and running you may add another primary storage and >> there you have list of choices where RBD type is present. >> After RBD is added you may assign this as default storage for particular >> cluster assuming that each cluster has its own primary storage. >> >> PS. I have also used Ceph object gateway to connect secondary storage. It >> works as expected. It would be nice to know which way are you going to >> integrate Ceph into CS? >> >> Vadim. >> >> On 2015-09-09 08:09, Shetty, Pradeep wrote: >> >> Please share steps for adding ceph RBD as primary storage using KVM >>> hypervisor in cloudstack. >>> >>> We are well aware of the best practices/recommendations for cloudstack. >>> >>> Regards, >>> Pradeep >>> >>> -Original Message- >>> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >>> Sent: 08 September 2015 12:14 >>> To: users@cloudstack.apache.org >>> Subject: RE: Unable to add Ceph KVM node in cloudstack >>> >>> Pradeep, >>> >>> From documentation : >>> Warning -- It is NOT recommended to run services on this host not >>> controlled by CloudStack. >>> >>> Good practice -- KVM host must be virgin-clean. >>> >>> Vadim. >>> >>> On 2015-09-08 09:14, Shetty, Pradeep wrote: >>> >>> Hello Vadim, >>> >>> We have followed the same guide and using OVS version 2.3.2 . >>> >>> Ceph storage cluster is running on same KVM hypervisor and it fails >>> when adding to cloudstack. >>> >>> Regards, >>> Pradeep >>> >>> -Original Message- >>> From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] >>> Sent: 08 September 2015 11:34 >>> To: users@cloudstack.apache.org >>> Cc: Shetty, Pradeep >>> Subject: Re: Unable to add Ceph KVM node in cloudstack >>> >>> Hello Shetty, >>> >>> It seems you agent or libvirt configuration is wrong. Please, specify >>> also do you use OVS or Linux bridge? >>> Did you follow this guide? >>> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv >>> [1] >>> [1 [1]] >>> m.html >>> [1 [2]] >>> >>> Regards, >>> >>> Vadim >>> >>> On 2015-09-08 08:18, Shetty, Pradeep wrote: >>> >>> Hello Folks, >>> >>> We are working on ceph(hammer, 0.94.3) implementation on kvm >>> hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), >>> >>> --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, >>> openvswitch 2.3.2) + ceph hammer, cluster health is Ok. >>> >>> We are unable to add the KVM host(new cluster) in cloudstack in an >>> existing zone, below is the error log. >>> >>> > agent log-- libvirt: XML-RPC error : Cannot write data: Broken >>> pipe >>> 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return >>> value of 143 >>> >>> Please let us know the exact steps to add Ceph KVM host to cloudstack. >>> >>> Thanks, >>> Pradeep >>> >> >> Links: >> -- >> [1] >> >> >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html >> [2] >> [2 [2]] >> >> Links: >> -- >> [1] >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv >> [1] >> [2] >> >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html >> [2] >> > > > > Links: > -- > [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv > [2] > http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html > -- Andrija Panić
RE: Unable to add Ceph KVM node in cloudstack
For KVM hosts RBD comes "out of the box" with libvirt. For XenServer Ceph (RBD) storage type is not yet implemented. I would say it is very hypervisor-specific and does not depend much from Cloudstack. Regards, Vadim. On 2015-09-11 08:55, Shetty, Pradeep wrote: I have one question: Is ceph rbd protocol supported by default on libvirt version 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 09 September 2015 12:07 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [1 [1]] m.html [1 [2]] Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] [2 [2]] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
RE: Unable to add Ceph KVM node in cloudstack
I have one question: Is ceph rbd protocol supported by default on libvirt version 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd within cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 09 September 2015 12:07 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: > Please share steps for adding ceph RBD as primary storage using KVM > hypervisor in cloudstack. > > We are well aware of the best practices/recommendations for cloudstack. > > Regards, > Pradeep > > -Original Message- > From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] > Sent: 08 September 2015 12:14 > To: users@cloudstack.apache.org > Subject: RE: Unable to add Ceph KVM node in cloudstack > > Pradeep, > > From documentation : > Warning -- It is NOT recommended to run services on this host not > controlled by CloudStack. > > Good practice -- KVM host must be virgin-clean. > > Vadim. > > On 2015-09-08 09:14, Shetty, Pradeep wrote: > > Hello Vadim, > > We have followed the same guide and using OVS version 2.3.2 . > > Ceph storage cluster is running on same KVM hypervisor and it fails > when adding to cloudstack. > > Regards, > Pradeep > > -Original Message- > From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] > Sent: 08 September 2015 11:34 > To: users@cloudstack.apache.org > Cc: Shetty, Pradeep > Subject: Re: Unable to add Ceph KVM node in cloudstack > > Hello Shetty, > > It seems you agent or libvirt configuration is wrong. Please, specify > also do you use OVS or Linux bridge? > Did you follow this guide? > http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv > [1] > m.html > [1 [2]] > > Regards, > > Vadim > > On 2015-09-08 08:18, Shetty, Pradeep wrote: > > Hello Folks, > > We are working on ceph(hammer, 0.94.3) implementation on kvm > hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), > > --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, > openvswitch 2.3.2) + ceph hammer, cluster health is Ok. > > We are unable to add the KVM host(new cluster) in cloudstack in an > existing zone, below is the error log. > > > agent log-- libvirt: XML-RPC error : Cannot write data: Broken > pipe > 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return > value of 143 > > Please let us know the exact steps to add Ceph KVM host to cloudstack. > > Thanks, > Pradeep Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
RE: Unable to add Ceph KVM node in cloudstack
Pradeep, I have installed CS 4.2 with Ceph storage successfully around year ago. There were no problem with configuration except the fact that you can't configure RBD storage as primary storage during Zone set-up. It must be NFS. After Zone is set-up and running you may add another primary storage and there you have list of choices where RBD type is present. After RBD is added you may assign this as default storage for particular cluster assuming that each cluster has its own primary storage. PS. I have also used Ceph object gateway to connect secondary storage. It works as expected. It would be nice to know which way are you going to integrate Ceph into CS? Vadim. On 2015-09-09 08:09, Shetty, Pradeep wrote: Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [1] m.html [1 [2]] Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [2] Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv [2] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
RE: Unable to add Ceph KVM node in cloudstack
Please share steps for adding ceph RBD as primary storage using KVM hypervisor in cloudstack. We are well aware of the best practices/recommendations for cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 12:14 To: users@cloudstack.apache.org Subject: RE: Unable to add Ceph KVM node in cloudstack Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: > Hello Vadim, > > We have followed the same guide and using OVS version 2.3.2 . > > Ceph storage cluster is running on same KVM hypervisor and it fails > when adding to cloudstack. > > Regards, > Pradeep > > -Original Message- > From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] > Sent: 08 September 2015 11:34 > To: users@cloudstack.apache.org > Cc: Shetty, Pradeep > Subject: Re: Unable to add Ceph KVM node in cloudstack > > Hello Shetty, > > It seems you agent or libvirt configuration is wrong. Please, specify > also do you use OVS or Linux bridge? > Did you follow this guide? > http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kv > m.html > [1] > > Regards, > > Vadim > > On 2015-09-08 08:18, Shetty, Pradeep wrote: > >> Hello Folks, >> >> We are working on ceph(hammer, 0.94.3) implementation on kvm >> hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), >> >> --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, >> openvswitch 2.3.2) + ceph hammer, cluster health is Ok. >> >> We are unable to add the KVM host(new cluster) in cloudstack in an >> existing zone, below is the error log. >> >> > agent log-- libvirt: XML-RPC error : Cannot write data: Broken >> pipe >> 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return >> value of 143 >> >> Please let us know the exact steps to add Ceph KVM host to cloudstack. >> >> Thanks, >> Pradeep Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
RE: Unable to add Ceph KVM node in cloudstack
Pradeep, From documentation : Warning -- It is NOT recommended to run services on this host not controlled by CloudStack. Good practice -- KVM host must be virgin-clean. Vadim. On 2015-09-08 09:14, Shetty, Pradeep wrote: Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html [1] Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep Links: -- [1] http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html
RE: Unable to add Ceph KVM node in cloudstack
Hello Vadim, We have followed the same guide and using OVS version 2.3.2 . Ceph storage cluster is running on same KVM hypervisor and it fails when adding to cloudstack. Regards, Pradeep -Original Message- From: Vadim Kimlaychuk [mailto:va...@kickcloud.net] Sent: 08 September 2015 11:34 To: users@cloudstack.apache.org Cc: Shetty, Pradeep Subject: Re: Unable to add Ceph KVM node in cloudstack Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: > Hello Folks, > > We are working on ceph(hammer, 0.94.3) implementation on kvm > hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), > > --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, > openvswitch 2.3.2) + ceph hammer, cluster health is Ok. > > We are unable to add the KVM host(new cluster) in cloudstack in an > existing zone, below is the error log. > > > agent log-- libvirt: XML-RPC error : Cannot write data: Broken > pipe > 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return > value of 143 > > Please let us know the exact steps to add Ceph KVM host to cloudstack. > > Thanks, > Pradeep
Re: Unable to add Ceph KVM node in cloudstack
Hello Shetty, It seems you agent or libvirt configuration is wrong. Please, specify also do you use OVS or Linux bridge? Did you follow this guide? http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html Regards, Vadim On 2015-09-08 08:18, Shetty, Pradeep wrote: Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep
Unable to add Ceph KVM node in cloudstack
Hello Folks, We are working on ceph(hammer, 0.94.3) implementation on kvm hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, openvswitch 2.3.2) + ceph hammer, cluster health is Ok. We are unable to add the KVM host(new cluster) in cloudstack in an existing zone, below is the error log. > agent log-- libvirt: XML-RPC error : Cannot write data: Broken pipe 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return value of 143 Please let us know the exact steps to add Ceph KVM host to cloudstack. Thanks, Pradeep
