Re: [ceph-users] Live Migrations with cephFS
Thank you, Sebastien Han. I am sure many are thankful you've published your thoughts and experiences with Ceph and even OpemStack.Thanks Bo! :)If I may, I would like to reword my question/statement with greater clarity: To force all instances to always boot from RBD volumes, would a person would have to make changes to something more than Horizon (demonstration GUI)? If the changes need only be in Horizon, the provider would then likely need to restrict or deny their customers access to their unmodified APIs. If they do not, then the unchanged APIs would allow for behavior the provider does not want.Thoughts? Corrections? Feel free to teach.This is correct. Forcing the boot from volume requires a modified version of the API which kinda tricky and GUI modifications. There are 2 cases:1. you're an ISP (public provider), you should forget about the idea unless you want to provide a _really_ close service.2. you're the only one managing your platform (private cloud) this might be doable but even so you'll encounter a lot of problems while upgrading. At the end it's up to you, if you're 100% sure that you have the complete control of your infra and that you know when, who and how new instances are booted (and occasionally don't care about update and compatibility). You can always hack the dashboard but it's more than that you have to automate the action that each time someone is booting a VM you have to create a volume from an image for this. This will prolong the process. At this point, I'll recommend you to push this blueprint, it'll run all the VM through ceph even the one not using the boot-from-volume option. https://blueprints.launchpad.net/nova/+spec/bring-rbd-support-libvirt-images-typeAn article is coming next week and will cover the entire subject.Cheers!Sébastien HanCloud Engineer"Always give 100%. Unless you're giving blood."Phone : +33 (0)1 49 70 99 72 – Mobile : +33 (0)6 52 84 44 70Email : sebastien@enovance.com – Skype : han.sbastienAddress : 10, rue de la Victoire – 75009 ParisWeb : www.enovance.com – Twitter : @enovance On Jun 17, 2013, at 8:00 AM, Wolfgang Hennerbichlerwrote:On 06/14/2013 08:00 PM, Ilja Maslov wrote:Hi,Is live migration supported with RBD and KVM/OpenStack?Always wanted to know but was afraid to ask :)totally works in my productive setup. but we don't use openstack in thisinstallation, just KVM/RBD.Pardon brevity and formatting, replying from the phone.Cheers,IljaRobert Sander wrote:On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote:By default, openstack uses NFS but… other options are available….can weuse cephFS instead of NFS?Wouldn't you use qemu-rbd for your virtual guests in OpenStack?AFAIK CephFS is not needed for KVM/qemu virtual machines.Regards--Robert SanderHeinlein Support GmbHSchwedter Str. 8/9b, 10119 Berlinhttp://www.heinlein-support.deTel: 030 / 405051-43Fax: 030 / 405051-19Zwangsangaben lt. §35a GmbHG:HRB 93818 B / Amtsgericht Berlin-Charlottenburg,Geschäftsführer: Peer Heinlein -- Sitz: Berlin___ceph-users mailing listceph-users@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.comThis email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please note that any review, dissemination, disclosure, alteration, printing, circulation, retention or transmission of this e-mail and/or any file or attachment transmitted with it, is prohibited and may be unlawful. If you have received this e-mail or any file or attachment transmitted with it in error please notify postmas...@openet.com. Although Openet has taken reasonable precautions to ensure no viruses are present in this email, we cannot accept responsibility for any loss or damage arising from the use of this email or attachments.___ceph-users mailing listceph-users@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com-- DI (FH) Wolfgang HennerbichlerSoftware DevelopmentUnit Advanced Computing TechnologiesRISC Software GmbHA company of the Johannes Kepler University LinzIT-CenterSoftwarepark 354232 HagenbergAustriaPhone: +43 7236 3343 245Fax: +43 7236 3343 250wolfgang.hennerbich...@risc-software.athttp://www.risc-software.at___ceph-users mailing listceph-users@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
OpenStack grizzly VM can be started on rbd(0.61.3) with no problem. I didn't try live migration though. -- Original -- From: "Wolfgang Hennerbichler"; Date: Mon, Jun 17, 2013 02:00 PM To: "ceph-users"; Subject: Re: [ceph-users] Live Migrations with cephFS On 06/14/2013 08:00 PM, Ilja Maslov wrote: > Hi, > > Is live migration supported with RBD and KVM/OpenStack? > Always wanted to know but was afraid to ask :) totally works in my productive setup. but we don't use openstack in this installation, just KVM/RBD. > Pardon brevity and formatting, replying from the phone. > > Cheers, > Ilja > > > Robert Sander wrote: > > > On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > >> By default, openstack uses NFS but… other options are available….can we >> use cephFS instead of NFS? > > Wouldn't you use qemu-rbd for your virtual guests in OpenStack? > AFAIK CephFS is not needed for KVM/qemu virtual machines. > > Regards > -- > Robert Sander > Heinlein Support GmbH > Schwedter Str. 8/9b, 10119 Berlin > > http://www.heinlein-support.de > > Tel: 030 / 405051-43 > Fax: 030 / 405051-19 > > Zwangsangaben lt. §35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Geschäftsführer: Peer Heinlein -- Sitz: Berlin > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. If > you are not the intended recipient, please note that any review, > dissemination, disclosure, alteration, printing, circulation, retention or > transmission of this e-mail and/or any file or attachment transmitted with > it, is prohibited and may be unlawful. If you have received this e-mail or > any file or attachment transmitted with it in error please notify > postmas...@openet.com. Although Openet has taken reasonable precautions to > ensure no viruses are present in this email, we cannot accept responsibility > for any loss or damage arising from the use of this email or attachments. > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- DI (FH) Wolfgang Hennerbichler Software Development Unit Advanced Computing Technologies RISC Software GmbH A company of the Johannes Kepler University Linz IT-Center Softwarepark 35 4232 Hagenberg Austria Phone: +43 7236 3343 245 Fax: +43 7236 3343 250 wolfgang.hennerbich...@risc-software.at http://www.risc-software.at ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
On 06/14/2013 08:00 PM, Ilja Maslov wrote: > Hi, > > Is live migration supported with RBD and KVM/OpenStack? > Always wanted to know but was afraid to ask :) totally works in my productive setup. but we don't use openstack in this installation, just KVM/RBD. > Pardon brevity and formatting, replying from the phone. > > Cheers, > Ilja > > > Robert Sander wrote: > > > On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > >> By default, openstack uses NFS but… other options are available….can we >> use cephFS instead of NFS? > > Wouldn't you use qemu-rbd for your virtual guests in OpenStack? > AFAIK CephFS is not needed for KVM/qemu virtual machines. > > Regards > -- > Robert Sander > Heinlein Support GmbH > Schwedter Str. 8/9b, 10119 Berlin > > http://www.heinlein-support.de > > Tel: 030 / 405051-43 > Fax: 030 / 405051-19 > > Zwangsangaben lt. §35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Geschäftsführer: Peer Heinlein -- Sitz: Berlin > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. If > you are not the intended recipient, please note that any review, > dissemination, disclosure, alteration, printing, circulation, retention or > transmission of this e-mail and/or any file or attachment transmitted with > it, is prohibited and may be unlawful. If you have received this e-mail or > any file or attachment transmitted with it in error please notify > postmas...@openet.com. Although Openet has taken reasonable precautions to > ensure no viruses are present in this email, we cannot accept responsibility > for any loss or damage arising from the use of this email or attachments. > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- DI (FH) Wolfgang Hennerbichler Software Development Unit Advanced Computing Technologies RISC Software GmbH A company of the Johannes Kepler University Linz IT-Center Softwarepark 35 4232 Hagenberg Austria Phone: +43 7236 3343 245 Fax: +43 7236 3343 250 wolfgang.hennerbich...@risc-software.at http://www.risc-software.at ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
Thank you, Sebastien Han. I am sure many are thankful you've published your thoughts and experiences with Ceph and even OpemStack. If I may, I would like to reword my question/statement with greater clarity: To force all instances to always boot from RBD volumes, would a person would have to make changes to something more than Horizon (demonstration GUI)? If the changes need only be in Horizon, the provider would then likely need to restrict or deny their customers access to their unmodified APIs. If they do not, then the unchanged APIs would allow for behavior the provider does not want. Thoughts? Corrections? Feel free to teach. -bo On Jun 16, 2013 9:44 AM, "Sebastien Han" wrote: > In OpenStack, a VM booted from a volume (where the disk is located on RBD) > supports the live-migration without any problems. > > > Sébastien Han > Cloud Engineer > > "Always give 100%. Unless you're giving blood." > > > > > > > > > > *Phone : *+33 (0)1 49 70 99 72 – *Mobile : *+33 (0)6 52 84 44 70 > *Email :* sebastien@enovance.com – *Skype : *han.sbastien > *Address :* 10, rue de la Victoire – 75009 Paris > *Web : *www.enovance.com – *Twitter : *@enovance > > On Jun 14, 2013, at 11:36 PM, Bo wrote: > > If I am not mistaken, one would need to modify OpenStack source to force > Nova to boot from RBD volumes. Is this no longer the case? > > Modifying OpenStack's source is a wonderful idea especially if you push > your changes upstream for review. However, it does add to your work when > you want to pull updated code from upstream into your deployment. > > -bo > > > > > On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > > > By default, openstack uses NFS but… other options are available….can we > > use cephFS instead of NFS? > > Wouldn't you use qemu-rbd for your virtual guests in OpenStack? > AFAIK CephFS is not needed for KVM/qemu virtual machines. > > Regards > -- > Robert Sander > Heinlein Support GmbH > Schwedter Str. 8/9b, 10119 Berlin > > http://www.heinlein-support.de > > Tel: 030 / 405051-43 > Fax: 030 / 405051-19 > > Zwangsangaben lt. §35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Geschäftsführer: Peer Heinlein -- Sitz: Berlin > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -- > > "But God demonstrates His own love toward us, in that while we were yet > sinners, Christ died for us. Much more then, having now been justified by > His blood, we shall be saved from the wrath of God through Him." Romans > 5:8-9 > All have sinned, broken God's law, and deserve eternal torment. Jesus > Christ, the Son of God, died for the sins of those that will believe, > purchasing our salvation, and defeated death so that we all may spend > eternity in heaven. Do you desire freedom from hell and be with God in His > love for eternity? > "If you confess with your mouth Jesus as Lord, and believe in your heart > that God raised Him from the dead, you will be saved." Romans 10:9 > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > <>___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
In OpenStack, a VM booted from a volume (where the disk is located on RBD) supports the live-migration without any problems.Sébastien HanCloud Engineer"Always give 100%. Unless you're giving blood."Phone : +33 (0)1 49 70 99 72 – Mobile : +33 (0)6 52 84 44 70Email : sebastien@enovance.com – Skype : han.sbastienAddress : 10, rue de la Victoire – 75009 ParisWeb : www.enovance.com – Twitter : @enovance On Jun 14, 2013, at 11:36 PM, Bowrote:If I am not mistaken, one would need to modify OpenStack source to force Nova to boot from RBD volumes. Is this no longer the case?Modifying OpenStack's source is a wonderful idea especially if you push your changes upstream for review. However, it does add to your work when you want to pull updated code from upstream into your deployment.-boOn 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote:> By default, openstack uses NFS but… other options are available….can we> use cephFS instead of NFS?Wouldn't you use qemu-rbd for your virtual guests in OpenStack?AFAIK CephFS is not needed for KVM/qemu virtual machines.Regards-- Robert SanderHeinlein Support GmbHSchwedter Str. 8/9b, 10119 Berlinhttp://www.heinlein-support.deTel: 030 / 405051-43Fax: 030 / 405051-19Zwangsangaben lt. §35a GmbHG:HRB 93818 B / Amtsgericht Berlin-Charlottenburg,Geschäftsführer: Peer Heinlein -- Sitz: Berlin___ceph-users mailing listceph-users@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com-- "But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Much more then, having now been justified by His blood, we shall be saved from the wrath of God through Him." Romans 5:8-9All have sinned, broken God's law, and deserve eternal torment. Jesus Christ, the Son of God, died for the sins of those that will believe, purchasing our salvation, and defeated death so that we all may spend eternity in heaven. Do you desire freedom from hell and be with God in His love for eternity?"If you confess with your mouth Jesus as Lord, and believe in your heart that God raised Him from the dead, you will be saved." Romans 10:9___ceph-users mailing listceph-users@lists.ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
If I am not mistaken, one would need to modify OpenStack source to force Nova to boot from RBD volumes. Is this no longer the case? Modifying OpenStack's source is a wonderful idea especially if you push your changes upstream for review. However, it does add to your work when you want to pull updated code from upstream into your deployment. -bo On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > By default, openstack uses NFS but… other options are available….can we > use cephFS instead of NFS? Wouldn't you use qemu-rbd for your virtual guests in OpenStack? AFAIK CephFS is not needed for KVM/qemu virtual machines. Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- "But God demonstrates His own love toward us, in that while we were yet sinners, Christ died for us. Much more then, having now been justified by His blood, we shall be saved from the wrath of God through Him." Romans 5:8-9 *All *have sinned, broken God's law, and deserve eternal torment. Jesus Christ, the Son of God, died for the sins of those that will believe, purchasing our salvation, and defeated death so that we all may spend eternity in heaven. Do you desire freedom from hell and be with God in His love for eternity? "If you confess with your mouth Jesus as Lord, and believe in your heart that God raised Him from the dead, you will be saved." Romans 10:9 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
Hi, Is live migration supported with RBD and KVM/OpenStack? Always wanted to know but was afraid to ask :) Pardon brevity and formatting, replying from the phone. Cheers, Ilja Robert Sander wrote: On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > By default, openstack uses NFS but… other options are available….can we > use cephFS instead of NFS? Wouldn't you use qemu-rbd for your virtual guests in OpenStack? AFAIK CephFS is not needed for KVM/qemu virtual machines. Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please note that any review, dissemination, disclosure, alteration, printing, circulation, retention or transmission of this e-mail and/or any file or attachment transmitted with it, is prohibited and may be unlawful. If you have received this e-mail or any file or attachment transmitted with it in error please notify postmas...@openet.com. Although Openet has taken reasonable precautions to ensure no viruses are present in this email, we cannot accept responsibility for any loss or damage arising from the use of this email or attachments. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Live Migrations with cephFS
On 14.06.2013 12:55, Alvaro Izquierdo Jimeno wrote: > By default, openstack uses NFS but… other options are available….can we > use cephFS instead of NFS? Wouldn't you use qemu-rbd for your virtual guests in OpenStack? AFAIK CephFS is not needed for KVM/qemu virtual machines. Regards -- Robert Sander Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com