Re: [Xen-devel] Save/Restore is not working properly
I have tested it both on Xen 4.8 and Debian Stretch : Kernel version 4.6 Xen 4.6 and Debian Jessie : Kernel version 3.19 and also Xen 4.6 and 4.7 on ubuntu Kernel 3.16 On Wed, Aug 24, 2016 at 3:38 PM, Roger Pau Monné wrote: > On Fri, Aug 19, 2016 at 08:45:49PM +0430, Cendrin Sa wrote: > > Hi again, > > So save/restore has a bug or not? I still have problem with it when i use > > LVM. > > Which kernel version are you using on the Dom0? > > Roger. > ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Fri, Aug 19, 2016 at 08:45:49PM +0430, Cendrin Sa wrote: > Hi again, > So save/restore has a bug or not? I still have problem with it when i use > LVM. Which kernel version are you using on the Dom0? Roger. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Fri, Aug 19, 2016 at 08:45:49PM +0430, Cendrin Sa wrote: > Hi again, > So save/restore has a bug or not? I still have problem with it when i use > LVM. > I think we need more logs. Which task hung? Can you provide kernel backtrace? Wei. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
Hi again, So save/restore has a bug or not? I still have problem with it when i use LVM. On Sat, Aug 13, 2016 at 11:10 AM, Cendrin Sa wrote: > I used save without any option when my VM was in running state, save won't > work if I pause a VM. > > > On Sat, Aug 13, 2016 at 11:04 AM, Cendrin Sa wrote: > >> >>- I'm using Xen unstable 4.8 manually compiled on debian , I create a >>debian netinst guest using the following config file and then just use >>save/restore, after restoring a machine *kernel hangout task happens*. >> >> >>- We've test it With Xen 4.7 manually compiled on ubuntu 14.04 and >>the same thing happened. the guest VM was ubuntu 14.04 with GUI, after >>restoring we were able to move the mouse but the VM was crashed. >> >> >>- Also, the same *kernel hangout task *happened on CentOS (also its >>kernel is 2.6...) and with Xen 4.2. >> >> These is important to note that after creating VMs using a raw image file >> created with both "qemu-img" and "dd" the problem solved and save/restore >> is working properly. >> It seems there is a problem related to LVM. >> >> >>1. >>2. builder = "hvm" >>3. memory = 1024 >>4. vcpus = 2 >>5. name = "debian64" >>6. vif = [ 'bridge=xenbr0' ] >>7. disk = [ >>8. 'file:/dev/vg0/debian64_clone.img,xvda,rw', >>9. >> 'file:/home/lisbeth/src/debian-8.5.0-amd64-netinst.iso,xvdc:cdrom,r' >>10.] >>11. >>12. boot = "c" >> >> >> On Thu, Aug 11, 2016 at 7:48 PM, Wei Liu wrote: >> >>> On Wed, Aug 10, 2016 at 02:24:09PM +0100, George Dunlap wrote: >>> > On Wed, Aug 10, 2016 at 12:11 PM, Roger Pau Monné < >>> roger@citrix.com> wrote: >>> > > On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: >>> > >> Hi, >>> > >> I was searching a way to clone a machine using both memory and disk >>> > >> approach. >>> > >> I checked xen save/restore but after restoring, I can only work some >>> > >> seconds with my machine and it will crash with >>> the_kernel_task_hang_up. >>> > >> using an script* to clone a machine is not working either. >>> > >> so is it a bug or something or I'm cloning the wrong way? >>> > > >>> > > Hello, >>> > > >>> > > I've not tried to perform cloning myself, but I have a little script >>> to >>> > > perform VM checkpoints (so that you can restore the VM to any given >>> point in >>> > > time). It's based on FreeBSD so it uses ZFS, but it should work with >>> LVM >>> > > also if you replace it with the appropriate runes. AFAICT it should >>> be quite >>> > > easy to expand it to also do VM cloning. This is transparent from a >>> VM point >>> > > of view. >>> > >>> > FWIW on a recent version of Xen-unstable, "xl save -c" appears to be >>> > broken, at least with me CentOS 6 VM. If I do "xl save" then "xl >>> > restore", everything works fine; but if I do "xl save -c", then the >>> > save appears to work as normal, and after it's done the guest console >>> > has output similar to the output it has when restoring, but processes >>> > which access the disk hang, and in 2 minutes I get "hung process" >>> > output as Cendrin described. >>> > >>> > I do get some warning messages though: >>> > >>> > Using NULL legacy PIC >>> > WARNING: g.e. still in use! >>> > WARNING: leaking g.e. and page still in use! >>> > WARNING: g.e. still in use! >>> > WARNING: leaking g.e. and page still in use! >>> > WARNING: g.e. still in use! >>> > WARNING: leaking g.e. and page still in use! >>> > Changing capacity of (202, 0) to 4194288 sectors >>> > >>> > This is the stock CentOS 6.6 kernel: 2.6.32-504.16.2.el6.x86_64 >>> > >>> >>> It looks like the guest kernel is trying to free up all the grant >>> references. >>> >>> In the case of xl save -c my impression is that it shouldn't be doing >>> that because the suspend is supposed to be canceled from guest's PoV. >>> >>> See comment in xenctrl.h for xc_domain_resume. >>> >>> Also related: 8903a7a5f6a47cc40c1c204a1cc28b0030b04486 >>> >>> Wei. >>> >>> > -George >>> >> >> > ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
I used save without any option when my VM was in running state, save won't work if I pause a VM. On Sat, Aug 13, 2016 at 11:04 AM, Cendrin Sa wrote: > >- I'm using Xen unstable 4.8 manually compiled on debian , I create a >debian netinst guest using the following config file and then just use >save/restore, after restoring a machine *kernel hangout task happens*. > > >- We've test it With Xen 4.7 manually compiled on ubuntu 14.04 and >the same thing happened. the guest VM was ubuntu 14.04 with GUI, after >restoring we were able to move the mouse but the VM was crashed. > > >- Also, the same *kernel hangout task *happened on CentOS (also its >kernel is 2.6...) and with Xen 4.2. > > These is important to note that after creating VMs using a raw image file > created with both "qemu-img" and "dd" the problem solved and save/restore > is working properly. > It seems there is a problem related to LVM. > > >1. >2. builder = "hvm" >3. memory = 1024 >4. vcpus = 2 >5. name = "debian64" >6. vif = [ 'bridge=xenbr0' ] >7. disk = [ >8. 'file:/dev/vg0/debian64_clone.img,xvda,rw', >9. > 'file:/home/lisbeth/src/debian-8.5.0-amd64-netinst.iso,xvdc:cdrom,r' >10.] >11. >12. boot = "c" > > > On Thu, Aug 11, 2016 at 7:48 PM, Wei Liu wrote: > >> On Wed, Aug 10, 2016 at 02:24:09PM +0100, George Dunlap wrote: >> > On Wed, Aug 10, 2016 at 12:11 PM, Roger Pau Monné >> wrote: >> > > On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: >> > >> Hi, >> > >> I was searching a way to clone a machine using both memory and disk >> > >> approach. >> > >> I checked xen save/restore but after restoring, I can only work some >> > >> seconds with my machine and it will crash with >> the_kernel_task_hang_up. >> > >> using an script* to clone a machine is not working either. >> > >> so is it a bug or something or I'm cloning the wrong way? >> > > >> > > Hello, >> > > >> > > I've not tried to perform cloning myself, but I have a little script >> to >> > > perform VM checkpoints (so that you can restore the VM to any given >> point in >> > > time). It's based on FreeBSD so it uses ZFS, but it should work with >> LVM >> > > also if you replace it with the appropriate runes. AFAICT it should >> be quite >> > > easy to expand it to also do VM cloning. This is transparent from a >> VM point >> > > of view. >> > >> > FWIW on a recent version of Xen-unstable, "xl save -c" appears to be >> > broken, at least with me CentOS 6 VM. If I do "xl save" then "xl >> > restore", everything works fine; but if I do "xl save -c", then the >> > save appears to work as normal, and after it's done the guest console >> > has output similar to the output it has when restoring, but processes >> > which access the disk hang, and in 2 minutes I get "hung process" >> > output as Cendrin described. >> > >> > I do get some warning messages though: >> > >> > Using NULL legacy PIC >> > WARNING: g.e. still in use! >> > WARNING: leaking g.e. and page still in use! >> > WARNING: g.e. still in use! >> > WARNING: leaking g.e. and page still in use! >> > WARNING: g.e. still in use! >> > WARNING: leaking g.e. and page still in use! >> > Changing capacity of (202, 0) to 4194288 sectors >> > >> > This is the stock CentOS 6.6 kernel: 2.6.32-504.16.2.el6.x86_64 >> > >> >> It looks like the guest kernel is trying to free up all the grant >> references. >> >> In the case of xl save -c my impression is that it shouldn't be doing >> that because the suspend is supposed to be canceled from guest's PoV. >> >> See comment in xenctrl.h for xc_domain_resume. >> >> Also related: 8903a7a5f6a47cc40c1c204a1cc28b0030b04486 >> >> Wei. >> >> > -George >> > > ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
- I'm using Xen unstable 4.8 manually compiled on debian , I create a debian netinst guest using the following config file and then just use save/restore, after restoring a machine *kernel hangout task happens*. - We've test it With Xen 4.7 manually compiled on ubuntu 14.04 and the same thing happened. the guest VM was ubuntu 14.04 with GUI, after restoring we were able to move the mouse but the VM was crashed. - Also, the same *kernel hangout task *happened on CentOS (also its kernel is 2.6...) and with Xen 4.2. These is important to note that after creating VMs using a raw image file created with both "qemu-img" and "dd" the problem solved and save/restore is working properly. It seems there is a problem related to LVM. 1. 2. builder = "hvm" 3. memory = 1024 4. vcpus = 2 5. name = "debian64" 6. vif = [ 'bridge=xenbr0' ] 7. disk = [ 8. 'file:/dev/vg0/debian64_clone.img,xvda,rw', 9. 'file:/home/lisbeth/src/debian-8.5.0-amd64-netinst.iso,xvdc:cdrom,r' 10.] 11. 12. boot = "c" On Thu, Aug 11, 2016 at 7:48 PM, Wei Liu wrote: > On Wed, Aug 10, 2016 at 02:24:09PM +0100, George Dunlap wrote: > > On Wed, Aug 10, 2016 at 12:11 PM, Roger Pau Monné > wrote: > > > On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: > > >> Hi, > > >> I was searching a way to clone a machine using both memory and disk > > >> approach. > > >> I checked xen save/restore but after restoring, I can only work some > > >> seconds with my machine and it will crash with > the_kernel_task_hang_up. > > >> using an script* to clone a machine is not working either. > > >> so is it a bug or something or I'm cloning the wrong way? > > > > > > Hello, > > > > > > I've not tried to perform cloning myself, but I have a little script to > > > perform VM checkpoints (so that you can restore the VM to any given > point in > > > time). It's based on FreeBSD so it uses ZFS, but it should work with > LVM > > > also if you replace it with the appropriate runes. AFAICT it should be > quite > > > easy to expand it to also do VM cloning. This is transparent from a VM > point > > > of view. > > > > FWIW on a recent version of Xen-unstable, "xl save -c" appears to be > > broken, at least with me CentOS 6 VM. If I do "xl save" then "xl > > restore", everything works fine; but if I do "xl save -c", then the > > save appears to work as normal, and after it's done the guest console > > has output similar to the output it has when restoring, but processes > > which access the disk hang, and in 2 minutes I get "hung process" > > output as Cendrin described. > > > > I do get some warning messages though: > > > > Using NULL legacy PIC > > WARNING: g.e. still in use! > > WARNING: leaking g.e. and page still in use! > > WARNING: g.e. still in use! > > WARNING: leaking g.e. and page still in use! > > WARNING: g.e. still in use! > > WARNING: leaking g.e. and page still in use! > > Changing capacity of (202, 0) to 4194288 sectors > > > > This is the stock CentOS 6.6 kernel: 2.6.32-504.16.2.el6.x86_64 > > > > It looks like the guest kernel is trying to free up all the grant > references. > > In the case of xl save -c my impression is that it shouldn't be doing > that because the suspend is supposed to be canceled from guest's PoV. > > See comment in xenctrl.h for xc_domain_resume. > > Also related: 8903a7a5f6a47cc40c1c204a1cc28b0030b04486 > > Wei. > > > -George > ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Wed, Aug 10, 2016 at 02:24:09PM +0100, George Dunlap wrote: > On Wed, Aug 10, 2016 at 12:11 PM, Roger Pau Monné > wrote: > > On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: > >> Hi, > >> I was searching a way to clone a machine using both memory and disk > >> approach. > >> I checked xen save/restore but after restoring, I can only work some > >> seconds with my machine and it will crash with the_kernel_task_hang_up. > >> using an script* to clone a machine is not working either. > >> so is it a bug or something or I'm cloning the wrong way? > > > > Hello, > > > > I've not tried to perform cloning myself, but I have a little script to > > perform VM checkpoints (so that you can restore the VM to any given point in > > time). It's based on FreeBSD so it uses ZFS, but it should work with LVM > > also if you replace it with the appropriate runes. AFAICT it should be quite > > easy to expand it to also do VM cloning. This is transparent from a VM point > > of view. > > FWIW on a recent version of Xen-unstable, "xl save -c" appears to be > broken, at least with me CentOS 6 VM. If I do "xl save" then "xl > restore", everything works fine; but if I do "xl save -c", then the > save appears to work as normal, and after it's done the guest console > has output similar to the output it has when restoring, but processes > which access the disk hang, and in 2 minutes I get "hung process" > output as Cendrin described. > > I do get some warning messages though: > > Using NULL legacy PIC > WARNING: g.e. still in use! > WARNING: leaking g.e. and page still in use! > WARNING: g.e. still in use! > WARNING: leaking g.e. and page still in use! > WARNING: g.e. still in use! > WARNING: leaking g.e. and page still in use! > Changing capacity of (202, 0) to 4194288 sectors > > This is the stock CentOS 6.6 kernel: 2.6.32-504.16.2.el6.x86_64 > It looks like the guest kernel is trying to free up all the grant references. In the case of xl save -c my impression is that it shouldn't be doing that because the suspend is supposed to be canceled from guest's PoV. See comment in xenctrl.h for xc_domain_resume. Also related: 8903a7a5f6a47cc40c1c204a1cc28b0030b04486 Wei. > -George ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Wed, Aug 10, 2016 at 12:11 PM, Roger Pau Monné wrote: > On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: >> Hi, >> I was searching a way to clone a machine using both memory and disk >> approach. >> I checked xen save/restore but after restoring, I can only work some >> seconds with my machine and it will crash with the_kernel_task_hang_up. >> using an script* to clone a machine is not working either. >> so is it a bug or something or I'm cloning the wrong way? > > Hello, > > I've not tried to perform cloning myself, but I have a little script to > perform VM checkpoints (so that you can restore the VM to any given point in > time). It's based on FreeBSD so it uses ZFS, but it should work with LVM > also if you replace it with the appropriate runes. AFAICT it should be quite > easy to expand it to also do VM cloning. This is transparent from a VM point > of view. FWIW on a recent version of Xen-unstable, "xl save -c" appears to be broken, at least with me CentOS 6 VM. If I do "xl save" then "xl restore", everything works fine; but if I do "xl save -c", then the save appears to work as normal, and after it's done the guest console has output similar to the output it has when restoring, but processes which access the disk hang, and in 2 minutes I get "hung process" output as Cendrin described. I do get some warning messages though: Using NULL legacy PIC WARNING: g.e. still in use! WARNING: leaking g.e. and page still in use! WARNING: g.e. still in use! WARNING: leaking g.e. and page still in use! WARNING: g.e. still in use! WARNING: leaking g.e. and page still in use! Changing capacity of (202, 0) to 4194288 sectors This is the stock CentOS 6.6 kernel: 2.6.32-504.16.2.el6.x86_64 -George ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Sun, Aug 07, 2016 at 07:51:14PM +0430, Cendrin Sa wrote: > Hi, > I was searching a way to clone a machine using both memory and disk > approach. > I checked xen save/restore but after restoring, I can only work some > seconds with my machine and it will crash with the_kernel_task_hang_up. > using an script* to clone a machine is not working either. > so is it a bug or something or I'm cloning the wrong way? Hello, I've not tried to perform cloning myself, but I have a little script to perform VM checkpoints (so that you can restore the VM to any given point in time). It's based on FreeBSD so it uses ZFS, but it should work with LVM also if you replace it with the appropriate runes. AFAICT it should be quite easy to expand it to also do VM cloning. This is transparent from a VM point of view. Roger. --- #!/bin/sh -e if [ "$#" -ne 1 ]; then echo "Illegal number of parameters" exit fi stamp=`date +%y-%m-%d-%H:%M.%S` xl save $1 $1.$stamp zfs snapshot tank/$1@$stamp xl restore $1.$stamp ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
On Sun, Aug 7, 2016 at 4:21 PM, Cendrin Sa wrote: > Hi, > I was searching a way to clone a machine using both memory and disk > approach. > I checked xen save/restore but after restoring, I can only work some seconds > with my machine and it will crash with the_kernel_task_hang_up. > using an script* to clone a machine is not working either. > so is it a bug or something or I'm cloning the wrong way? Thanks for the bug report -- but you're missing a lot of important information to help us figure out what's going wrong. You need to let us know what version of Xen you're running, attach all the relevant config files, and include the relevant console output. So to begin with -- just to understand -- if you do a simple save / restore, does that work? Also, if you just do a save (without pausing), and then restore with the alternate VM config, does that work? Thanks, -George > > > > * > #!/bin/bash > > : ${2? "Usage: $0 "} > echo "$1 is source domain id"; > echo "$2 is the destination file to store the image"; > xl save -p $1 $2 ; > lvremove -f /dev/vg0/vm-clone-lv; > lvcreate -L10G -s -n vm-clone-lv /dev/vg0/orig-vm ; > xl restore -e -p /etc/xen/vm-clone.cfg $2; > xl unpause $1; > > -- > Regards > > > ___ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel > ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] Save/Restore is not working properly
Hi, I was searching a way to clone a machine using both memory and disk approach. I checked xen save/restore but after restoring, I can only work some seconds with my machine and it will crash with the_kernel_task_hang_up. using an script* to clone a machine is not working either. so is it a bug or something or I'm cloning the wrong way? * #!/bin/bash : ${2? "Usage: $0 "} echo "$1 is source domain id"; echo "$2 is the destination file to store the image"; xl save -p $1 $2 ; lvremove -f /dev/vg0/vm-clone-lv; lvcreate -L10G -s -n vm-clone-lv /dev/vg0/orig-vm ; xl restore -e -p /etc/xen/vm-clone.cfg $2; xl unpause $1; -- Regards ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel