Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
On 06/13/2013 02:14 PM, Stefan Hajnoczi wrote: On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all 5. It started the copying the storage from souce to destination (conitinously monitored it was growing) 6. Guest on the destination was paused and was running on the source 7. At some point the VM on the source shutdown and got an error on the vnc display as Viewport:write: Broken pipe (32) and the VM on the destination was undefined. Below is the libvirt debug log, please let me with your comments. Debug log: -- What about /var/log/libvirt/qemu/rhel64-64.log? That is the QEMU command-line and stderr log. I have attached all source and destination logs, including the sosreport of both source and destination in the bug. https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1192499 Also can you try without copy-storage-all just to see if migration completes successfully? The guest will act weird once it migrates since the disk is zeroed but it will isolate the failure to --copy-storage-all. Without copy-storage-all (meaning with NFS shared storage the migration works fine). Stefan Please let me know if you need more info. Thanks, Shastri
Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
On 06/14/2013 03:15 AM, Paolo Bonzini wrote: Il 13/06/2013 13:26, chandrashekar shastri ha scritto: On 06/13/2013 02:14 PM, Stefan Hajnoczi wrote: On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 Please try with libvirt 1.0.5.2. I tried with libvirt 1.0.6 and it is still failing, hence reported the bug in the launchpad. Bug #1192499 : virsh migration copy-storage-all fails with Unable to read from monitor: Connection reset by peer virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all I recently found a bug here related to IPv4/IPv6. I need to understand if it is in QEMU or libvirt. Paolo Chandrashekar
Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
On 06/14/2013 03:15 AM, Paolo Bonzini wrote: Il 13/06/2013 13:26, chandrashekar shastri ha scritto: On 06/13/2013 02:14 PM, Stefan Hajnoczi wrote: On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 Please try with libvirt 1.0.5.2. I pulled the latest libvirt from git to test this and some other issues as suggested by commuinty. But I am stuck and not able to make any progress because libvirt compliation is failing with below error: Sorry for Laszlo, Stefan, Martin, Paolo and others for not following the things what they have suggessted. Below is the error what I am getting when I tried to compile the libvirt: ### You may need to use the following Makefile variables when linking. Use them in program_LDADD when linking a program, or in library_a_LDFLAGS or library_la_LDFLAGS when linking a library. $(GETADDRINFO_LIB) $(GETHOSTNAME_LIB) $(HOSTENT_LIB) $(INET_NTOP_LIB) $(INET_PTON_LIB) $(LDEXP_LIBM) $(LIBSOCKET) $(LIB_CLOCK_GETTIME) $(LIB_EXECINFO) $(LIB_FDATASYNC) $(LIB_POLL) $(LIB_PTHREAD) $(LIB_PTHREAD_SIGMASK) $(LIB_SELECT) $(LTLIBINTL) when linking with libtool, $(LIBINTL) otherwise $(LTLIBTHREAD) when linking with libtool, $(LIBTHREAD) otherwise $(PTY_LIB) $(SERVENT_LIB) Don't forget to - include gnulib.mk from within gnulib/lib/Makefile.am, - include gnulib.mk from within gnulib/tests/Makefile.am, - mention -I gnulib/m4 in ACLOCAL_AMFLAGS in Makefile.am, - mention gnulib/m4/gnulib-cache.m4 in EXTRA_DIST in Makefile.am, - invoke gl_EARLY in ./configure.ac, right after AC_PROG_CC, - invoke gl_INIT in ./configure.ac. running: AUTOPOINT=true LIBTOOLIZE=true autoreconf --verbose --install --force -I gnulib/m4 --no-recursive autoreconf: Entering directory `.' autoreconf: running: true --force autoreconf: running: aclocal -I m4 -I gnulib/m4 --force -I m4 -I gnulib/m4 autoreconf: configure.ac: tracing autoreconf: running: true --copy --force autoreconf: running: /usr/bin/autoconf --include=gnulib/m4 --force autoreconf: running: /usr/bin/autoheader --include=gnulib/m4 --force autoreconf: running: automake --add-missing --copy --force-missing configure.ac:2424: error: required file 'gnulib/lib/Makefile.in' not found configure.ac:2424: error: required file 'gnulib/tests/Makefile.in' not found autoreconf: automake failed with exit status: 1 ### I didn't face any issue with libvirt 1.0.5. Please let me know if I am missing anything here or it is really a bug with gnulib. Thanks, Chandrashekar virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all I recently found a bug here related to IPv4/IPv6. I need to understand if it is in QEMU or libvirt. Paolo
Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all 5. It started the copying the storage from souce to destination (conitinously monitored it was growing) 6. Guest on the destination was paused and was running on the source 7. At some point the VM on the source shutdown and got an error on the vnc display as Viewport:write: Broken pipe (32) and the VM on the destination was undefined. Below is the libvirt debug log, please let me with your comments. Debug log: -- What about /var/log/libvirt/qemu/rhel64-64.log? That is the QEMU command-line and stderr log. Also can you try without copy-storage-all just to see if migration completes successfully? The guest will act weird once it migrates since the disk is zeroed but it will isolate the failure to --copy-storage-all. Stefan
Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
On 06/13/2013 02:14 PM, Stefan Hajnoczi wrote: On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all 5. It started the copying the storage from souce to destination (conitinously monitored it was growing) 6. Guest on the destination was paused and was running on the source 7. At some point the VM on the source shutdown and got an error on the vnc display as Viewport:write: Broken pipe (32) and the VM on the destination was undefined. Below is the libvirt debug log, please let me with your comments. Debug log: -- What about /var/log/libvirt/qemu/rhel64-64.log? That is the QEMU command-line and stderr log. Also can you try without copy-storage-all just to see if migration completes successfully? The guest will act weird once it migrates since the disk is zeroed but it will isolate the failure to --copy-storage-all. I have scheduled live migration with shared storage (nfs), looks like that is only not working properly. I have turned on the verbose it goes to 99% and sometimes even reaches 100% again comes back 96% very inconsistent. I will update the result by tomorrow. Thanks, Chandrashekar Stefan
Re: [Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
Il 13/06/2013 13:26, chandrashekar shastri ha scritto: On 06/13/2013 02:14 PM, Stefan Hajnoczi wrote: On Thu, Jun 13, 2013 at 10:31:04AM +0530, chandrashekar shastri wrote: We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 Please try with libvirt 1.0.5.2. virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all I recently found a bug here related to IPv4/IPv6. I need to understand if it is in QEMU or libvirt. Paolo
[Qemu-devel] virsh live migration w/o shared storage fails with error as vm is not running
Hi All, We are testing the upstream KVM with : Kernel, Qemu, Libvirt, Virt-Manager is built from the source (git). kernel version : 3.9.0+ qemu version : QEMU emulator version 1.5.0 libvirt version : 1.0.5 virt-install : 0.600.3 I have followed the below steps to test the Live migration w/o shared storage feature : 1. Created the qemu-img create -f qcow2 vm.qcow2 12G on the destination host which is same as the source. 2. Started the guest on the source 3. Started the vncdisplay to monitor the guest 4. Initiated the migration virsh migrate --live rhel64-64 qemu+ssh://9.126.89.202/system --verbose --copy-storage-all 5. It started the copying the storage from souce to destination (conitinously monitored it was growing) 6. Guest on the destination was paused and was running on the source 7. At some point the VM on the source shutdown and got an error on the vnc display as Viewport:write: Broken pipe (32) and the VM on the destination was undefined. Below is the libvirt debug log, please let me with your comments. Debug log: -- When the copy operation started: 2013-06-12 14:49:43.640+: 1696: info : libvirt version: 1.0.5 2013-06-12 14:49:43.640+: 1696: debug : virGlobalInit:439 : register drivers 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5cd6a0 name=Test 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering Test as driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNetworkDriver:616 : registering Test as network driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterInterfaceDriver:643 : registering Test as interface driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterStorageDriver:670 : registering Test as storage driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNodeDeviceDriver:697 : registering Test as device driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterSecretDriver:724 : registering Test as secret driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNWFilterDriver:751 : registering Test as network filter driver 0 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5ced60 name=OPENVZ 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering OPENVZ as driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5cf340 name=VMWARE 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering VMWARE as driver 2 2013-06-12 14:49:43.640+: 1696: debug : vboxRegister:131 : VBoxCGlueInit failed, using dummy driver 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5cf920 name=VBOX 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering VBOX as driver 3 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNetworkDriver:616 : registering VBOX as network driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterStorageDriver:670 : registering VBOX as storage driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5d3080 name=ESX 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering ESX as driver 4 2013-06-12 14:49:43.640+: 1696: debug : virRegisterInterfaceDriver:643 : registering ESX as interface driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNetworkDriver:616 : registering ESX as network driver 2 2013-06-12 14:49:43.640+: 1696: debug : virRegisterStorageDriver:670 : registering ESX as storage driver 2 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNodeDeviceDriver:697 : registering ESX as device driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterSecretDriver:724 : registering ESX as secret driver 1 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNWFilterDriver:751 : registering ESX as network filter driver 1 2013-06-12 14:49:43.640+: 1696: debug : parallelsRegister:2448 : Can't find prlctl command in the PATH env 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:769 : driver=0x7f2a6a5ce0a0 name=remote 2013-06-12 14:49:43.640+: 1696: debug : virRegisterDriver:781 : registering remote as driver 5 2013-06-12 14:49:43.640+: 1696: debug : virRegisterNetworkDriver:616 : registering remote as network driver 3 2013-06-12 14:49:43.640+: 1696: debug : virRegisterInterfaceDriver:643 : registering remote as interface driver 2 2013-06-12 14:49:43.641+: 1696: debug : virRegisterStorageDriver:670 : registering remote as storage driver 3 2013-06-12 14:49:43.641+: 1696: debug : virRegisterNodeDeviceDriver:697 : registering remote as device driver 2 2013-06-12 14:49:43.641+: 1696: debug : virRegisterSecretDriver:724 : registering remote as secret driver 2 2013-06-12 14:49:43.641+: 1696: debug : virRegisterNWFilterDriver:751 : registering remote as network filter driver 2 2013-06-12
