Sorry, I kind of lost track of what the problem is The "KeyError: 'appsList'" issue is a known bug[1]
If a manual (not via vdsm) run of qemu-img is actually stuck, then let's involve the qemu-discuss list, with the version of the relevant packages (qemu, qemu-img, kernel, your distro) and the output of gdb commands [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1690301 On Sat, Apr 20, 2019 at 1:36 AM Callum Smith <cal...@well.ox.ac.uk> wrote: > > Dear Benny and others, > > So it seems I wasn’t being patient with GDB and it does show me some output. > This error of qemu-img convert even is failing and preventing updating > ovirt-node version from 4.3.2 to 4.3.3.1. I get a feeling this is an > unrelated error, but I thought I’d be complete: > > Excuse any typos, im having to type this manually from a remote session, but > the error: > > [733272.427922] hid-generic 0003:0624:0249.0001: usb_submit_urb(ctrl) failed: > -19 > > If this bug is preventing even a local yum updatei can’t see how it’s any > issue other than somehow involved with the hardware of the hypervisor, our > network and storage configuration must be irrelevant to this fact at this > stage? > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 11 Apr 2019, at 12:00, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Without the sudo and running in a dir where the root has access to, gdb has > zero output: > > <Screenshot 2019-04-11 at 12.00.27.png> > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 11 Apr 2019, at 11:54, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Some more information: > > running qemu-img convert manually having captured the failed attempt from the > previous: > > sudo -u vdsm /usr/bin/qemu-img convert -p -t none -T none -f raw > /rhev/data-center/mnt/10.141.15.248:_export_instruct_vm__storage/0e01f014-530b-4067-aa1d-4e9378626a9d/images/6597eede-9fa0-4451-84fc-9f9c070cb5f3/765fa48b-2e77-4637-b4ca-e1affcd71e48 > -O raw > /rhev/data-center/mnt/10.141.15.248:_export_instruct_vm__storage/0e01f014-530b-4067-aa1d-4e9378626a9d/images/9cc99110-70a2-477f-b3ef-1031a912d12b/c2776107-4579-43a6-9d60-93a5ea9c64c5 > -W > > Added the -W flag just to see what would happen: > > gdb -p 79913 -batch -ex "t a a bt" > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > 0x00007f528d7661f0 in __poll_nocancel () from /lib64/libc.so.6 > > Thread 1 (Thread 0x7f528e6bb840 (LWP 79913)): > #0 0x00007f528d7661f0 in __poll_nocancel () from /lib64/libc.so.6 > #1 0x00007f528dc510fb in sudo_ev_scan_impl () from > /usr/libexec/sudo/libsudo_util.so.0 > #2 0x00007f528dc49b44 in sudo_ev_loop_v1 () from > /usr/libexec/sudo/libsudo_util.so.0 > #3 0x000055e94aa0e271 in exec_nopty () > #4 0x000055e94aa0afda in sudo_execute () > #5 0x000055e94aa18a12 in run_command () > #6 0x000055e94aa0969e in main () > > > And without -W: > > gdb -p 85235 -batch -ex "t a a bt" > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > 0x00007fc0cc69b1f0 in __poll_nocancel () from /lib64/libc.so.6 > > Thread 1 (Thread 0x7fc0cd5f0840 (LWP 85235)): > #0 0x00007fc0cc69b1f0 in __poll_nocancel () from /lib64/libc.so.6 > #1 0x00007fc0ccb860fb in sudo_ev_scan_impl () from > /usr/libexec/sudo/libsudo_util.so.0 > #2 0x00007fc0ccb7eb44 in sudo_ev_loop_v1 () from > /usr/libexec/sudo/libsudo_util.so.0 > #3 0x00005610f4397271 in exec_nopty () > #4 0x00005610f4393fda in sudo_execute () > #5 0x00005610f43a1a12 in run_command () > #6 0x00005610f439269e in main () > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 11 Apr 2019, at 09:57, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Dear Benny, > > It would seem that even cloning a VM is failing, creating a VM works on the > same storage. This is the only error i could find: > > ERROR Internal server error > Traceback (most > recent call last): > File > "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in > _handle_request > res = > method(**params) > File > "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 194, in > _dynamicMethod > result = > fn(*methodArgs) > File > "<string>", line 2, in getAllVmStats > File > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method > ret = > func(*args, **kwargs) > File > "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1388, in getAllVmStats > statsList = > self._cif.getAllVmStats() > File > "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 567, in > getAllVmStats > return > [v.getStats() for v in self.vmContainer.values()] > File > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1766, in getStats > oga_stats = > self._getGuestStats() > File > "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1967, in > _getGuestStats > stats = > self.guestAgent.getGuestInfo() > File > "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 505, in > getGuestInfo > del > qga['appsList'] > KeyError: > 'appsList' > > > It's the qemu-img convert for sure that's just failing to do anything, this > is the command from the clone: > /usr/bin/qemu-img convert -p -t none -T none -f raw > /rhev/data-center/mnt/10.141.15.248:_export_instruct_vm__storage/0e01f014-530b-4067-aa1d-4e9378626a9d/images/6597eede-9fa0-4451-84fc-9f9c070cb5f3/765fa48b-2e77-4637-b4ca-e1affcd71e48 > -O raw > /rhev/data-center/mnt/10.141.15.248:_export_instruct_vm__storage/0e01f014-530b-4067-aa1d-4e9378626a9d/images/f0700631-e60b-4c2a-a6f5-a6c818ae7651/d4fb05ec-7c78-4d89-9a66-614c093c6e16 > > gdb has a blank output for this though. This means 4.3.2 is fairly unusable > for us, so two questions, can I downgrade to 4.2, and is there a fix coming > in 4.3.3 for this? > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 10 Apr 2019, at 11:22, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Creating a disk on the target share works fine. - This seems to specifically > be an issue to do with moving a disk to/from a share. > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 10 Apr 2019, at 09:53, Callum Smith <cal...@well.ox.ac.uk> wrote: > > gdb -p $(pidof qemu-img convert) -batch -ex "t a a bt" > 289444: No such file or directory. > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 10 Apr 2019, at 09:36, Benny Zlotnik <bzlot...@redhat.com> wrote: > > Can you run: > $ gdb -p $(pidof qemu-img convert) -batch -ex "t a a bt" > > > On Wed, Apr 10, 2019 at 11:26 AM Callum Smith <cal...@well.ox.ac.uk> wrote: > > > Dear All, > > Further to this, I can't migrate a disk to different storage using the GUI. > Both disks are configured identically and on the same physical NFS provider. > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 9 Apr 2019, at 12:12, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Dear All, > > It would seem this is a bug in 4.3.? - As upgrading the old oVirt HE to 4.3 > (from 4.2.latest) now means that the export of VMs to export domain no longer > works. > > Again qemu-img convert is using some cpu, but no network. Progress is 0. > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > On 8 Apr 2019, at 15:42, Callum Smith <cal...@well.ox.ac.uk> wrote: > > Dear All, > > We've exported some VMs from our old oVirt infrastructure and want to import > them into the new one, but qemu-img appears to be failing. We have mounted an > export domain populated from the old oVirt in the new hosted engine and are > using the GUI to import the VM. Manually running the command sits at 16% CPU, > 0% network usage and no progress. It appears to lock the NFS mount and ls and > lsof both hang. > > sudo -u vdsm /usr/bin/qemu-img convert -p -t none -T none -f raw <path to > source> -O raw <path to dest> > > Conversely a simple cp will work (ruling out file permissions errors): > sudo -u vddm cp <path to source> <path to test> > > What might we be doing wrong? > > Regards, > Callum > > -- > > Callum Smith > Research Computing Core > Wellcome Trust Centre for Human Genetics > University of Oxford > e. cal...@well.ox.ac.uk > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZUBKWQIGGSJETPVRWN42R4J7COPFV6GS/ > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/6EE6X43GTAJ6L4QBH2XQJ4LVPIXCZC3T/ > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/GD2XCKEIPNCXXQWMR4OP5IKGCORJQEGB/ > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/SV3V34DQJZ7A3MH4CQMMIZ4GPPHWFB7E/ > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/MICXCNEDJL5NOFUNS7R7W5CYDNEFINVK/ > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/KRV6NPWUCOAMRXOV5S7PIHBMJ3ZOCAAD/ > > > > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/R53H5T4ZHFGTSWPJAOORMTAZYD355SBF/ > > _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NAXU4YW5E2HHR44NSTJNHNVRTW5SN2L3/
