vdsmd 4.4.0 throws an exception in asyncore.py while updating OVF data. First
exception occured on December 14th and happens every hour since then.
An hour before the first exception, we updated engine's and host's rpms
packages. We did not run engine-backup so we cannot restore oVirt database if
it got corrupted.
Currently, both engine (RHEL 7.7) and host (RHEL 8.1) have the most recent
packages (and were rebooted). Neither the updates, nor reboots resolve the
issue.
In general, oVirt is working. We can run virtual machines with disks in the
storage domain. We can create new virtual machines. But some functionality is
not available, for example, we cannot move disks between domains.
The storage domain in question is on NFS. After the issue appeared, we
successfully created Glusterfs domain but its OVF update also failed.
Interesting that it was possible to move disks to it before OVF failure.
Here is the vdsmd traceback for your convinience
2019-12-17 16:36:58,393-0600 ERROR (Reactor thread) [vds.dispatcher] uncaptured
python exception, closing channel
<yajsonrpc.betterAsyncore.Dispatcher connected ('::ffff:172.20.1.142', 38002,
0, 0) at 0x7fbda865ed30> (
<class 'TypeError'>:object of type 'NoneType' has no len()
[/usr/lib64/python3.6/asyncore.py|readwrite|108]
[/usr/lib64/python3.6/asyncore.py|handle_read_event|423]
[/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|handle_read|71]
[/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|168]
[/usr/lib/python3.6/site-packages/vdsm/protocoldetector.py|handle_read|115]
)
(betterAsyncore:179)
I appreciate your help,
Alexander Murashkin
------ Current Versions (that have the exception) ------
Engine
ovirt-engine-4.4.0-0.0.master.20191204120550.git04d5d05.el7.noarch
Host
vdsm-4.40.0-1363.gitf6a1ba0a0.el8.x86_64
vdsm-python-4.40.0-1363.gitf6a1ba0a0.el8.noarch
vdsm-yajsonrpc-4.40.0-1363.gitf6a1ba0a0.el8.noarch
python3-libs-3.6.8-15.1.el8.x86_64
------ December 13th Versions (that did not have the exception) ------
Engine
ovirt-engine-4.4.0-0.0.master.20191204120550.git04d5d05.el7.noarch ---
not sure, but probably the same as now
Host
vdsm-4.40.0-1360.git821afbbc2.el8.x86_64
vdsm-python-4.40.0-1360.git821afbbc2.el8.noarch
vdsm-yajsonrpc-4.40.0-1360.git821afbbc2.el8.noarch
python3-libs-3.6.8-15.1.el8.x86_64
------ vdsm.log ------
2019-12-17 16:36:58,393-0600 ERROR (Reactor thread) [vds.dispatcher] uncaptured
python exception, closing channel <yajsonrpc.betterAsyncore.Dispatcher
connected ('::ffff:172.20.1.142', 38002, 0, 0) at 0x7fbda865ed30> (<class
'TypeError'>:object of type 'NoneType' has no len()
[/usr/lib64/python3.6/asyncore.py|readwrite|108]
[/usr/lib64/python3.6/asyncore.py|handle_read_event|423]
[/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|handle_read|71]
[/usr/lib/python3.6/site-packages/yajsonrpc/betterAsyncore.py|_delegate_call|168]
[/usr/lib/python3.6/site-packages/vdsm/protocoldetector.py|handle_read|115])
(betterAsyncore:179)
----- engine.log ------
2019-12-17 16:36:58,395-06 ERROR
[org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-95)
[6c787bf3] Command
'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' failed:
EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
java.net.SocketException: Connection reset (Failed with error VDS_NETWORK_ERROR
and code 5022)
------ Web Interface Events ------
Storage Domain Events
Dec 17, 2019, 4:36:58 PM
Failed to update VMs/Templates OVF data for Storage Domain storedom3 in
Data Center Default.
36b41d9c
oVirt
Dec 17, 2019, 4:36:58 PM
Failed to update OVF disks fcc661df-b2e3-4625-be40-52b65033c6d7, OVF
data isn't updated on those OVF stores (Data Center Default, Storage Domain
storedom3).
36b41d9c
oVirt
Host Events
Dec 17, 2019, 4:38:20 PM
Status of host poplar was set to Up.
399a2181
oVirt
Dec 17, 2019, 4:36:58 PM
Host poplar is not responding. Host cannot be fenced automatically
because power management for the host is disabled.
oVirt
_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/ARLNATXWKYDJBCZ4WWYS5EKTFA6IQ3SK/
