zap51 opened a new issue, #8211:
URL: https://github.com/apache/cloudstack/issues/8211
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete
the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
Storage, Ceph, RBD
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main
branch.
-->
~~~
4.16.1.0
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network,
advanced networking, etc. N/A otherwise
-->
Advanced Networking
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
Ubuntu Server 20.04 LTS on both Management and Hypervisors
Hypervisor: KVM
##### SUMMARY
<!-- Explain the problem/feature briefly -->
libvirtd throws error messages like `Oct 25 13:38:11 hv-01 libvirtd[9464]:
failed to open the RBD image '087bb114-448a-41d2-9f5d-6865b62eed15': No such
file or directory`
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal
test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
N/A
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
N/A
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
It appears that one of our clusters having 8 hosts is having the issue. We
have HCI on these 8 hosts and there are approximately 700+ VMs running. But
strange enough, there are these logs like below on hosts.
Oct 25 13:38:11 hv-01 libvirtd[9464]: failed to open the RBD image
'087bb114-448a-41d2-9f5d-6865b62eed15': No such file or directory
Oct 25 20:35:22 hv-01 libvirtd[9464]: failed to open the RBD image
'ccc1168a-5ffa-4b6d-a953-8e0ac788ebc5': No such file or directory
Oct 26 09:48:33 hv-01 libvirtd[9464]: failed to open the RBD image
'a3fe82f8-afc9-4604-b55e-91b676514a18': No such file or directory
Oct 26 10:38:17 hv-01 libvirtd[9464]: End of file while reading data:
Input/output error
We've got DNS servers on which there is an`A` record resolving to all the
IPv4 Addresses of 8 monitors and there have not been any issues with the DNS
resolution. But the issue of "failed to open the RBD image
'ccc1168a-5ffa-4b6d-a953-8e0ac788ebc5': No such file or directory" gets more
weird because the VM that is making use of that RBD image lets say
"087bb114-448a-41d2-9f5d-6865b62eed15" is running on altogether different host
like "hv-06". On further inspection of that specific Virtual Machine, it has
been running on that host "hv-06" for more than 4 months or so (looked at "Last
updated" field). Fortunately, the Virtual Machine has no issues and has been
running since then.
We're noticing the same "failed to open the RBD image" on all the hosts in
that cluster. There are no network issues observed or any issues on the hosts.
I was thinking of getting this host into maintenance but the same is there on
all the hosts in that cluster. I've not gotten a chance to look at the
management server logs thus seeking assistance.
~~~
Thanks,
Jayanth Reddy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
