GitHub user tatay188 closed a discussion: CloudStack 4.20 IPv6 Primary Storage CEPH Initial VMs on KVM not created - Not Converting to RBD - IPv6 addresses show truncated
### problem I am having the following problem, The Libvirt is unable to convert from Secondary Storage to CEPH RBD. CEPH is purely IPV6, The RBD was created, There are no error on the CEPH side. The Initial VM Creation starts and then stops, they never come enabled, the initial VMs are deleted and the process start in a loop. systemctl status libvirtd ● libvirtd.service - Virtualization daemon Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2025-04-01 01:19:22 UTC; 8h ago Docs: man:libvirtd(8) https://libvirt.org Main PID: 2271 (libvirtd) Tasks: 19 (limit: 32768) Memory: 98.5M CPU: 9.064s CGroup: /system.slice/libvirtd.service └─2271 /usr/sbin/libvirtd --listen Apr 01 02:32:57 kvmvcompatl2001 libvirtd[2271]: invalid argument: Connections from inside daemon must be direct Apr 01 02:32:57 kvmvcompatl2001 libvirtd[2271]: End of file while reading data: Input/output error Apr 01 02:33:02 kvmvcompatl2001 libvirtd[2271]: invalid argument: Connections from inside daemon must be direct Apr 01 02:33:02 kvmvcompatl2001 libvirtd[2271]: End of file while reading data: Input/output error Apr 01 02:33:27 kvmvcompatl2001 libvirtd[2271]: invalid argument: Connections from inside daemon must be direct Apr 01 02:33:27 kvmvcompatl2001 libvirtd[2271]: End of file while reading data: Input/output error Apr 01 02:33:32 kvmvcompatl2001 libvirtd[2271]: invalid argument: Connections from inside daemon must be direct the agent.log is showing the following in a loop: 2025-04-01 02:33:54,480 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:d64e452c) Starting copy from source image /mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 to RBD image 3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0 2025-04-01 02:33:54,480 DEBUG [utils.script.Script] (agentRequest-Handler-4:[]) (logid:d64e452c) Executing command [qemu-img convert -O raw -U --image-opts driver=qcow2,file.filename=/mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30 ]. 2025-04-01 02:33:54,532 WARN [utils.script.Script] (agentRequest-Handler-4:[]) (logid:d64e452c) Execution of process [7500] for command [qemu-img convert -O raw -U --image-opts driver=qcow2,file.filename=/mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30 ] failed. 2025-04-01 02:33:54,532 DEBUG [utils.script.Script] (agentRequest-Handler-4:[]) (logid:d64e452c) Exit value of process [7500] for command [qemu-img convert -O raw -U --image-opts driver=qcow2,file.filename=/mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30 ] is [1]. 2025-04-01 02:33:54,532 WARN [utils.script.Script] (agentRequest-Handler-4:[]) (logid:d64e452c) Process [7500] for command [qemu-img convert -O raw -U --image-opts driver=qcow2,file.filename=/mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30 ] encountered the error: [qemu-img: rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30: error while converting raw: invalid conf option 550:5607:fff0::22:24;20XX:YYYY:ZZZZ:LLLL::OO:26;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported: No such file or directory]. 2025-04-01 02:33:54,532 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:d64e452c) Failed to convert from /mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 to rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30 the error was: qemu-img: rbd:3cephUserandPool/f32a0f81-5661-41ac-832f-f5dfffa8b1e0:mon_host=20XX:YYYY:ZZZZ:LLLL::OO:24\;20XX:YYYY:ZZZZ:LLLL::OO:26\;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported=cephx:id=3cephUserandPool:key=KEYGENERATEDBYCEPH:rbd_default_format=2:client_mount_timeout=30: error while converting raw: invalid conf option 550:5607:fff0::22:24;20XX:YYYY:ZZZZ:LLLL::OO:26;20XX:YYYY:ZZZZ:LLLL::OO:auth_supported: No such file or directory 2025-04-01 02:33:54,532 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:d64e452c) Attempting to remove storage pool a1aa3257-b554-3896-a168-a593ebde9994 from libvirt 2025-04-01 02:33:54,532 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:[]) (logid:d64e452c) Looking for libvirtd connection at: qemu:///system 2025-04-01 02:33:54,534 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:d64e452c) Storage pool a1aa3257-b554-3896-a168-a593ebde9994 has no corresponding secret. Not removing any secret. 2025-04-01 02:33:54,577 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-4:[]) (logid:d64e452c) Storage pool a1aa3257-b554-3896-a168-a593ebde9994 was successfully removed from libvirt. 2025-04-01 02:33:54,579 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:[]) (logid:d64e452c) Seq 3-8729383452727574659: { Ans: , MgmtId: 250977680725600, via: 3, Ver: v1, Flags: 110, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","details":"com.cloud.utils.exception.CloudRuntimeException: Failed to copy /mnt/a1aa3257-b554-3896-a168-a593ebde9994/5ccb81a1-26ec-4d57-a02a-37f81e09be08.qcow2 to f32a0f81-5661-41ac-832f-f5dfffa8b1e0","wait":"0","bypassHostMaintenance":"false"}}] } 2025-04-01 02:33:54,621 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:[]) (logid:d64e452c) Request:Seq 3-8729383452727574660: { Cmd , MgmtId: 250977680725600, via: 3, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","controlIp":"169.254.138.135","forceStop":"false","volumesToDisconnect":[],"vmName":"v-172-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2025-04-01 02:33:54,621 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:[]) (logid:d64e452c) Processing command: com.cloud.agent.api.StopCommand 2025-04-01 02:33:54,621 DEBUG [resource.wrapper.LibvirtStopCommandWrapper] (agentRequest-Handler-3:[]) (logid:d64e452c) backing up the cmdline 2025-04-01 02:33:57,691 DEBUG [resource.wrapper.LibvirtStopCommandWrapper] (agentRequest-Handler-3:[]) (logid:d64e452c) Failed to backup cmdline file due to There was a problem while connecting to 169.254.138.135:3922 Error logs showing: libvirt: Remote Driver error : invalid argument: Connections from inside daemon must be direct libvirt: QEMU Driver error : Domain not found: no domain with matching name 'v-172-VM' libvirt: QEMU Driver error : Domain not found: no domain with matching name 'v-172-VM' libvirt: QEMU Driver error : Domain not found: no domain with matching name 'v-172-VM' libvirt: QEMU Driver error : Domain not found: no domain with matching name 'v-172-VM' libvirt: QEMU Driver error : Domain not found: no domain with matching name 'v-172-VM' libvirt: Storage Driver error : Storage pool not found: no storage pool with matching uuid 'a1aa3257-b554-3896-a168-a593ebde9994' libvirt: Secrets Driver error : Secret not found: no secret with matching uuid 'a1aa3257-b554-3896-a168-a593ebde9994' The Host VM reached the IPV6 ip. ### versions ACS 4.20.0 Hypervisor KVM Primary stoarge CEPH Squid RBD Secondary NFS EMC VXLAN Only no VLANS anywhere ### The steps to reproduce the bug 1.Install Ceph, Management, KVM host 2.Primary storage using CEPH RBD 3.Adding Host, Infrastructure VMs start, but do not complete ... ### What to do about it? Any help, or guidance, Seems a bug. I am unable to try with IPv4, and the CEPH is settle as IPv6. GitHub link: https://github.com/apache/cloudstack/discussions/10680 ---- This is an automatically sent email for users@cloudstack.apache.org. To unsubscribe, please send an email to: users-unsubscr...@cloudstack.apache.org