GitHub user jonathaanbastin edited a comment on the discussion: CloudStack-KVM
integration for Veeam B&R
Hi, I have been testing this, and aside from the setup complications, it's
working OK. I found that the hosts won't output more than 200**Mb/s** from the
image server, which is confusing, since disk tests show ~300**MB/s** with under
2ms. So, a substantial speed difference. I am also having a VM that won't back
up. Another issue I came across was that setting a shared network in the same
network as the host killed all networking between Cloudstack and the host agent.
2026-10-02 13:22:48,492 WARN [utils.script.Script] (AgentRequest-Handler-1:[])
(logid:56f55ba3) Execution of process [22017] for command [/bin/bash -c virsh
checkpoint-delete i-2-14786-VM ckp-63debab3 ] failed.
2026-10-02 13:22:48,492 WARN [utils.script.Script] (AgentRequest-Handler-1:[])
(logid:56f55ba3) Process [22017] for command [/bin/bash -c virsh
checkpoint-delete i-2-14786-VM ckp-63debab3 ] encountered the error: [error:
Domain checkpoint not found: no domain checkpoint with matching name
'ckp-63debab3'].
2026-10-02 13:22:48,613 WARN [utils.script.Script] (AgentRequest-Handler-3:[])
(logid:56f55ba3) Execution of process [22019] for command [/bin/bash -c virsh
checkpoint-dumpxml --domain i-2-14786-VM --checkpointname ckp-4e6495e2
--no-domain ] failed.
2026-10-02 13:22:48,613 WARN [utils.script.Script] (AgentRequest-Handler-3:[])
(logid:56f55ba3) Process [22019] for command [/bin/bash -c virsh
checkpoint-dumpxml --domain i-2-14786-VM --checkpointname ckp-4e6495e2
--no-domain ] encountered the error: [error: Domain checkpoint not found: no
domain checkpoint with matching name 'ckp-4e6495e2'].
2026-10-02 13:22:58,677 WARN
[resource.wrapper.LibvirtCreateImageTransferCommandWrapper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Execution of process [22032] for
command [/bin/bash -c systemctl is-active --quiet cloudstack-image-server ]
failed.
2026-10-02 13:22:58,677 WARN
[resource.wrapper.LibvirtCreateImageTransferCommandWrapper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Process [22032] for command
[/bin/bash -c systemctl is-active --quiet cloudstack-image-server ] encountered
the error: [].
2026-10-02 13:22:58,753 WARN [kvm.resource.ImageServerControlSocket]
(AgentRequest-Handler-5:[]) (logid:65798c41) Execution of process [22037] for
command [/bin/bash -c echo '{"action":"status"}' | socat -t5 -
UNIX-CONNECT:/var/run/cloudstack/image-server.sock ] failed.
2026-10-02 13:22:58,754 WARN [kvm.resource.ImageServerControlSocket]
(AgentRequest-Handler-5:[]) (logid:65798c41) Exception [Stream closed] occurred
when attempting to run command [/bin/bash -c echo '{"action":"status"}' | socat
-t5 - UNIX-CONNECT:/var/run/cloudstack/image-server.sock ].
java.io.IOException: Stream closed
at
java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:287)
at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:330)
at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:190)
at java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)
at java.base/java.io.BufferedReader.fill(BufferedReader.java:162)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)
at
com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:41)
at com.cloud.utils.script.Script.executeInternal(Script.java:374)
at com.cloud.utils.script.Script.execute(Script.java:248)
at
com.cloud.hypervisor.kvm.resource.ImageServerControlSocket.sendMessage(ImageServerControlSocket.java:58)
at
com.cloud.hypervisor.kvm.resource.ImageServerControlSocket.isReady(ImageServerControlSocket.java:117)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.startImageServerIfNotRunning(LibvirtCreateImageTransferCommandWrapper.java:101)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.execute(LibvirtCreateImageTransferCommandWrapper.java:166)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.execute(LibvirtCreateImageTransferCommandWrapper.java:39)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2407)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2026-10-02 13:22:58,754 ERROR [kvm.resource.ImageServerControlSocket]
(AgentRequest-Handler-5:[]) (logid:65798c41) Control socket communication
failed: java.io.IOException: Stream closed
at
java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:287)
at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:330)
at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:190)
at java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)
at java.base/java.io.BufferedReader.fill(BufferedReader.java:162)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)
at
com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:41)
at com.cloud.utils.script.Script.executeInternal(Script.java:374)
at com.cloud.utils.script.Script.execute(Script.java:248)
at
com.cloud.hypervisor.kvm.resource.ImageServerControlSocket.sendMessage(ImageServerControlSocket.java:58)
at
com.cloud.hypervisor.kvm.resource.ImageServerControlSocket.isReady(ImageServerControlSocket.java:117)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.startImageServerIfNotRunning(LibvirtCreateImageTransferCommandWrapper.java:101)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.execute(LibvirtCreateImageTransferCommandWrapper.java:166)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtCreateImageTransferCommandWrapper.execute(LibvirtCreateImageTransferCommandWrapper.java:39)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2407)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2026-10-02 13:22:59,769 INFO
[resource.wrapper.LibvirtCreateImageTransferCommandWrapper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Image server control socket is
ready (attempt 2)
2026-10-02 13:22:59,769 INFO [storage.resource.IpTablesHelper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Adding rule [-p tcp -m state
--state NEW -m tcp --dport 54322 -j ACCEPT] to [INPUT] if required.
2026-10-02 13:22:59,778 WARN [storage.resource.IpTablesHelper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Execution of process [22046] for
command [/bin/bash -c iptables -C INPUT -p tcp -m state --state NEW -m tcp
--dport 54322 -j ACCEPT ] failed.
2026-10-02 13:22:59,778 WARN [storage.resource.IpTablesHelper]
(AgentRequest-Handler-5:[]) (logid:65798c41) Process [22046] for command
[/bin/bash -c iptables -C INPUT -p tcp -m state --state NEW -m tcp --dport
54322 -j ACCEPT ] encountered the error: [iptables: Bad rule (does a matching
rule exist in that chain?).].
2026-10-02 13:23:00,204 INFO
[resource.wrapper.LibvirtFinalizeImageTransferCommandWrapper]
(AgentRequest-Handler-4:[]) (logid:11c417b5) Image server
cloudstack-image-server stopped
2026-10-02 13:23:00,237 INFO
[resource.wrapper.LibvirtFinalizeImageTransferCommandWrapper]
(AgentRequest-Handler-4:[]) (logid:11c417b5) Firewall rule removed for port
54322 (or did not exist)
2026-10-02 13:23:00,571 WARN [utils.script.Script] (AgentRequest-Handler-2:[])
(logid:25d20b3e) Execution of process [22090] for command [/bin/bash -c virsh
domjobabort i-2-14786-VM ] failed.
2026-10-02 13:23:00,571 WARN [utils.script.Script] (AgentRequest-Handler-2:[])
(logid:25d20b3e) Process [22090] for command [/bin/bash -c virsh domjobabort
i-2-14786-VM ] encountered the error: [error: failed to get domain
'i-2-14786-VM'].
2026-10-02 13:23:10,500 WARN [utils.script.Script] (AgentRequest-Handler-3:[])
(logid:56f55ba3) Execution of process [22104] for command [/bin/bash -c virsh
checkpoint-delete i-2-14786-VM ckp-5c613880 ] failed.
2026-10-02 13:23:10,500 WARN [utils.script.Script] (AgentRequest-Handler-3:[])
(logid:56f55ba3) Process [22104] for command [/bin/bash -c virsh
checkpoint-delete i-2-14786-VM ckp-5c613880 ] encountered the error: [error:
failed to get domain 'i-2-14786-VM'].
>journalctl --since "2026-10-02 13:20:00" --until "2026-10-02 13:25:00" | \
egrep -i 'error|fail|i/o|scsi|iscsi|multipath|reset|timeout|qemu|libvirt'
Oct 02 13:22:48 host libvirtd[1893]: Domain checkpoint not found: no domain
checkpoint with matching name 'ckp-63debab3'
Oct 02 13:22:48 host libvirtd[1893]: Domain checkpoint not found: no domain
checkpoint with matching name 'ckp-4e6495e2'
Oct 02 13:22:48 host libvirtd[1893]: Domain id=18 name='i-2-14786-VM'
uuid=ec651411-6b77-458d-ace2-a2e5d95e9585 is tainted: custom-monitor
Oct 02 13:23:00 host libvirtd[1893]: internal error: End of file from agent
socket
Oct 02 13:23:00 host python3[22036]: 2026-10-02 13:23:00,104 ERROR EXTENTS
error image_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f
err=Error('nbd_connect_unix: recv: Connection reset by peer', 104)
Oct 02 13:23:00 host python3[22036]: 2026-10-02 13:23:00,104 ERROR HTTP failure
status=500 method=GET path=/images/bab3a2b9-efb5-4d62-9e8f-8b8348ad502f/extents
message=backend error
Oct 02 13:23:00 host systemd[1]:
machine-qemu\x2d18\x2di\x2d2\x2d14786\x2dVM.scope: Deactivated successfully.
Oct 02 13:23:00 host systemd[1]:
machine-qemu\x2d18\x2di\x2d2\x2d14786\x2dVM.scope: Consumed 1min 38.226s CPU
time.
Oct 02 13:23:00 host systemd-machined[1421]: Machine qemu-18-i-2-14786-VM
terminated.
Oct 02 13:23:16 host libvirtd[1893]: no connection driver available for lxc:///
Oct 02 13:23:16 host libvirtd[1893]: End of file while reading data:
Input/output error
Oct 02 13:24:16 host libvirtd[1893]: no connection driver available for lxc:///
Oct 02 13:24:16 host libvirtd[1893]: End of file while reading data:
Input/output error
>journalctl _PID=22036 --since "2026-10-02 13:20:00" --until "2026-10-02
>13:24:00"
Oct 02 13:22:58 frn1-q05-s02 python3[22036]: 2026-10-02 13:22:58,817 INFO
control socket listening on /var/run/cloudstack/image-server.sock
Oct 02 13:22:58 frn1-q05-s02 python3[22036]: 2026-10-02 13:22:58,830 INFO
listening on https://10.40.10.2:54322
Oct 02 13:22:59 frn1-q05-s02 python3[22036]: 2026-10-02 13:22:59,798 INFO
registered transfer_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f active=1
Oct 02 13:23:00 frn1-q05-s02 python3[22036]: 2026-10-02 13:23:00,047 INFO
EXTENTS start image_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f context=None
Oct 02 13:23:00 frn1-q05-s02 python3[22036]: 2026-10-02 13:23:00,104 ERROR
EXTENTS error image_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f
err=Error('nbd_connect_unix: recv: Connection reset by peer', 104)
Oct 02 13:23:00 frn1-q05-s02 python3[22036]: 2026-10-02 13:23:00,104 ERROR HTTP
failure status=500 method=GET
path=/images/bab3a2b9-efb5-4d62-9e8f-8b8348ad502f/extents message=backend error
Oct 02 13:23:00 frn1-q05-s02 python3[22036]: 2026-10-02 13:23:00,105 INFO
EXTENTS end image_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f duration_s=0.058
Oct 02 13:23:00 frn1-q05-s02 python3[22036]: 2026-10-02 13:23:00,141 INFO
unregistered transfer_id=bab3a2b9-efb5-4d62-9e8f-8b8348ad502f active=0
After the backup has failed, the domain is no longer found. The VM stays
running, but you are no longer able to show snapshots, and sometimes this can
be fixed with a live migration; however, it can fail to migrate as well, and it
can't find the domain.
Veeam 13.1
Cloudstack 4.23 server and agent
Host Ubuntu 24.04.05
GitHub link:
https://github.com/apache/cloudstack/discussions/12869#discussioncomment-18718839
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
