GitHub user sptest232 created a discussion: Error deleting VM. ACS 4.22.0.0
KVM Ceph
ACS 4.22.0.0, KVM host OS - Oracle Linux Server 10.1 (Agent version -
4.22.0.0), ceph (20.2.0 tentacle).
An error occurs when deleting a VM.
2025-12-10 16:34:15,666 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Stopping","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
state transited from [Stopping] to [Stopping] with event [StopRequested]. VM's
original host: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"},
new host: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"},
host before state transition: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"}
2025-12-10 16:34:15,713 DEBUG [c.c.a.t.Request] (AgentManager-Handler-8:[])
(logid:) Seq 77387-94012642221359648: Processing: { Ans: , MgmtId:
108597816265214, via: 77387, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"false","details":"java.lang.IllegalArgumentException:
No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getDisks(LibvirtComputingResource.java:4818)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmDiskStat(LibvirtComputingResource.java:4883)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmDiskStatsCommandWrapper.execute(LibvirtGetVmDiskStatsCommandWrapper.java:50)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmDiskStatsCommandWrapper.execute(LibvirtGetVmDiskStatsCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
","wait":"0","bypassHostMaintenance":"false"}}] }
2025-12-10 16:34:15,713 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Seq 77387-94012642221359648: Received: { Ans: , MgmtId:
108597816265214, via: 77387(kvm-sr246), Ver: v1, Flags: 10, { Answer } }
2025-12-10 16:34:15,714 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Details from executing class
com.cloud.agent.api.GetVmDiskStatsCommand: java.lang.IllegalArgumentException:
No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getDisks(LibvirtComputingResource.java:4818)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmDiskStat(LibvirtComputingResource.java:4883)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmDiskStatsCommandWrapper.execute(LibvirtGetVmDiskStatsCommandWrapper.java:50)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmDiskStatsCommandWrapper.execute(LibvirtGetVmDiskStatsCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
2025-12-10 16:34:15,714 WARN [c.c.v.UserVmManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Error while collecting disk stats for vm: VM instance {"id":695
9,"instanceName":"i-332-6959-VM","state":"Stopping","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
from host: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-
e11e-4f96-9dcf-6355f723922f"} java.lang.ClassCastException: class
com.cloud.agent.api.Answer cannot be cast to class
com.cloud.agent.api.GetVmDiskStatsAnswer (com.cloud.agent.api.Answer and
com.cloud.agent.api.Ge
tVmDiskStatsAnswer are in unnamed module of loader 'app')
at
com.cloud.vm.UserVmManagerImpl.collectVmDiskStatistics(UserVmManagerImpl.java:6023)
at
com.cloud.vm.UserVmManagerImpl.collectVmDiskAndNetworkStatistics(UserVmManagerImpl.java:9924)
at
com.cloud.vm.UserVmManagerImpl.collectVmDiskAndNetworkStatistics(UserVmManagerImpl.java:9916)
at
com.cloud.vm.UserVmManagerImpl.prepareStop(UserVmManagerImpl.java:9293)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:2502)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:2355)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:5970)
at jdk.internal.reflect.GeneratedMethodAccessor1474.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-10 16:34:15,768 DEBUG [c.c.a.t.Request] (AgentManager-Handler-11:[])
(logid:) Seq 77387-94012642221359649: Processing: { Ans: , MgmtId:
108597816265214, via: 77387, Ver: v1, Flags: 10, [{"com.cloud.agent
.api.Answer":{"result":"false","details":"java.lang.IllegalArgumentException:
No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getInterfaces(LibvirtComputingResource.java:4796)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmNetworkStat(LibvirtComputingResource.java:4859)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmNetworkStatsCommandWrapper.execute(LibvirtGetVmNetworkStatsCommandWrapper.java:50)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmNetworkStatsCommandWrapper.execute(LibvirtGetVmNetworkStatsCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
","wait":"0","bypassHostMaintenance":"false"}}] }
2025-12-10 16:34:15,768 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Seq 77387-94012642221359649: Received: { Ans: , MgmtId:
10859781626521
4, via: 77387(kvm-sr246), Ver: v1, Flags: 10, { Answer } }
2025-12-10 16:34:15,768 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Details from executing class com.cloud.agent.api.GetV
mNetworkStatsCommand: java.lang.IllegalArgumentException: No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getInterfaces(LibvirtComputingResource.java:4796)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmNetworkStat(LibvirtComputingResource.java:4859)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmNetworkStatsCommandWrapper.execute(LibvirtGetVmNetworkStatsCommandWrapper.java:50)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmNetworkStatsCommandWrapper.execute(LibvirtGetVmNetworkStatsCommandWrapper.java:36)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
2025-12-10 16:34:15,768 WARN [c.c.v.UserVmManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Error while collecting network stats for vm: VM instance {"id":
6959,"instanceName":"i-332-6959-VM","state":"Stopping","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
from host: Host {"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bc
e1-e11e-4f96-9dcf-6355f723922f"} java.lang.ClassCastException: class
com.cloud.agent.api.Answer cannot be cast to class
com.cloud.agent.api.GetVmNetworkStatsAnswer (com.cloud.agent.api.Answer and
com.cloud.agent.
api.GetVmNetworkStatsAnswer are in unnamed module of loader 'app')
at
com.cloud.vm.UserVmManagerImpl.collectVmNetworkStatistics(UserVmManagerImpl.java:5142)
at
com.cloud.vm.UserVmManagerImpl.collectVmDiskAndNetworkStatistics(UserVmManagerImpl.java:9925)
at
com.cloud.vm.UserVmManagerImpl.collectVmDiskAndNetworkStatistics(UserVmManagerImpl.java:9916)
at
com.cloud.vm.UserVmManagerImpl.prepareStop(UserVmManagerImpl.java:9293)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:2502)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:2355)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:5970)
at jdk.internal.reflect.GeneratedMethodAccessor1474.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-10 16:34:15,780 DEBUG [c.c.v.ClusteredVirtualMachineManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) VM instance {"id":6959,"instanceName":"i-332-6
959-VM","state":"Stopping","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
is a [User], returning null for control Nic IP.
2025-12-10 16:34:15,780 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Wait time setting on com.cloud.agent.api.StopCommand
is 1800 seconds
2025-12-10 16:34:15,780 DEBUG [c.c.a.m.ClusteredAgentAttache]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Seq 77394-94012642221359650: Routed from 108597816265214
2025-12-10 16:34:15,781 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Seq 77387-94012642221359650: Sending { Cmd , MgmtId:
108597816265214,
via: 77387(kvm-sr246), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"2250":"true"},"exp
ungeVM":"false","volumesToDisconnect":[],"vmName":"i-332-6959-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
}
2025-12-10 16:34:15,803 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(StatsCollector-6:[ctx-756eb558]) (logid:1943dd78) Wait time setting on
com.cloud.agent.api.GetVmStatsCommand is 1800 seconds
2025-12-10 16:34:15,804 DEBUG [c.c.a.m.ClusteredDirectAgentAttache]
(StatsCollector-6:[ctx-756eb558]) (logid:1943dd78) Seq
77394-6782983988773400411: Routed from 108597816265214
2025-12-10 16:34:15,804 DEBUG [c.c.a.m.D.Task] (DirectAgent-106:[ctx-7d46979e])
(logid:838c6948) Seq 77394-6782983988773400411: Executing request
2025-12-10 16:34:15,824 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:[])
(logid:) Seq 77387-94012642221359650: Processing: { Ans: , MgmtId:
108597816265214, via: 77387, Ver: v1, Flags: 10, [{"com.cloud.agent
.api.Answer":{"result":"false","details":"java.lang.IllegalArgumentException:
No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getDisks(LibvirtComputingResource.java:4818)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:87)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStopCommandWrapper.execute(LibvirtStopCommandWrapper.java:47)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
","wait":"0","bypassHostMaintenance":"false"}}] }
2025-12-10 16:34:15,824 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Seq 77387-94012642221359650: Received: { Ans: , MgmtId:
10859781626521
4, via: 77387(kvm-sr246), Ver: v1, Flags: 10, { Answer } }
2025-12-10 16:34:15,824 WARN [c.c.v.ClusteredVirtualMachineManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Unable to stop vm VM instance {"id":6959,"inst
anceName":"i-332-6959-VM","state":"Stopping","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
2025-12-10 16:34:15,831 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) VM instance {"id":6959,"instanceName":"i-332-6959-VM","state"
:"Running","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"} state
transited from [Stopping] to [Running] with event [OperationFailed]. VM's
original host: Host {"id":77387,"name":"kvm-sr246-u1617.aisr
p.local","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"}, new
host: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"},
host befo
re state transition: Host
{"id":77387,"name":"kvm-sr246","type":"Routing","uuid":"8843bce1-e11e-4f96-9dcf-6355f723922f"}
2025-12-10 16:34:15,832 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Invocation exception, caused by: com.cloud.utils.exception.
CloudRuntimeException: Unable to stop VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Running","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
2025-12-10 16:34:15,832 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719, ctx-5fcf1916])
(logid:56c78da9) Rethrow exception com.cloud.utils.exception.CloudRuntimeExc
eption: Unable to stop VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Running","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
2025-12-10 16:34:15,832 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Done with run of VM work job: com.cloud.vm.VmWorkStop for VM 6959, job orig
in: 124718
2025-12-10 16:34:15,832 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Unable to complete AsyncJob {"accountId":36,"cmd":"com.cloud.vm.VmWorkStop"
,"cmdInfo":"rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4
cAAAAAAAAAAkAAAAAAAAACcAAAAAAAAbL3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA","cmdVersion":0,"completeMsid":null,"created":"Wed
Dec 10 16:34:15 MSK
2025","id":124719,"initMsid":108597816265214,"instanceId":null,"inst
anceType":null,"lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":39,"uuid":"816c3866-50ab-4d22-bcb2-6ebb379421ed"},
job origin: 124
718 com.cloud.utils.exception.CloudRuntimeException: Unable to stop VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Running","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"}
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:2551)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:2355)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:5970)
at jdk.internal.reflect.GeneratedMethodAccessor1474.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-10 16:34:15,836 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Publish async job-124719 complete on message bus
2025-12-10 16:34:15,836 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Wake up jobs related to job-124719
2025-12-10 16:34:15,836 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Update db status for job-124719
2025-12-10 16:34:15,837 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Wake up jobs joined with job-124719 and disjoin all subjobs created f
rom job- 124719
2025-12-10 16:34:15,845 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Done executing com.cloud.vm.VmWorkStop for job-124719
2025-12-10 16:34:15,846 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-144:[ctx-c925c4b8, job-124718/job-124719]) (logid:56c78da9)
Remove job-124719 from job monitoring
2025-12-10 16:34:15,862 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-40:[ctx-ef1768c5, job-124718]) (logid:56c78da9) Unexpected
exception while executing org.apache.cloudstack.api.command.admin.account.D
isableAccountCmd com.cloud.utils.exception.CloudRuntimeException: Unable to
stop VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Running","type":"User","uuid":"70847e83-b8d3-4116-bb2c-1e32079a8f0f"
}
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:2551)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:2355)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:5970)
at jdk.internal.reflect.GeneratedMethodAccessor1474.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:6075)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:698)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:646)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
2025-12-10 16:34:15,863 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-40:[ctx-ef1768c5, job-124718]) (logid:56c78da9) Complete
async job-124718, jobStatus: FAILED, resultCode: 530, result: org.apache.
cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Unable
to stop VM instance
{"id":6959,"instanceName":"i-332-6959-VM","state":"Running","type":"User","uuid":"70847e83-b
8d3-4116-bb2c-1e32079a8f0f"}"}
2025-12-10 16:34:16,007 ERROR [o.a.c.f.m.MessageBusBase]
(pool-17015-thread-1:[]) (logid:) NO EVENT PUBLISH CAN BE WRAPPED WITHIN DB
TRANSACTION! com.cloud.utils.exception.CloudRuntimeException: NO EVENT PUBLISH
CAN BE WRAPPED WITHIN DB TRANSACTION!
at
org.apache.cloudstack.framework.messagebus.MessageBusBase.publish(MessageBusBase.java:168)
at
com.cloud.capacity.CapacityManagerImpl.updateCapacityForHost(CapacityManagerImpl.java:786)
at
com.cloud.alert.AlertManagerImpl$1.doInTransactionWithoutResult(AlertManagerImpl.java:320)
at
com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
at
com.cloud.alert.AlertManagerImpl.lambda$recalculateHostCapacities$0(AlertManagerImpl.java:316)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
==========================================================
agent.log:
WARN [cloud.agent.Agent] (AgentRequest-Handler-4:[]) (logid:fd97e4d1) Caught:
java.lang.IllegalArgumentException: No enum constant
com.cloud.hypervisor.kvm.resource.LibvirtVMDef.DiskDef.DiskCacheMode.
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at
com.cloud.hypervisor.kvm.resource.LibvirtVMDef$DiskDef$DiskCacheMode.valueOf(LibvirtVMDef.java:709)
at
com.cloud.hypervisor.kvm.resource.LibvirtDomainXMLParser.parseDomainXML(LibvirtDomainXMLParser.java:114)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVncPort(LibvirtComputingResource.java:4668)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVncPortCommandWrapper.execute(LibvirtGetVncPortCommandWrapper.java:41)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVncPortCommandWrapper.execute(LibvirtGetVncPortCommandWrapper.java:32)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
========================================================
agent.err:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
libvirt: Storage Driver error : Storage pool not found: no storage pool with
matching uuid 'c0d74da4-e790-42a6-b44d-e395ba50d1bf'
libvirt: Storage Driver error : Storage pool not found: no storage pool with
matching uuid '234a8740-7c52-3a83-b995-e2a7b85f2967'
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-287-6943-VM'
libvirt: error : no connection driver available for lxc:///
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-332-6952-VM'
libvirt: error : no connection driver available for lxc:///
libvirt: QEMU Driver error : Domain not found: no domain with matching uuid
'6beeb063-6461-4b44-8442-67bc99cc13ca' (i-287-6943-VM)
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-287-6943-VM'
libvirt: QEMU Driver error : Domain not found: no domain with matching uuid
'9d280bb8-218b-49be-9afe-2d434467c516' (i-332-6952-VM)
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-332-6952-VM'
libvirt: Storage Driver error : Storage volume not found: no storage vol with
matching name 'b2b05820-6c0a-4bc3-a3a1-9c792c12c678'
libvirt: QEMU Driver error : Domain not found: no domain with matching name
'i-332-6959-VM'
libvirt: Storage Driver error : Storage volume not found: no storage vol with
matching name 'b4f08622-7459-4860-8aad-2e50cbafc761'
libvirt: Storage Driver error : Storage volume not found: no storage vol with
matching name 'cb822720-8ceb-4601-b104-073f065eb61c'
libvirt: Storage Driver error : failed to remove volume
'sr241-u5-u14-1/03b56a1a-c2d6-41db-ad3b-ecadc7fa448a': No such file or directory
libvirt: Storage Driver error : failed to remove volume
'sr241-u5-u14-1/b4f08622-7459-4860-8aad-2e50cbafc761': No such file or directory
libvirt: Storage Driver error : Storage volume not found: no storage vol with
matching name '480f1b9a-82ab-4278-96c3-6ceb02416e05'
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
========================
agent.out
WARNING: JAVA version is 21. JAVA 17 is recommended.
WARNING: JAVA version is 21. JAVA 17 is recommended.
WARNING: JAVA version is 21. JAVA 17 is recommended.
GitHub link: https://github.com/apache/cloudstack/discussions/12226
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
