Re: Cloudstack Agent 4.18.1 metrics issue
Thanks Wei, Unsuspending helped Cheers Jon :) On Wed, Dec 13, 2023 at 4:07 PM Wei ZHOU wrote: > yes. > > the domain state is not supported in libvirt-java > I just created a PR > https://gitlab.com/libvirt/libvirt-java/-/merge_requests/40 > > > -Wei > > On Wed, 13 Dec 2023 at 11:36, Joan g wrote: > > > yes. restarting libvirtd does not help. > > > > "virsh list" command output. Is the "pmsuspended" causing issue ? > > > > root@kvm-1:~# virsh list > > IdName State > > --- > > 11i-2-281-VMrunning > > 25i-2-228-VMrunning > > 27i-2-230-VMrunning > > 52i-2-483-VMrunning > > 121 i-2-585-VMpmsuspended > > 122 i-2-584-VMrunning > > 123 i-2-590-VMrunning > > 139 i-2-635-VMrunning > > 145 i-2-479-VMrunning > > 164 i-8-703-VM running > > > > > > Jon > > > > On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > > > > > Can you restart libvirtd ? If it does not work, can you share the > result > > of > > > "virsh list" command ? > > > > > > > > > > > > -Wei > > > > > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > > > > > Hi Wei. > > > > > > > > OS : Ubuntu 20.04.6 LTS > > > > > > > > root@kvm-1:~# virsh version > > > > Compiled against library: libvirt 6.0.0 > > > > Using library: libvirt 6.0.0 > > > > Using API: QEMU 6.0.0 > > > > Running hypervisor: QEMU 4.2.1 > > > > > > > > Regards, > > > > Jon > > > > > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU > > wrote: > > > > > > > > > What's the OS and libvirt/qemu version ? > > > > > > > > > > -Wei > > > > > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Recently below is getting logged in my cloudstack agent and the > > > manager > > > > > is > > > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds > for > > > > > length > > > > > > 7 > > > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > > > at > > > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > > > at > > > > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > > > > > Regards, > > > > > > Jon > > > > > > > > > > > > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
yes. the domain state is not supported in libvirt-java I just created a PR https://gitlab.com/libvirt/libvirt-java/-/merge_requests/40 -Wei On Wed, 13 Dec 2023 at 11:36, Joan g wrote: > yes. restarting libvirtd does not help. > > "virsh list" command output. Is the "pmsuspended" causing issue ? > > root@kvm-1:~# virsh list > IdName State > --- > 11i-2-281-VMrunning > 25i-2-228-VMrunning > 27i-2-230-VMrunning > 52i-2-483-VMrunning > 121 i-2-585-VMpmsuspended > 122 i-2-584-VMrunning > 123 i-2-590-VMrunning > 139 i-2-635-VMrunning > 145 i-2-479-VMrunning > 164 i-8-703-VM running > > > Jon > > On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > > > Can you restart libvirtd ? If it does not work, can you share the result > of > > "virsh list" command ? > > > > > > > > -Wei > > > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > > > Hi Wei. > > > > > > OS : Ubuntu 20.04.6 LTS > > > > > > root@kvm-1:~# virsh version > > > Compiled against library: libvirt 6.0.0 > > > Using library: libvirt 6.0.0 > > > Using API: QEMU 6.0.0 > > > Running hypervisor: QEMU 4.2.1 > > > > > > Regards, > > > Jon > > > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU > wrote: > > > > > > > What's the OS and libvirt/qemu version ? > > > > > > > > -Wei > > > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > > > Hi, > > > > > > > > > > Recently below is getting logged in my cloudstack agent and the > > manager > > > > is > > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > > > length > > > > > 7 > > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > > at > > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > > at > > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > at > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > > at > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > > > Regards, > > > > > Jon > > > > > > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
yes. restarting libvirtd does not help. "virsh list" command output. Is the "pmsuspended" causing issue ? root@kvm-1:~# virsh list IdName State --- 11i-2-281-VMrunning 25i-2-228-VMrunning 27i-2-230-VMrunning 52i-2-483-VMrunning 121 i-2-585-VMpmsuspended 122 i-2-584-VMrunning 123 i-2-590-VMrunning 139 i-2-635-VMrunning 145 i-2-479-VMrunning 164 i-8-703-VM running Jon On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > Can you restart libvirtd ? If it does not work, can you share the result of > "virsh list" command ? > > > > -Wei > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > Hi Wei. > > > > OS : Ubuntu 20.04.6 LTS > > > > root@kvm-1:~# virsh version > > Compiled against library: libvirt 6.0.0 > > Using library: libvirt 6.0.0 > > Using API: QEMU 6.0.0 > > Running hypervisor: QEMU 4.2.1 > > > > Regards, > > Jon > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > > > > > What's the OS and libvirt/qemu version ? > > > > > > -Wei > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > Hi, > > > > > > > > Recently below is getting logged in my cloudstack agent and the > manager > > > is > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > > length > > > > 7 > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > at > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > at > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > at > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > at > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > Regards, > > > > Jon > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
Can you restart libvirtd ? If it does not work, can you share the result of "virsh list" command ? -Wei On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > Hi Wei. > > OS : Ubuntu 20.04.6 LTS > > root@kvm-1:~# virsh version > Compiled against library: libvirt 6.0.0 > Using library: libvirt 6.0.0 > Using API: QEMU 6.0.0 > Running hypervisor: QEMU 4.2.1 > > Regards, > Jon > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > > > What's the OS and libvirt/qemu version ? > > > > -Wei > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > Hi, > > > > > > Recently below is getting logged in my cloudstack agent and the manager > > is > > > not able to get any VM metrics. Can Someone help on below > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > length > > > 7 > > > at org.libvirt.DomainInfo.(Unknown Source) > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > at > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > at > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > at > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > at > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > Regards, > > > Jon > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
Hi Wei. OS : Ubuntu 20.04.6 LTS root@kvm-1:~# virsh version Compiled against library: libvirt 6.0.0 Using library: libvirt 6.0.0 Using API: QEMU 6.0.0 Running hypervisor: QEMU 4.2.1 Regards, Jon On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > What's the OS and libvirt/qemu version ? > > -Wei > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > Hi, > > > > Recently below is getting logged in my cloudstack agent and the manager > is > > not able to get any VM metrics. Can Someone help on below > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > length > > 7 > > at org.libvirt.DomainInfo.(Unknown Source) > > at org.libvirt.Domain.getInfo(Unknown Source) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > at > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > at com.cloud.utils.nio.Task.call(Task.java:83) > > at com.cloud.utils.nio.Task.call(Task.java:29) > > at > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > Regards, > > Jon > > >
Re: Cloudstack Agent 4.18.1 metrics issue
What's the OS and libvirt/qemu version ? -Wei On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > Hi, > > Recently below is getting logged in my cloudstack agent and the manager is > not able to get any VM metrics. Can Someone help on below > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for length > 7 > at org.libvirt.DomainInfo.(Unknown Source) > at org.libvirt.Domain.getInfo(Unknown Source) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > at com.cloud.agent.Agent.processRequest(Agent.java:662) > at > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > at com.cloud.utils.nio.Task.call(Task.java:83) > at com.cloud.utils.nio.Task.call(Task.java:29) > at > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > > Regards, > Jon >
Cloudstack Agent 4.18.1 metrics issue
Hi, Recently below is getting logged in my cloudstack agent and the manager is not able to get any VM metrics. Can Someone help on below 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:58ef1261) Caught: java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for length 7 at org.libvirt.DomainInfo.(Unknown Source) at org.libvirt.Domain.getInfo(Unknown Source) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) at com.cloud.agent.Agent.processRequest(Agent.java:662) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Regards, Jon
