GitHub user leduyquy added a comment to the discussion: Host stuck state:
Alert, When add Secondary storage
Can the host reach the NFS?
=> Host can connect to nfs server, i tried mounting nfs folder manually on
hosts to make sure it works properly
Can you share the agent log for the host?
=> Below is the agent log on host.
>
> 2025-04-11 10:42:56,783 INFO [cloud.agent.Agent] (Agent-Handler-1:[])
> (logid:) Reconnecting to host:<IP Managerment>
> 2025-04-11 10:42:56,783 INFO [utils.nio.NioClient] (Agent-Handler-1:[])
> (logid:) Connecting to <IP Managerment>:8250
> 2025-04-11 10:42:56,784 INFO [utils.nio.Link] (Agent-Handler-1:[]) (logid:)
> Conf file found: /etc/cloudstack/agent/agent.properties
> 2025-04-11 10:42:56,825 INFO [utils.nio.NioClient] (Agent-Handler-1:[])
> (logid:) SSL: Handshake done
> 2025-04-11 10:42:56,825 INFO [utils.nio.NioClient] (Agent-Handler-1:[])
> (logid:) Connected to <IP Managerment>:8250
> 2025-04-11 10:42:56,877 INFO [utils.linux.KVMHostInfo] (Agent-Handler-1:[])
> (logid:) Fetching CPU speed from command "lscpu".
> 2025-04-11 10:42:56,914 ERROR [utils.linux.KVMHostInfo] (Agent-Handler-1:[])
> (logid:) Unable to retrieve the CPU speed from lscpu.
> java.lang.NullPointerException
>
> 2025-04-11 10:42:56,914 INFO [utils.linux.KVMHostInfo] (Agent-Handler-1:[])
> (logid:) Fetching CPU speed from file
> [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency].
> 2025-04-11 10:42:56,915 INFO [utils.linux.KVMHostInfo] (Agent-Handler-1:[])
> (logid:) Retrieved value [2000000] from file
> [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency]. This corresponds to a
> CPU speed of [2000] MHz.
> 2025-04-11 10:42:56,915 INFO [utils.linux.KVMHostInfo] (Agent-Handler-1:[])
> (logid:) Fetching host CPU arch
> 2025-04-11 10:42:56,926 INFO [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-1:[]) (logid:) Host uses control group [cgroup2fs].
> 2025-04-11 10:42:56,926 INFO [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-1:[]) (logid:) Calculating the max shares of the host.
> 2025-04-11 10:42:56,926 INFO [kvm.resource.LibvirtComputingResource]
> (Agent-Handler-1:[]) (logid:) The max shares of the host is [64000].
> 2025-04-11 10:42:57,009 WARN [utils.script.Script] (Agent-Handler-1:[])
> (logid:) Execution of process [1715251] for command [/bin/bash -c virt-v2v
> --version ] failed.
> 2025-04-11 10:42:57,009 WARN [utils.script.Script] (Agent-Handler-1:[])
> (logid:) Process [1715251] for command [/bin/bash -c virt-v2v --version ]
> encountered the error: [127].
> 2025-04-11 10:42:57,030 INFO [kvm.storage.LibvirtStorageAdaptor]
> (Agent-Handler-1:[]) (logid:) Attempting to create storage pool
> 4952e569-130a-4dd8-817c-78f945323cfa (Filesystem) in libvirt
> 2025-04-11 10:42:57,052 INFO [kvm.storage.LibvirtStorageAdaptor]
> (Agent-Handler-1:[]) (logid:) Found existing defined storage pool
> 4952e569-130a-4dd8-817c-78f945323cfa, using it.
> 2025-04-11 10:42:57,052 INFO [kvm.storage.LibvirtStorageAdaptor]
> (Agent-Handler-1:[]) (logid:) Trying to fetch storage pool
> 4952e569-130a-4dd8-817c-78f945323cfa from libvirt
> 2025-04-11 10:42:57,078 INFO [cloud.agent.Agent] (Agent-Handler-2:[])
> (logid:) Process agent startup answer, agent id = 0
> 2025-04-11 10:42:57,079 INFO [cloud.agent.Agent] (Agent-Handler-2:[])
> (logid:) Startup Response Received: agent id = 0
> 2025-04-11 10:42:57,577 WARN [cloud.agent.Agent] (Agent-Handler-5:[])
> (logid:8c3b34cc) Unable to send response: null
here is the log from the management server i found. hope it helps
> 2025-04-11 09:46:30,192 DEBUG [o.a.c.c.p.RootCACustomTrustManager]
> (pool-13580-thread-1:[]) (logid:) Client/agent connection from ip=<IP host
> agent> accepted without certificate validation.
> 2025-04-11 09:46:30,465 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-7:[ctx-003f58d4]) (logid:) Seq 0-34315: Scheduling the
> first command { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 1,
> [{"com.cloud.agent.api.StartupRoutingCommand":{"cpuSockets":"1","cpus":"32","speed":"2000","cpuArch":"x86_64","memory":"268672163840","dom0MinMemory":"1073741824","poolSync":"false","supportsClonedVolumes":"false","caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"Host.OS.Kernel.Version":"5.15.0-135-generic","com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS.Version":"22.04","host.volume.encryption":"true","host.instance.conversion":"false","secured":"true","Host.OS":"Ubuntu"},"hostTags":[],"groupDetails":{},"type":"Routing","dataCenter":"9","pod":"9","cluster":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","iqn":"iqn.2004-10.com.ubuntu:01:52c55542e014","privateIpAddress":"<I
P host
agent>","privateMacAddress":"1a:b7:0d:b8:19:7f","privateNetmask":"255.255.255.0","storageIpAddress":"<IP
host
agent>","storageNetmask":"255.255.255.0","storageMacAddress":"1a:b7:0d:b8:19:7f","resourceName":"LibvirtComputingResource","gatewayIpAddress":"<IP
gateway>","msHostList":"<IP
Managerment>@static","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.StartupStorageCommand":{"totalSize":"(0
bytes)
0","poolInfo":{"uuid":"4952e569-130a-4dd8-817c-78f945323cfa","host":"<IP host
agent>","localPath":"/data","hostPath":"/data","poolType":"Filesystem","capacityBytes":"(3.4908
TB) 3838228365312","availableBytes":"(3.4629 TB)
3807542657024"},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"9","pod":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","resourceName":"LibvirtComputingResource","msHostList":"<IP
Managerment>@static","wait":"0","bypassHostMaintenance":"fal
se"}}] }
> 2025-04-11 09:46:30,468 DEBUG [c.c.a.t.Request]
> (AgentConnectTaskPool-13571:[ctx-a28ca8e4]) (logid:4b22fb41) Seq 0-34315:
> Processing the first command { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags: 1,
> [{"com.cloud.agent.api.StartupRoutingCommand":{"cpuSockets":"1","cpus":"32","speed":"2000","cpuArch":"x86_64","memory":"268672163840","dom0MinMemory":"1073741824","poolSync":"false","supportsClonedVolumes":"false","caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"Host.OS.Kernel.Version":"5.15.0-135-generic","com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS.Version":"22.04","host.volume.encryption":"true","host.instance.conversion":"false","secured":"true","Host.OS":"Ubuntu"},"hostTags":[],"groupDetails":{},"type":"Routing","dataCenter":"9","pod":"9","cluster":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","iqn":"iqn.2004-10.com.ubuntu:01:52c55542e014","privateIp
Address":"<IP host
agent>","privateMacAddress":"1a:b7:0d:b8:19:7f","privateNetmask":"255.255.255.0","storageIpAddress":"<IP
host
agent>","storageNetmask":"255.255.255.0","storageMacAddress":"1a:b7:0d:b8:19:7f","resourceName":"LibvirtComputingResource","gatewayIpAddress":"<IP
gateway>","msHostList":"<IP
Managerment>@static","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.StartupStorageCommand":{"totalSize":"(0
bytes)
0","poolInfo":{"uuid":"4952e569-130a-4dd8-817c-78f945323cfa","host":"<IP host
agent>","localPath":"/data","hostPath":"/data","poolType":"Filesystem","capacityBytes":"(3.4908
TB) 3838228365312","availableBytes":"(3.4629 TB)
3807542657024"},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"9","pod":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","resourceName":"LibvirtComputingResource","msHostList":"<IP
Managerment>@static","wait":"0","bypassHostMaint
enance":"false"}}] }
> 2025-04-11 09:46:30,706 WARN [c.c.a.m.A.HandleAgentConnectTask]
> (AgentConnectTaskPool-13571:[ctx-a28ca8e4]) (logid:4b22fb41) Unable to create
> attache for agent: Seq 0-34315: { Cmd , MgmtId: -1, via: 0, Ver: v1, Flags:
> 1,
> [{"com.cloud.agent.api.StartupRoutingCommand":{"cpuSockets":"1","cpus":"32","speed":"2000","cpuArch":"x86_64","memory":"268672163840","dom0MinMemory":"1073741824","poolSync":"false","supportsClonedVolumes":"false","caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"Host.OS.Kernel.Version":"5.15.0-135-generic","com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS.Version":"22.04","host.volume.encryption":"true","host.instance.conversion":"false","secured":"true","Host.OS":"Ubuntu"},"hostTags":[],"groupDetails":{},"type":"Routing","dataCenter":"9","pod":"9","cluster":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","iqn":"iqn.2004-10.com.ubuntu:01:
52c55542e014","privateIpAddress":"<IP host
agent>","privateMacAddress":"1a:b7:0d:b8:19:7f","privateNetmask":"255.255.255.0","storageIpAddress":"<IP
host
agent>","storageNetmask":"255.255.255.0","storageMacAddress":"1a:b7:0d:b8:19:7f","resourceName":"LibvirtComputingResource","gatewayIpAddress":"<IP
gateway>","msHostList":"<IP
Managerment>@static","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.StartupStorageCommand":{"totalSize":"(0
bytes)
0","poolInfo":{"uuid":"4952e569-130a-4dd8-817c-78f945323cfa","host":"<IP host
agent>","localPath":"/data","hostPath":"/data","poolType":"Filesystem","capacityBytes":"(3.4908
TB) 3838228365312","availableBytes":"(3.4629 TB)
3807542657024"},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"9","pod":"9","guid":"92426e62-ab7a-3ef4-9d7b-0c0d02d93121-LibvirtComputingResource","name":"cs-gpu-01","id":"0","version":"4.20.0.0","resourceName":"LibvirtComputingResource","msHostList":"<IP
Managerment>@static","wa
it":"0","bypassHostMaintenance":"false"}}] }
GitHub link:
https://github.com/apache/cloudstack/discussions/10718#discussioncomment-12839041
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscr...@cloudstack.apache.org
