Now the log changed.
--------2019-10-04 13:39:50,484 DEBUG [c.c.a.ApiServlet] (qtp504527234-315:ctx-e1adf333) (logid:431d6db7) ===START=== 10.20.11.22 -- POST command=addHost&response=json 2019-10-04 13:39:50,490 DEBUG [c.c.a.ApiServer] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 13:39:50,496 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Received unknown parameters for command addHost. Unknown parameters : clustertype 2019-10-04 13:39:50,499 INFO [c.c.r.ResourceManagerImpl] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Trying to add a new host at http://10.20.11.206 in data center 2 2019-10-04 13:39:50,728 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Executing cmd: lsmod|grep kvm 2019-10-04 13:39:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-59d21754) (logid:f5f3e369) Begin cleanup expired async-jobs 2019-10-04 13:39:51,272 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-59d21754) (logid:f5f3e369) End cleanup expired async-jobs 2019-10-04 13:39:52,171 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) SSH command: lsmod|grep kvm
SSH command output:kvm_intel 188644 0 kvm 621480 1 kvm_intel irqbypass 13503 1 kvm2019-10-04 13:39:52,178 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Executing cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/ 2019-10-04 13:39:54,099 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) SSH command: sudo /usr/share/cloudstack-common/scripts/util/keystore-setup /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
SSH command output:2019-10-04 13:39:54,125 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Executing cmd: sudo /usr/share/cloudstack-common/scripts/util/keystore-cert-import /etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/ 2019-10-04 13:39:56,299 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Succeeded to import certificate in the keystore for agent on the KVM host: 10.20.11.206. Agent secured and trusted. 2019-10-04 13:39:56,302 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Executing cmd: cloudstack-setup-agent -m 10.20.11.203 -z 2 -p 2 -c 2 -g 2d5145f6-a4cf-3100-a196-60971d23c751 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm 2019-10-04 13:39:57,770 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-75569c1b) (logid:a5a1e15e) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:39:57,782 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-ad34ea9e) (logid:b8699705) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:39:58,317 DEBUG [c.c.u.s.SSHCmdHelper] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) SSH command: cloudstack-setup-agent -m 10.20.11.203 -z 2 -p 2 -c 2 -g 2d5145f6-a4cf-3100-a196-60971d23c751 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=kvm
SSH command output:Starting to configure your system: Configure SElinux ... [OK] Configure Network ... [Failed] dev: enp2s0 10.20.11.1 is not a network device Try to restore your system: Restore SElinux ... [OK] Restore Network ... [OK]2019-10-04 13:40:01,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-296deb1f) (logid:9540e79e) Begin cleanup expired async-jobs 2019-10-04 13:40:01,271 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-296deb1f) (logid:9540e79e) End cleanup expired async-jobs 2019-10-04 13:40:11,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-066073ca) (logid:a1e23b92) Begin cleanup expired async-jobs 2019-10-04 13:40:11,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-066073ca) (logid:a1e23b92) End cleanup expired async-jobs 2019-10-04 13:40:11,300 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-85cdfb69) (logid:75004b11) Snapshot scheduler.poll is being called at 2019-10-04 17:40:11 GMT 2019-10-04 13:40:11,301 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-85cdfb69) (logid:75004b11) Got 0 snapshots to be executed at 2019-10-04 17:40:11 GMT 2019-10-04 13:40:11,303 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-41c566cf) (logid:d61216ce) External devices are not used. Skipping external device usage collection 2019-10-04 13:40:11,346 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-2f5d23ff) (logid:3da0f3be) Found 0 running routers. 2019-10-04 13:40:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2c33e001) (logid:8c5304fc) Found 0 routers to update status. 2019-10-04 13:40:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2c33e001) (logid:8c5304fc) Found 0 VPC networks to update Redundant State. 2019-10-04 13:40:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2c33e001) (logid:8c5304fc) Found 0 networks to update RvR status. 2019-10-04 13:40:11,418 INFO [c.c.r.ResourceLimitManagerImpl] (ResourceCountChecker-1:ctx-3e377a5f) (logid:619c3f3d) Started resource counters recalculation periodic task. 2019-10-04 13:40:21,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7fbfe59b) (logid:bc5c2116) Begin cleanup expired async-jobs 2019-10-04 13:40:21,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7fbfe59b) (logid:bc5c2116) End cleanup expired async-jobs 2019-10-04 13:40:26,532 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-652ec0db) (logid:c9c69765) AutoScaling Monitor is running... 2019-10-04 13:40:26,698 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-9d8eaa0f) (logid:efd9904b) StorageCollector is running... 2019-10-04 13:40:26,699 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-af254b83) (logid:20af0cb6) HostStatsCollector is running... 2019-10-04 13:40:27,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-347a6de6) (logid:b730d7e6) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:40:27,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-7b9cf2d4) (logid:636ba47f) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:40:31,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-24eb284d) (logid:79bcec7a) Begin cleanup expired async-jobs 2019-10-04 13:40:31,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-24eb284d) (logid:79bcec7a) End cleanup expired async-jobs 2019-10-04 13:40:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-582e52b4) (logid:f6016d06) Begin cleanup expired async-jobs 2019-10-04 13:40:41,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-582e52b4) (logid:f6016d06) End cleanup expired async-jobs 2019-10-04 13:40:41,300 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (HostReservationReleaseChecker:ctx-090c2dca) (logid:7dea3c6c) Checking if any host reservation can be released ... 2019-10-04 13:40:41,301 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (HostReservationReleaseChecker:ctx-090c2dca) (logid:7dea3c6c) Done running HostReservationReleaseChecker ... 2019-10-04 13:40:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Running Capacity Checker ... 2019-10-04 13:40:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) recalculating system capacity 2019-10-04 13:40:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Executing cpu/ram capacity update 2019-10-04 13:40:41,347 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done executing cpu/ram capacity update 2019-10-04 13:40:41,347 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Executing storage capacity update 2019-10-04 13:40:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-b307cbf6) (logid:b7fadfcb) Found 0 routers to update status. 2019-10-04 13:40:41,348 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done executing storage capacity update 2019-10-04 13:40:41,348 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Executing capacity updates for public ip and Vlans 2019-10-04 13:40:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-b307cbf6) (logid:b7fadfcb) Found 0 VPC networks to update Redundant State. 2019-10-04 13:40:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-b307cbf6) (logid:b7fadfcb) Found 0 networks to update RvR status. 2019-10-04 13:40:41,355 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done capacity updates for public ip and Vlans 2019-10-04 13:40:41,355 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Executing capacity updates for private ip 2019-10-04 13:40:41,359 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done executing capacity updates for private ip 2019-10-04 13:40:41,360 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done recalculating system capacity 2019-10-04 13:40:41,392 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-e28be756) (logid:1ec6852f) Done running Capacity Checker ... 2019-10-04 13:40:46,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Resetting hosts suitable for reconnect 2019-10-04 13:40:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Completed resetting hosts suitable for reconnect 2019-10-04 13:40:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Acquiring hosts for clusters already owned by this management server 2019-10-04 13:40:46,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Completed acquiring hosts for clusters already owned by this management server 2019-10-04 13:40:46,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Acquiring hosts for clusters not owned by any management server 2019-10-04 13:40:46,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-4f4eb004) (logid:85109635) Completed acquiring hosts for clusters not owned by any management server 2019-10-04 13:40:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-9b7c8714) (logid:17bbd7e1) Begin cleanup expired async-jobs 2019-10-04 13:40:51,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-9b7c8714) (logid:17bbd7e1) End cleanup expired async-jobs 2019-10-04 13:40:57,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-013af53a) (logid:842b2e4f) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:40:57,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-1ee74604) (logid:6772c94a) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:41:01,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2f24436b) (logid:8d283633) Begin cleanup expired async-jobs 2019-10-04 13:41:01,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2f24436b) (logid:8d283633) End cleanup expired async-jobs 2019-10-04 13:41:11,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b042aefe) (logid:563f0d63) Begin cleanup expired async-jobs 2019-10-04 13:41:11,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b042aefe) (logid:563f0d63) End cleanup expired async-jobs 2019-10-04 13:41:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-fbc5a62e) (logid:8f5e81a5) Found 0 routers to update status. 2019-10-04 13:41:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-fbc5a62e) (logid:8f5e81a5) Found 0 VPC networks to update Redundant State. 2019-10-04 13:41:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-fbc5a62e) (logid:8f5e81a5) Found 0 networks to update RvR status. 2019-10-04 13:41:21,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-569328bb) (logid:43f7424b) Begin cleanup expired async-jobs 2019-10-04 13:41:21,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-569328bb) (logid:43f7424b) End cleanup expired async-jobs 2019-10-04 13:41:26,533 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-fd3477f7) (logid:00d3c836) AutoScaling Monitor is running... 2019-10-04 13:41:26,700 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-bd83e1c7) (logid:7a88c33a) StorageCollector is running... 2019-10-04 13:41:26,702 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-aa403673) (logid:448ff1bf) HostStatsCollector is running... 2019-10-04 13:41:27,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-048af1b9) (logid:c3965f0e) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:41:27,784 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-61b3a6f2) (logid:8ecd351f) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:41:31,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4434c56b) (logid:7e27f5bd) Begin cleanup expired async-jobs 2019-10-04 13:41:31,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4434c56b) (logid:7e27f5bd) End cleanup expired async-jobs 2019-10-04 13:41:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-a96b1e24) (logid:f789c648) Begin cleanup expired async-jobs 2019-10-04 13:41:41,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-a96b1e24) (logid:f789c648) End cleanup expired async-jobs 2019-10-04 13:41:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-da69e2a9) (logid:df397cfb) Found 0 routers to update status. 2019-10-04 13:41:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-da69e2a9) (logid:df397cfb) Found 0 VPC networks to update Redundant State. 2019-10-04 13:41:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-da69e2a9) (logid:df397cfb) Found 0 networks to update RvR status. 2019-10-04 13:41:51,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c5f85d65) (logid:c7a7fad6) Begin cleanup expired async-jobs 2019-10-04 13:41:51,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c5f85d65) (logid:c7a7fad6) End cleanup expired async-jobs 2019-10-04 13:41:57,773 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-981da045) (logid:fa73d6e1) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:41:57,784 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-e8561d79) (logid:d37229d3) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:42:01,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-615030b1) (logid:573c8ede) Begin cleanup expired async-jobs 2019-10-04 13:42:01,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-615030b1) (logid:573c8ede) End cleanup expired async-jobs 2019-10-04 13:42:11,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e2c012d7) (logid:53e2ec2f) Begin cleanup expired async-jobs 2019-10-04 13:42:11,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e2c012d7) (logid:53e2ec2f) End cleanup expired async-jobs 2019-10-04 13:42:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-4847a789) (logid:dfb2d55f) Found 0 routers to update status. 2019-10-04 13:42:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-4847a789) (logid:dfb2d55f) Found 0 VPC networks to update Redundant State. 2019-10-04 13:42:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-4847a789) (logid:dfb2d55f) Found 0 networks to update RvR status. 2019-10-04 13:42:16,306 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Resetting hosts suitable for reconnect 2019-10-04 13:42:16,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Completed resetting hosts suitable for reconnect 2019-10-04 13:42:16,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Acquiring hosts for clusters already owned by this management server 2019-10-04 13:42:16,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Completed acquiring hosts for clusters already owned by this management server 2019-10-04 13:42:16,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Acquiring hosts for clusters not owned by any management server 2019-10-04 13:42:16,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-7af1ffd8) (logid:241bc11d) Completed acquiring hosts for clusters not owned by any management server 2019-10-04 13:42:21,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7ef3eaa6) (logid:03c3dc4d) Begin cleanup expired async-jobs 2019-10-04 13:42:21,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7ef3eaa6) (logid:03c3dc4d) End cleanup expired async-jobs 2019-10-04 13:42:26,535 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-893e4ad6) (logid:f0afca9f) AutoScaling Monitor is running... 2019-10-04 13:42:26,703 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-6c696186) (logid:99429c97) StorageCollector is running... 2019-10-04 13:42:26,705 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-561072b5) (logid:2295be80) HostStatsCollector is running... 2019-10-04 13:42:27,771 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-114c404c) (logid:a5bca723) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:42:27,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-4f1eb3f5) (logid:e7d78e47) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:42:31,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-11a2b49f) (logid:4430b5d4) Begin cleanup expired async-jobs 2019-10-04 13:42:31,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-11a2b49f) (logid:4430b5d4) End cleanup expired async-jobs 2019-10-04 13:42:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-965731bd) (logid:5fc08bc8) Begin cleanup expired async-jobs 2019-10-04 13:42:41,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-965731bd) (logid:5fc08bc8) End cleanup expired async-jobs 2019-10-04 13:42:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a0694257) (logid:337909ba) Found 0 routers to update status. 2019-10-04 13:42:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a0694257) (logid:337909ba) Found 0 VPC networks to update Redundant State. 2019-10-04 13:42:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-a0694257) (logid:337909ba) Found 0 networks to update RvR status. 2019-10-04 13:42:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-3ce922c7) (logid:693f7e0e) Begin cleanup expired async-jobs 2019-10-04 13:42:51,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-3ce922c7) (logid:693f7e0e) End cleanup expired async-jobs 2019-10-04 13:42:57,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-16643bb4) (logid:9cc0093e) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:42:57,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-b649dc3f) (logid:c75f861f) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:43:01,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-1692dd40) (logid:caa65499) Begin cleanup expired async-jobs 2019-10-04 13:43:01,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-1692dd40) (logid:caa65499) End cleanup expired async-jobs 2019-10-04 13:43:11,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2ed2a0af) (logid:aea3f1d2) Begin cleanup expired async-jobs 2019-10-04 13:43:11,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2ed2a0af) (logid:aea3f1d2) End cleanup expired async-jobs 2019-10-04 13:43:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-672800f1) (logid:524fc907) Found 0 routers to update status. 2019-10-04 13:43:11,348 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-672800f1) (logid:524fc907) Found 0 VPC networks to update Redundant State. 2019-10-04 13:43:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-672800f1) (logid:524fc907) Found 0 networks to update RvR status. 2019-10-04 13:43:21,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7a192109) (logid:cb97bed3) Begin cleanup expired async-jobs 2019-10-04 13:43:21,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7a192109) (logid:cb97bed3) End cleanup expired async-jobs 2019-10-04 13:43:26,536 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-582902c6) (logid:011e1750) AutoScaling Monitor is running... 2019-10-04 13:43:26,705 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-1559d46f) (logid:b3f16c7d) StorageCollector is running... 2019-10-04 13:43:26,707 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-6eec9079) (logid:24dc2ded) HostStatsCollector is running... 2019-10-04 13:43:27,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-344e7747) (logid:0da47e9d) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:43:27,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-65f64fb8) (logid:d954d164) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:43:31,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0384bf09) (logid:7b745814) Begin cleanup expired async-jobs 2019-10-04 13:43:31,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0384bf09) (logid:7b745814) End cleanup expired async-jobs 2019-10-04 13:43:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4d968065) (logid:47d4a190) Begin cleanup expired async-jobs 2019-10-04 13:43:41,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4d968065) (logid:47d4a190) End cleanup expired async-jobs 2019-10-04 13:43:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee68bacc) (logid:144f16d4) Found 0 routers to update status. 2019-10-04 13:43:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee68bacc) (logid:144f16d4) Found 0 VPC networks to update Redundant State. 2019-10-04 13:43:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-ee68bacc) (logid:144f16d4) Found 0 networks to update RvR status. 2019-10-04 13:43:46,306 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Resetting hosts suitable for reconnect 2019-10-04 13:43:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Completed resetting hosts suitable for reconnect 2019-10-04 13:43:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Acquiring hosts for clusters already owned by this management server 2019-10-04 13:43:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Completed acquiring hosts for clusters already owned by this management server 2019-10-04 13:43:46,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Acquiring hosts for clusters not owned by any management server 2019-10-04 13:43:46,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-029e4dba) (logid:e06c13a4) Completed acquiring hosts for clusters not owned by any management server 2019-10-04 13:43:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-65cd849d) (logid:21d1979c) Begin cleanup expired async-jobs 2019-10-04 13:43:51,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-65cd849d) (logid:21d1979c) End cleanup expired async-jobs 2019-10-04 13:43:57,773 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-d54a9521) (logid:d522d901) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:43:57,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-60b58efe) (logid:7ef377b3) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:44:01,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d9d602cf) (logid:25abd3ec) Begin cleanup expired async-jobs 2019-10-04 13:44:01,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d9d602cf) (logid:25abd3ec) End cleanup expired async-jobs 2019-10-04 13:44:11,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-062514cc) (logid:08602b2a) Begin cleanup expired async-jobs 2019-10-04 13:44:11,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-062514cc) (logid:08602b2a) End cleanup expired async-jobs 2019-10-04 13:44:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-41534d40) (logid:87b101c3) Found 0 routers to update status. 2019-10-04 13:44:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-41534d40) (logid:87b101c3) Found 0 VPC networks to update Redundant State. 2019-10-04 13:44:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-41534d40) (logid:87b101c3) Found 0 networks to update RvR status. 2019-10-04 13:44:21,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b5243c4b) (logid:33f5ea7d) Begin cleanup expired async-jobs 2019-10-04 13:44:21,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b5243c4b) (logid:33f5ea7d) End cleanup expired async-jobs 2019-10-04 13:44:26,538 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-9c933699) (logid:3263e358) AutoScaling Monitor is running... 2019-10-04 13:44:26,708 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-13054cde) (logid:78a24aeb) StorageCollector is running... 2019-10-04 13:44:26,710 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-25e4b83a) (logid:57f9b51d) HostStatsCollector is running... 2019-10-04 13:44:27,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-0f309e6d) (logid:845a5f17) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:44:27,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-88814105) (logid:8e89e055) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:44:31,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-237080cf) (logid:c2bcfe68) Begin cleanup expired async-jobs 2019-10-04 13:44:31,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-237080cf) (logid:c2bcfe68) End cleanup expired async-jobs 2019-10-04 13:44:41,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e86e4cea) (logid:5f3c2c0d) Begin cleanup expired async-jobs 2019-10-04 13:44:41,271 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e86e4cea) (logid:5f3c2c0d) End cleanup expired async-jobs 2019-10-04 13:44:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-04668c3e) (logid:eb0f139c) Found 0 routers to update status. 2019-10-04 13:44:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-04668c3e) (logid:eb0f139c) Found 0 VPC networks to update Redundant State. 2019-10-04 13:44:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-04668c3e) (logid:eb0f139c) Found 0 networks to update RvR status. 2019-10-04 13:44:51,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-5ede47b6) (logid:fd67dbbf) Begin cleanup expired async-jobs 2019-10-04 13:44:51,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-5ede47b6) (logid:fd67dbbf) End cleanup expired async-jobs 2019-10-04 13:44:57,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-1f3c6086) (logid:3e8632a5) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:44:57,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-c80962c7) (logid:5beb37f3) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:44:58,328 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Timeout, to wait for the host connecting to mgt svr, assuming it is failed 2019-10-04 13:44:58,330 WARN [c.c.r.ResourceManagerImpl] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Unable to find the server resources at http://10.20.11.206 2019-10-04 13:44:58,330 INFO [c.c.u.e.CSExceptionErrorCode] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions 2019-10-04 13:44:58,332 WARN [o.a.c.a.c.a.h.AddHostCmd] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Exception:
com.cloud.exception.DiscoveryException: Unable to add the hostat com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:825) at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:611)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy174.discoverHosts(Unknown Source)at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:732) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:556)at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:310)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130)at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:127) at com.cloud.api.ApiServlet.doPost(ApiServlet.java:94) at javax.servlet.http.HttpServlet.service(HttpServlet.java:706) at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)2019-10-04 13:44:58,334 INFO [c.c.a.ApiServer] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) Unable to add the host 2019-10-04 13:44:58,334 DEBUG [c.c.a.ApiServlet] (qtp504527234-315:ctx-e1adf333 ctx-b67fe6c4) (logid:431d6db7) ===END=== 10.20.11.22 -- POST command=addHost&response=json 2019-10-04 13:45:01,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-1bdb4b3c) (logid:ff631f77) Begin cleanup expired async-jobs 2019-10-04 13:45:01,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-1bdb4b3c) (logid:ff631f77) End cleanup expired async-jobs 2019-10-04 13:45:11,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-53166fce) (logid:299bf51e) Begin cleanup expired async-jobs 2019-10-04 13:45:11,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-53166fce) (logid:299bf51e) End cleanup expired async-jobs 2019-10-04 13:45:11,300 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-7d3fbdf9) (logid:e9bd4b95) Snapshot scheduler.poll is being called at 2019-10-04 17:45:11 GMT 2019-10-04 13:45:11,301 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-7d3fbdf9) (logid:e9bd4b95) Got 0 snapshots to be executed at 2019-10-04 17:45:11 GMT 2019-10-04 13:45:11,303 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-3fafc1c0) (logid:7914d74d) External devices are not used. Skipping external device usage collection 2019-10-04 13:45:11,306 INFO [c.c.h.HighAvailabilityManagerImpl] (HA-2:ctx-87e9260d) (logid:086e0569) checking health of usage server 2019-10-04 13:45:11,307 DEBUG [c.c.h.HighAvailabilityManagerImpl] (HA-2:ctx-87e9260d) (logid:086e0569) usage server running? false, heartbeat: null 2019-10-04 13:45:11,308 WARN [o.a.c.alerts] (HA-2:ctx-87e9260d) (logid:086e0569) AlertType:: 13 | dataCenterId:: 0 | podId:: 0 | clusterId:: null | message:: No usage server process running 2019-10-04 13:45:11,310 DEBUG [c.c.a.AlertManagerImpl] (HA-2:ctx-87e9260d) (logid:086e0569) Have already sent: 1 emails for alert type '13' -- skipping send email 2019-10-04 13:45:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-b17d98d3) (logid:8fa5dce2) Found 0 running routers. 2019-10-04 13:45:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1fd79560) (logid:68479c14) Found 0 routers to update status. 2019-10-04 13:45:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1fd79560) (logid:68479c14) Found 0 VPC networks to update Redundant State. 2019-10-04 13:45:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1fd79560) (logid:68479c14) Found 0 networks to update RvR status. 2019-10-04 13:45:11,364 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-4e07db9e) (logid:2576cd8d) Found 0 running routers. 2019-10-04 13:45:11,418 INFO [c.c.r.ResourceLimitManagerImpl] (ResourceCountChecker-1:ctx-e997e708) (logid:973aa4a4) Started resource counters recalculation periodic task. 2019-10-04 13:45:11,455 INFO [o.a.c.e.o.NetworkOrchestrator] (Network-Scavenger-1:ctx-147f010a) (logid:fde3d70e) NetworkGarbageCollector uses '600' seconds for GC interval. 2019-10-04 13:45:16,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Resetting hosts suitable for reconnect 2019-10-04 13:45:16,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Completed resetting hosts suitable for reconnect 2019-10-04 13:45:16,308 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Acquiring hosts for clusters already owned by this management server 2019-10-04 13:45:16,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Completed acquiring hosts for clusters already owned by this management server 2019-10-04 13:45:16,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Acquiring hosts for clusters not owned by any management server 2019-10-04 13:45:16,309 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-c0badc3a) (logid:86b52c12) Completed acquiring hosts for clusters not owned by any management server 2019-10-04 13:45:21,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b6ee3d89) (logid:225d6d59) Begin cleanup expired async-jobs 2019-10-04 13:45:21,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-b6ee3d89) (logid:225d6d59) End cleanup expired async-jobs 2019-10-04 13:45:21,437 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-8b9fa761) (logid:c9a1ac88) LB HealthCheck Manager is running and getting the updates from LB providers and updating service status 2019-10-04 13:45:21,451 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-8b9fa761) (logid:c9a1ac88) LB HealthCheck Manager is running and getting the updates from LB providers and updating service status 2019-10-04 13:45:26,539 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-df4639fe) (logid:e2c5e1dc) AutoScaling Monitor is running... 2019-10-04 13:45:26,711 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-daa6d07a) (logid:b27f7934) StorageCollector is running... 2019-10-04 13:45:26,713 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-b069b620) (logid:7962cac8) HostStatsCollector is running... 2019-10-04 13:45:27,771 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8a09ac57) (logid:96b379c9) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:45:27,784 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-03cfc09b) (logid:1e1159e3) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:45:31,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-282f3dbb) (logid:7cea4d11) Begin cleanup expired async-jobs 2019-10-04 13:45:31,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-282f3dbb) (logid:7cea4d11) End cleanup expired async-jobs 2019-10-04 13:45:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bebe5d2e) (logid:7503543a) Begin cleanup expired async-jobs 2019-10-04 13:45:41,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bebe5d2e) (logid:7503543a) End cleanup expired async-jobs 2019-10-04 13:45:41,300 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (HostReservationReleaseChecker:ctx-bcaa8cc8) (logid:6ddca66c) Checking if any host reservation can be released ... 2019-10-04 13:45:41,300 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (HostReservationReleaseChecker:ctx-bcaa8cc8) (logid:6ddca66c) Done running HostReservationReleaseChecker ... 2019-10-04 13:45:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Running Capacity Checker ... 2019-10-04 13:45:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) recalculating system capacity 2019-10-04 13:45:41,346 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Executing cpu/ram capacity update 2019-10-04 13:45:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-eafb9532) (logid:569a166e) Found 0 routers to update status. 2019-10-04 13:45:41,347 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done executing cpu/ram capacity update 2019-10-04 13:45:41,347 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Executing storage capacity update 2019-10-04 13:45:41,348 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done executing storage capacity update 2019-10-04 13:45:41,348 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Executing capacity updates for public ip and Vlans 2019-10-04 13:45:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-eafb9532) (logid:569a166e) Found 0 VPC networks to update Redundant State. 2019-10-04 13:45:41,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-eafb9532) (logid:569a166e) Found 0 networks to update RvR status. 2019-10-04 13:45:41,355 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done capacity updates for public ip and Vlans 2019-10-04 13:45:41,355 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Executing capacity updates for private ip 2019-10-04 13:45:41,359 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done executing capacity updates for private ip 2019-10-04 13:45:41,359 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done recalculating system capacity 2019-10-04 13:45:41,391 DEBUG [c.c.a.AlertManagerImpl] (CapacityChecker:ctx-fccfc367) (logid:019af20e) Done running Capacity Checker ... 2019-10-04 13:45:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-f2c5fa09) (logid:a23cbc06) Begin cleanup expired async-jobs 2019-10-04 13:45:51,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-f2c5fa09) (logid:a23cbc06) End cleanup expired async-jobs 2019-10-04 13:45:57,773 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-dbf17d3c) (logid:8e95d630) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 13:45:57,784 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-855a9404) (logid:849122a1) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 13:46:01,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-63b8adf5) (logid:65f26f38) Begin cleanup expired async-jobs 2019-10-04 13:46:01,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-63b8adf5) (logid:65f26f38) End cleanup expired async-jobs
On 10/4/2019 4:57 PM, Eriel Perez wrote:
Thanks friend for answering. I send photos of my settings. What do you think is wrong? On 10/4/2019 4:40 PM, Andrija Panic wrote:Hi Eriel, for beginning, ensure your firewall is completely switched off. I can see specific error of not being able to ssh to the host - I assume this is your KVM (amd management server) host: 2019-10-04 11:50:04,829 WARN [c.c.h.k.d.LibvirtServerDiscoverer] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) can't setup agent, due to java.io.IOException: There was a problem while connecting to 10.20.11.253:22 - There was a problem while connecting to 10.20.11.253:22 So check your ssh/password with root user, that ACS need to setup agent. The guide is OK, except that for CloudStack 4.13 version, you should be installing systemVM template version 4.11.3, and not 4.11.2 that is mentioned in the guide. Double and triple check your setup against the guide please and update us here. (btw mostly English speaking persons here and WELCOME to the community!) Andrija On Fri, 4 Oct 2019, 22:00 Eriel Perez,<erielperezg...@gmail.com> wrote:Hello friends. I just finished installing CloudStack ( http://docs.cloudstack.apache.org/en/4.13.0.0/quickinstallationguide/qig.html), and I am unable to operate a host or configure primary storage. I've seen manuals and videos but I can't make it work. Anyone know if that guide has problems? Please I need help. I insert the logs below. Thank you. Hola amigos. Recien acabo de instalar CloudStack ( http://docs.cloudstack.apache.org/en/4.13.0.0/quickinstallationguide/qig.html), y no logro poner funcionar un host ni configurar el primary storage. He visto manuales y videos pero no logro hacerlo funcinar. Alguien sabe si esa guia tiene problemas? Por favor necesito ayuda. Abajo insterto los logs. Gracias. ---------------- 2019-10-04 11:49:20,758 DEBUG [c.c.a.ApiServlet] (qtp504527234-289:ctx-1e5d4f2b) (logid:acd38e93) ===START=== 10.20.11.22 -- GET command=listNetworkOfferings&id=a21264b9-a8cd-41d9-8cae-a71e7ff51f68&response=json&_=1570218577221 2019-10-04 11:49:20,763 DEBUG [c.c.a.ApiServer] (qtp504527234-289:ctx-1e5d4f2b ctx-5daf456d) (logid:acd38e93) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:20,775 DEBUG [c.c.a.ApiServlet] (qtp504527234-289:ctx-1e5d4f2b ctx-5daf456d) (logid:acd38e93) ===END=== 10.20.11.22 -- GET command=listNetworkOfferings&id=a21264b9-a8cd-41d9-8cae-a71e7ff51f68&response=json&_=1570218577221 2019-10-04 11:49:21,267 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-12529967) (logid:c7fad834) Begin cleanup expired async-jobs 2019-10-04 11:49:21,272 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-12529967) (logid:c7fad834) End cleanup expired async-jobs 2019-10-04 11:49:23,082 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-fa28dc7b) (logid:0abfd9a5) ===START=== 10.20.11.22 -- GET command=listInfrastructure&response=json&_=1570218579507 2019-10-04 11:49:23,087 DEBUG [c.c.a.ApiServer] (qtp504527234-18:ctx-fa28dc7b ctx-0acdd12f) (logid:0abfd9a5) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:23,097 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-fa28dc7b ctx-0acdd12f) (logid:0abfd9a5) ===END=== 10.20.11.22 -- GET command=listInfrastructure&response=json&_=1570218579507 2019-10-04 11:49:24,575 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-1e577188) (logid:4a9558de) ===START=== 10.20.11.22 -- GET command=listHosts&response=json&type=routing&listAll=true&page=1&pagesize=20&_=1570218581056 2019-10-04 11:49:24,581 DEBUG [c.c.a.ApiServer] (qtp504527234-13:ctx-1e577188 ctx-7c50ebc8) (logid:4a9558de) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:24,585 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp504527234-13:ctx-1e577188 ctx-7c50ebc8) (logid:4a9558de) Received unknown parameters for command listHosts. Unknown parameters : listall 2019-10-04 11:49:24,585 DEBUG [c.c.a.q.QueryManagerImpl] (qtp504527234-13:ctx-1e577188 ctx-7c50ebc8) (logid:4a9558de) >>>Searching for hosts>>> 2019-10-04 11:49:24,589 DEBUG [c.c.a.q.QueryManagerImpl] (qtp504527234-13:ctx-1e577188 ctx-7c50ebc8) (logid:4a9558de) >>>Generating Response>>> 2019-10-04 11:49:24,589 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-1e577188 ctx-7c50ebc8) (logid:4a9558de) ===END=== 10.20.11.22 -- GET command=listHosts&response=json&type=routing&listAll=true&page=1&pagesize=20&_=1570218581056 2019-10-04 11:49:26,374 DEBUG [c.c.s.StatsCollector] (StatsCollector-6:ctx-ab32d7be) (logid:292ba088) AutoScaling Monitor is running... 2019-10-04 11:49:26,409 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-cee1dafa) (logid:cbf747f9) StorageCollector is running... 2019-10-04 11:49:26,411 DEBUG [c.c.s.StatsCollector] (StatsCollector-5:ctx-db751f5e) (logid:fdb2b819) HostStatsCollector is running... 2019-10-04 11:49:27,660 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-4fc5887b) (logid:5a0223e0) ===START=== 10.20.11.22 -- GET command=listZones&response=json&_=1570218584145 2019-10-04 11:49:27,665 DEBUG [c.c.a.ApiServer] (qtp504527234-18:ctx-4fc5887b ctx-a3dbb063) (logid:5a0223e0) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:27,676 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-4fc5887b ctx-a3dbb063) (logid:5a0223e0) ===END=== 10.20.11.22 -- GET command=listZones&response=json&_=1570218584145 2019-10-04 11:49:27,695 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-fca1f1a6) (logid:3c210e32) ===START=== 10.20.11.22 -- GET command=listDomains&response=json&listAll=true&details=min&_=1570218584156 2019-10-04 11:49:27,698 DEBUG [c.c.a.ApiServlet] (qtp504527234-289:ctx-fc61d64a) (logid:d097d4f3) ===START=== 10.20.11.22 -- GET command=listHostTags&response=json&_=1570218584163 2019-10-04 11:49:27,699 DEBUG [c.c.a.ApiServer] (qtp504527234-13:ctx-fca1f1a6 ctx-c107034f) (logid:3c210e32) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:27,702 DEBUG [c.c.a.ApiServer] (qtp504527234-289:ctx-fc61d64a ctx-01a3b01c) (logid:d097d4f3) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:27,706 DEBUG [c.c.a.ApiServlet] (qtp504527234-289:ctx-fc61d64a ctx-01a3b01c) (logid:d097d4f3) ===END=== 10.20.11.22 -- GET command=listHostTags&response=json&_=1570218584163 2019-10-04 11:49:27,709 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-fca1f1a6 ctx-c107034f) (logid:3c210e32) ===END=== 10.20.11.22 -- GET command=listDomains&response=json&listAll=true&details=min&_=1570218584156 2019-10-04 11:49:27,714 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-2d0c807e) (logid:dda8fe8e) ===START=== 10.20.11.22 -- GET command=listPods&zoneid=7aa776df-5b3e-41ab-9c0c-d5754d231f70&response=json&_=1570218584196 2019-10-04 11:49:27,718 DEBUG [c.c.a.ApiServer] (qtp504527234-18:ctx-2d0c807e ctx-bf58736a) (logid:dda8fe8e) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:27,724 DEBUG [c.c.a.ApiServlet] (qtp504527234-18:ctx-2d0c807e ctx-bf58736a) (logid:dda8fe8e) ===END=== 10.20.11.22 -- GET command=listPods&zoneid=7aa776df-5b3e-41ab-9c0c-d5754d231f70&response=json&_=1570218584196 2019-10-04 11:49:27,772 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-e4634f65) (logid:7a4ed6ab) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 11:49:27,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-e4b87c58) (logid:213b20f0) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 11:49:27,793 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-19dcffa6) (logid:8379445f) ===START=== 10.20.11.22 -- GET command=listClusters&podid=50e419db-bb71-461f-add6-3d4e45ec3467&response=json&_=1570218584279 2019-10-04 11:49:27,796 DEBUG [c.c.a.ApiServer] (qtp504527234-13:ctx-19dcffa6 ctx-467c56bf) (logid:8379445f) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:49:27,805 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-19dcffa6 ctx-467c56bf) (logid:8379445f) ===END=== 10.20.11.22 -- GET command=listClusters&podid=50e419db-bb71-461f-add6-3d4e45ec3467&response=json&_=1570218584279 2019-10-04 11:49:31,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0f1dfeb8) (logid:4bd93bd8) Begin cleanup expired async-jobs 2019-10-04 11:49:31,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-0f1dfeb8) (logid:4bd93bd8) End cleanup expired async-jobs 2019-10-04 11:49:41,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-39f92880) (logid:4c45fb13) Begin cleanup expired async-jobs 2019-10-04 11:49:41,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-39f92880) (logid:4c45fb13) End cleanup expired async-jobs 2019-10-04 11:49:41,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2de21e5d) (logid:4ecd9bb6) Found 0 routers to update status. 2019-10-04 11:49:41,348 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2de21e5d) (logid:4ecd9bb6) Found 0 VPC networks to update Redundant State. 2019-10-04 11:49:41,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-2de21e5d) (logid:4ecd9bb6) Found 0 networks to update RvR status. 2019-10-04 11:49:46,300 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Resetting hosts suitable for reconnect 2019-10-04 11:49:46,301 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Completed resetting hosts suitable for reconnect 2019-10-04 11:49:46,302 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Acquiring hosts for clusters already owned by this management server 2019-10-04 11:49:46,302 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Completed acquiring hosts for clusters already owned by this management server 2019-10-04 11:49:46,302 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Acquiring hosts for clusters not owned by any management server 2019-10-04 11:49:46,303 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-bb83df01) (logid:d638791c) Completed acquiring hosts for clusters not owned by any management server 2019-10-04 11:49:51,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e7f85ef7) (logid:c711bd46) Begin cleanup expired async-jobs 2019-10-04 11:49:51,271 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-e7f85ef7) (logid:c711bd46) End cleanup expired async-jobs 2019-10-04 11:49:57,771 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-92c0d08c) (logid:2f3a30c3) Zone 1 is not ready to launch secondary storage VM yet 2019-10-04 11:49:57,783 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-9f26c595) (logid:2f313a73) Skip capacity scan as there is no Primary Storage in 'Up' state 2019-10-04 11:50:01,266 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-da068a38) (logid:30d435c7) Begin cleanup expired async-jobs 2019-10-04 11:50:01,270 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-da068a38) (logid:30d435c7) End cleanup expired async-jobs 2019-10-04 11:50:01,809 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-8db1a27e) (logid:5ea85676) ===START=== 10.20.11.22 -- POST command=addHost&response=json 2019-10-04 11:50:01,814 DEBUG [c.c.a.ApiServer] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) CIDRs from which account 'Acct[b908fe58-e6b9-11e9-835d-7427ea50bf96-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2019-10-04 11:50:01,820 WARN [c.c.a.d.ParamGenericValidationWorker] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Received unknown parameters for command addHost. Unknown parameters : clustertype 2019-10-04 11:50:01,822 INFO [c.c.r.ResourceManagerImpl] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Trying to add a new host athttp://10.20.11.253 in data center 1 2019-10-04 11:50:04,829 WARN [c.c.h.k.d.LibvirtServerDiscoverer] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) can't setup agent, due to java.io.IOException: There was a problem while connecting to 10.20.11.253:22 - There was a problem while connecting to 10.20.11.253:22 2019-10-04 11:50:04,829 WARN [c.c.r.ResourceManagerImpl] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Unable to find the server resources athttp://10.20.11.253 2019-10-04 11:50:04,829 INFO [c.c.u.e.CSExceptionErrorCode] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Could not find exception: com.cloud.exception.DiscoveryException in error code list for exceptions 2019-10-04 11:50:04,829 WARN [o.a.c.a.c.a.h.AddHostCmd] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Exception: com.cloud.exception.DiscoveryException: Unable to add the host at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:825) at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:611) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy174.discoverHosts(Unknown Source) at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:732) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:556) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:310) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:127) at com.cloud.api.ApiServlet.doPost(ApiServlet.java:94) at javax.servlet.http.HttpServlet.service(HttpServlet.java:706) at javax.servlet.http.HttpServlet.service(HttpServlet.java:791) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 2019-10-04 11:50:04,831 INFO [c.c.a.ApiServer] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) Unable to add the host 2019-10-04 11:50:04,831 DEBUG [c.c.a.ApiServlet] (qtp504527234-13:ctx-8db1a27e ctx-8d4528ea) (logid:5ea85676) ===END=== 10.20.11.22 -- POST command=addHost&response=json 2019-10-04 11:50:11,265 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-87bcd0b1) (logid:ae570817) Begin cleanup expired async-jobs 2019-10-04 11:50:11,269 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-87bcd0b1) (logid:ae570817) End cleanup expired async-jobs 2019-10-04 11:50:11,300 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-dc2f0451) (logid:2cee066f) Snapshot scheduler.poll is being called at 2019-10-04 15:50:11 GMT 2019-10-04 11:50:11,302 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-dc2f0451) (logid:2cee066f) Got 0 snapshots to be executed at 2019-10-04 15:50:11 GMT 2019-10-04 11:50:11,303 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-1b1883ee) (logid:9e4923e8) External devices are not used. Skipping external device usage collection 2019-10-04 11:50:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-5bdc97fc) (logid:74179713) Found 0 running routers. 2019-10-04 11:50:11,347 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-9f4bfcfa) (logid:4b04ea4f) Found 0 routers to update status. 2019-10-04 11:50:11,349 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-9f4bfcfa) (logid:4b04ea4f) Found 0 VPC networks to update Redundant State. 2019-10-04 11:50:11,350 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-9f4bfcfa) (logid:4b04ea4f) Found 0 networks to update RvR status. 2019-10-04 11:50:11,418 INFO [c.c.r.ResourceLimitManagerImpl] (ResourceCountChecker-1:ctx-06208675) (logid:94f24d7a) Started resource counters recalculation periodic task.
