Can you check if agent.properties on kvm hosts is correct ? the value of "host" should be the ip of management server.
- Wei On Tuesday, 3 January 2023, Mevludin Blazevic <[email protected]> wrote: > Cloudstack Management Server is reachable by the Webbrowser, it seems > everything is worked except port 8250 (or other).. > > Am 03.01.2023 um 18:56 schrieb Mevludin Blazevic: > >> Hi, >> >> what I currently get is one of these Error messages: >> >> 2023-01-03 17:52:14,669 ERROR [c.c.h.KVMFencer] (HA-Worker-0:ctx-b5398a32 >> work-12311) (logid:e87b21e4) Unable to fence off VM instance {id: "419", >> name: "i-4-419-VM", uuid: "3277af4d-c460-4bc3-83a0-0af3469d5f32", >> type="User"} on Host {"id": "40", "name": "sparci-cpu14", "uuid": >> "3774ecc7-68f2-4f3c-a0ad-4ccf3eb55e12", "type"="Routing"} >> 2023-01-03 17:52:42,024 ERROR [c.c.s.StatsCollector] >> (StatsCollector-2:ctx-3c5241a0) (logid:bd04d7c3) db statistics >> collection failed due to For input string: "3727527314" >> 2023-01-03 17:53:14,677 ERROR [c.c.h.KVMFencer] (HA-Worker-4:ctx-2cb2fd7f >> work-12740) (logid:d6240bca) Unable to fence off VM instance {id: "199", >> name: "i-9-199-VM", uuid: "f5609614-5aa9-4cc6-a0b0-7b232bdae35b", >> type="User"} on Host {"id": "35", "name": "sparci-cpu12", "uuid": >> "3c1eff13-7aa1-427d-90bd-fd46700cb29e", "type"="Routing"} >> 2023-01-03 17:53:14,681 ERROR [c.c.h.KVMFencer] (HA-Worker-2:ctx-5686f4cc >> work-12741) (logid:098ccc6d) Unable to fence off VM instance {id: "458", >> name: "i-35-458-VM", uuid: "6a439b25-58f3-4732-b138-2f547e70d0f2", >> type="User"} on Host {"id": "35", "name": "sparci-cpu12", "uuid": >> "3c1eff13-7aa1-427d-90bd-fd46700cb29e", "type"="Routing"} >> 2023-01-03 17:53:42,021 ERROR [c.c.s.StatsCollector] >> (StatsCollector-3:ctx-4fb6b9be) (logid:8632364f) db statistics >> collection failed due to For input string: "3727543779" >> 2023-01-03 17:54:14,684 ERROR [c.c.h.KVMFencer] (HA-Worker-3:ctx-3668f04d >> work-12839) (logid:f386015e) Unable to fence off VM instance {id: "199", >> name: "i-9-199-VM", uuid: "f5609614-5aa9-4cc6-a0b0-7b232bdae35b", >> type="User"} on Host {"id": "35", "name": "sparci-cpu12", "uuid": >> "3c1eff13-7aa1-427d-90bd-fd46700cb29e", "type"="Routing"} >> 2023-01-03 17:54:42,022 ERROR [c.c.s.StatsCollector] >> (StatsCollector-1:ctx-8a742ef7) (logid:a2f5ccd9) db statistics >> collection failed due to For input string: "3727558372" >> 2023-01-03 17:55:14,692 ERROR [c.c.h.KVMFencer] (HA-Worker-1:ctx-37927b46 >> work-7473) (logid:5cc55a35) Unable to fence off VM instance {id: "199", >> name: "i-9-199-VM", uuid: "f5609614-5aa9-4cc6-a0b0-7b232bdae35b", >> type="User"} on Host {"id": "35", "name": "sparci-cpu12", "uuid": >> "3c1eff13-7aa1-427d-90bd-fd46700cb29e", "type"="Routing"} >> 2023-01-03 17:55:14,695 ERROR [c.c.h.KVMFencer] (HA-Worker-0:ctx-f59d96cb >> work-7474) (logid:c22a9177) Unable to fence off VM instance {id: "458", >> name: "i-35-458-VM", uuid: "6a439b25-58f3-4732-b138-2f547e70d0f2", >> type="User"} on Host {"id": "35", "name": "sparci-cpu12", "uuid": >> "3c1eff13-7aa1-427d-90bd-fd46700cb29e", "type"="Routing"} >> >> Due to the host alert problem, all system VMs are disconnected except the >> VR which stays in running state but "requires update". However I cannot >> migrate any System VMs. >> >> Example output of one of the failing hosts: >> >> 2023-01-03 18:54:39,741 WARN [utils.nio.NioConnection] >> (Agent-Handler-3:null) (logid:) Unable to connect to remote: is there a >> server running on port 8250 >> 2023-01-03 18:54:39,741 ERROR [utils.nio.NioConnection] >> (Agent-Handler-4:null) (logid:) Unable to initialize the threads. >> java.nio.channels.AsynchronousCloseException >> at java.base/java.nio.channels.spi.AbstractInterruptibleChannel >> .end(AbstractInterruptibleChannel.java:202) >> at java.base/sun.nio.ch.SocketChannelImpl.endRead(SocketChannel >> Impl.java:331) >> at java.base/sun.nio.ch.SocketChannelImpl.endConnect(SocketChan >> nelImpl.java:659) >> at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannel >> Impl.java:695) >> at com.cloud.utils.nio.NioClient.init(NioClient.java:56) >> at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95) >> at com.cloud.agent.Agent.reconnect(Agent.java:535) >> at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1118) >> at com.cloud.utils.nio.Task.call(Task.java:83) >> at com.cloud.utils.nio.Task.call(Task.java:29) >> at java.base/java.util.concurrent.FutureTask.run(FutureTask. >> java:264) >> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1128) >> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker. >> run(ThreadPoolExecutor.java:628) >> at java.base/java.lang.Thread.run(Thread.java:829) >> 2023-01-03 18:54:39,741 WARN [utils.nio.NioConnection] >> (Agent-Handler-5:null) (logid:) Unable to connect to remote: is there a >> server running on port 8250 >> 2023-01-03 18:54:39,741 ERROR [utils.nio.NioConnection] >> (Agent-Handler-3:null) (logid:) Unable to initialize the threads. >> java.nio.channels.AsynchronousCloseException >> at java.base/java.nio.channels.spi.AbstractInterruptibleChannel >> .end(AbstractInterruptibleChannel.java:202) >> at java.base/sun.nio.ch.SocketChannelImpl.endRead(SocketChannel >> Impl.java:331) >> at java.base/sun.nio.ch.SocketChannelImpl.endConnect(SocketChan >> nelImpl.java:659) >> at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannel >> Impl.java:695) >> at com.cloud.utils.nio.NioClient.init(NioClient.java:56) >> at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95) >> at com.cloud.agent.Agent.reconnect(Agent.java:535) >> at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1118) >> at com.cloud.utils.nio.Task.call(Task.java:83) >> at com.cloud.utils.nio.Task.call(Task.java:29) >> at java.base/java.util.concurrent.FutureTask.run(FutureTask. >> java:264) >> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1128) >> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker. >> run(ThreadPoolExecutor.java:628) >> at java.base/java.lang.Thread.run(Thread.java:829) >> 2023-01-03 18:54:39,741 ERROR [utils.nio.NioConnection] >> (Agent-Handler-4:null) (logid:) Unable to initialize the threads. >> java.nio.channels.AsynchronousCloseException >> at java.base/java.nio.channels.spi.AbstractInterruptibleChannel >> .end(AbstractInterruptibleChannel.java:202) >> at java.base/sun.nio.ch.SocketChannelImpl.endRead(SocketChannel >> Impl.java:331) >> at java.base/sun.nio.ch.SocketChannelImpl.endConnect(SocketChan >> nelImpl.java:659) >> at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannel >> Impl.java:695) >> at com.cloud.utils.nio.NioClient.init(NioClient.java:56) >> at com.cloud.utils.nio.NioConnection.start(NioConnection.java:95) >> at com.cloud.agent.Agent.reconnect(Agent.java:535) >> at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1118) >> at com.cloud.utils.nio.Task.call(Task.java:83) >> at com.cloud.utils.nio.Task.call(Task.java:29) >> at java.base/java.util.concurrent.FutureTask.run(FutureTask. >> java:264) >> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1128) >> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker. >> run(ThreadPoolExecutor.java:628) >> at java.base/java.lang.Thread.run(Thread.java:829) >> 2023-01-03 18:54:39,741 WARN [utils.nio.NioConnection] >> (Agent-Handler-4:null) (logid:) Unable to connect to remote: is there a >> server running on port 8250 >> 2023-01-03 18:54:39,741 WARN [utils.nio.NioConnection] >> (Agent-Handler-4:null) (logid:) Unable to connect to remote: is there a >> server running on port 8250 >> >> Regards, >> >> Mevludin >> >> Am 03.01.2023 um 17:11 schrieb Wei ZHOU: >> >>> Hi, >>> >>> Is the management service working ? Are you able to access the GUI ? >>> Can you share more management server logs ? >>> >>> -Wei >>> >>> >>> On Tuesday, 3 January 2023, Mevludin Blazevic <[email protected]> >>> wrote: >>> >>> Hi all, >>>> >>>> after upgrading to ACS 4.17.2, the Management Server Port 8250 is not >>>> reachable by the hosts anymore. All hosts are in an Alert >>>> <https://www.google.com/maps/search/reachable+by+the+hosts+anymore.+All+hosts+are+in+an+Alert+?entry=gmail&source=g>state. >>>> Is someone >>>> facing the same problem? Nothing was changer, neither sshd_config nor >>>> the >>>> underlying OS.. >>>> >>>> Regards >>>> >>>> Mevludin >>>> >>>> >>>> >>> -- > Mevludin Blazevic, M.Sc. > > University of Koblenz-Landau > Computing Centre (GHRKO) > Universitaetsstrasse 1 > D-56070 Koblenz, Germany > Room A023 > Tel: +49 261/287-1326 > >
