hstan77 commented on issue #10574:
URL: https://github.com/apache/cloudstack/issues/10574#issuecomment-2725635658
This is detail with debug log
2025-03-14 19:58:24,165 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-50:ctx-8c72f876 job-488650) (logid:345a719c) Add job-488650
into job monitoring
2025-03-14 19:58:24,169 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8844,
power state: PowerOn
2025-03-14 19:58:24,173 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(qtp1886491834-1113:ctx-cdaaea5e ctx-3c16e021) (logid:a4093f5d) submit async
job-488650, details: AsyncJobVO {id:488650, userId: 2, accountId: 2,
instanceType: None, instanceId: null, cmd:
org.apache.cloudstack.api.command.user.network.RestartNetworkCmd, cmdInfo:
{"makeredundant":"false","cleanup":"false","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"3079878","id":"a3cca856-973e-4a6a-aa9d-d4c908938cc7","ctxDetails":"{\"interface
com.cloud.network.Network\":\"a3cca856-973e-4a6a-aa9d-d4c908938cc7\"}","ctxAccountId":"2","uuid":"a3cca856-973e-4a6a-aa9d-d4c908938cc7","cmdEventType":"NETWORK.RESTART"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 2130706689, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null, removed: null}
2025-03-14 19:58:24,175 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8848,
power state: PowerOn
2025-03-14 19:58:24,176 DEBUG [c.c.a.ApiServlet]
(qtp1886491834-1113:ctx-cdaaea5e ctx-3c16e021) (logid:a4093f5d) ===END===
180.75.249.24 -- GET
id=a3cca856-973e-4a6a-aa9d-d4c908938cc7&cleanup=false&makeredundant=false&command=restartNetwork&response=json
2025-03-14 19:58:24,180 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1940,
power state: PowerOn
2025-03-14 19:58:24,181 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-50:ctx-8c72f876 job-488650) (logid:3fef167a) Executing
AsyncJobVO {id:488650, userId: 2, accountId: 2, instanceType: None, instanceId:
null, cmd: org.apache.cloudstack.api.command.user.network.RestartNetworkCmd,
cmdInfo:
{"makeredundant":"false","cleanup":"false","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"3079878","id":"a3cca856-973e-4a6a-aa9d-d4c908938cc7","ctxDetails":"{\"interface
com.cloud.network.Network\":\"a3cca856-973e-4a6a-aa9d-d4c908938cc7\"}","ctxAccountId":"2","uuid":"a3cca856-973e-4a6a-aa9d-d4c908938cc7","cmdEventType":"NETWORK.RESTART"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 2130706689, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null, removed: null}
2025-03-14 19:58:24,185 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 9621,
power state: PowerOn
2025-03-14 19:58:24,191 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8857,
power state: PowerOn
2025-03-14 19:58:24,197 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 9628,
power state: PowerOn
2025-03-14 19:58:24,204 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1181,
power state: PowerOn
2025-03-14 19:58:24,210 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1951,
power state: PowerOn
2025-03-14 19:58:24,215 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1445,
power state: PowerOn
2025-03-14 19:58:24,221 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 9642,
power state: PowerOn
2025-03-14 19:58:24,228 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1196,
power state: PowerOn
2025-03-14 19:58:24,235 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1199,
power state: PowerOn
2025-03-14 19:58:24,242 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 433,
power state: PowerOn
2025-03-14 19:58:24,249 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8627,
power state: PowerOn
2025-03-14 19:58:24,253 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Restarting network 217...
2025-03-14 19:58:24,255 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 7352,
power state: PowerOn
2025-03-14 19:58:24,258 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Implementing the network Ntwk[217|Guest|6] elements and resources as a part of
network restart without cleanup
2025-03-14 19:58:24,260 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1209,
power state: PowerOn
2025-03-14 19:58:24,266 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 1212,
power state: PowerOn
2025-03-14 19:58:24,271 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 7358,
power state: PowerOn
2025-03-14 19:58:24,277 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8640,
power state: PowerOn
2025-03-14 19:58:24,283 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 960,
power state: PowerOn
2025-03-14 19:58:24,289 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 7620,
power state: PowerOn
2025-03-14 19:58:24,295 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 965,
power state: PowerOn
2025-03-14 19:58:24,300 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 967,
power state: PowerOn
2025-03-14 19:58:24,306 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 970,
power state: PowerOn
2025-03-14 19:58:24,311 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 5834,
power state: PowerOn
2025-03-14 19:58:24,317 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 8139,
power state: PowerOn
2025-03-14 19:58:24,323 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 2000,
power state: PowerOn
2025-03-14 19:58:24,325 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Asking SecurityGroupProvider to implemenet Ntwk[217|Guest|6]
2025-03-14 19:58:24,328 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 980,
power state: PowerOn
2025-03-14 19:58:24,333 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 981,
power state: PowerOn
2025-03-14 19:58:24,339 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 983,
power state: PowerOn
2025-03-14 19:58:24,345 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 2007,
power state: PowerOn
2025-03-14 19:58:24,351 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 984,
power state: PowerOn
2025-03-14 19:58:24,356 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 6106,
power state: PowerOn
2025-03-14 19:58:24,361 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 2012,
power state: PowerOn
2025-03-14 19:58:24,367 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 991,
power state: PowerOn
2025-03-14 19:58:24,372 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 2017,
power state: PowerOn
2025-03-14 19:58:24,375 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Asking VirtualRouter to implemenet Ntwk[217|Guest|6]
2025-03-14 19:58:24,378 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 2019,
power state: PowerOn
2025-03-14 19:58:24,384 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 996,
power state: PowerOn
2025-03-14 19:58:24,388 ERROR [o.c.n.r.d.RouterDeploymentDefinition]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Didn't support redundant virtual router without public network!
2025-03-14 19:58:24,389 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 7142,
power state: PowerOn
2025-03-14 19:58:24,395 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-12:null) (logid:) VM state report. host: 19, vm id: 7147,
power state: PowerOn
2025-03-14 19:58:24,395 WARN [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Failed to implement network Ntwk[217|Guest|6] elements and resources as a part
of network restart due to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is
unreachable: Can't find all necessary running routers!
at
com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:251)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElements(NetworkOrchestrator.java:1392)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1327)
at
org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:3166)
at
com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1994)
at
com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:2012)
at jdk.internal.reflect.GeneratedMethodAccessor716.invoke(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy256.restartNetwork(Unknown Source)
at
org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:99)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2025-03-14 19:58:24,397 WARN [c.c.n.NetworkServiceImpl]
(API-Job-Executor-50:ctx-8c72f876 job-488650 ctx-e17cb53d) (logid:3fef167a)
Network id=217 failed to restart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
