Right. Another user did same thing here on ML (Jevgeni, from 19th March), and had exact same issues (on basic zone) - please check it.
I did advised back then to try to "undelete' the router, by changing a DB/table values in vm_instance table, to mark VR's "state" column back to "Stopped" and set the NULL to the "removed" column for the VR. After that I would try to restart Network (with cleanup) which should trigger proper delete/recreated of the VR - this is how things SHOULD work, but again I'm not using Basic Zone actively... On Fri, 29 Mar 2019 at 12:09, Fariborz Navidan <mdvlinqu...@gmail.com> wrote: > My zone is a basic one. Unfortunately.,It does not work for me It think > router exists for the network but it does not! > > On Fri, Mar 29, 2019 at 3:18 PM Andrija Panic <andrija.pa...@gmail.com> > wrote: > > > what ACS version, what kind of network/zone ? > > > > In 4.11.2, Advanced Zone, Isolated network (KVM), I have deleted router, > > then started a VM, which triggered a creation of new VM. > > > > Again, destroyed VR, restarted network withOUT the "clean up" option, and > > again new VR was created. > > > > On Fri, 29 Mar 2019 at 11:39, Fariborz Navidan <mdvlinqu...@gmail.com> > > wrote: > > > > > It fails to restart networl. Log says it cannot find virtual router. > > Bellow > > > is the log: > > > > > > 2019-03-29 10:53:41,621 DEBUG [c.c.c.ClusterManagerImpl] > > > (Cluster-Heartbeat-1:ctx-277a1d9d) (logid:9902f1d7) Detected management > > > node left, id:8, nodeIP:178.33.230.41 > > > 2019-03-29 10:53:41,621 INFO [c.c.c.ClusterManagerImpl] > > > (Cluster-Heartbeat-1:ctx-277a1d9d) (logid:9902f1d7) Trying to connect > to > > > 178.33.230.41 > > > 2019-03-29 10:53:41,621 INFO [c.c.c.ClusterManagerImpl] > > > (Cluster-Heartbeat-1:ctx-277a1d9d) (logid:9902f1d7) Management node 8 > is > > > detected inactive by timestamp but is pingable > > > 2019-03-29 10:53:42,526 DEBUG [c.c.a.ApiServlet] > > > (qtp788117692-21:ctx-219919fa) (logid:c3be28b0) ===START=== > 2.190.177.97 > > > -- GET > > > > > > > > > command=restartNetwork&response=json&id=6e644551-5aee-4c9b-a75f-134f544ee97c&cleanup=false&makeredundant=false&_=1553853224533 > > > 2019-03-29 10:53:42,528 DEBUG [c.c.a.ApiServer] > > > (qtp788117692-21:ctx-219919fa ctx-3b77ed02) (logid:c3be28b0) CIDRs from > > > which account 'Acct[27cd01ef-3907-11e9-87ab-a4bf012ed1a6-admin]' is > > allowed > > > to perform API calls: 0.0.0.0/0,::/0 > > > 2019-03-29 10:53:42,536 INFO [o.a.c.f.j.i.AsyncJobMonitor] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321) (logid:b95430aa) Add > > job-16321 > > > into job monitoring > > > 2019-03-29 10:53:42,539 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (qtp788117692-21:ctx-219919fa ctx-3b77ed02) (logid:c3be28b0) submit > async > > > job-16321, details: AsyncJobVO {id:16321, userId: 2, accountId: 2, > > > instanceType: None, instanceId: null, cmd: > > > org.apache.cloudstack.api.command.user.network.RestartNetworkCmd, > > cmdInfo: > > > > > > > > > {"makeredundant":"false","cleanup":"false","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"3200","id":"6e644551-5aee-4c9b-a75f-134f544ee97c","ctxDetails":"{\"interface > > > > > > > > > com.cloud.network.Network\":\"6e644551-5aee-4c9b-a75f-134f544ee97c\"}","ctxAccountId":"2","uuid":"6e644551-5aee-4c9b-a75f-134f544ee97c","cmdEventType":"NETWORK.RESTART","_":"1553853224533"}, > > > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, > > > result: null, initMsid: 279278805450993, completeMsid: null, > lastUpdated: > > > null, lastPolled: null, created: null} > > > 2019-03-29 10:53:42,540 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321) (logid:155f349b) Executing > > > AsyncJobVO {id:16321, userId: 2, accountId: 2, instanceType: None, > > > instanceId: null, cmd: > > > org.apache.cloudstack.api.command.user.network.RestartNetworkCmd, > > cmdInfo: > > > > > > > > > {"makeredundant":"false","cleanup":"false","response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventId":"3200","id":"6e644551-5aee-4c9b-a75f-134f544ee97c","ctxDetails":"{\"interface > > > > > > > > > com.cloud.network.Network\":\"6e644551-5aee-4c9b-a75f-134f544ee97c\"}","ctxAccountId":"2","uuid":"6e644551-5aee-4c9b-a75f-134f544ee97c","cmdEventType":"NETWORK.RESTART","_":"1553853224533"}, > > > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, > > > result: null, initMsid: 279278805450993, completeMsid: null, > lastUpdated: > > > null, lastPolled: null, created: null} > > > 2019-03-29 10:53:42,541 DEBUG [c.c.a.ApiServlet] > > > (qtp788117692-21:ctx-219919fa ctx-3b77ed02) (logid:c3be28b0) ===END=== > > > 2.190.177.97 -- GET > > > > > > > > > command=restartNetwork&response=json&id=6e644551-5aee-4c9b-a75f-134f544ee97c&cleanup=false&makeredundant=false&_=1553853224533 > > > 2019-03-29 10:53:42,551 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321 ctx-a30b75bf) > (logid:155f349b) > > > Restarting network 204... > > > 2019-03-29 10:53:42,552 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321 ctx-a30b75bf) > (logid:155f349b) > > > Implementing the network Ntwk[204|Guest|7] elements and resources as a > > part > > > of network restart without cleanup > > > 2019-03-29 10:53:42,554 DEBUG [o.a.c.e.o.NetworkOrchestrator] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321 ctx-a30b75bf) > (logid:155f349b) > > > Asking VirtualRouter to implemenet Ntwk[204|Guest|7] > > > 2019-03-29 10:53:42,558 ERROR [o.c.n.r.d.RouterDeploymentDefinition] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321 ctx-a30b75bf) > (logid:155f349b) > > > Didn't support redundant virtual router without public network! > > > 2019-03-29 10:53:42,560 WARN [o.a.c.e.o.NetworkOrchestrator] > > > (API-Job-Executor-2:ctx-5a2a6bbe job-16321 ctx-a30b75bf) > (logid:155f349b) > > > Failed to implement network Ntwk[204|Guest|7] elements and resources > as a > > > part of network restart due to > > > com.cloud.exception.ResourceUnavailableException: Resource > [DataCenter:1] > > > is unreachable: Can't find all necessary running routers! > > > at > > > > > > > > > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:243) > > > at > > > > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElements(NetworkOrchestrator.java:1203) > > > at > > > > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1138) > > > at > > > > > > > > > org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2862) > > > at > > > > > > > > > com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1883) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:498) > > > at > > > > > > > > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > > at > > > > > > > > > org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174) > > > at > > > > > > > > > com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174) > > > at > > > > > > > > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) > > > at > > > > > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) > > > at > > > > > > > > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > > at com.sun.proxy.$Proxy229.restartNetwork(Unknown Source) > > > at > > > > > > > > > org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:99) > > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150) > > > at > > > > > > com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108) > > > at > > > > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581) > > > at > > > > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > > > at > > > > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > > > at > > > > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > > > at > > > > > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > > > at > > > > > > > > > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > > > at > > > > > > > > > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529) > > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > at java.lang.Thread.run(Thread.java:748) > > > > > > On Fri, Mar 29, 2019 at 11:08 AM Thomas Joseph <thomas.jo...@gmail.com > > > > > wrote: > > > > > > > Restart the network. > > > > > > > > On Thu, 28 Mar 2019, 10:59 pm Fariborz Navidan, < > mdvlinqu...@gmail.com > > > > > > > wrote: > > > > > > > > > Hello, > > > > > I stopped and deleted virtual router to see how it affects the > > changed > > > > > network settings. I tought CS will recreate it automatically but it > > > > didn't > > > > > and now I cannot start new VMs because it complains about > > insufficient > > > > data > > > > > capacity. How do I force CS to re-create the router? > > > > > > > > > > > > > > > > > > -- > > > > Andrija Panić > > > -- Andrija Panić