Rayees Namathponnan created CLOUDSTACK-5295: -----------------------------------------------
Summary: [Automation] Router deployment failed with null pointer exception, while calling VirtualNetworkApplianceManagerImpl.getVpnCidr Key: CLOUDSTACK-5295 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5295 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Automation Affects Versions: 4.3.0 Reporter: Rayees Namathponnan Priority: Blocker Fix For: 4.3.0 This issue observed in autoamtion environment; many router deployment failed below error 2013-11-27 02:34:02,194 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Sending network shutdown to VirtualRouter 2013-11-27 02:34:02,196 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Stopping router VM[DomainRouter|r-586-QA] 2013-11-27 02:34:02,198 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) VM is already stopped: VM[DomainRouter|r-586-QA] 2013-11-27 02:34:02,200 DEBUG [c.c.a.ApiServlet] (catalina-exec-7:ctx-436ed958) ===START=== 10.223.240.194 -- GET signature=DFJW4CnPHf9BbR%2BTYevb%2FA8%2BZpM%3D&apiKey=1fASWWJnTkiQdjWWd9ex6s pm2-D7xsAQvkXh8vBMHIay-aW6dYeUqWsBoAcK-jkfQKPvBaDJLQDEDra4cfGfaA&command=queryAsyncJobResult&response=json&jobid=cb269b12-6c73-44dd-82aa-1303bf3d33fb 2013-11-27 02:34:02,204 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Network id=489 is shutdown successfully, cleaning up corresponding resources now. 2013-11-27 02:34:02,209 DEBUG [c.c.n.g.GuestNetworkGuru] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Releasing vnet for the network id=489 2013-11-27 02:34:02,221 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Lock is released for network Ntwk[489|Guest|8] as a part of network shutdown 2013-11-27 02:34:02,222 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Lock is released for network id 489 as a part of network implement 2013-11-27 02:34:02,222 INFO [c.c.v.VirtualMachineManagerImpl] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Unable to contact resource. com.cloud.exception.AgentUnavailableException: Resource [Host:2] is unreachable: Host 2: Unable to start instance due to null at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1011) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:706) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2667) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1767) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1867) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1845) at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetwork(NetworkOrchestrator.java:960) at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1222) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:899) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:706) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:552) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3465) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3046) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3032) at sun.reflect.GeneratedMethodAccessor383.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at $Proxy169.startVirtualMachine(Unknown Source) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:449) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161) at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109) at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.NullPointerException at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.getVpnCidr(VirtualNetworkApplianceManagerImpl.java:3277) at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.getVpnCidr(VpcVirtualNetworkApplianceManagerImpl.java:1408) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.createApplyVpnCommands(VirtualNetworkApplianceManagerImpl.java:3287) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2399) at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:897) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2209) at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:719) at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2173) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:918) ... 59 more 2013-11-27 02:34:02,306 DEBUG [c.c.a.ApiServlet] (catalina-exec-7:ctx-436ed958 ctx-de4055c2 ctx-abb8a254) ===END=== 10.223.240.194 -- GET signature=DFJW4CnPHf9BbR%2BTYevb%2FA8%2BZpM%3D&apiKey=1fASWWJnTkiQdjWWd9ex6spm2-D7xsAQvkXh8vBMHIay-aW6dYeUqWsBoAcK-jkfQKPvBaDJLQDEDra4cfGfaA&command=queryAsyncJobResult&response=json&jobid=cb269b12-6c73-44dd-82aa-1303bf3d33fb 2013-11-27 02:34:02,306 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-36:ctx-73116225 ctx-d59c6c54) Cleaning up resources for the vm VM[User|QA-87396a27-9eb5-4bb6-b851-6ad08df25a9a] in Starting state 2013-11-27 02:34:02,309 DEBUG [c -- This message was sent by Atlassian JIRA (v6.1#6144)