[ https://issues.apache.org/jira/browse/CLOUDSTACK-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15063313#comment-15063313 ]
Andrei Mikhailovsky commented on CLOUDSTACK-8927: ------------------------------------------------- I am experiencing this issue on 3 out of 22 virtual routers. This started happening after upgrading from 4.5.2 to 4.6.2 tonight. > [VPC]Executing command in VR: /opt/cloud/bin/router_proxy.sh is failing > whenever there is a configuration change in VR > ---------------------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-8927 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8927 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Network Controller > Affects Versions: 4.6.0 > Reporter: manasaveloori > Assignee: Wilder Rodrigues > Priority: Blocker > Fix For: 4.6.0 > > Attachments: management-server.rar, management-server.site-site.gz > > > Whenever there is a configuration change in VPC VR observing the connectivity > issues with VR. > Case1: > Created VPC and tier network with default allow. > Now created a new ACL list and rules. Changed the ACL list for the tier > network.Reboot VR > 2015-09-30 04:35:39,553 ERROR [c.c.u.s.SshHelper] > (DirectAgent-336:ctx-b9e5cdf1) SSH execution of command > /opt/cloud/bin/router_proxy.sh update_config.py 169.254.3.89 > guest_network.json has an error status code in return. result output: > 2015-09-30 04:35:39,554 DEBUG [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-336:ctx-b9e5cdf1) Processing ScriptConfigItem, executing > update_config.py guest_network.json took 21165ms > 2015-09-30 04:35:39,554 WARN [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-336:ctx-b9e5cdf1) Expected 1 answers while executing > SetupGuestNetworkCommand but received 2 > 2015-09-30 04:35:45,769 ERROR [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-94:ctx-56b18174 job-227/job-228 ctx-f92247d7) Failed to > start instance VM[DomainRouter|r-22-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[DomainRouter|r-22-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576) > at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at > com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4732) > at > com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at org.apache.cloudstack.managed.context.impl.Def > Case2: > Reboot VR with remote access VPN enabled on VPC VR: > Created VPC ,enabled vpn and rebooted the VR. > ERROR in logs: > 2015-09-30 04:46:18,663 ERROR [c.c.u.s.SshHelper] > (DirectAgent-46:ctx-3c355a22) SSH execution of command > /opt/cloud/bin/router_proxy.sh update_config.py 169.254.0.95 > vpn_user_list.json has an error status code in return. result output: > 2015-09-30 04:46:18,664 DEBUG [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-46:ctx-3c355a22) Processing ScriptConfigItem, executing > update_config.py vpn_user_list.json took 21168ms > 2015-09-30 04:46:18,664 WARN [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-46:ctx-3c355a22) Expected 1 answers while executing > VpnUsersCfgCommand but received 2 > 015-09-30 04:46:24,821 ERROR [c.c.v.VirtualMachineManagerImpl] > (Work-Job-Executor-101:ctx-fecf4919 job-240/job-242 ctx-44fde71b) Failed to > start instance VM[DomainRouter|r-23-VM] > com.cloud.utils.exception.ExecutionException: Unable to start > VM[DomainRouter|r-23-VM] due to error in finalizeStart, not retrying > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083) > at > com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576) > at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) > at com.cloud.vm.VirtualMach > Case3: > Created site to site VPN > Enable vpn on VPCA and then enabling VPN on VPCB is throwing following error: > Observed the following error: > 2015-09-29 12:17:40,600 ERROR [c.c.u.s.SshHelper] > (DirectAgent-392:ctx-d373204a) Timed out in waiting SSH execution result > 2015-09-29 12:17:40,607 DEBUG [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-392:ctx-d373204a) Processing ScriptConfigItem, executing > update_config.py site_2_site_vpn.json took 120148ms > 2015-09-29 12:17:40,607 WARN [c.c.a.r.v.VirtualRoutingResource] > (DirectAgent-392:ctx-d373204a) Expected 1 answers while executing > Site2SiteVpnCfgCommand but received 2 > 2015-09-29 12:17:40,607 DEBUG [c.c.a.m.DirectAgentAttache] > (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212: Response Received: > 2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] > (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212: Processing: { Ans: , > MgmtId: 6877316382886, via: 4, Ver: v1, Flags: 100, > [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success: > ","null - failed: Timed out in waiting SSH execution > result"],"result":false,"wait":0}}] } > 2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] > (API-Job-Executor-94:ctx-cc4f9605 job-194 ctx-8bccbe8f) Seq > 4-50384020831211212: Received: { Ans: , MgmtId: 6877316382886, via: 4, Ver: > v1, Flags: 100, { GroupAnswer } } > 2015-09-29 12:17:40,610 DEBUG [c.c.a.m.AgentAttache] > (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212: No more commands found > 2015-09-29 12:17:40,644 WARN [o.a.c.a.c.u.v.CreateVpnConnectionCmd] > (API-Job-Executor-94:ctx-cc4f9605 job-194 ctx-8bccbe8f) Exception: > com.cloud.exception.ResourceUnavailableException: Resource > [Site2SiteVpnConnection:1] is unreachable: Failed to apply site-to-site VPN > at > com.cloud.network.vpn.Site2SiteVpnManagerImpl.startVpnConnection(Site2SiteVpnManagerImpl.java:345) -- This message was sent by Atlassian JIRA (v6.3.4#6332)