venkata swamybabu budumuru created CLOUDSTACK-2115: ------------------------------------------------------
Summary: [BasicZone-XenServer] Unable to add host to basic zone that is configured with bridge Key: CLOUDSTACK-2115 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2115 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: XenServer Affects Versions: 4.2.0 Environment: commit 81e1ba3bb406f9546f0d46ccbfa3fbbb35b762e2 Reporter: venkata swamybabu budumuru Priority: Critical Fix For: 4.2.0 Steps to reproduce: 1. Create a basic zone 2. Add a XenServer 6.1 host to CloudStack Note : before adding, have changed the following - xe-switch-network-backend bridge - update sysctl.conf with the following # Disable *tables rules for bridge traffic to increase performance net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-arptables = 1 - sysctl -p /etc/sysctl.conf Observations: (i) It failed to add the host with the following error 2013-04-19 07:24:25,283 DEBUG [agent.transport.Request] (DirectAgent-80:null) Seq 1-1241186308: Processing: { Ans: , MgmtId: 7280707764394, via: 1, Ver: v1, Flags: 10, [{"ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false,"result":true,"wait":0}}] } 2013-04-19 07:24:25,424 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-5:null) XmlRpcException for method: host.call_plugin due to Failed to create input stream: Read timed out. Reconnecting...retry=1 2013-04-19 07:24:25,424 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-5:null) callHostPlugin failed for cmd: setLinkLocalIP with args brName: xapi1, due to Failed to create input stream: Read timed out 2013-04-19 07:24:25,424 WARN [agent.manager.DirectAgentAttache] (DirectAgent-5:null) Seq 1-1241186311: Exception Caught while executing command com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd: setLinkLocalIP with args brName: xapi1, due to Failed to create input stream: Read timed out at com.cloud.hypervisor.xen.resource.CitrixResourceBase.callHostPlugin(CitrixResourceBase.java:3971) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4651) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3159) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) (ii) [root@Rack1Pod1Host11 log]# !cat cat /etc/xensource/network.conf bridge (iii) Found the following in the SMlog [17790] 2013-04-19 05:58:59.443668 ['cat', '/etc/xensource/network.conf'] [17790] 2013-04-19 05:58:59.451187 pread SUCCESS [17790] 2013-04-19 05:58:59.451275 ['ovs-vsctl', 'add-br', 'xapi1'] (iv) the above output proves that though the network.conf has value "bridge", it is trying to fire OVS commands (v) After further debugging the /etc/xapi.d/plugins/vmops, found that while reading the network.conf, it ended up reading with a "\n" character which is making it picking the ovs commands. Here is the snippet of code : try: cmd = ['cat', '/etc/xensource/network.conf'] result = util.pread2(cmd) except: return 'can not cat network.conf' if result.lower() == "bridge": ===> result.lower() is returning "bridge\n" hence this condition is failing, changing this to result.lower().strip() resolved the issue for me. try: cmd = ["brctl", "addbr", brName] txt = util.pread2(cmd) except: pass else: try: cmd = ["ovs-vsctl", "add-br", brName] txt = util.pread2(cmd) except: pass Attaching the management-server log, api.log, vmops script, SMlog to the bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira