[ https://issues.apache.org/jira/browse/CLOUDSTACK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sailaja Mada updated CLOUDSTACK-4584: ------------------------------------- Attachment: upgradeissuelogs.rar > [VMWARE]System VM's failed to start if nexus is enabled at cloud level before > upgrade but all the VMWARE clusters are with Standard vSwitch > ------------------------------------------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-4584 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4584 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Network Devices, VMware > Affects Versions: 4.2.1 > Reporter: Sailaja Mada > Fix For: 4.2.1 > > Attachments: upgradeissuelogs.rar > > > Steps: > 1. Configure two zones with VMWARE cluster in 307 Patch B > 2. Create two physical networks (1 for mgmt. 2 for public/guest ) > 3. Deploy VM's on these two zones > 4. Enable nexus flag With no clusters on nexus vSwitch > 5. Upgrade to 4.2 > Observation: > System VM's failed to start if nexus is enabled at cloud level before upgrade > but all the VMWARE clusters are with Standard vSwitch > Root Cause: We are assuming that at cloud level if nexus is enabled then all > clusters switch type should get modified to Nexus only. > With that cluster_details table is updating the Switch type for public/Guest > traffic all the VMWARE clusters in the cloud to nexus type. > mysql> select * from cluster_details; > +----+------------+-----------------------+------------------------------------------+ > | id | cluster_id | name | value > | > +----+------------+-----------------------+------------------------------------------+ > | 1 | 1 | username | administrator > | > | 2 | 1 | password | XZCFHHifMgSGvFarPBx16cwqI9Y9Oadr > | > | 3 | 1 | url | > http://10.102.192.248/307dc1/307c1 | > | 5 | 3 | username | administrator > | > | 6 | 3 | password | 6RJmckhfL31Y9BYv6DuCLZ094BWzE6ed > | > | 7 | 3 | url | > http://10.102.192.248/legacydc1/legacyc1 | > | 9 | 1 | guestvswitchtype | nexusdvs > | > | 10 | 1 | guestvswitchname | vSwitch1 > | > | 11 | 1 | publicvswitchtype | nexusdvs > | > | 12 | 1 | publicvswitchname | vSwitch1 > | > | 13 | 1 | guestvswitchtype | nexusdvs > | > | 14 | 1 | guestvswitchname | vSwitch1 > | > | 15 | 1 | publicvswitchtype | nexusdvs > | > | 16 | 1 | publicvswitchname | vSwitch1 > | > | 17 | 3 | guestvswitchtype | nexusdvs > | > | 18 | 3 | guestvswitchname | vSwitch1 > | > | 19 | 3 | publicvswitchtype | nexusdvs > | > | 20 | 3 | publicvswitchname | vSwitch1 > | > | 21 | 1 | cpuOvercommitRatio | 1 > | > | 22 | 1 | memoryOvercommitRatio | 1 > | > | 23 | 2 | cpuOvercommitRatio | 1 > | > | 24 | 2 | memoryOvercommitRatio | 1 > | > | 25 | 3 | cpuOvercommitRatio | 1 > | > | 26 | 3 | memoryOvercommitRatio | 1 > | > | 33 | 1 | NativeHA | false > | > | 34 | 3 | NativeHA | false > | > +----+------------+-----------------------+------------------------------------------+ > 26 rows in set (0.00 sec) > 2013-08-31 16:10:33,109 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Preparing NIC device on network > cloud.private.untagged.0.1-vSwitch0 > 2013-08-31 16:10:33,109 DEBUG [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare NIC at new device > {"operation":"ADD","device":{"addressType":"Manual","macAddress":"02:00:07:e2:00:02","key":-3,"backing":{"network":{"value":"network-12406","type":"Network"},"deviceName":"cloud.private.untagged.0.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true},"unitNumber":0}} > 2013-08-31 16:10:33,109 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare NIC device based on NicTO: > {"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"b5a549b4-caed-4e1c-a55d-d512fb073ace","ip":"10.102.195.114","netmask":"255.255.252.0","gateway":"10.102.192.1","mac":"06:ca:4e:00:00:05","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"vSwitch0"} > 2013-08-31 16:10:33,113 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare network on vmwaresvs > P[vSwitch0:untagged] with name prefix: cloud.private > 2013-08-31 16:10:33,177 INFO [vmware.mo.HypervisorHostHelper] > (DirectAgent-42:10.102.192.20) Network cloud.private.untagged.0.1-vSwitch0 is > ready on vSwitch vSwitch0 > 2013-08-31 16:10:33,177 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Preparing NIC device on network > cloud.private.untagged.0.1-vSwitch0 > 2013-08-31 16:10:33,177 DEBUG [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare NIC at new device > {"operation":"ADD","device":{"addressType":"Manual","macAddress":"06:ca:4e:00:00:05","key":-4,"backing":{"network":{"value":"network-12406","type":"Network"},"deviceName":"cloud.private.untagged.0.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true},"unitNumber":1}} > 2013-08-31 16:10:33,177 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare NIC device based on NicTO: > {"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"uuid":"66657016-b62e-4061-840b-3515fa33f583","ip":"10.102.197.230","netmask":"255.255.255.0","gateway":"10.102.197.1","mac":"06:04:06:00:00:07","dns1":"10.103.128.15","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://101","isolationUri":"vlan://101","isSecurityGroupEnabled":false,"name":"vSwitch1"} > 2013-08-31 16:10:33,181 INFO [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) Prepare network on nexusdvs > P[vSwitch1:untagged] with name prefix: cloud.public > 2013-08-31 16:10:33,188 ERROR [vmware.mo.HypervisorHostHelper] > (DirectAgent-42:10.102.192.20) Unable to find Ethernet port profile vSwitch1 > 2013-08-31 16:10:33,188 WARN [vmware.resource.VmwareResource] > (DirectAgent-42:10.102.192.20) StartCommand failed due to Exception: > java.lang.Exception > Message: Unable to find Ethernet port profile vSwitch1 > java.lang.Exception: Unable to find Ethernet port profile vSwitch1 > at > com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.prepareNetwork(HypervisorHostHelper.java:567) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.prepareNetworkFromNicInfo(VmwareResource.java:3530) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2896) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:519) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-08-31 16:10:33,190 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-42:null) Seq 6-340918310: Cancelling because one of the answers > is false and it is stop on error. > 2013-08-31 16:10:33,190 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-42:null) Seq 6-340918310: Response Received: > 2013-08-31 16:10:33,191 DEBUG [agent.transport.Request] (DirectAgent-42:null) > Seq 6-340918310: Processing: { Ans: , MgmtId: 227594284004867, via: 6, Ver: > v1, Flags: 110, > [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":7,"name":"v-7-VM","bootloader":"HVM","type":"ConsoleProxy","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"hostName":"v-7-VM","arch":"x86_64","os":"Debian > GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP type=consoleproxy > host=10.102.192.221 port=8250 name=v-7-VM premium=true zone=2 pod=2 > guid=Proxy.7 proxy_vm=7 disable_rp_filter=true eth2ip=10.102.197.230 > eth2mask=255.255.255.0 gateway=10.102.197.1 eth0ip=0.0.0.0 eth0mask=0.0.0.0 > eth1ip=10.102.195.114 eth1mask=255.255.252.0 mgmtcidr=10.102.192.0/22 > localgw=10.102.192.1 internaldns1=10.103.128.15 > dns1=10.103.128.15","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"2857ec9fcb4d655d","params":{"nicAdapter":"E1000","vmware.reserve.cpu":"false","vmware.reserve.mem":"false"},"uuid":"de2ecf56-bb07-451c-9123-4be19fe97778","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"30d602e6-8b76-43f8-a0e4-d41b9ba503fc","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"55e6d095-7e22-3f4a-b533-874a9fede8df","id":202,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/vmwareps1","port":2049}},"name":"ROOT-7","size":0,"path":"ROOT-7","volumeId":20,"vmName":"v-7-VM","accountId":1,"format":"OVA","id":20,"hypervisorType":"VMware"}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"uuid":"66657016-b62e-4061-840b-3515fa33f583","ip":"10.102.197.230","netmask":"255.255.255.0","gateway":"10.102.197.1","mac":"06:04:06:00:00:07","dns1":"10.103.128.15","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://101","isolationUri":"vlan://101","isSecurityGroupEnabled":false,"name":"vSwitch1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"6c7a9d3a-a50f-4fa4-954f-dc57a7da9243","mac":"02:00:07:e2:00:02","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch0"},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"b5a549b4-caed-4e1c-a55d-d512fb073ace","ip":"10.102.195.114","netmask":"255.255.252.0","gateway":"10.102.192.1","mac":"06:ca:4e:00:00:05","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"vSwitch0"}]},"result":false,"details":"StartCommand > failed due to Exception: java.lang.Exception\nMessage: Unable to find > Ethernet port profile vSwitch1\n","wait":0}}] } > WorkAround: > 1. Disable Zone > 2.Update the Switch type to vmwaresvs - Standard Vswitch) > 3 Stop and Start Management server > 4. Enable Zone > After this workaround system VM's got started with no issues -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira