Sailaja Mada created CLOUDSTACK-4414: ----------------------------------------
Summary: [VMWARE] System VM's are failed to start with new Nexus enabled Zone after upgrade (Nexus Details provided are not stored in DB) Key: CLOUDSTACK-4414 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4414 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Network Devices, Upgrade, VMware Affects Versions: 4.2.0 Reporter: Sailaja Mada Priority: Blocker Attachments: alllogs.rar Steps: 1. Upgraded from 3.0.7 to 4.2 ( VMWARE Zone with Standard vSwitch) 2. Enable Nexus global config variable 3. Tried to add new Zone with VMWARE Nexus enabled 4. Added two physical networks ( 1 - Mgmt - vSwitch0,,vmwaresvs 2- Public & guest -> sailajanewpp,,nexusdvs) 5. Provided VSM details while adding cluster Observation: System VM's are failed to start with Nexus enabled Zone saying Nexus details can not be retrieved from DB. 2013-08-20 15:52:15,497 INFO [vmware.resource.VmwareResource] (DirectAgent-345:10.102.192.18) Prepare NIC device based on NicTO: {"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"uuid":"3ad57763-dfe8-467a-b21d-07de6ff64135","ip":"10.102.196.221","netmask":"255.255.255.0","gateway":"10.102.196.1","mac":"06:62:f8:00:00:08","dns1":"10.103.128.15","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":false,"name":"sailajanewpp,,nexusdvs"} 2013-08-20 15:52:15,500 INFO [vmware.resource.VmwareResource] (DirectAgent-345:10.102.192.18) Prepare network on nexusdvs P[sailajanewpp:untagged] with name prefix: cloud.public 2013-08-20 15:52:15,508 INFO [vmware.mo.HypervisorHostHelper] (DirectAgent-345:10.102.192.18) Found Ethernet port profile sailajanewpp 2013-08-20 15:52:15,512 INFO [vmware.mo.HypervisorHostHelper] (DirectAgent-345:10.102.192.18) Port profile cloud.public.100.0.1-sailajanewpp not found. 2013-08-20 15:52:15,512 ERROR [vmware.mo.HypervisorHostHelper] (DirectAgent-345:10.102.192.18) Failed to retrieve required credentials of Nexus VSM from database. 2013-08-20 15:52:15,513 WARN [vmware.resource.VmwareResource] (DirectAgent-345:10.102.192.18) StartCommand failed due to Exception: java.lang.Exception Message: Failed to retrieve required credentials of Nexus VSM from database. java.lang.Exception: Failed to retrieve required credentials of Nexus VSM from database. at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.getValidatedVsmCredentials(HypervisorHostHelper.java:183) at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.createPortProfile(HypervisorHostHelper.java:201) at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.prepareNetwork(HypervisorHostHelper.java:584) at com.cloud.hypervisor.vmware.resource.VmwareResource.prepareNetworkFromNicInfo(VmwareResource.java:3308) at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2904) at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:514) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2013-08-20 15:52:15,515 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-345:null) Seq 10-374145270: Cancelling because one of the answers is false and it is stop on error. 2013-08-20 15:52:15,515 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-345:null) Seq 10-374145270: Response Received: 2013-08-20 15:52:15,516 DEBUG [agent.transport.Request] (DirectAgent-345:null) Seq 10-374145270: Processing: { Ans: , MgmtId: 227594284004867, via: 10, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":30,"name":"v-30-VM","bootloader":"HVM","type":"ConsoleProxy","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"hostName":"v-30-VM","arch":"x86_64","os":"Debian GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP type=consoleproxy host=10.102.192.221 port=8250 name=v-30-VM premium=true zone=4 pod=3 guid=Proxy.30 proxy_vm=30 disable_rp_filter=true eth2ip=10.102.196.221 eth2mask=255.255.255.0 gateway=10.102.196.1 eth0ip=0.0.0.0 eth0mask=0.0.0.0 eth1ip=10.102.195.152 eth1mask=255.255.252.0 mgmtcidr=10.102.192.0/22 localgw=10.102.192.1 internaldns1=10.103.128.15 dns1=10.103.128.15","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"55fb7bf05d9fa2a6","params":{"nicAdapter":"E1000"},"uuid":"778a22ca-438c-46f3-b3c8-620340f0a99c","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1c10799a-e31a-4f86-8868-e58c62867b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"19fde31f-a662-309e-84f7-6a0dec9e5ca3","id":203,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/nexusps1","port":2049}},"name":"ROOT-30","size":0,"path":"ROOT-30","volumeId":107,"vmName":"v-30-VM","accountId":1,"format":"OVA","id":107,"hypervisorType":"VMware"}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"uuid":"3ad57763-dfe8-467a-b21d-07de6ff64135","ip":"10.102.196.221","netmask":"255.255.255.0","gateway":"10.102.196.1","mac":"06:62:f8:00:00:08","dns1":"10.103.128.15","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://100","isolationUri":"vlan://100","isSecurityGroupEnabled":false,"name":"sailajanewpp,,nexusdvs"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"d4940cd6-177f-4de4-9cf9-17ba411881de","mac":"02:00:52:a1:00:02","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false,"name":"vSwitch0,,vmwaresvs"},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"9f6ec612-71c5-4638-a8ed-7547d2e3b2ff","ip":"10.102.195.152","netmask":"255.255.252.0","gateway":"10.102.192.1","mac":"06:2b:a2:00:00:03","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"vSwitch0,,vmwaresvs"}]},"result":false,"details":"StartCommand failed due to Exception: java.lang.Exception\nMessage: Failed to retrieve required credentials of Nexus VSM from database.\n","wait":0}}] } 2013-08-20 15:52:15,517 DEBUG [agent.transport.Request] (consoleproxy-1:null) Seq 10-374145270: Received: { Ans: , MgmtId: 227594284004867, via: 10, Ver: v1, Flags: 10, { StartAnswer } } -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira