[
https://issues.apache.org/jira/browse/CLOUDSTACK-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745520#comment-13745520
]
ASF subversion and git services commented on CLOUDSTACK-4376:
-------------------------------------------------------------
Commit d7c49e485a02d9b03de4bca10cc150fdc34d0407 in branch refs/heads/4.2 from
[~kelveny]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=d7c49e4 ]
CLOUDSTACK-4376: fix cloud-early-config to workaround vSwitch stall in VPC
mode. fix potential NPE in DirectAgentAttache in processing answers from
resource
> [Automation] [vmware] Failed to deploy VPC router in parallel deployment
> scenario
> ---------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4376
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4376
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: API, Automation
> Affects Versions: 4.2.0
> Environment: Vmware
> 4.2 build
> Automation enviroenment
> Reporter: Rayees Namathponnan
> Assignee: Kelven Yang
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: CLOUDSTACK-4376.rar, management-server.zip
>
>
> In Automation environment VPC test cases failed during VPC router deployment;
> we are running 5 tests in parallel
> Attached log for more information; here the example for router " r-278",
> In below log, after creating router r-278, trying ssh connection, its failed
> 2013-08-16 07:57:43,108 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Asking
> BareMetalUserdata to prepare for
> Nic[536-278-4893da91-fa2d-4675-ad74-25c1d83f9ce5-10.223.250.170]
> 2013-08-16 07:57:43,110 DEBUG [cloud.storage.VolumeManagerImpl]
> (Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Checking
> if we need to prepare 1 volumes f
> or VM[DomainRouter|r-278-TestVM]
> 2013-08-16 07:57:43,165 DEBUG [agent.transport.Request]
> (Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Seq
> 3-806750713: Executing: { Cmd , MgmtId: 90928106758026, via: 3, Ver: v1,
> Flags: 100011,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"772bc80f5a773aafa713c41017b167e4","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova","uuid":"4dff74ac-0633-11e3-a733-52b2d980df8a","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM
> Template
> (vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"routing-8","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"95cb6848-a917-4012-b1e0-faceba7cb374","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"ROOT-278","size":2097152000,"volumeId":316,"vmName":"r-278-TestVM","accountId":183,"format":"OVA","id":316,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}]
> }
> 2013-08-16 07:57:43,167 DEBUG [cloud.api.ApiServlet] (catalina-exec-7:null)
> ===END=== 10.223.240.195 -- GET
> signature=dabpu%2BmE%2B3ToMpP%2BHLlBT0IFPEI%3D&apiKey=i4-XPSPRUUp7UQhz7PkZ7AsThtQ1d4LTEqLlFgg8yQxdtPaekrzmnPfjMyMUoCsd_AUFYGSzr7AqZAji5kNR1w&command=queryAsyncJobResult&response=json&jobid=fb6335e4-f30d-49eb-8b66-a5fd684363dd
> 2013-08-16 07:57:43,167 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-311:null) Seq 3-806750713: Executing request
> 2013-08-16 07:58:05,152 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-71:job-908
> = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Boot Args for
> VM[DomainRouter|r-278-TestVM]: vpccidr=10.0.0.1/24 domain=csb7auto.advanced
> dns1=8.8.8.8 template=domP name=r-278-TestVM eth0ip=10.223.250.170
> eth0mask=255.255.255.192 mgmtcidr=10.223.49.192/26 localgw=10.223.250.129
> type=vpcrouter disable_rp_filter=true extra_pubnics=2
> 2013-08-16 07:58:05,214 DEBUG
> [network.router.VpcVirtualNetworkApplianceManagerImpl]
> (Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Found 0
> static routes to apply as a part of vpc route VM[DomainRouter|r-278-TestVM]
> start
> 2013-08-16 08:22:35,131 INFO [storage.resource.VmwareStorageLayoutHelper]
> (DirectAgent-61:10.223.250.130) sync [16c8ee2e3f2630628878b7d6c53c5797]
> ROOT-278-delta.vmdk->[16c8ee
> 2e3f2630628878b7d6c53c5797] r-278-TestVM/ROOT-278-delta.vmdk
> 2013-08-16 08:22:35,383 INFO [storage.resource.VmwareStorageLayoutHelper]
> (DirectAgent-61:10.223.250.130) sync [16c8ee2e3f2630628878b7d6c53c5797]
> ROOT-278.vmdk->[16c8ee2e3f26
> 30628878b7d6c53c5797] r-278-TestVM/ROOT-278.vmdk
> 2013-08-16 08:22:35,654 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Prepare volume at new device
> {"capacityInKB":0,"key":-2,"backing":{"diskMode":"p
> ersistent","fileName":"[16c8ee2e3f2630628878b7d6c53c5797]
> r-278-TestVM/ROOT-278.vmdk","datastore":{"value":"datastore-26","type":"Datastore"}},"connectable":{"startConnected":
> true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":0}
> 2013-08-16 08:22:35,654 INFO [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Prepare NIC device based on NicTO:
> {"deviceId":0,"networkRateMbps":-1,"defaultNi
> c":false,"uuid":"1313c17c-eabe-4fb4-b672-e7d3f10713b4","ip":"10.223.250.170","netmask":"255.255.255.192","gateway":"10.223.250.129","mac":"02:00:75:9b:00:8d","broadcastType":"
> Native","type":"Control","isSecurityGroupEnabled":false}
> 2013-08-16 08:22:35,658 INFO [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Prepare network on vmwaresvs P[null:untagged]
> with name prefix: cloud.private
> 2013-08-16 08:22:35,658 INFO [vmware.mo.HypervisorHostHelper]
> (DirectAgent-61:10.223.250.130) Detected vswitch name as undefined.
> Defaulting to vSwitch0
> 2013-08-16 08:22:35,749 INFO [vmware.mo.HypervisorHostHelper]
> (DirectAgent-61:10.223.250.130) Network cloud.private.untagged.0.1-vSwitch0
> is ready on vSwitch vSwitch0
> 2013-08-16 08:22:35,749 INFO [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Preparing NIC device on network
> cloud.private.untagged.0.1-vSwitch0
> 2013-08-16 08:22:35,749 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Prepare NIC at new device
> {"operation":"ADD","device":{"addressType":"Manual","m
> acAddress":"02:00:75:9b:00:8d","key":-3,"backing":{"network":{"value":"network-24","type":"Network"},"deviceName":"cloud.private.untagged.0.1-vSwitch0"},"connectable":{"startC
> onnected":true,"allowGuestControl":true,"connected":true},"unitNumber":0}}
> 2013-08-16 08:22:35,749 DEBUG [vmware.mo.HostMO]
> (DirectAgent-61:10.223.250.130) find VM r-278-TestVM on host
> 2013-08-16 08:22:35,749 DEBUG [vmware.mo.HostMO]
> (DirectAgent-61:10.223.250.130) VM r-278-TestVM found in host cache
> 2013-08-16 08:22:35,788 INFO [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Configure VNC port for VM r-278-TestVM, port:
> 5950, host: 10.223.250.130
> 2013-08-16 08:48:32,246 ERROR [vmware.resource.VmwareResource]
> (DirectAgent-61:10.223.250.130) Can not ping System vm r-278-TestVMdue
> to:Unable to connect
> 2013-08-16 08:48:32,246 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-61:null) Seq 3-806750716: Cancelling because one of the answers
> is false and it is stop on error.
> 2013-08-16 08:48:32,246 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-61:null) Seq 3-806750716: Response Received:
> 2013-08-16 08:48:32,247 DEBUG [agent.transport.Request] (DirectAgent-61:null)
> Seq 3-806750716: Processing: { Ans: , MgmtId: 90928106758026, via: 3, Ver:
> v1, Flags: 110,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":278,"name":"r-278-TestVM","bootloader":"HVM","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"hostName":"r-278-TestVM","arch":"i686","os":"Debian
> GNU/Linux 5.0 (32-bit)","bootArgs":" vpccidr=10.0.0.1/24
> domain=csb7auto.advanced dns1=8.8.8.8 template=domP name=r-278-TestVM
> eth0ip=10.223.250.170 eth0mask=255.255.255.192 mgmtcidr=10.223.49.192/26
> localgw=10.223.250.129 type=vpcrouter disable_rp_filter=true extra_pubnics=2
> nic_macs=02:00:75:9b:00:8d","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"4af860eb17e29895","params":{"nicAdapter":"E1000"},"uuid":"881e4c0b-264d-40ea-8859-a4195f67ca11","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"95cb6848-a917-4012-b1e0-faceba7cb374","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"ROOT-278","size":2097152000,"path":"ROOT-278","volumeId":316,"vmName":"r-278-TestVM","accountId":183,"format":"OVA","id":316,"hypervisorType":"VMware"}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"1313c17c-eabe-4fb4-b672-e7d3f10713b4","ip":"10.223.250.170","netmask":"255.255.255.192","gateway":"10.223.250.129","mac":"02:00:75:9b:00:8d","broadcastType":"Native","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":false,"details":"Can
> not ping System vm r-278-TestVMdue to:Unable to connect","wait":0}}] }
> 2013-08-16 08:48:32,261 WARN
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-71:job-908
> = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Unable to ssh to the VM: Can not
> ping System vm r-278-TestVMdue to:Unable to connect
> 2013-08-16 08:48:32,262 INFO [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) The guru
> did not like the answers so stopping VM[DomainRouter|r-278-TestVM]
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
