Chandan Purushothama created CLOUDSTACK-7743: ------------------------------------------------
Summary: [Automation][KVM] Migration of VM failed due to LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device" Key: CLOUDSTACK-7743 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7743 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Automation, KVM Affects Versions: 4.5.0 Reporter: Chandan Purushothama Assignee: edison su Priority: Critical Fix For: 4.5.0 Use Case: 1. VM Migration from One Host to Another - Fails currently due to LibvirtException ============= LibvirtException: ============= {noformat} 2014-10-13 19:26:36,930 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 1-8599060538510540956: Sending { Cmd , MgmtId: 86095456037899, via: 1(cl07-11), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":32,"name":"i-13-32-VM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"CentOS 5.5 (64-bit)","platformEmulator":"CentOS 5.5","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"UgJEzSDcfsSllX2nVfeMqQ==","params":{},"uuid":"fe6ef18e-37da-4730-9794-03760f325082","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"c031588c-51be-4be8-9c30-4535b3428d2f","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3ce1672-a3c4-3396-a86b-025c13ee2b8c","id":1,"poolType":"NetworkFilesystem","host":"10.220.128.55","path":"/vol/xenrtnfs/861825-jkTv6e","port":2049,"url":"NetworkFilesystem://10.220.128.55/vol/xenrtnfs/861825-jkTv6e/?ROLE=Primary&STOREUUID=e3ce1672-a3c4-3396-a86b-025c13ee2b8c"}},"name":"ROOT-32","size":8589934592,"path":"c031588c-51be-4be8-9c30-4535b3428d2f","volumeId":35,"vmName":"i-13-32-VM","accountId":13,"format":"QCOW2","provisioningType":"THIN","id":35,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"c031588c-51be-4be8-9c30-4535b3428d2f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.220.128.55","volumeSize":"8589934592"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"47696919-1d62-416f-9f2c-1bfa72ef2330","uuid":"85218f30-0f90-4d63-8553-b8431cdf60aa","ip":"192.168.200.220","netmask":"255.255.255.0","gateway":"192.168.200.1","mac":"02:00:7a:45:00:05","dns1":"10.220.128.11","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://3011","isolationUri":"vlan://3011","isSecurityGroupEnabled":false}]},"wait":0}}] } 2014-10-13 19:26:36,993 DEBUG [c.c.a.t.Request] (AgentManager-Handler-19:null) Seq 1-8599060538510540956: Processing: { Ans: , MgmtId: 86095456037899, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.PrepareForMigrationAnswer":{"result":true,"wait":0}}] } 2014-10-13 19:26:36,993 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-19:null) Seq 1-8599060538510540956: No more commands found 2014-10-13 19:26:36,993 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 1-8599060538510540956: Received: { Ans: , MgmtId: 86095456037899, via: 1, Ver: v1, Flags: 110, { PrepareForMigrationAnswer } } 2014-10-13 19:26:37,000 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original host id: 2 new host id: 1 host id before state transition: 2 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Hosts's actual total CPU: 38400 and CPU after applying overprovisioning: 38400 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) We are allocating VM, increasing the used capacity of this host:1 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Current Used CPU: 2500 , Free CPU:35900 ,Requested CPU: 100 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Current Used RAM: 2818572288 , Free RAM:13891837952 ,Requested RAM: 268435456 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) CPU STATS after allocation: for host: 1, old used: 2500, old reserved: 0, actual total: 38400, total with overprovisioning: 38400; new used:2600, reserved:0; requested cpu:100,alloc_from_last:false 2014-10-13 19:26:37,005 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) RAM STATS after allocation: for host: 1, old used: 2818572288, old reserved: 0, total: 16710410240; new used: 3087007744, reserved: 0; requested mem: 268435456,alloc_from_last:false 2014-10-13 19:26:37,013 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 2-7200411379235291376: Sending { Cmd , MgmtId: 86095456037899, via: 2(cl07-09), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-13-32-VM","destIp":"10.220.131.110","hostGuid":"7d1db5c7-3c83-35a7-a5ec-a750c01f2ea1-LibvirtComputingResource","isWindows":false,"vmTO":{"id":32,"name":"i-13-32-VM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"CentOS 5.5 (64-bit)","platformEmulator":"CentOS 5.5","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"UgJEzSDcfsSllX2nVfeMqQ==","params":{},"uuid":"fe6ef18e-37da-4730-9794-03760f325082","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"c031588c-51be-4be8-9c30-4535b3428d2f","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3ce1672-a3c4-3396-a86b-025c13ee2b8c","id":1,"poolType":"NetworkFilesystem","host":"10.220.128.55","path":"/vol/xenrtnfs/861825-jkTv6e","port":2049,"url":"NetworkFilesystem://10.220.128.55/vol/xenrtnfs/861825-jkTv6e/?ROLE=Primary&STOREUUID=e3ce1672-a3c4-3396-a86b-025c13ee2b8c"}},"name":"ROOT-32","size":8589934592,"path":"c031588c-51be-4be8-9c30-4535b3428d2f","volumeId":35,"vmName":"i-13-32-VM","accountId":13,"format":"QCOW2","provisioningType":"THIN","id":35,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"c031588c-51be-4be8-9c30-4535b3428d2f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.220.128.55","volumeSize":"8589934592"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"47696919-1d62-416f-9f2c-1bfa72ef2330","uuid":"85218f30-0f90-4d63-8553-b8431cdf60aa","ip":"192.168.200.220","netmask":"255.255.255.0","gateway":"192.168.200.1","mac":"02:00:7a:45:00:05","dns1":"10.220.128.11","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://3011","isolationUri":"vlan://3011","isSecurityGroupEnabled":false}]},"executeInSequence":false,"wait":0}}] } 2014-10-13 19:26:37,545 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) Seq 2-7200411379235291376: Processing: { Ans: , MgmtId: 86095456037899, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"org.libvirt.LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device","wait":0}}] } 2014-10-13 19:26:37,545 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 2-7200411379235291376: Received: { Ans: , MgmtId: 86095456037899, via: 2, Ver: v1, Flags: 10, { MigrateAnswer } } 2014-10-13 19:26:37,545 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Migration was unsuccessful. Cleaning up: VM[User|i-13-32-VM] 2014-10-13 19:26:37,551 WARN [o.a.c.alerts] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) alertType:: 17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm i-13-32-VM from host cl07-09 in zone XenRT-Zone-0 and pod XenRT-Zone-0-Pod-0 2014-10-13 19:26:37,557 DEBUG [c.c.a.m.AgentAttache] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 1-8599060538510540957: Routed from 86095456037899 2014-10-13 19:26:37,558 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Seq 1-8599060538510540957: Sending { Cmd , MgmtId: 86095456037899, via: 1(cl07-11), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-13-32-VM","wait":0}}] } 2014-10-13 19:26:37,562 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) VM state transitted from :Migrating to Running with event: OperationFailedvm's original host id: 2 new host id: 2 host id before state transition: 1 2014-10-13 19:26:37,568 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Hosts's actual total CPU: 38400 and CPU after applying overprovisioning: 38400 2014-10-13 19:26:37,568 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Hosts's actual total RAM: 16710410240 and RAM after applying overprovisioning: 16710410240 2014-10-13 19:26:37,568 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) release cpu from host: 1, old used: 2600,reserved: 0, actual total: 38400, total with overprovisioning: 38400; new used: 2500,reserved:0; movedfromreserved: false,moveToReserveredfalse 2014-10-13 19:26:37,568 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) release mem from host: 1, old used: 3087007744,reserved: 0, total: 16710410240; new used: 2818572288,reserved:0; movedfromreserved: false,moveToReserveredfalse 2014-10-13 19:26:37,573 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to org.libvirt.LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device 2014-10-13 19:26:37,573 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198 ctx-066f6b6a) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to org.libvirt.LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device 2014-10-13 19:26:37,573 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Done with run of VM work job: com.cloud.vm.VmWorkMigrate for VM 32, job origin: 197 2014-10-13 19:26:37,573 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Unable to complete AsyncJobVO {id:198, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTGphdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAIHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAnNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXEAfgAJcQB-AAlwcQB-AAk, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 86095456037899, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Oct 13 19:26:36 UTC 2014}, job origin:197 com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to org.libvirt.LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device at com.cloud.vm.VirtualMachineManagerImpl.migrate(VirtualMachineManagerImpl.java:1907) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrate(VirtualMachineManagerImpl.java:1807) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrate(VirtualMachineManagerImpl.java:4614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4746) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:513) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:470) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 2014-10-13 19:26:37,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Complete async job-198, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QAjEpvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gbWlncmF0ZSBkdWUgdG8gb3JnLmxpYnZpcnQuTGlidmlydEV4Y2VwdGlvbjogQ2Fubm90IGdldCBpbnRlcmZhY2UgTVRVIG9uICdicmV0aDAtMzAxMSc6IE5vIHN1Y2ggZGV2aWNldXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAAOc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAESQAKbGluZU51bWJlckwADmRlY2xhcmluZ0NsYXNzcQB-AARMAAhmaWxlTmFtZXEAfgAETAAKbWV0aG9kTmFtZXEAfgAEeHAAAABydAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-AAsAAAIBdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgALAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgALAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4ACwAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ABp0AA9jYWxsV2l0aENvbnRleHRzcQB-AAsAAAA1cQB-AB1xAH4AGnQADnJ1bldpdGhDb250ZXh0c3EAfgALAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAWcQB-ABdzcQB-AAsAAAHWcQB-ABFxAH4AEnEAfgAXc3EAfgALAAAB13QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgAbc3EAfgALAAABTnQAJGphdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2skU3luY3QAD0Z1dHVyZVRhc2suamF2YXQACGlubmVyUnVuc3EAfgALAAAApnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2txAH4AKXEAfgAXc3EAfgALAAAEVnQAJ2phdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvcnQAF1RocmVhZFBvb2xFeGVjdXRvci5qYXZhdAAJcnVuV29ya2Vyc3EAfgALAAACW3QALmphdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvciRXb3JrZXJxAH4AL3EAfgAXc3EAfgALAAAC0nQAEGphdmEubGFuZy5UaHJlYWR0AAtUaHJlYWQuamF2YXEAfgAXc3IAJmphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVMaXN0_A8lMbXsjhACAAFMAARsaXN0cQB-AAZ4cgAsamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUNvbGxlY3Rpb24ZQgCAy173HgIAAUwAAWN0ABZMamF2YS91dGlsL0NvbGxlY3Rpb247eHBzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhwAAAAAHcEAAAAAHhxAH4AO3g 2014-10-13 19:26:37,576 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Publish async job-198 complete on message bus 2014-10-13 19:26:37,576 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Wake up jobs related to job- 198 2014-10-13 19:26:37,576 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Update db status for job- 198 2014-10-13 19:26:37,578 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Wake up jobs joined with job- 198 and disjoin all subjobs created from job- 198 2014-10-13 19:26:37,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Done executing com.cloud.vm.VmWorkMigrate for job-198 2014-10-13 19:26:37,594 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Sync queue (63) is currently empty 2014-10-13 19:26:37,595 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-65:ctx-3cedea49 job-197/job-198) Remove job-198 from job monitoring 2014-10-13 19:26:37,596 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-99:ctx-b771b262 job-197) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd java.lang.RuntimeException: Job failed due to exception Unable to migrate due to org.libvirt.LibvirtException: Cannot get interface MTU on 'breth0-3011': No such device at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:513) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:470) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 2014-10-13 19:26:37,602 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-99:ctx-b771b262 job-197) Complete async job-197, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Job failed due to exception Unable to migrate due to org.libvirt.LibvirtException: Cannot get interface MTU on \u0027breth0-3011\u0027: No such device"} {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)