[ https://issues.apache.org/jira/browse/CLOUDSTACK-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devdeep Singh resolved CLOUDSTACK-5639. --------------------------------------- Resolution: Fixed > [Hyper-v] cold storage migration does not work in hyperv > -------------------------------------------------------- > > Key: CLOUDSTACK-5639 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5639 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Hypervisor Controller, Management Server > Affects Versions: 4.3.0 > Environment: Latest build from 4.3 branch with > commit:4b2b6544255ee7fe12b4b4732cacd08e4ee6a1b7 > Storage: SMB for both primary and secondary > Hypervisor: Hyper-v > Reporter: Sanjeev N > Assignee: Devdeep Singh > Priority: Critical > Labels: hyper-V,, hyper-v, hyperv > Fix For: 4.3.0 > > Attachments: cloudstack-agent.rar, management-server.rar, > management-server.rar > > > [Hyper-v] cold storage does not work in hyperv > Steps to Reproduce: > ================ > 1.Bring up CS in advanced zone with 2 hosts in a hyper-v cluster using CIFS > for both primary and secondary storage > 2.Deploy one or two guest vms using default cent os template. > 3.Add one more primary storage (say primary2) to the cluster > 4.Stop one of the vms deployed at step2 > 5.Migrate the storage from source primary to new primary storage primary2 > Expected Result: > ============= > Storage migration should succeed when the vm is in stopped state > Actual Result: > =========== > Storage migration failed with the error "Failed to migration: > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7] is > unreachable" > Observations: > ============ > Log snippet from the MS log: > 2013-12-24 19:11:55,475 DEBUG [c.c.a.ApiServlet] > (catalina-exec-11:ctx-717ba369) ===START=== 10.146.0.134 -- GET > command=migrateVirtualMachine&storageid=fe97ca45-5cba-329a-ae79-fa02649421c1&virtualmachineid=3a1b4f23-7a43-4806-8fd5-43a9c1130872&response=json&sessionkey=uHhzduF1EppEHm%2Bibsj2%2Bqf%2Fzto%3D&_=1387892514178 > 2013-12-24 19:11:55,661 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (catalina-exec-11:ctx-717ba369 ctx-d805adb4) submit async job-33, details: > AsyncJobVO {id:33, userId: 2, accountId: 2, instanceType: None, instanceId: > null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: > {"response":"json","sessionkey":"uHhzduF1EppEHm+ibsj2+qf/zto\u003d","virtualmachineid":"3a1b4f23-7a43-4806-8fd5-43a9c1130872","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"fe97ca45-5cba-329a-ae79-fa02649421c1","httpmethod":"GET","_":"1387892514178","ctxAccountId":"2","ctxStartEventId":"99"}, > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: > null, initMsid: 132129494109518, completeMsid: null, lastUpdated: null, > lastPolled: null, created: null} > 2013-12-24 19:11:55,663 INFO [o.a.c.f.j.i.AsyncJobMonitor] > (Job-Executor-29:ctx-893d8661) Add job-33 into job monitoring > 2013-12-24 19:11:55,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (Job-Executor-29:ctx-893d8661) Executing AsyncJobVO {id:33, userId: 2, > accountId: 2, instanceType: None, instanceId: null, cmd: > org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: > {"response":"json","sessionkey":"uHhzduF1EppEHm+ibsj2+qf/zto\u003d","virtualmachineid":"3a1b4f23-7a43-4806-8fd5-43a9c1130872","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"fe97ca45-5cba-329a-ae79-fa02649421c1","httpmethod":"GET","_":"1387892514178","ctxAccountId":"2","ctxStartEventId":"99"}, > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: > null, initMsid: 132129494109518, completeMsid: null, lastUpdated: null, > lastPolled: null, created: null} > 2013-12-24 19:11:55,663 DEBUG [c.c.a.ApiServlet] > (catalina-exec-11:ctx-717ba369 ctx-d805adb4) ===END=== 10.146.0.134 -- GET > command=migrateVirtualMachine&storageid=fe97ca45-5cba-329a-ae79-fa02649421c1&virtualmachineid=3a1b4f23-7a43-4806-8fd5-43a9c1130872&response=json&sessionkey=uHhzduF1EppEHm%2Bibsj2%2Bqf%2Fzto%3D&_=1387892514178 > 2013-12-24 19:11:55,794 DEBUG [c.c.c.CapacityManagerImpl] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) VM state transitted from :Stopped > to Migrating with event: StorageMigrationRequestedvm's original host id: 4 > new host id: null host id before state transition: null > 2013-12-24 19:11:56,026 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) copyAsync inspecting src type > VOLUME copyAsync inspecting dest type VOLUME > 2013-12-24 19:11:56,199 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 > ctx-d805adb4) Seq 1-1652228440: Sending { Cmd , MgmtId: 132129494109518, > via: 1(10.147.40.14), Ver: v1, Flags: 100011, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9e132ff7-f9ef-3f17-bbc2-c44c323d96e7","id":1,"poolType":"NetworkFilesystem","host":"Host13","path":"/hyperv/primary?user=sanjeev&password=Password_123&domain=BLR","port":445,"url":"NetworkFilesystem://Host13//hyperv/primary?user=sanjeev&password=Password_123&domain=BLR/?ROLE=Primary&STOREUUID=9e132ff7-f9ef-3f17-bbc2-c44c323d96e7"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"path":"volumes/2/3","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] > } > 2013-12-24 19:11:56,199 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 > ctx-d805adb4) Seq 1-1652228440: Executing: { Cmd , MgmtId: 132129494109518, > via: 1(10.147.40.14), Ver: v1, Flags: 100011, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9e132ff7-f9ef-3f17-bbc2-c44c323d96e7","id":1,"poolType":"NetworkFilesystem","host":"Host13","path":"/hyperv/primary?user=sanjeev&password=Password_123&domain=BLR","port":445,"url":"NetworkFilesystem://Host13//hyperv/primary?user=sanjeev&password=Password_123&domain=BLR/?ROLE=Primary&STOREUUID=9e132ff7-f9ef-3f17-bbc2-c44c323d96e7"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"path":"volumes/2/3","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] > } > 2013-12-24 19:11:56,208 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 > ctx-d805adb4) Seq 1-1652228440: Received: { Ans: , MgmtId: 132129494109518, > via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } } > 2013-12-24 19:11:56,208 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) copy to image store failed: > org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value > cannot be null.^M > Parameter name: VolumeObjectTO missing primary dataStore in spec {^M > "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M > "volumeType": "ROOT",^M > "dataStore": {^M > "com.cloud.agent.api.to.NfsTO": {^M > "_url": > "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M > "_role": "Image"^M > }^M > },^M > "name": "ROOT-3",^M > "size": 5368709120,^M > "path": "volumes/2/3",^M > "volumeId": 3,^M > "vmName": "i-2-3-VM",^M > "accountId": 2,^M > "id": 3,^M > "deviceId": 0,^M > "hypervisorType": "Hyperv"^M > }2013-12-24 19:11:56,394 DEBUG [c.c.a.t.Request] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 3-177144205: Sending { Cmd , > MgmtId: 132129494109518, via: 3(s-2-VM), Ver: v1, Flags: 100011, > [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"wait":0}}] > } > 2013-12-24 19:11:56,438 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 > ctx-d805adb4) Seq 3-177144205: Received: { Ans: , MgmtId: 132129494109518, > via: 3, Ver: v1, Flags: 10, { Answer } } > 2013-12-24 19:11:56,439 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Volume 3 is not found on image > store 1, so no need to delete > 2013-12-24 19:11:56,505 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Unsupported data object (VOLUME, > org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@10d466d5), no > need to delete from object in store ref table > 2013-12-24 19:11:56,735 DEBUG [o.a.c.s.v.VolumeServiceImpl] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Marking volume that was never > created as destroyed: Vol[9|vm=3|ROOT] > 2013-12-24 19:11:56,788 ERROR [o.a.c.e.o.VolumeOrchestrator] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) migrate volume > failed:org.apache.cloudstack.storage.command.CopyCommand failed on exception, > Value cannot be null. > Parameter name: VolumeObjectTO missing primary dataStore in spec {^M > "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M > "volumeType": "ROOT",^M > "dataStore": {^M > "com.cloud.agent.api.to.NfsTO": {^M > "_url": > "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M > "_role": "Image"^M > }^M > },^M > "name": "ROOT-3",^M > "size": 5368709120,^M > "path": "volumes/2/3",^M > "volumeId": 3,^M > "vmName": "i-2-3-VM",^M > "accountId": 2,^M > "id": 3,^M > "deviceId": 0,^M > "hypervisorType": "Hyperv"^M > } > 2013-12-24 19:11:56,788 DEBUG [c.c.v.VirtualMachineManagerImpl] > (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Failed to migration: > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7] is > unreachable: migrate volume failed: > org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value > cannot be null.^M > Agent log: > ======== > 2013-12-24 11:25:55,436 [6] ERROR HypervResource.HypervResourceController > [d72f6849-8022-4183-87f1-3202083ed907] - > com.cloud.agent.api.GetStorageStatsCommand failed on exceptionIllegal > characters in path.^M > System.ArgumentException: Illegal characters in path.^M > at System.IO.Path.CheckInvalidPathChars(String path, Boolean > checkAdditional)^M > at > System.Security.Permissions.FileIOPermission.CheckIllegalCharacters(String[] > str)^M > at > System.Security.Permissions.FileIOPermission.AddPathList(FileIOPermissionAccess > access, AccessControlActions control, String[] pathListOrig, Boolean > checkForDuplicates, Boolean needFullPath, Boolean copyPathList)^M > at > System.Security.Permissions.FileIOPermission..ctor(FileIOPermissionAccess > access, String[] pathList, Boolean checkForDuplicates, Boolean needFullPath)^M > at System.IO.Path.GetFullPath(String path)^M > at HypervResource.HypervResourceController.GetCapacityForLocalPath(String > localStoragePath, Int64& capacityBytes, Int64& availableBytes)^M > at HypervResource.HypervResourceController.GetStorageStatsCommand(Object > cmd)^M > 2013-12-24 11:25:55,436 [6] INFO HypervResource.HypervResourceController > [d72f6849-8022-4183-87f1-3202083ed907] - {^M > "com.cloud.agent.api.GetStorageStatsAnswer": {^M > "result": false,^M > "details": "com.cloud.agent.api.GetStorageStatsCommand failed on > exceptionIllegal characters in path.",^M > "capacity": 0,^M > "used": 0,^M > "contextMap": {}^M > }^M > }^M > 2013-12-24 11:26:20,795 [35] ERROR HypervResource.VolumeObjectTO > [1d5b7262-afc4-4b65-ab5a-a5b189817c5e] - VolumeObjectTO missing primary > dataStore in spec {^M > "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M > "volumeType": "ROOT",^M > "dataStore": {^M > "com.cloud.agent.api.to.NfsTO": {^M > "_url": > "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M > "_role": "Image"^M > }^M > },^M > "name": "ROOT-3",^M > "size": 5368709120,^M > "path": "volumes/2/3",^M > "volumeId": 3,^M > "vmName": "i-2-3-VM",^M > "accountId": 2,^M > "id": 3,^M > "deviceId": 0,^M > "hypervisorType": "Hyperv"^M > }^M > 2013-12-24 11:26:20,795 [35] ERROR HypervResource.HypervResourceController > [1d5b7262-afc4-4b65-ab5a-a5b189817c5e] - > org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value > cannot be null.^M > Parameter name: VolumeObjectTO missing primary dataStore in spec {^M > "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M > "volumeType": "ROOT",^M > "dataStore": {^M > "com.cloud.agent.api.to.NfsTO": {^M > "_url": > "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M > "_role": "Image"^M > }^M > },^M > "name": "ROOT-3",^M > "size": 5368709120,^M > "path": "volumes/2/3",^M > "volumeId": 3,^M > "vmName": "i-2-3-VM",^M > "accountId": 2,^M > "id": 3,^M > "deviceId": 0,^M > "hypervisorType": "Hyperv"^M > }^M > System.ArgumentNullException: Value cannot be null.^M > Parameter name: VolumeObjectTO missing primary dataStore in spec {^M > "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M > "volumeType": "ROOT",^M > "dataStore": {^M > "com.cloud.agent.api.to.NfsTO": {^M > "_url": > "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M > "_role": "Image"^M > }^M > },^M > "name": "ROOT-3",^M > "size": 5368709120,^M > "path": "volumes/2/3",^M > "volumeId": 3,^M > "vmName": "i-2-3-VM",^M > "accountId": 2,^M > "id": 3,^M > "deviceId": 0,^M > "hypervisorType": "Hyperv"^M > }^M -- This message was sent by Atlassian JIRA (v6.1.5#6160)