Abhinav Roy created CLOUDSTACK-6868: ---------------------------------------
Summary: [Hyper-V] Download of an uploaded volume is failing Key: CLOUDSTACK-6868 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6868 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server, Storage Controller Affects Versions: 4.4.0 Environment: Hyper-V Reporter: Abhinav Roy Priority: Critical Fix For: 4.4.0 Steps: ============= 1. Deploy an advanced zone Hyper-V setup. 2. Create a VM v1 3. Upload a volume upload1 4. Download upload1 Expected behavior : ============= Download should pass. Observed behavior : ============= Download fails with 2014-06-09 14:06:16,740 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-2b2fe04f) ===START=== 10.144.7.13 -- GET command=extractVolume&id=7282784d-75bf-46a4-b6a5-33fae5cda75c&zoneid=622e7c06-e62e-4a6b-8de6-a462a3ee7cf7&mode=HTTP_DOWNLOAD&response=json&sessionkey=RGKWL0km%2FibmrEZH39fWJsmJeeU%3D&_=1402302332046 2014-06-09 14:06:16,783 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (catalina-exec-13:ctx-2b2fe04f ctx-676b8816) submit async job-113, details: AsyncJobVO {id:113, userId: 2, accountId: 2, instanceType: Volume, instanceId: 18, cmd: org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd, cmdInfo: {"response":"json","id":"7282784d-75bf-46a4-b6a5-33fae5cda75c","sessionkey":"RGKWL0km/ibmrEZH39fWJsmJeeU\u003d","ctxDetails":"{\"com.cloud.storage.Volume\":18,\"Volume\":\"7282784d-75bf-46a4-b6a5-33fae5cda75c\",\"com.cloud.dc.DataCenter\":1}","cmdEventType":"VOLUME.EXTRACT","ctxUserId":"2","zoneid":"622e7c06-e62e-4a6b-8de6-a462a3ee7cf7","httpmethod":"GET","_":"1402302332046","uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","ctxAccountId":"2","ctxStartEventId":"197","mode":"HTTP_DOWNLOAD"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2014-06-09 14:06:16,783 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-2b2fe04f ctx-676b8816) ===END=== 10.144.7.13 -- GET command=extractVolume&id=7282784d-75bf-46a4-b6a5-33fae5cda75c&zoneid=622e7c06-e62e-4a6b-8de6-a462a3ee7cf7&mode=HTTP_DOWNLOAD&response=json&sessionkey=RGKWL0km%2FibmrEZH39fWJsmJeeU%3D&_=1402302332046 2014-06-09 14:06:16,785 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-28:ctx-774021c1 job-113) Add job-113 into job monitoring 2014-06-09 14:06:16,786 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:ctx-774021c1 job-113) Executing AsyncJobVO {id:113, userId: 2, accountId: 2, instanceType: Volume, instanceId: 18, cmd: org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd, cmdInfo: {"response":"json","id":"7282784d-75bf-46a4-b6a5-33fae5cda75c","sessionkey":"RGKWL0km/ibmrEZH39fWJsmJeeU\u003d","ctxDetails":"{\"com.cloud.storage.Volume\":18,\"Volume\":\"7282784d-75bf-46a4-b6a5-33fae5cda75c\",\"com.cloud.dc.DataCenter\":1}","cmdEventType":"VOLUME.EXTRACT","ctxUserId":"2","zoneid":"622e7c06-e62e-4a6b-8de6-a462a3ee7cf7","httpmethod":"GET","_":"1402302332046","uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","ctxAccountId":"2","ctxStartEventId":"197","mode":"HTTP_DOWNLOAD"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2014-06-09 14:06:16,795 WARN [c.c.a.d.ParamGenericValidationWorker] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) Received unknown parameters for command extractVolume. Unknown parameters : ctxdetails 2014-06-09 14:06:16,833 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) copyAsync inspecting src type VOLUME copyAsync inspecting dest type VOLUME 2014-06-09 14:06:16,850 DEBUG [c.c.a.t.Request] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) Seq 1-9177491615650939381: Sending { Cmd , MgmtId: 213737702773493, via: 1(10.102.244.20), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"641da331-d6d6-3224-b53a-0dad93667d30","id":2,"poolType":"SMB","host":"SMB19","path":"/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR","port":445,"url":"SMB://SMB19/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR/?ROLE=Primary&STOREUUID=641da331-d6d6-3224-b53a-0dad93667d30"}},"name":"upload3","size":21474836480,"volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.102.192.19/hyperv-share/abhinav-ss1?user=abhinavroy&domain=BLR","_role":"Image"}},"name":"upload3","size":21474836480,"path":"volumes/2/18","volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] } 2014-06-09 14:06:16,852 DEBUG [c.c.a.t.Request] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) Seq 1-9177491615650939381: Executing: { Cmd , MgmtId: 213737702773493, via: 1(10.102.244.20), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"641da331-d6d6-3224-b53a-0dad93667d30","id":2,"poolType":"SMB","host":"SMB19","path":"/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR","port":445,"url":"SMB://SMB19/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR/?ROLE=Primary&STOREUUID=641da331-d6d6-3224-b53a-0dad93667d30"}},"name":"upload3","size":21474836480,"volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.102.192.19/hyperv-share/abhinav-ss1?user=abhinavroy&domain=BLR","_role":"Image"}},"name":"upload3","size":21474836480,"path":"volumes/2/18","volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] } 2014-06-09 14:06:16,852 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-396:ctx-3fcd7b13) Seq 1-9177491615650939381: Executing request 2014-06-09 14:06:16,853 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-396:ctx-3fcd7b13) POST request to https://10.102.244.20:8250/api/HypervResource/org.apache.cloudstack.storage.command.CopyCommand with contents {"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"641da331-d6d6-3224-b53a-0dad93667d30","id":2,"poolType":"SMB","host":"SMB19","path":"/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR","port":445,"url":"SMB://SMB19/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR/?ROLE=Primary&STOREUUID=641da331-d6d6-3224-b53a-0dad93667d30"}},"name":"upload3","size":21474836480,"volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.102.192.19/hyperv-share/abhinav-ss1?user=abhinavroy&domain=BLR","_role":"Image"}},"name":"upload3","size":21474836480,"path":"volumes/2/18","volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"contextMap":{},"wait":10800} 2014-06-09 14:06:16,856 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-396:ctx-3fcd7b13) Sending cmd to https://10.102.244.20:8250/api/HypervResource/org.apache.cloudstack.storage.command.CopyCommand cmd data:{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"641da331-d6d6-3224-b53a-0dad93667d30","id":2,"poolType":"SMB","host":"SMB19","path":"/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR","port":445,"url":"SMB://SMB19/hyperv-share/abhinav-ps1?user=abhinavroy&domain=BLR/?ROLE=Primary&STOREUUID=641da331-d6d6-3224-b53a-0dad93667d30"}},"name":"upload3","size":21474836480,"volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"7282784d-75bf-46a4-b6a5-33fae5cda75c","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.102.192.19/hyperv-share/abhinav-ss1?user=abhinavroy&domain=BLR","_role":"Image"}},"name":"upload3","size":21474836480,"path":"volumes/2/18","volumeId":18,"accountId":2,"format":"VHDX","id":18,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"contextMap":{},"wait":10800} 2014-06-09 14:06:16,937 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-396:ctx-3fcd7b13) POST response is [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Local template file missing from \\smb19\hyperv-share\abhinav-ps1\7282784d-75bf-46a4-b6a5-33fae5cda75c.vhdx","newData":null,"contextMap":{}}}] 2014-06-09 14:06:16,938 DEBUG [c.c.h.h.r.HypervDirectConnectResource] (DirectAgent-396:ctx-3fcd7b13) executeRequest received response [Lcom.cloud.agent.api.Answer;@477df73e 2014-06-09 14:06:16,938 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-396:ctx-3fcd7b13) Seq 1-9177491615650939381: Response Received: 2014-06-09 14:06:16,938 DEBUG [c.c.a.t.Request] (DirectAgent-396:ctx-3fcd7b13) Seq 1-9177491615650939381: Processing: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Local template file missing from \\\\smb19\\hyperv-share\\abhinav-ps1\\7282784d-75bf-46a4-b6a5-33fae5cda75c.vhdx","wait":0}}] } 2014-06-09 14:06:16,938 DEBUG [c.c.a.t.Request] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) Seq 1-9177491615650939381: Received: { Ans: , MgmtId: 213737702773493, via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } } 2014-06-09 14:06:16,943 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (API-Job-Executor-28:ctx-774021c1 job-113 ctx-351bc29f) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@25eff6cc), no need to delete from object in store ref table 2014-06-09 14:06:16,964 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-28:ctx-774021c1 job-113) Unexpected exception while executing org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd com.cloud.utils.exception.CloudRuntimeException: Failed to copy the volume from the source primary storage pool to secondary storage. at com.cloud.storage.VolumeApiServiceImpl.extractVolume(VolumeApiServiceImpl.java:1900) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at $Proxy181.extractVolume(Unknown Source) at org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd.execute(ExtractVolumeCmd.java:137) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 2014-06-09 14:06:16,966 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:ctx-774021c1 job-113) Complete async job-113, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to copy the volume from the source primary storage pool to secondary storage."} 2014-06-09 14:06:16,982 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-28:ctx-774021c1 job-113) Done executing org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd for job-113 NOTE : ============== The volume was uploaded as .vhd. In the primary storage also we see that the type is .vhd but in DB it is stored as .vhdx mysql> select * from volumes where id=18 \G; *************************** 1. row *************************** id: 18 account_id: 2 domain_id: 1 pool_id: 2 last_pool_id: NULL instance_id: NULL device_id: NULL name: upload3 uuid: 7282784d-75bf-46a4-b6a5-33fae5cda75c size: 21474836480 folder: NULL path: NULL pod_id: NULL data_center_id: 1 iscsi_name: NULL host_ip: NULL volume_type: DATADISK pool_type: NULL disk_offering_id: 6 template_id: NULL first_snapshot_backup_uuid: NULL recreatable: 0 created: 2014-06-09 06:42:37 attached: NULL updated: 2014-06-09 06:51:29 removed: NULL state: Ready chain_info: NULL update_count: 4 disk_type: NULL vm_snapshot_chain_size: NULL iso_id: NULL display_volume: 1 format: VHDX min_iops: NULL max_iops: NULL hv_ss_reserve: NULL 1 row in set (0.00 sec) -- This message was sent by Atlassian JIRA (v6.2#6252)