[ https://issues.apache.org/jira/browse/CLOUDSTACK-4475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajesh Battala updated CLOUDSTACK-4475: --------------------------------------- Priority: Critical (was: Major) > [ZWPS] attaching an uploaded volume to a VM is always going to first primary > storage added > ------------------------------------------------------------------------------------------ > > Key: CLOUDSTACK-4475 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4475 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Doc, Storage Controller > Affects Versions: 4.2.1 > Environment: vmware esxi 5.1 > Reporter: Srikanteswararao Talluri > Priority: Critical > Labels: ReleaseNote > Fix For: 4.4.0 > > > Steps to reproduce: > ============== > 1. Have an advanced zone deployment with two cluster one host and one cluster > scoped primary storage each > 2. add two more zone wide primary storages > 3. create a deployment on zone scoped primary storage > 4. upload a volume. > 5. attach uploaded volume to VM created in step 3. > Observation: > ========= > While attaching volume, volume is always copied to first available primary > storage in the storage_pool table and as a result attaching a volume created > on cluster scoped primary storage to a VM with its root volume on zone wide > primary storage fails. > mysql> select * from storage_pool; > +----+--------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+------------------------------------------------+---------------------+---------+-------------+-------------+-----------------------+---------+------------+---------+---------------+ > | id | name | uuid | pool_type > | port | data_center_id | pod_id | cluster_id | used_bytes | > capacity_bytes | host_address | user_info | path > | created | removed | update_time | status | > storage_provider_name | scope | hypervisor | managed | capacity_iops | > +----+--------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+------------------------------------------------+---------------------+---------+-------------+-------------+-----------------------+---------+------------+---------+---------------+ > | 1 | primaryclus1 | 722e6181-8497-3d31-9933-a0a267ae376c | > NetworkFilesystem | 2049 | 1 | 1 | 1 | > 1678552014848 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/primary | 2013-08-23 12:11:12 | NULL > | NULL | Maintenance | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 2 | pimaryclu2 | 9fd9b0fc-c9fd-39b8-8d66-06372c5ff6d2 | > NetworkFilesystem | 2049 | 1 | 1 | 2 | > 1676566495232 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/clus1primary | 2013-08-23 12:18:14 | NULL > | NULL | Up | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 3 | clus1p2 | 22e0c3fe-a390-38fa-8ff7-e1d965a36309 | > NetworkFilesystem | 2049 | 1 | 1 | 1 | > 1660903886848 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/clus1p2 | 2013-08-23 14:30:32 | NULL > | NULL | Up | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 4 | clus1p3 | f2d9fb6b-c433-3c03-acf8-8f73eac48fae | > NetworkFilesystem | 2049 | 1 | 1 | 1 | > 1660901400576 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/clus1p3 | 2013-08-23 14:31:05 | NULL > | NULL | Up | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 5 | clus2p2 | 13bf579c-51f3-317b-893a-98ff6ca8f486 | > NetworkFilesystem | 2049 | 1 | 1 | 2 | > 1660900147200 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/clus2p2 | 2013-08-23 14:31:38 | NULL > | NULL | Up | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 7 | clus2p3 | 294ae9ff-cb02-33a0-8f31-21fdd8ff34db | > NetworkFilesystem | 2049 | 1 | 1 | 2 | > 1660894195712 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/clus2p3 | 2013-08-23 14:33:03 | NULL > | NULL | Up | DefaultPrimary | CLUSTER | NULL | > 0 | NULL | > | 8 | z1 | 035188ce-7269-3440-8090-8c13da2158ba | > NetworkFilesystem | 2049 | 1 | NULL | NULL | > 1626954428416 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/z1 | 2013-08-23 16:57:56 | NULL > | NULL | Up | DefaultPrimary | ZONE | VMware | > 0 | NULL | > | 9 | z2 | f0840b39-e42e-3e48-8c36-e9d121512efd | > NetworkFilesystem | 2049 | 1 | NULL | NULL | > 1627066220544 | 5902284800000 | 10.147.28.7 | NULL | > /export/home/talluri/vmware.campo/z2 | 2013-08-23 16:58:25 | NULL > | NULL | Up | DefaultPrimary | ZONE | VMware | > 0 | NULL | > +----+--------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+------------------------------------------------+---------------------+---------+-------------+-------------+-----------------------+---------+------------+---------+---------------+ > 8 rows in set (0.00 sec) > attach volume command: > ===START=== 10.101.255.7 -- GET > command=attachVolume&id=bcba9a9e-de07-433f-bbad-4a9a03d5283f&virtualMachineId=67dbcb09-dbe6-4b8d-892d-e0bb02eecca4&response=json&sessionkey=Kzq5%2F2XXjxGbr%2FF4ahPYYcgKZDU%3D&_=1377264362757 > 2013-08-24 00:20:09,974 DEBUG [cloud.async.AsyncJobManagerImpl] > (catalina-exec-19:null) submit async job-62 = [ > 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ], details: AsyncJobVO {id:62, userId: > 2, accountId: 2, sessionKey: null, instanceType: Volume, instanceId: 21, cmd: > org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd, cmdOriginator: > null, cmdInfo: > {"response":"json","id":"bcba9a9e-de07-433f-bbad-4a9a03d5283f","sessionkey":"Kzq5/2XXjxGbr/F4ahPYYcgKZDU\u003d","cmdEventType":"VOLUME.ATTACH","ctxUserId":"2","virtualMachineId":"67dbcb09-dbe6-4b8d-892d-e0bb02eecca4","httpmethod":"GET","_":"1377264362757","ctxAccountId":"2","ctxStartEventId":"181"}, > cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, > processStatus: 0, resultCode: 0, result: null, initMsid: 7363452993625, > completeMsid: null, lastUpdated: null, lastPolled: null, created: null} > 2013-08-24 00:20:09,976 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null) > ===END=== 10.101.255.7 -- GET > command=attachVolume&id=bcba9a9e-de07-433f-bbad-4a9a03d5283f&virtualMachineId=67dbcb09-dbe6-4b8d-892d-e0bb02eecca4&response=json&sessionkey=Kzq5%2F2XXjxGbr%2FF4ahPYYcgKZDU%3D&_=1377264362757 > 2013-08-24 00:20:09,979 DEBUG [cloud.async.AsyncJobManagerImpl] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Executing > org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd for job-62 = [ > 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ] > 2013-08-24 00:20:10,022 DEBUG [storage.allocator.LocalStoragePoolAllocator] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) > LocalStoragePoolAllocator trying to find storage pool to fit the vm > 2013-08-24 00:20:10,023 DEBUG > [storage.allocator.ClusterScopeStoragePoolAllocator] (Job-Executor-62:job-62 > = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) ClusterScopeStoragePoolAllocator > looking for storage pool > 2013-08-24 00:20:10,024 DEBUG > [storage.allocator.ClusterScopeStoragePoolAllocator] (Job-Executor-62:job-62 > = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Looking for pools in dc: 1 pod:1 > cluster:null > 2013-08-24 00:20:10,034 DEBUG > [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-62:job-62 = [ > 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Checking if storage pool is suitable, > name: null ,poolId: 2 > 2013-08-24 00:20:10,040 DEBUG [cloud.storage.StorageManagerImpl] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Checking > pool 2 for storage, totalSize: 5902284800000, usedBytes: 4282525622272, > usedPct: 0.725570820010583, disable threshold: 0.85 > 2013-08-24 00:20:10,048 DEBUG [cloud.storage.StorageManagerImpl] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Checking > pool: 2 for volume allocation [Vol[21|vm=null|DATADISK]], maxSize : > 11804569600000, totalAllocatedSize : 2097152000, askingSize : 2097152000, > allocated disable threshold: 0.85 > 2013-08-24 00:20:10,048 DEBUG > [storage.allocator.ClusterScopeStoragePoolAllocator] (Job-Executor-62:job-62 > = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) FirstFitStoragePoolAllocator > returning 1 suitable storage pools > 2013-08-24 00:20:10,078 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) copyAsync > inspecting src type VOLUME copyAsync inspecting dest type VOLUME > 2013-08-24 00:20:10,104 DEBUG [agent.transport.Request] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Seq > 3-1450508329: Sending { Cmd , MgmtId: 7363452993625, via: 3, Ver: v1, Flags: > 100011, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"bcba9a9e-de07-433f-bbad-4a9a03d5283f","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/talluri/vmware.campo/secondary","_role":"Image"}},"name":"uv","size":2097152000,"path":"volumes/2/21/da6a4f3f-fd51-3a23-ac2e-53d584936e92.ova","volumeId":21,"accountId":2,"format":"OVA","id":21,"hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"bcba9a9e-de07-433f-bbad-4a9a03d5283f","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9fd9b0fc-c9fd-39b8-8d66-06372c5ff6d2","id":2,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/talluri/vmware.campo/clus1primary","port":2049}},"name":"uv","size":2097152000,"volumeId":21,"accountId":2,"format":"OVA","id":21,"hypervisorType":"VMware"}},"executeInSequence":false,"wait":10800}}] > } > Error: > 2013-08-24 00:21:00,632 DEBUG [agent.transport.Request] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Seq > 3-1450508329: Received: { Ans: , MgmtId: 7363452993625, via: 3, Ver: v1, > Flags: 10, { CopyCmdAnswer } } > 2013-08-24 00:21:00,690 ERROR [cloud.async.AsyncJobManagerImpl] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) > Unexpected exception while executing > org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd > com.cloud.utils.exception.CloudRuntimeException: Can't move volume between > scope: CLUSTER and ZONE > at > com.cloud.storage.VolumeManagerImpl.needMoveVolume(VolumeManagerImpl.java:1596) > at > com.cloud.storage.VolumeManagerImpl.attachVolumeToVM(VolumeManagerImpl.java:1871) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd.execute(AttachVolumeCmd.java:122) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-08-24 00:21:00,692 DEBUG [cloud.async.AsyncJobManagerImpl] > (Job-Executor-62:job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ]) Complete > async job-62 = [ 556e947b-98bf-4e8f-a3cd-80ba46cd9e9d ], jobStatus: 2, > resultCode: 530, result: Error Code: 530 Error text: Can't move volume > between scope: CLUSTER and ZONE -- This message was sent by Atlassian JIRA (v6.2#6252)