Keiichi Yusa created CLOUDSTACK-8085: ----------------------------------------
Summary: Fails to attach a volume (is made from a snapshot) to a VM with using local storage as primary storage. Key: CLOUDSTACK-8085 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8085 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: KVM, Snapshot, Volumes Affects Versions: Future, 4.5.0, 4.6.0, 4.4.3, 4.3.2 Environment: - Builds up with 155 hosts and a cloudstack server. - Each host equips these. + Xeon E5-2680 (10 cores, 20 threads). + 128GB memory (and is not set swap space). + 600GB Solid State Drive. - All hosts are networked by InfiniBand and 10GbE. - Using CentOS6.6 (64bit) - Using local storage on all hosts as primary storage. - Using S3 compatible storage (RadosGW) as secondary storage. Reporter: Keiichi Yusa Priority: Critical Fix For: Future, 4.5.0, 4.6.0, 4.4.3, 4.3.2 CloudStack fails to attach a DATADISK volume (This DATADISK volume is made from a snapshot) to a VM with using local storage as primary storage. Reproduction procedure: # To use local storage as primary storage. # Take a snapshot of a VM's ROOT/DATADISK volume. (By clicking [Take Snapshot] button on WebUI.) # Create a new DATADISK volume from this snapshot. (By clicking [Create Volume] button on WebUI.) # Try to attach this new DATADISK volume to a VM. (By Clicking [Attach Disk] button on WebUI.) When we execute these, CloudStack fails attach this new DATADISK volume to the VM with occuring the following error: {noformat} "Failed to attach local data volume snapshot-test-std-stop to VM snapshot-test-0001 as migration of local data volume is not allowed" {noformat} At this time, Cloudstack puts a exception in management-server.log. (Please see [1]) We investigate our CloudStack environment about this problem. We notice that Cloudstack behaves as follows: * CloudStack creates a new DATADISK volume at primary storage of any one of the hosts when a user creates the volume from a snapshot by executing [Create Volume]. (This primary storage is local storage on the host.) * Now, the user deploys a new VM (or uses a existing VM). In many cases, the new DATADISK volume is deployed to a host different from the host that is deployed the VM. * User will attach the new DATADISK volume to the VM. When the user executes [Attach Disk], CloudStack tries to migrate the new DATADISK volume to the host that is deployed the VM in order to attach the new DATADISK volume to the VM. * However, CloudStack fails this migration because we use local storage as primary storage. [1] Exception in managemnet-server.log {noformat} 2014-12-02 19:37:10,807 ERROR [c.c.a.ApiAsyncJobDispatcher] (Job-Executor-50:ctx-a0ef6a5a) Unexpected exception while executing org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd com.cloud.utils.exception.CloudRuntimeException: Failed to attach local data volume snapshot-test-std-stop to VM snapshot-test-0001 as migration of local data volume is not allowed at com.cloud.storage.VolumeApiServiceImpl.attachVolumeToVM(VolumeApiServiceImpl.java:1292) at com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:1156) at com.cloud.storage.VolumeApiServiceImpl.attachVolumeToVM(VolumeApiServiceImpl.java:1126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy197.attachVolumeToVM(Unknown Source) at org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd.execute(AttachVolumeCmd.java:123) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161) at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109) at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)