disclamer - I did not read your logs, but.,,: CentOS 6.5 version of qemu-img is missing some flag, I beleive "-s" flag (redhat removed itd) - and cloudstack still uses that flag... So if this is also problem in your case - the snapshtos should be created, but not backup-ed.
Meaning, downgrate qemu-img to CentOs 6.4 version...or use the one in the folowing way, as described by Lucin / Nux: http://www.nux.ro/archive/2014/01/Taking_KVM_volume_snapshots_with_Cloudstack_4_2_on_CentOS_6_5.html On 26 November 2014 at 16:19, Jeff Barnett <jbarn...@prokarma.com> wrote: > The OS is CentOS 6.5, Storage is NFS not hosted on the controller. I will > also note that there is two zones, with the same configuration just using > different NFS devices one zone can take snapshots but the one that can no > longer take them has been working for 8 months with no issues. > > Here is the section from the log file when I try to take a snapshot: > > > 2014-11-26 07:00:48,308 DEBUG [cloud.api.ApiServlet] > (catalina-exec-4:null) ===START=== 172.16.28.20 -- GET > command=createSnapshot&volumeid=ec6460c4-896c-46d1-8ae5-75d6431a6608&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014045703 > 2014-11-26 07:00:48,362 DEBUG [cloud.async.AsyncJobManagerImpl] > (catalina-exec-4:null) submit async job-147 = [ > 2007bb4e-03c9-4ff7-874d-e50641318ace ], details: AsyncJobVO {id:147, > userId: 2, accountId: 2, sessionKey: null, instanceType: Snapshot, > instanceId: 128, cmd: > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, > cmdOriginator: null, cmdInfo: > {"id":"128","response":"json","sessionkey":"ysbTjq7whHzoePtvcRBRSzsJ6zw\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"2","httpmethod":"GET","_":"1417014045703","volumeid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","ctxAccountId":"2","ctxStartEventId":"5425"}, > cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, > processStatus: 0, resultCode: 0, result: null, initMsid: 345048644703, > completeMsid: null, lastUpdated: null, lastPolled: null, created: null} > 2014-11-26 07:00:48,363 DEBUG [cloud.async.AsyncJobManagerImpl] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) > Executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd > for job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ] > 2014-11-26 07:00:48,363 DEBUG [cloud.api.ApiServlet] > (catalina-exec-4:null) ===END=== 172.16.28.20 -- GET > command=createSnapshot&volumeid=ec6460c4-896c-46d1-8ae5-75d6431a6608&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014045703 > 2014-11-26 07:00:48,386 INFO [user.snapshot.CreateSnapshotCmd] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) VOLSS: > createSnapshotCmd starts:1417014048386 > 2014-11-26 07:00:48,437 DEBUG [agent.transport.Request] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq > 7-1985284483: Sending { Cmd , MgmtId: 345048644703, via: 7, Ver: v1, > Flags: 100011, > [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02] > i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"wait":0}}] > } > 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq > 7-1985284483: Executing: { Cmd , MgmtId: 345048644703, via: 7, Ver: v1, > Flags: 100011, > [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02] > i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"wait":0}}] > } > 2014-11-26 07:00:48,438 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-311:null) Seq 7-1985284483: Executing request > 2014-11-26 07:00:48,438 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-311:null) Seq 7-1985284483: Response Received: > 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request] > (DirectAgent-311:null) Seq 7-1985284483: Processing: { Ans: , MgmtId: > 345048644703, via: 7, Ver: v1, Flags: 10, > [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"b7c08601-0654-4749-b7e0-ccc380458e3d","id":0}},"result":true,"wait":0}}] > } > 2014-11-26 07:00:48,438 DEBUG [agent.transport.Request] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq > 7-1985284483: Received: { Ans: , MgmtId: 345048644703, via: 7, Ver: v1, > Flags: 10, { CreateObjectAnswer } } > 2014-11-26 07:00:48,488 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) > copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type > SNAPSHOT > 2014-11-26 07:00:48,535 DEBUG [agent.transport.Request] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq > 9-1742209940: Sending { Cmd , MgmtId: 345048644703, via: 9, Ver: v1, > Flags: 100111, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"b7c08601-0654-4749-b7e0-ccc380458e3d","volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02] > i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"parentSnapshotPath":"2e7af1a5-a31c-4fe6-b2f7-21965c1a86b9","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/334","volume":{"uuid":"ec6460c4-896c-46d1-8ae5-75d6431a6608","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"055b61d2-5260-3856-92fc-48facf5dfbaf","id":2,"poolType":"VMFS","host":"VMFS > datastore: > /Labs/CS-PRI-02","path":"/Labs/CS-PRI-02","port":0}},"name":"ROOT-297","size":21474836480,"path":"ROOT-297-000001","volumeId":334,"vmName":"i-2-297-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[CS-PRI-02] > i-2-297-VM/ROOT-297.vmdk\"]}","format":"OVA","id":334,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs:// > 172.16.25.84/data/CloudStackSEC2","_role":"Image"}},"vmName":"i-2-297-VM","name":"new-import_ROOT-297_20141126150048","hypervisorType":"VMware","id":128}},"executeInSequence":true,"wait":21600}}] > } > 2014-11-26 07:00:51,404 DEBUG [cloud.api.ApiServlet] > (catalina-exec-25:null) ===START=== 172.16.28.20 -- GET > command=queryAsyncJobResult&jobId=2007bb4e-03c9-4ff7-874d-e50641318ace&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014048864 > 2014-11-26 07:00:51,428 DEBUG [cloud.api.ApiServlet] > (catalina-exec-25:null) ===END=== 172.16.28.20 -- GET > command=queryAsyncJobResult&jobId=2007bb4e-03c9-4ff7-874d-e50641318ace&response=json&sessionkey=ysbTjq7whHzoePtvcRBRSzsJ6zw%3D&_=1417014048864 > 2014-11-26 07:00:51,840 DEBUG [agent.manager.AgentManagerImpl] > (AgentManager-Handler-2:null) SeqA 3-1302876: Processing Seq 3-1302876: { > Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11, > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n > \"connections\": []\n}","wait":0}}] } > 2014-11-26 07:00:51,846 DEBUG [agent.manager.AgentManagerImpl] > (AgentManager-Handler-2:null) SeqA 3-1302876: Sending Seq 3-1302876: { > Ans: , MgmtId: 345048644703, via: 3, Ver: v1, Flags: 100010, > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } > 2014-11-26 07:00:52,150 DEBUG [agent.transport.Request] > (AgentManager-Handler-15:null) Seq 9-1742209940: Processing: { Ans: , > MgmtId: 345048644703, via: 9, Ver: v1, Flags: 110, > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"backup > snapshot exception: Exception: java.lang.Exception\nMessage: unable to > prepare snapshot backup directory\n","wait":0}}] } > 2014-11-26 07:00:52,150 DEBUG [agent.transport.Request] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Seq > 9-1742209940: Received: { Ans: , MgmtId: 345048644703, via: 9, Ver: v1, > Flags: 110, { CopyCmdAnswer } } > 2014-11-26 07:00:52,155 DEBUG [agent.manager.AgentAttache] > (AgentManager-Handler-15:null) Seq 9-1742209940: No more commands found > 2014-11-26 07:00:52,171 DEBUG [storage.snapshot.SnapshotManagerImpl] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Failed > to create snapshot > com.cloud.utils.exception.CloudRuntimeException: backup snapshot > exception: Exception: java.lang.Exception > Message: unable to prepare snapshot backup directory > > at > org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280) > at > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136) > at > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279) > at > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1007) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311) > at > com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2773) > at > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2014-11-26 07:00:52,184 DEBUG [storage.volume.VolumeServiceImpl] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Take > snapshot: 334 failed > com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot > at > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1034) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311) > at > com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2773) > at > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: com.cloud.utils.exception.CloudRuntimeException: backup > snapshot exception: Exception: java.lang.Exception > Message: unable to prepare snapshot backup directory > > at > org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280) > at > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136) > at > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279) > at > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1007) > ... 15 more > 2014-11-26 07:00:52,184 DEBUG [user.snapshot.CreateSnapshotCmd] > (Job-Executor-65:job-147 = [ 2007bb4e-03c9-4ff7-874d-e50641318ace ]) Failed > to create snapshot > org.apache.cloudstack.api.ServerApiException > at > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:176) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > -----Original Message----- > From: Andrija Panic [mailto:andrija.pa...@gmail.com] > Sent: Tuesday, November 25, 2014 11:55 PM > To: users@cloudstack.apache.org > Subject: Re: Unable to take Snapshot CS 4.2.1 > > Please attach the management logs for further invetigation...what OS, what > storage, etc... > > On 26 November 2014 at 01:51, Jeff Barnett <jbarn...@prokarma.com> wrote: > > > When I try to take a snapshot now I just get an error like this, > > Failed to create a snapshot due to an internal error creating snapshot > for volume ... > > has anyone run into this, if so what was the fix, I tried restarting > > the cloud services on the SSVM, but that did not help. > > > > > > > > > > > -- > > Andrija Panić > -- Andrija Panić
