Hi Guys, I have issues of snapshot on some vms, I using cloudstack 4.1 and xenserver 6.1. The following are the error logs, any ideas or suggestions will be great appreciated. Any other question, if I want clean the error snapshot, how can I achieve it? As I try to clean it in Cloudstack management UI, but after refresh, it still there.
Great thanks, William ###########################################Cloudstack management server log############################################################################## [root@mgt-cld ~]# less /var/log/cloudstack/management/management-server.log |grep "Job-Executor-62:job-3754" 2014-01-16 06:16:19,450 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-62:job-3754) Executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for job-3754 2014-01-16 06:16:19,455 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-62:job-3754) Access to Acct[5-PrjAcct-Tubes-1] granted to Acct[5-PrjAcct-Tubes-1] by DomainChecker_EnhancerByCloudStack_f81f8595 2014-01-16 06:16:19,477 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-62:job-3754) Access to Vol[82|vm=63|DATADISK] granted to Acct[5-PrjAcct-Tubes-1] by DomainChecker_EnhancerByCloudStack_f81f8595 2014-01-16 06:16:19,481 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) Failed to update snapshot state due to Unable to transition to a new state from Creating via CreateRequested 2014-01-16 06:16:19,511 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396163: Sending { Cmd , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 100011, [{"ManageSnapshotCommand":{"_commandSwitch":"-c","_volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","_pool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"_snapshotName":"vm111_DATA-63_20140116111619","_snapshotId":1525,"_vmName":"i-5-63-VM","wait":0}}] } 2014-01-16 06:16:19,512 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396163: Executing: { Cmd , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 100011, [{"ManageSnapshotCommand":{"_commandSwitch":"-c","_volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","_pool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"_snapshotName":"vm111_DATA-63_20140116111619","_snapshotId":1525,"_vmName":"i-5-63-VM","wait":0}}] } 2014-01-16 06:17:52,373 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396163: Received: { Ans: , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 10, { ManageSnapshotAnswer } } 2014-01-16 06:17:52,391 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396168: Sending { Cmd , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:17:52,391 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396168: Executing: { Cmd , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:18:17,228 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 4-823396168: Received: { Ans: , MgmtId: 110492071566, via: 4, Ver: v1, Flags: 10, { BackupSnapshotAnswer } } 2014-01-16 06:18:17,228 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Task failed! Task record: uuid: f390af0e-32a9-aea7-c35d-d7aa15c4f163 2014-01-16 06:20:17,228 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) Retrying... 2014-01-16 06:20:17,230 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 1-733415213: Sending { Cmd , MgmtId: 110492071566, via: 1, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:20:17,231 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 1-733415213: Executing: { Cmd , MgmtId: 110492071566, via: 1, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:20:46,127 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 1-733415213: Received: { Ans: , MgmtId: 110492071566, via: 1, Ver: v1, Flags: 10, { BackupSnapshotAnswer } } 2014-01-16 06:20:46,127 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Task failed! Task record: uuid: a81edf2f-0889-fedd-ebe6-a46502591e52 2014-01-16 06:22:46,127 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) Retrying... 2014-01-16 06:22:46,130 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 112-1384318777: Sending { Cmd , MgmtId: 110492071566, via: 112, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:22:46,130 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 112-1384318777: Executing: { Cmd , MgmtId: 110492071566, via: 112, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:23:09,309 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 112-1384318777: Received: { Ans: , MgmtId: 110492071566, via: 112, Ver: v1, Flags: 10, { BackupSnapshotAnswer } } 2014-01-16 06:23:09,309 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Task failed! Task record: uuid: 217d8761-ff27-aca6-eb7f-511e9d67342c 2014-01-16 06:25:09,309 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) Retrying... 2014-01-16 06:25:09,312 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 116-479725007: Sending { Cmd , MgmtId: 110492071566, via: 116, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:25:09,313 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 116-479725007: Executing: { Cmd , MgmtId: 110492071566, via: 116, Ver: v1, Flags: 100011, [{"BackupSnapshotCommand":{"isVolumeInactive":false,"vmName":"i-5-63-VM","snapshotId":1525,"primaryStoragePoolNameLabel":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","primaryPool":{"id":202,"uuid":"2865ed4f-e8da-31f2-9353-c0abd93d8aba","host":"10.3.34.5","path":"/iqn.2001-05.com.equallogic:0-1cb196-305b0190a-84600000019511bd-cloudstackprimary/0","port":3260,"type":"IscsiLUN"},"snapshotUuid":"f8af702e-dae8-4ffd-ab29-4f4255e160ff","snapshotName":"vm111_DATA-63_20140116111619","secondaryStorageUrl":"nfs://mgt-cld.na.manwin.local/home/secondary","dcId":1,"accountId":5,"volumeId":82,"volumePath":"59eb6fcc-6e44-428d-8db5-f602384a4141","wait":21600}}] } 2014-01-16 06:25:33,440 DEBUG [agent.transport.Request] (Job-Executor-62:job-3754) Seq 116-479725007: Received: { Ans: , MgmtId: 110492071566, via: 116, Ver: v1, Flags: 10, { BackupSnapshotAnswer } } 2014-01-16 06:25:33,440 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) The result for com.cloud.agent.api.BackupSnapshotCommand is BackupSnapshot Failed due to Task failed! Task record: uuid: 2bed03a7-50aa-c821-9fe2-a27cd53ae333 2014-01-16 06:27:33,440 DEBUG [storage.snapshot.SnapshotManagerImpl] (Job-Executor-62:job-3754) Retrying... This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately. Ce courrier électronique est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courrier électronique par erreur, veuillez m'en aviser immédiatement, par retour de courrier électronique ou par un autre moyen.