Hi,
For some reason, we can no longer convert snapshots to template if we
use Swift as secondary storage. I don't understand what's going on since
this was working couple weeks ago, and we didn't change anything... afaik!
The behavior we see is the snapshot is not pushed back to the image
store prior attempting to run the privatetemplate_from_snapshot_xen script:
On the Management Server:
2014-10-22 10:07:53,839 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) copyAsync
inspecting src type SNAPSHOT copyAsync inspecting dest type TEMPLATE
2014-10-22 10:07:53,858 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151)
needCacheStorage true, dest at template/tmpl/40/229 dest role
ImageS-367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd src role Image
2014-10-22 10:07:53,881 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) copyAsync
inspecting src type SNAPSHOT copyAsync inspecting dest type SNAPSHOT
2014-10-22 10:07:53,957 DEBUG [c.c.a.t.Request]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) Seq
43-3461016313634228639: Sending { Cmd , MgmtId: 130589506760615, via:
43(s-345-VM), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"S-367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd","volume":{"uuid":"9fece68b-904e-496c-a1db-9107784b66c6","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"84235d47-ce49-3d9b-b60e-12afb358853e","id":7,"poolType":"IscsiLUN","host":"172.25.32.254","path":"/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0","port":3260,"url":"IscsiLUN://172.25.xx.xxx/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0/?ROLE=Primary&STOREUUID=84235d47-ce49-3d9b-b60e-12afb358853e"}},"name":"ROOT-174","size":17179869184,"path":"6d1c8c4b-c650-42be-a56c-925fc37cbed7","volumeId":367,"vmName":"i-40-174-VM","accountId":40,"format":"VHD","id":367,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.SwiftTO":{"id":4,"url":"STRIPPED","account":"xxx","userName":"xxx","key":"xxx"}},"vmName":"i-40-174-VM","name":"Sysbench_ROOT-174_20141022140400","hypervisorType":"XenServer","id":107,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/40/367","volume":{"uuid":"9fece68b-904e-496c-a1db-9107784b66c6","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"84235d47-ce49-3d9b-b60e-12afb358853e","id":7,"poolType":"IscsiLUN","host":"172.25.xx.xx","path":"/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0","port":3260,"url":"IscsiLUN://172.25.xx.xx/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0/?ROLE=Primary&STOREUUID=84235d47-ce49-3d9b-b60e-12afb358853e"}},"name":"ROOT-174","size":17179869184,"path":"6d1c8c4b-c650-42be-a56c-925fc37cbed7","volumeId":367,"vmName":"i-40-174-VM","accountId":40,"format":"VHD","id":367,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://172.25.xx.xx/data/secondary","_role":"ImageCache"}},"vmName":"i-40-174-VM","name":"Sysbench_ROOT-174_20141022140400","hypervisorType":"XenServer","id":107,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}]
}
2014-10-22 10:07:54,309 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-15:null) Seq 43-3461016313634228639: Processing:
{ Ans: , MgmtId: 130589506760615, via: 43, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}]
}
2014-10-22 10:07:54,309 DEBUG [c.c.a.t.Request]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) Seq
43-3461016313634228639: Received: { Ans: , MgmtId: 130589506760615,
via: 43, Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-10-22 10:07:54,350 DEBUG [c.c.a.t.Request]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) Seq
43-3461016313634228640: Sending { Cmd , MgmtId: 130589506760615, via:
43(s-345-VM), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd","volume":{"uuid":"9fece68b-904e-496c-a1db-9107784b66c6","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"84235d47-ce49-3d9b-b60e-12afb358853e","id":7,"poolType":"IscsiLUN","host":"172.25.xx.xx","path":"/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0","port":3260,"url":"IscsiLUN://172.25.xx.xx/iqn.2010-01.com.solidfire:tsdz.cca-c7-cluster1-sr01.5/0/?ROLE=Primary&STOREUUID=84235d47-ce49-3d9b-b60e-12afb358853e"}},"name":"ROOT-174","size":17179869184,"path":"6d1c8c4b-c650-42be-a56c-925fc37cbed7","volumeId":367,"vmName":"i-40-174-VM","accountId":40,"format":"VHD","id":367,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://172.25.xx.xx/data/secondary","_role":"ImageCache"}},"vmName":"i-40-174-VM","name":"Sysbench_ROOT-174_20141022140400","hypervisorType":"XenServer","id":107,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/40/229","uuid":"3a52db5c-00c5-45ec-b393-11fd5dba498c","id":229,"format":"RAW","accountId":40,"hvm":true,"displayText":"test","imageDataStore":{"com.cloud.agent.api.to.SwiftTO":{"id":4,"url":"http://STRIPPED","account":"xxx","userName":"xxx","key":"xxx"}},"name":"11098f6bcd-4621-3373-8ade-4e832627b4f6","hypervisorType":"XenServer"}},"executeInSequence":false,"options":{},"wait":10800}}]
}
2014-10-22 10:07:54,388 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-1:null) Seq 43-3461016313634228640: Processing: {
Ans: , MgmtId: 130589506760615, via: 43, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"30#failed
to query
/mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd","wait":0}}]
}
2014-10-22 10:07:54,388 DEBUG [c.c.a.t.Request]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) Seq
43-3461016313634228640: Received: { Ans: , MgmtId: 130589506760615,
via: 43, Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-10-22 10:07:54,402 DEBUG [c.c.t.TemplateManagerImpl]
(API-Job-Executor-11:ctx-61de07d0 job-8087 ctx-ed4c2151) Failed to
create template30#failed to query
/mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd
2014-10-22 10:07:54,425 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-11:ctx-61de07d0 job-8087) Unexpected exception while
executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
com.cloud.utils.exception.CloudRuntimeException: Failed to create
template30#failed to query
/mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd
at
com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1402)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
On the SSVM:
2014-10-22 14:27:59,230 DEBUG
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) Executing:
/usr/local/cloud/systemvm/scripts/storage/secondary/create_privatetemplate_from_snapshot_xen.sh
-p /mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/snapshots/40/367
-s 7193b0c9-a61c-46f0-accd-665050b02f46.vhd -n
7f4a0432-ce74-47c4-8e59-1dc9d2469238.vhd -t
/mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/template/tmpl/40/235
2014-10-22 14:27:59,238 DEBUG
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) Exit value is 2
2014-10-22 14:27:59,238 DEBUG
[storage.resource.NfsSecondaryStorageResource]
(agentRequest-Handler-1:null) 30#failed to query
/mnt/SecStorage/4dbd3ea3-688f-3eda-a48f-4c3062e3029f/snapshots/40/367/7193b0c9-a61c-46f0-accd-665050b02f46.vhd
Any Ideas?
--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775
CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_
