ACS version: 4.2.1
Hypervisors: KVM
Storage pool type: CLVM
Since we upgraded from 4.1 to 4.2.1 moving volumes to a different primary
storage pool fail. I've enabled debug on the agents side and I think there
is a problem with the format type conversion
Volume on database has format QCOW2
these are the parameters for the first step (CLVM -> NFS):
"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"cda46430-52d7-4bf0-b0c2-adfc78dd011c","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"655d6965-b3f3-4118-a970-d50cf6afc365","id":211,"poolType":"CLVM","host":"localhost","path":"/FC10KY1","port":0}},"name":"ROOT-4450","size":5368709120,"path":"39a25daf-23a1-4b65-99ac-fb98469ac197","volumeId":5937,"vmName":"i-402-4450-VM","accountId":402,"format":"QCOW2","id":5937,"hypervisorType":"KVM"}}
"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"cda46430-52d7-4bf0-b0c2-adfc78dd011c","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://
192.168.11.6/home/a1iwstack
","_role":"Image"}},"name":"ROOT-4450","size":5368709120,"path":"volumes/402/5937","volumeId":5937,"vmName":"i-402-4450-VM","accountId":402,"format":"QCOW2","id":5937,"hypervisorType":"KVM"}}
Those commads are translated into the agent:
DEBUG [utils.script.Script] (agentRequest-Handler-1:null) Executing:
qemu-img info /dev/FC10KY1/39a25daf-23a1-4b65-99ac-fb98469ac197
DEBUG [utils.script.Script] (agentRequest-Handler-1:null) Execution is
successful.
DEBUG [utils.script.Script] (agentRequest-Handler-1:null) Executing: */bin/bash
-c cp -f /dev/FC10KY1/39a25daf-23a1-4b65-99ac-fb98469ac197
/mnt/b8311c72-fe75-3832-98fc-975445028a12/5c713376-c418-478c-8a31-89c4181cb48e.qcow2*
With the result that the output file isn't a qcow2 file but a raw
partition, which in turn make the next step fail.
(NFS -> CLVM)
DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: qemu-img
info
/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2
DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is
successful.
DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: qemu-img
convert -f qcow2 -O
raw/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2
/dev/FCSTORAGE/da162325-467b-4e78-af07-4bad85470d66
DEBUG [utils.script.Script] (agentRequest-Handler-2:) Exit value is 1
DEBUG [utils.script.Script] (agentRequest-Handler-2:) qemu-img: Could not
open
'/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2'qemu-img:
Could not open
'/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2'
ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:) Failed
to convert
/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2
to /dev/FCSTORAGE/da162325-467b-4e78-af07-4bad85470d66 the error was:
qemu-img: Could not open
'/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2'qemu-img:
Could not open
'/mnt/b8311c72-fe75-3832-98fc-975445028a12/b9303d8d-cd51-4b6c-a244-43c405df4238.qcow2'
If I change on the database the format of the volume to RAW the effect is
even worse as *data is lost* in the process!
These are the parameter for the first step (CLVM => NFS)
"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"cda46430-52d7-4bf0-b0c2-adfc78dd011c","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"655d6965-b3f3-4118-a970d50cf6afc365","id":211,"poolType":"CLVM","host":"localhost","path":"/FC10KY1","port":0}},"name":"ROOT-4450"
,"size":5368709120,"path":"39a25daf-23a1-4b65-99ac-fb98469ac197","volumeId":5937,"vmName":"i-4024450VM","accountId":402,
"format":"RAW","id":5937,"hypervisorType":"KVM"}},
"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"cda46430-52d7-4bf0-b0c2-adfc78dd011c","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://
192.168.11.6/home/a1iwstack
","_role":"Image"}},"name":"ROOT4450","size":5368709120,"path":"volumes/402/5937","volumeId":5937,"vmName":"i-402-4450-VM","accountId":402,
"format":"RAW","id":5937,"hypervisorType":"KVM"}}
this time the output is converted to qcow2!
DEBUG [utils.script.Script] (agentRequest-Handler-3:null) Executing:
qemu-img info /dev/FC10KY1/39a25daf-23a1-4b65-99ac-fb98469ac197
DEBUG [utils.script.Script] (agentRequest-Handler-3:null) Execution is
successful.
DEBUG [utils.script.Script] (agentRequest-Handler-3:null) *Executing:
qemu-img convert -f raw -O qcow2
*/dev/FC10KY1/39a25daf-23a1-4b65-99ac-fb98469ac197
/mnt/b8311c72-fe75-3832-98fc-975445028a12/01ab129f-aaf6-4b1a-8e2a-093bee0b811c.raw
and *data is lost* in the next step (NFS -> CLVM):
"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"cda46430-52d7-4bf0-b0c2-adfc78dd011c","volumeType":"ROOT","dataStore":{"
com.cl
oud.agent.api.to.NfsTO":{"_url":"nfs://192.168.11.6/home/a1iwstack
","_role":"Image"}},"name":"ROOT4450","size":5368709120,"path":"volumes/402/5937/01ab129f-aaf6-4b1a-8e2a-093bee0b811c.raw","volumeId":5937,"vmName":"i-402-4450-VM","accountId":402,
"format":"RAW","id":5937,"hypervisorType":"KVM"}}
"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2d684689-efbc-4ce3-95ad-b38e80a1afcf","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"4d59fde3-4c93-4aa5-8041-fab797b90899","id":214,"poolType":
"CLVM","host":"localhost","path":"/STOREY1","port":0}},"name":"ROOT-4450","size":5368709120,"path":"39a25daf-23a1-4b65-99ac-fb98469ac197","volumeId":5967,"vmName":"i-402-4450-VM","accountId":402,
"format":"RAW","id":5967,"hypervisorType":"KVM"}}
DEBUG [utils.script.Script] (agentRequest-Handler-4:null) Executing:
qemu-img info
/mnt/b8311c72-fe75-3832-98fc-975445028a12/01ab129f-aaf6-4b1a-8e2a-093bee0b811c.raw
DEBUG [utils.script.Script] (agentRequest-Handler-4:null) Execution is
successful.
DEBUG [utils.script.Script] (agentRequest-Handler-4:null) Executing:*
/bin/bash -c cp -f
*/mnt/b8311c72-fe75-3832-98fc-975445028a12/01ab129f-aaf6-4b1a-8e2a-093bee0b811c.raw
/dev/STOREY1/1e1ae58c-ca3f-4d4a-b0fd-92d18d07fb7c
Since the original format was actually QCOW2 (from the previous step) this
lead to data loss once the volume is expunged from the previous pool and
from the nfs storage!
Migration worked before we migrated to 4.2.1.
I'll appreciate any help!
Thanks
Best,
Salvatore
