[
https://issues.apache.org/jira/browse/CLOUDSTACK-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718552#comment-14718552
]
Józef Budziński commented on CLOUDSTACK-8786:
---------------------------------------------
Cloudstack agent-log
2015-08-28 12:57:34,637 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null)
Request:Seq 1-1489436813: { Cmd , MgmtId: 161332321633, via: 1, Ver: v1,
Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e/2080032e-1871-4daa-8109-a5321e3bcd1e","volume":{"uuid":"498c9f27-4c80-4f35-9db1-bd6ca9dc751e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d","id":1,"poolType":"NetworkFilesystem","host":"37.72.168.58","path":"/export/primary","port":2049,"url":"NetworkFilesystem://37.72.168.58//export/primary/?ROLE=Primary&STOREUUID=6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d"}},"name":"ROOT-6","size":10737418240,"path":"498c9f27-4c80-4f35-9db1-bd6ca9dc751e","volumeId":8,"vmName":"i-3-6-VM","accountId":3,"format":"QCOW2","id":8,"deviceId":0,"hypervisorType":"KVM"},"parentSnapshotPath":"/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e/cdc1c8ea-1b1a-4410-ba7b-59feead5a7d4","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d","id":1,"poolType":"NetworkFilesystem","host":"37.72.168.58","path":"/export/primary","port":2049,"url":"NetworkFilesystem://37.72.168.58//export/primary/?ROLE=Primary&STOREUUID=6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d"}},"vmName":"i-3-6-VM","name":"cloudvps1344-ubuntu_ROOT-6_20150828125724","hypervisorType":"KVM","id":5,"quiescevm":false,"parents":["/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e/cdc1c8ea-1b1a-4410-ba7b-59feead5a7d4"],"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/3/8","volume":{"uuid":"498c9f27-4c80-4f35-9db1-bd6ca9dc751e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d","id":1,"poolType":"NetworkFilesystem","host":"37.72.168.58","path":"/export/primary","port":2049,"url":"NetworkFilesystem://37.72.168.58//export/primary/?ROLE=Primary&STOREUUID=6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d"}},"name":"ROOT-6","size":10737418240,"path":"498c9f27-4c80-4f35-9db1-bd6ca9dc751e","volumeId":8,"vmName":"i-3-6-VM","accountId":3,"format":"QCOW2","id":8,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://37.72.168.58/export/secondary","_role":"Image"}},"vmName":"i-3-6-VM","name":"cloudvps1344-ubuntu_ROOT-6_20150828125724","hypervisorType":"KVM","id":5,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}]
}
2015-08-28 12:57:34,638 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null)
Processing command: org.apache.cloudstack.storage.command.CopyCommand
2015-08-28 12:57:34,640 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) createStoragePool didn't find existing running
pool: org.libvirt.LibvirtException: Storage pool not found: no pool with
matching uuid, need to create it
2015-08-28 12:57:34,640 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Didn't find an existing storage pool
986db431-d956-350c-917b-ee3381fded7a by UUID, checking for pools with duplicate
paths
2015-08-28 12:57:34,640 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Checking path of existing pool
c56b1198-8baa-488b-898c-42fd34363fe8 against pool we want to create
2015-08-28 12:57:34,642 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Checking path of existing pool
6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d against pool we want to create
2015-08-28 12:57:34,643 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Checking path of existing pool
4ac88d63-b3bc-3ff2-9638-dfe7a7de998d against pool we want to create
2015-08-28 12:57:34,644 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Attempting to create storage pool
986db431-d956-350c-917b-ee3381fded7a
2015-08-28 12:57:34,644 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) <pool type='netfs'>
<name>986db431-d956-350c-917b-ee3381fded7a</name>
<uuid>986db431-d956-350c-917b-ee3381fded7a</uuid>
<source>
<host name='37.72.168.58'/>
<dir path='/export/secondary'/>
</source>
<target>
<path>/mnt/986db431-d956-350c-917b-ee3381fded7a</path>
</target>
</pool>
2015-08-28 12:57:34,839 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-1:null) Executing:
/usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b
/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e
-n 2080032e-1871-4daa-8109-a5321e3bcd1e -p
/mnt/986db431-d956-350c-917b-ee3381fded7a/snapshots/3/8 -t
2080032e-1871-4daa-8109-a5321e3bcd1e
2015-08-28 12:57:34,899 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-1:null) Exit value is 2
2015-08-28 12:57:34,900 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-1:null) Failed to backup
2080032e-1871-4daa-8109-a5321e3bcd1e for disk
/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e
to /mnt/986db431-d956-350c-917b-ee3381fded7a/snapshots/3/8
2015-08-28 12:57:34,900 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-1:null) Failed to backup snaptshot: Failed to backup
2080032e-1871-4daa-8109-a5321e3bcd1e for disk
/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e
to /mnt/986db431-d956-350c-917b-ee3381fded7a/snapshots/3/8
2015-08-28 12:57:35,087 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null)
Seq 1-1489436813: { Ans: , MgmtId: 161332321633, via: 1, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Failed
to backup 2080032e-1871-4daa-8109-a5321e3bcd1e for disk
/mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d/498c9f27-4c80-4f35-9db1-bd6ca9dc751e
to /mnt/986db431-d956-350c-917b-ee3381fded7a/snapshots/3/8","wait":0}}] }
2015-08-28 12:57:41,752 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-3:null) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py cleanup_rules
2015-08-28 12:57:41,839 DEBUG [kvm.resource.LibvirtComputingResource]
(Agent-Handler-3:null) Execution is successful.
2015-08-28 12:57:41,839 DEBUG [cloud.agent.Agent] (Agent-Handler-3:null) Watch
Sent: Seq 1-1489436675: { Ans: , MgmtId: 161332321633, via: 1, Ver: v1, Flags:
10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}] }
2015-08-28 12:57:55,127 DEBUG [kvm.resource.KVMHAMonitor] (Thread-43:null)
Found NFS storage pool 6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d in libvirt,
continuing
2015-08-28 12:57:55,127 DEBUG [kvm.resource.KVMHAMonitor] (Thread-43:null)
Executing:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
37.72.168.58 -p /export/primary -m /mnt/6ca6fe9a-2f07-3dc6-9e8a-61f3837bec2d -h
37.72.168.59
2015-08-28 12:57:55,143 DEBUG [kvm.resource.KVMHAMonitor] (Thread-43:null)
Execution is successful.
2015-08-28 12:58:00,042 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-5:null) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
get_rule_logs_for_vms
2015-08-28 12:58:00,126 DEBUG [kvm.resource.LibvirtComputingResource]
(UgentTask-5:null) Execution is successful.
2015-08-28 12:58:00,133 DEBUG [cloud.agent.Agent] (UgentTask-5:null) Sending
ping: Seq 1-44: { Cmd , MgmtId: -1, via: 1, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.PingRoutingWithNwGroupsCommand":{"newGroupStates":{"i-3-5-VM":[5,6],"i-3-6-VM":[6,2]},"newStates":{},"_hostVmStateReport":{"i-3-5-VM":{"state":"PowerOn","host":"hypervisor.swiftway.net"},"r-4-VM":{"state":"PowerOn","host":"hypervisor.swiftway.net"},"s-2-VM":{"state":"PowerOn","host":"hypervisor.swiftway.net"},"i-3-6-VM":{"state":"PowerOn","host":"hypervisor.swiftway.net"},"v-1-VM":{"state":"PowerOn","host":"hypervisor.swiftway.net"}},"_gatewayAccessible":true,"_vnetAccessible":true,"hostType":"Routing","hostId":1,"wait":0}}]
}
2015-08-28 12:58:00,140 DEBUG [cloud.agent.Agent] (Agent-Handler-4:null)
Received response: Seq 1-44: { Ans: , MgmtId: 161332321633, via: 1, Ver: v1,
Flags: 100010,
[{"com.cloud.agent.api.PingAnswer":{"_command":{"hostType":"Routing","hostId":1,"wait":0},"result":true,"wait":0}}]
}
2015-08-28 12:58:14,978 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null)
Processing command: com.cloud.agent.api.GetVmStatsCommand
2015-08-28 12:58:21,491 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null)
Processing command: com.cloud.agent.api.GetStorageStatsCommand
2015-08-28 12:58:31,831 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Processing command: com.cloud.agent.api.GetHostStatsCommand
2015-08-28 12:58:31,831 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing: /bin/bash -c idle=$(top -b -n 1| awk
-F, '/^[%]*[Cc]pu/{$0=$4; gsub(/[^0-9.,]+/,""); print }'); echo $idle
2015-08-28 12:58:32,352 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
2015-08-28 12:58:32,352 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing: /bin/bash -c freeMem=$(free|grep
cache:|awk '{print $4}');echo $freeMem
2015-08-28 12:58:32,355 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
2015-08-28 12:58:32,355 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing: /bin/bash -c free|grep Mem:|awk
'{print $2}'
2015-08-28 12:58:32,357 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
> Failed to create snapshot
> -------------------------
>
> Key: CLOUDSTACK-8786
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8786
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Hypervisor Controller, KVM, Snapshot
> Affects Versions: 4.2.1, 4.3.2
> Environment: Centos 6.6, Centos 6.7
> KVM
> Reporter: Józef Budziński
> Priority: Critical
>
> When atempt to create storage volume root snapshot achived
> Problem with snapshots VM ERROR: { "createsnapshotresponse" :
> {"uuidList":[],"errorcode":530,"cserrorcode":4250,"errortext":"Internal error
> executing command, please contact your system administrator"} }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
