Hi, just wondered, why one of my VMs had a completely broken filesystem. After some research I found an old instance of my template (I'm using persistent storage) running on one of my nodes. It looks like OpenNebula didn't handle the broken auth correctly (caused by NFS-issues) and marked the storage as free even if the VM is still running, isn't it?
-- Sat Jun 14 20:20:13 2014 [LCM][I]: New VM state is RUNNING Sat Jun 28 14:11:07 2014 [LCM][I]: New VM state is UNKNOWN Sat Jun 28 14:11:08 2014 [LCM][I]: New VM state is CLEANUP. Sat Jun 28 14:11:08 2014 [VMM][I]: Driver command for 74 cancelled Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail: /var/tmp/one/vmm/kvm/cancel 'one-74' 'node1' 74 node1 Sat Jun 28 14:11:09 2014 [VMM][I]: Warning: Permanently added 'node1,192.168.20.21' (ECDSA) to the list of known hosts. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password). Sat Jun 28 14:11:09 2014 [VMM][I]: ExitSSHCode: 255 Sat Jun 28 14:11:09 2014 [VMM][E]: Error connecting to node1 Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute virtualization driver operation: cancel. Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail: /var/tmp/one/vnm/dummy/clean PFZNPjxJRD43NDwvSUQ+PERFUExPWV9JRD5vbmUtNzQ8L0RFUExPWV9JRD48VEVNUExBVEU+PE5JQz48QlJJREdFPjwhW0NEQVRBW2JyMF1dPjwvQlJJREdFPjxJUD48IVtDREFUQVs4OS4yMzguODkuMzZdXT48L0lQPjxJUDZfTElOSz48IVtDREFUQVtmZTgwOjoyMTY6M2VmZjpmZWI5OjU2ZTBdXT48L0lQNl9MSU5LPjxNQUM+PCFbQ0RBVEFbMDA6MTY6M2U6Yjk6NTY6ZTBdXT48L01BQz48TkVUV09SSz48IVtDREFUQVs4OS4yMzguODkuMzItMjldXT48L05FVFdPUks+PE5FVFdPUktfSUQ+PCFbQ0RBVEFbNF1dPjwvTkVUV09SS19JRD48TkVUV09SS19VTkFNRT48IVtDREFUQVtvbmVhZG1pbl1dPjwvTkVUV09SS19VTkFNRT48TklDX0lEPjwhW0NEQVRBWzBdXT48L05JQ19JRD48VkxBTj48IVtDREFUQVtOT11dPjwvVkxBTj48L05JQz48L1RFTVBMQVRFPjxISVNUT1JZX1JFQ09SRFM+PEhJU1RPUlk+PEhPU1ROQU1FPm5vZGUxPC9IT1NUTkFNRT48L0hJU1RPUlk+PC9ISVNUT1JZX1JFQ09SRFM+PC9WTT4= Sat Jun 28 14:11:09 2014 [VMM][I]: Warning: Permanently added 'node1,192.168.20.21' (ECDSA) to the list of known hosts. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password). Sat Jun 28 14:11:09 2014 [VMM][I]: ExitSSHCode: 255 Sat Jun 28 14:11:09 2014 [VMM][E]: Error connecting to node1 Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute network driver operation: clean. Sat Jun 28 14:11:09 2014 [VMM][I]: Successfully execute transfer manager driver operation: tm_delete. Sat Jun 28 14:11:09 2014 [VMM][I]: Command execution fail: /var/lib/one/remotes/tm/shared/delete node1:/var/lib/one//datastores/0/74 74 0 Sat Jun 28 14:11:09 2014 [VMM][I]: delete: Deleting /var/lib/one/datastores/0/74 Sat Jun 28 14:11:09 2014 [VMM][E]: delete: Command "[ -e "/var/lib/one/datastores/0/74" ] || exit 0 Sat Jun 28 14:11:09 2014 [VMM][I]: Sat Jun 28 14:11:09 2014 [VMM][I]: times=10 Sat Jun 28 14:11:09 2014 [VMM][I]: function="rm -rf /var/lib/one/datastores/0/74" Sat Jun 28 14:11:09 2014 [VMM][I]: Sat Jun 28 14:11:09 2014 [VMM][I]: count=1 Sat Jun 28 14:11:09 2014 [VMM][I]: Sat Jun 28 14:11:09 2014 [VMM][I]: ret=$($function) Sat Jun 28 14:11:09 2014 [VMM][I]: error=$? Sat Jun 28 14:11:09 2014 [VMM][I]: Sat Jun 28 14:11:09 2014 [VMM][I]: while [ $count -lt $times -a "$error" != "0" ]; do Sat Jun 28 14:11:09 2014 [VMM][I]: sleep 1 Sat Jun 28 14:11:09 2014 [VMM][I]: count=$(( $count + 1 )) Sat Jun 28 14:11:09 2014 [VMM][I]: ret=$($function) Sat Jun 28 14:11:09 2014 [VMM][I]: error=$? Sat Jun 28 14:11:09 2014 [VMM][I]: done Sat Jun 28 14:11:09 2014 [VMM][I]: Sat Jun 28 14:11:09 2014 [VMM][I]: [ "x$error" = "x0" ]" failed: Warning: Permanently added 'node1,192.168.20.21' (ECDSA) to the list of known hosts. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied, please try again. Sat Jun 28 14:11:09 2014 [VMM][I]: Permission denied (publickey,password). Sat Jun 28 14:11:09 2014 [VMM][E]: Error deleting /var/lib/one/datastores/0/74 Sat Jun 28 14:11:09 2014 [VMM][I]: ExitCode: 255 Sat Jun 28 14:11:09 2014 [VMM][I]: Failed to execute transfer manager driver operation: tm_delete. Sat Jun 28 14:11:09 2014 [VMM][I]: Host successfully cleaned. Sat Jun 28 14:11:09 2014 [DiM][I]: New VM state is PENDING Sat Jun 28 14:11:33 2014 [DiM][I]: New VM state is ACTIVE. Sat Jun 28 14:11:33 2014 [LCM][I]: New VM state is PROLOG. Sat Jun 28 14:32:57 2014 [LCM][I]: New VM state is CLEANUP. Sat Jun 28 14:32:57 2014 [DiM][I]: New VM state is DONE Sat Jun 28 14:32:57 2014 [TM][W]: Ignored: LOG I 74 Driver command for 74 cancelled Sat Jun 28 14:32:59 2014 [TM][W]: Ignored: TRANSFER SUCCESS 74 - -- Fabian _______________________________________________ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org