[one-users] Copy image on Ceph to file-based datastore
Hi all, In OpenNebula, it seems we can easily set it up to use Ceph as a datastore for images, cloning non-persistent images to make temporary copies, etc. We can also have the images stored as files on the frontend computer and dolled out via SSH; and in the case of persistent images, copied back when the VM is undeployed. What I'm missing is a hybrid between the two. We've got a 3-node Ceph cluster which is working well. However, compared to a local SSD, the Ceph cluster with its 6 spinners is a bit on the slow side in comparison. There are VMs for which we don't care about resiliency, and so it makes sense to use the SSDs attached to the VM hosts, and store the image itself in Ceph. The thought is, when the VM starts up, rather than having it dished out from the frontend computer, the host does a: rbd export pool-name/image-name /var/lib/one/datastores/123/45/disk.0 Then, if the image is persistent, it does a rbd import pool-name/image-name /var/lib/one/datastores/123/45/disk.0 (Note: not sure if import will overwrite the image, it may be necessary to do a rbd rm first of the old image... or better, import the new copy then remove and rename.) I had a tinker looking at the ssh transfer driver, but couldn't quite figure out how to reach out to the other datastore without some ugly kludges. Anyone else done somthing like this? Regards, -- Stuart Longland Systems Engineer _ ___ \ /|_) | T: +61 7 3535 9619 \/ | \ | 38b Douglas StreetF: +61 7 3535 9699 SYSTEMSMilton QLD 4064 http://www.vrt.com.au ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] Database management
Dear OpenNebula users, I need some information about the ONE database management / maintenance, on which basis are old values removed to keep the DB small? Is it based on time intervals, DB size or something different? And furthermore, can it be configured? I looked through the net and unfortunately, there is no clear information provided. Thanks for your help, Michael - Michael Gienger Höchstleistungsrechenzentrum Stuttgart Universität Stuttgart Nobelstraße 19 70569 Stuttgart Phone: +49 711 685 63824 Fax: +49 711 685 65832 ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
[one-users] image pool duplicate key
Hello i had some hardware failure .. but i had some backup made .. so after installed full system back .. i had to recopy database from old backup .. and files .. added manualy in database the images .. to match the old configs and info .. but now if try to clone any image .. i get duplicate key .. i have try to change autoincrement for image_pool index but is not working any ideea ? SQL command was: INSERT INTO image_pool (oid, name, body, uid, gid, owner_u, group_u, other_u) VALUES (0,'testasd','IMAGEID0/IDUID0/UIDGID0/GIDUNAMEoneadmin/UNAMEGNAMEoneadmin/GNAMENAMEtestasd/NAMEPERMISSIONSOWNER_U1/OWNER_UOWNER_M1/OWNER_MOWNER_A0/OWNER_AGROUP_U0/GROUP_UGROUP_M0/GROUP_MGROUP_A0/GROUP_AOTHER_U0/OTHER_UOTHER_M0/OTHER_MOTHER_A0/OTHER_A/PERMISSIONSTYPE0/TYPEDISK_TYPE0/DISK_TYPEPERSISTENT1/PERSISTENTREGTIME1394716874/REGTIMESOURCE/SOURCEPATH/var/lib/one/datastores/1/1870e708127c10e3a81f4bc62c5afdd6/PATHFSTYPE/FSTYPESIZE8192/SIZESTATE4/STATERUNNING_VMS0/RUNNING_VMSCLONING_OPS0/CLONING_OPSCLONING_ID119/CLONING_IDDATASTORE_ID1/DATASTORE_IDDATASTOREdefault/DATASTOREVMS/VMSCLONES/CLONESTEMPLATEDEV_PREFIX![CDATA[hd]]/DEV_PREFIXFSTYPE![CDATA[]]/FSTYPE/TEMPLATE/IMAGE',0,0,1,0,0), error 1062 : Duplicate entry '0' for key 'PRIMARY' Thu Mar 13 14:21:14 2014 [ReM][E]: Req:9424 UID:0 ImageClone result FAILURE [ImageClone] Error allocating a new image. ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] image pool duplicate key
Indices are controlled by OpenNebula and the last id is saved to table pool_control. Modify it so the ID is greater that the latest image and start oned again. On Thu, Mar 13, 2014 at 2:34 PM, Ionut Popovici io...@hackaserver.com wrote: Hello i had some hardware failure .. but i had some backup made .. so after installed full system back .. i had to recopy database from old backup .. and files .. added manualy in database the images .. to match the old configs and info .. but now if try to clone any image .. i get duplicate key .. i have try to change autoincrement for image_pool index but is not working any ideea ? SQL command was: INSERT INTO image_pool (oid, name, body, uid, gid, owner_u, group_u, other_u) VALUES (0,'testasd','IMAGEID0/IDUID0/UIDGID0/GIDUNAMEoneadmin/UNAMEGNAMEoneadmin/GNAMENAMEtestasd/NAMEPERMISSIONSOWNER_U1/OWNER_UOWNER_M1/OWNER_MOWNER_A0/OWNER_AGROUP_U0/GROUP_UGROUP_M0/GROUP_MGROUP_A0/GROUP_AOTHER_U0/OTHER_UOTHER_M0/OTHER_MOTHER_A0/OTHER_A/PERMISSIONSTYPE0/TYPEDISK_TYPE0/DISK_TYPEPERSISTENT1/PERSISTENTREGTIME1394716874/REGTIMESOURCE/SOURCEPATH/var/lib/one/datastores/1/1870e708127c10e3a81f4bc62c5afdd6/PATHFSTYPE/FSTYPESIZE8192/SIZESTATE4/STATERUNNING_VMS0/RUNNING_VMSCLONING_OPS0/CLONING_OPSCLONING_ID119/CLONING_IDDATASTORE_ID1/DATASTORE_IDDATASTOREdefault/DATASTOREVMS/VMSCLONES/CLONESTEMPLATEDEV_PREFIX![CDATA[hd]]/DEV_PREFIXFSTYPE![CDATA[]]/FSTYPE/TEMPLATE/IMAGE',0,0,1,0,0), error 1062 : Duplicate entry '0' for key 'PRIMARY' Thu Mar 13 14:21:14 2014 [ReM][E]: Req:9424 UID:0 ImageClone result FAILURE [ImageClone] Error allocating a new image. ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Javier Fontán Muiños Developer OpenNebula - The Open Source Toolkit for Data Center Virtualization www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] image pool duplicate key
Hi, On Thu, Mar 13, 2014 at 2:34 PM, Ionut Popovici io...@hackaserver.com wrote: Hello i had some hardware failure .. but i had some backup made .. so after installed full system back .. i had to recopy database from old backup .. and files .. added manualy in database the images .. to match the old configs and info .. but now if try to clone any image .. i get duplicate key .. i have try to change autoincrement for image_pool index but is not working any ideea ? SQL command was: INSERT INTO image_pool (oid, name, body, uid, gid, owner_u, group_u, other_u) VALUES (0,'testasd','IMAGEID0/ IDUID0/UIDGID0/GIDUNAMEoneadmin/UNAMEGNAME oneadmin/GNAMENAMEtestasd/NAMEPERMISSIONSOWNER_U1 /OWNER_UOWNER_M1/OWNER_MOWNER_A0/OWNER_AGROUP_U0 /GROUP_UGROUP_M0/GROUP_MGROUP_A0/GROUP_AOTHER_U0 /OTHER_UOTHER_M0/OTHER_MOTHER_A0/OTHER_A/ PERMISSIONSTYPE0/TYPEDISK_TYPE0/DISK_TYPE PERSISTENT1/PERSISTENTREGTIME1394716874/REGTIME SOURCE/SOURCEPATH/var/lib/one/datastores/1/ 1870e708127c10e3a81f4bc62c5afdd6/PATHFSTYPE/FSTYPE SIZE8192/SIZESTATE4/STATERUNNING_VMS0/RUNNING_ VMSCLONING_OPS0/CLONING_OPSCLONING_ID119/CLONING_ IDDATASTORE_ID1/DATASTORE_IDDATASTOREdefault/ DATASTOREVMS/VMSCLONES/CLONESTEMPLATEDEV_PREFIX ![CDATA[hd]]/DEV_PREFIXFSTYPE![CDATA[]]/FSTYPE/TEMPLATE/IMAGE',0,0,1,0,0), error 1062 : Duplicate entry '0' for key 'PRIMARY' Thu Mar 13 14:21:14 2014 [ReM][E]: Req:9424 UID:0 ImageClone result FAILURE [ImageClone] Error allocating a new image. OpenNebula includes the onedb fsck tool to check and fix DB consistency problems: http://docs.opennebula.org/stable/administration/references/onedb.html Regards -- Carlos Martín, MSc Project Engineer OpenNebula - Flexible Enterprise Cloud Made Simple www.OpenNebula.org | cmar...@opennebula.org | @OpenNebulahttp://twitter.com/opennebulacmar...@opennebula.org ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Database management
* VMs are never deleted just marked as DONE. This data will be used later for accounting. * The amount of monitoring data can be configured in oned.conf. Check INTERVAL and EXPIRATION parameters [1]. * Other objects are deleted as soon as you issue a delete action. [1] http://docs.opennebula.org/stable/administration/references/oned_conf.html#daemon-configuration-attributes On Thu, Mar 13, 2014 at 12:02 PM, Michael Gienger gien...@hlrs.de wrote: Dear OpenNebula users, I need some information about the ONE database management / maintenance, on which basis are old values removed to keep the DB small? Is it based on time intervals, DB size or something different? And furthermore, can it be configured? I looked through the net and unfortunately, there is no clear information provided. Thanks for your help, Michael - Michael Gienger Höchstleistungsrechenzentrum Stuttgart Universität Stuttgart Nobelstraße 19 70569 Stuttgart Phone: +49 711 685 63824 Fax: +49 711 685 65832 ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org -- Javier Fontán Muiños Developer OpenNebula - The Open Source Toolkit for Data Center Virtualization www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Database management
Hi, On Thu, Mar 13, 2014 at 4:25 PM, Javier Fontan jfon...@opennebula.orgwrote: * VMs are never deleted just marked as DONE. This data will be used later for accounting. * The amount of monitoring data can be configured in oned.conf. Check INTERVAL and EXPIRATION parameters [1]. * Other objects are deleted as soon as you issue a delete action. [1] http://docs.opennebula.org/stable/administration/references/oned_conf.html#daemon-configuration-attributes On Thu, Mar 13, 2014 at 12:02 PM, Michael Gienger gien...@hlrs.de wrote: Dear OpenNebula users, I need some information about the ONE database management / maintenance, on which basis are old values removed to keep the DB small? Is it based on time intervals, DB size or something different? And furthermore, can it be configured? I looked through the net and unfortunately, there is no clear information provided. Thanks for your help, Michael Adding to what Javi said, the scalability guide discusses the DB storage requirements depending on the oned.conf configuration and your cloud size: http://docs.opennebula.org/stable/advanced_administration/scalability/one_scalability.html Regards -- Carlos Martín, MSc Project Engineer OpenNebula - Flexible Enterprise Cloud Made Simple www.OpenNebula.org http://www.opennebula.org/ | cmar...@opennebula.org | @OpenNebula http://twitter.com/opennebula ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
