Re: [one-users] Native GlusterFS support
Now that the packages for OpenNebula 4.6 beta are ready is anyone willing to give a shot to the gluster integration? Any feedback is welcome. Post: http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/ Packages: http://opennebula.org/software/ Documentation: http://docs.opennebula.org/4.6/administration/storage/gluster_ds.html On Fri, Mar 7, 2014 at 12:00 PM, Javier Fontan wrote: > Hi, > > One of the features we have been working on is the native support of > GlusterFS for VM images. The advantages here are faster IO as FUSE is > no longer used for running VMs and multiple replicas usage. Get some > more info in this post: > > http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/ > > Cheers > > -- > Javier Fontán Muiños > Developer > OpenNebula - The Open Source Toolkit for Data Center Virtualization > www.OpenNebula.org | @OpenNebula | github.com/jfontan -- Javier Fontán Muiños Developer OpenNebula - The Open Source Toolkit for Data Center Virtualization www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Quoting Javier Fontan (jfon...@opennebula.org): > Now that the packages for OpenNebula 4.6 beta are ready is anyone > willing to give a shot to the gluster integration? Any feedback is > welcome. I'll try to set up a Gluster cluster and a Ceph cluster this week with ONE 4.6 beta on Ubuntu Trusty Tahr / 14.04 LTS. I hope to have a demo ready at the upcoming techday @BIT. Gr. Stefan -- | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Quoting Javier Fontan (jfon...@opennebula.org): > Now that the packages for OpenNebula 4.6 beta are ready is anyone > willing to give a shot to the gluster integration? Any feedback is > welcome. > > Post: http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/ > Packages: http://opennebula.org/software/ > Documentation: > http://docs.opennebula.org/4.6/administration/storage/gluster_ds.html I'm trying glusterfs on Ubuntu Saucy (frontend) and Ubuntu Trusty (nodes). I've followed the documentation but something is not working. Might be me missing something here. DATASTORE 101 INFORMATION ID : 101 NAME : gluster_gv0_ds USER : oneadmin GROUP : oneadmin CLUSTER: - TYPE : IMAGE DS_MAD : shared TM_MAD : shared BASE PATH : /var/lib/one//datastores/101 DISK_TYPE : DATASTORE CAPACITY TOTAL: : 0M FREE: : 0M USED: : 0M LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH="/var/lib/one//datastores/" CLONE_TARGET="SYSTEM" DISK_TYPE="GLUSTER" DS_MAD="shared" GLUSTER_HOST="gluster1:24007" GLUSTER_VOLUME="gv0" LN_TARGET="NONE" TM_MAD="shared" TYPE="IMAGE_DS" oneadmin@oned1:~$ onedatastore list ID NAMESIZE AVAIL CLUSTER IMAGES TYPE DS TM 0 system215.9G 92% - 0 sys -shared 1 default 215.9G 92% - 1 img fs shared 2 files 17.6G 38% - 0 fil fs ssh 100 ceph_one_ds 1T 91% - 2 img ceph ceph 101 gluster_gv0_d 0M - - 0 img shared shared It reports "0M" - AVAIL. The "system" and "default" datastores are mounted glusterfs volume (so glusterfs does work on frontend/nodes). If I try to import an image from marketplace I get the following error: [ImageAllocate] Cannot determine Image SIZE. Datastore driver 'shared' not available. Gr. Stefan -- | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Maybe I have a bug in the documentation. Set the ds mad to "fs" and check if it works. On Mon, Mar 24, 2014 at 9:03 PM, Stefan Kooman wrote: > Quoting Javier Fontan (jfon...@opennebula.org): >> Now that the packages for OpenNebula 4.6 beta are ready is anyone >> willing to give a shot to the gluster integration? Any feedback is >> welcome. >> >> Post: http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/ >> Packages: http://opennebula.org/software/ >> Documentation: >> http://docs.opennebula.org/4.6/administration/storage/gluster_ds.html > > I'm trying glusterfs on Ubuntu Saucy (frontend) and Ubuntu Trusty > (nodes). I've followed the documentation but something is not working. > Might be me missing something here. > > DATASTORE 101 INFORMATION > ID : 101 > NAME : gluster_gv0_ds > USER : oneadmin > GROUP : oneadmin > CLUSTER: - > TYPE : IMAGE > DS_MAD : shared > TM_MAD : shared > BASE PATH : /var/lib/one//datastores/101 > DISK_TYPE : > > DATASTORE CAPACITY > TOTAL: : 0M > FREE: : 0M > USED: : 0M > LIMIT: : - > > PERMISSIONS > OWNER : um- > GROUP : u-- > OTHER : --- > > DATASTORE TEMPLATE > BASE_PATH="/var/lib/one//datastores/" > CLONE_TARGET="SYSTEM" > DISK_TYPE="GLUSTER" > DS_MAD="shared" > GLUSTER_HOST="gluster1:24007" > GLUSTER_VOLUME="gv0" > LN_TARGET="NONE" > TM_MAD="shared" > TYPE="IMAGE_DS" > > oneadmin@oned1:~$ onedatastore list > ID NAMESIZE AVAIL CLUSTER IMAGES TYPE DS TM >0 system215.9G 92% - 0 sys -shared >1 default 215.9G 92% - 1 img fs shared >2 files 17.6G 38% - 0 fil fs ssh > 100 ceph_one_ds 1T 91% - 2 img ceph ceph > 101 gluster_gv0_d 0M - - 0 img shared shared > > It reports "0M" - AVAIL. The "system" and "default" datastores are mounted > glusterfs volume (so glusterfs does work on frontend/nodes). > > If I try to import an image from marketplace I get the following error: > > [ImageAllocate] Cannot determine Image SIZE. Datastore driver 'shared' > not available. > > Gr. Stefan > > > -- > | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 > | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl -- Javier Fontán Muiños Developer OpenNebula - The Open Source Toolkit for Data Center Virtualization www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Quoting Javier Fontan (jfon...@opennebula.org): > Maybe I have a bug in the documentation. Set the ds mad to "fs" and > check if it works. I changed the setting and that helped :). Unfortunately qemu logs an error: qemu-system-x86_64: -drive file=gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4,if=none,id=drive-virtio-disk0,format=raw,cache=none: Gluster connection failed for server=gluster1 port=24007 volume=gv0 image=53f35f5ca22540d558404caa3b9907d4 transport=tcp ... while the gluster volume is mounted correctly on this hypervisor I'll try to debug this later. Gr. Stefan -- | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
It could be a permission problem. Try this command with both root and oneadmin: $ qemu-img info gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 If just oneadmin does not work you may need to change some configuration parameters to let non-root users access glusterfs: http://docs.opennebula.org/4.6/administration/storage/gluster_ds.html#glusterfs-volume-configuration On Tue, Mar 25, 2014 at 4:52 PM, Stefan Kooman wrote: > Quoting Javier Fontan (jfon...@opennebula.org): >> Maybe I have a bug in the documentation. Set the ds mad to "fs" and >> check if it works. > > I changed the setting and that helped :). Unfortunately qemu logs an error: > > qemu-system-x86_64: -drive > file=gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4,if=none,id=drive-virtio-disk0,format=raw,cache=none: > Gluster connection failed for server=gluster1 port=24007 volume=gv0 > image=53f35f5ca22540d558404caa3b9907d4 transport=tcp > > ... while the gluster volume is mounted correctly on this hypervisor > > I'll try to debug this later. > > Gr. Stefan > > -- > | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 > | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl -- Javier Fontán Muiños Developer OpenNebula - The Open Source Toolkit for Data Center Virtualization www.OpenNebula.org | @OpenNebula | github.com/jfontan ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Quoting Javier Fontan (jfon...@opennebula.org): > It could be a permission problem. Try this command with both root and > oneadmin: > > $ qemu-img info gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 qemu-img info gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 qemu-img: Could not open 'gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4': Unknown protocol It turns out that stock libvirt/qemu in Ubuntu Trusty do _not_ have gluster support. I've installed a PPA [1], with that PPA installed it does work: qemu-img info gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 image: gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 file format: raw virtual size: 40M (41943040 bytes) disk size: 40M (ttylinux image). I'll go ahead and install the PPA on the other node too. Thanks for helping me out. Gr. Stefan -- | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Can you share that ppa for the community please? I am assuming the qemu-img ppa that has gluster support. FYI the fedora releases 19 and 20 do have glusterfs support in qemu without extra packages. On Tue, Mar 25, 2014 at 1:09 PM, Stefan Kooman wrote: > Quoting Javier Fontan (jfon...@opennebula.org): > > It could be a permission problem. Try this command with both root and > oneadmin: > > > > $ qemu-img info > gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 > > qemu-img info > gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 > qemu-img: Could not open > 'gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4': Unknown > protocol > > It turns out that stock libvirt/qemu in Ubuntu Trusty do _not_ have > gluster support. I've installed a PPA [1], with that PPA installed it > does work: > > qemu-img info > gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 > image: gluster://gluster1:24007/gv0/53f35f5ca22540d558404caa3b9907d4 > file format: raw > virtual size: 40M (41943040 bytes) > disk size: 40M > > (ttylinux image). > > I'll go ahead and install the PPA on the other node too. Thanks for > helping me out. > > Gr. Stefan > > -- > | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 > | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl > ___ > Users mailing list > Users@lists.opennebula.org > http://lists.opennebula.org/listinfo.cgi/users-opennebula.org > ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
Re: [one-users] Native GlusterFS support
Quoting Shankhadeep Shome (shank15...@gmail.com): > Can you share that ppa for the community please? I am assuming the qemu-img > ppa that has gluster support. FYI the fedora releases 19 and 20 do have > glusterfs support in qemu without extra packages. https://launchpad.net/~semiosis/+archive/ubuntu-qemu-glusterfs It's not my PPA but it works. I've run into issues (1 UNIT test failed) while enabeling support for glusterfs in ubuntu trusty deb. Gr. Stefan -- | BIT BV http://www.bit.nl/Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / i...@bit.nl ___ Users mailing list Users@lists.opennebula.org http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
