cool, thanks.
2013/9/11 John Skinner <john.skin...@appcore.com> > I ran each test independently for each block size. > > iozone -I -i 0 -i 1 -i 2 -r 4k -s 1G > > In order: -I to specify direct-IO, -i 0 to specify write/rewrite, -i 1 to > specify read/re-read, -i 2 to specify random read/write, -r specify block > size, -s to specify file size. > > The report is pretty thorough. What I put in the email was different, I > just took the throughput from each test and put it into a table outside of > the full report. > > ----- Original Message ----- > > From: "Rafael Weingartner" <rafaelweingart...@gmail.com> > To: users@cloudstack.apache.org > Sent: Wednesday, September 11, 2013 4:16:23 PM > Subject: Re: Cloustack 4.1.0 + GlusterFS > > I have never used iozone before, > > How did you get that report? > > I tried: iozone -s 1024m -t 1 -R > > But the report was pretty different from yours. > > > 2013/9/11 John Skinner <john.skin...@appcore.com> > > > I currently have GlusterFS deployed into an 8 node KVM cluster running on > > CloudStack 4.1 for primary storage. Gluster is deployed on 28 1TB drives > > across 2 separate storage appliances using a distributed-replicated > volume > > with the replica set to 2. The storage network is 10Gb copper. > > > > These are the options I have configured for the volume in Gluster, most > of > > them are from a Red Hat document on configuring Red Hat Enterprise > Storage > > for VM hosting: > > > > > > > > performance.io-thread-count: 32 > > performance.cache-size: 1024MB > > performance.write-behind-window-size: 5MB > > performance.write-behind: on > > network.remote-dio: on > > cluster.eager-lock: enable > > performance.stat-prefetch: off > > performance.io-cache: on > > performance.read-ahead: on > > performance.quick-read: on > > > > Here are some of the numbers I was getting when benchmarking the storage > > from the KVM node directly (not a VM) > > > > The below table is in KB/s. The test is single stream 1GB file utilizing > > Direct I/O (no cache). I used iozone to run the benchmark. > > > > Write 4k 45729 > > Read 4k 10189 > > Random Write 4k 31983 > > Random Read 4k 9859 > > Write 16k 182246 > > Read 16k 37146 > > Random Write 16k 113026 > > Random Read 16k 37237 > > Write 64k 420908 > > Read 64k 125315 > > Random Write 64k 383848 > > Random Read 64k 125218 > > Write 256k 567501 > > Read 256k 218413 > > Random Write 256k 508650 > > Random Read 256k 229117 > > > > In the above results, I have the volume mounted to each KVM host as a > FUSE > > glusterfs file system. They are added to CloudStack as a shared mount > > point. In the future it would be great to utilize GlusterFS qemu libvirt > > integration with libgfapi so I could bypass fuse altogether. However, > that > > would require adding that code to CloudStack to support that. > > > > I maybe have 15 or so VMs running from the storage now and it is still > > pretty snappy. Need to do some more testing though and really get it > loaded. > > > > ----- Original Message ----- > > > > > From: "Rafael Weingartner" <rafaelweingart...@gmail.com> > > To: users@cloudstack.apache.org > > Sent: Wednesday, September 11, 2013 8:48:07 AM > > Subject: Re: Cloustack 4.1.0 + GlusterFS > > > > Right now I can think in three main reasons: > > > > The first reason, performance (I do not know Gluster and its performance > > and if the read and write speed are satisfactory). Please if you can > make a > > test, post the results. > > > > Second consistency, I do not know Gluster, but swift that is also a > > Distributed File System is not consistency and they make it pretty clear > on > > their page (http://docs.openstack.org/developer/swift/) > > > > "Swift is a highly available, distributed, eventually consistent > > object/blob store...". > > > > I would not accept to storage my VMs images on a FS that is eventually > > consistent. > > > > Third, network, I haven't used this kind of FS, but I can image that it > > uses a lot of bandwidth to keep synchronizing, managing and securing the > > data. So, managing the networking would be a pain. > > > > > > > > 2013/9/11 Olivier Mauras <oliv...@core-hosting.net> > > > > > > > > > > > Hi, > > > > > > Those thinking that it's not a good idea, do you mind > > > explaining your point of view? > > > GlusterFS seems like the only real > > > alternative to a highly priced SAN for the primary storage... > > > > > > > > > Thanks, > > > Olivier > > > > > > On 2013-09-11 15:08, Rafael Weingartner wrote: > > > > > > > I > > > totally agree with Tomasz, I do not think that using a distributed FS > > > as > > > > primary storage is a good idea, but as a secondary it sounds > > > interesting. > > > > > > > > But, off course you can try *;*) > > > > > > > > 2013/9/11 > > > Shanker Balan <shanker.ba...@shapeblue.com> > > > > > > > >> On 11-Sep-2013, at > > > 5:14 PM, Tomasz Zięba <t.a.zi...@gmail.com [1]> wrote: > > > >> > > > >>> Hi, Some > > > time ago I tried to use GlusterFS as storage for cloudstacka but I > > > noticed that cloudstack uses the default settings for mount command. By > > > default mount command is using the UDP protocol but glusterfs works > > > >> > > > only > > > >> > > > >>> using tcp. I think, if cloudstack developers could add "-o > > > proto=tcp" to code glusterfs should works. For example: /bin/mount -t > > > nfs -o proto=tcp IP:/share /mnt/gluster/ If you are using CitrixXen you > > > should mount the share and make it as SR. For cloudstacka is clear > > > because you should use the option PreSetup when creating > PrimaryStorage. > > > Personally, I doubt that using GlusterFS as a primary storage is a good > > > solution but for secondary storage it should be very usefull. > > > >> And > > > maybe as a Swift backend. -- @shankerbalan M: +91 98860 60539 | O: +91 > > > (80) 67935867 shanker.ba...@shapeblue.com [2] | www.shapeblue.com [3] > | > > > Twitter:@shapeblue ShapeBlue Services India LLP, 22nd floor, Unit > 2201A, > > > World Trade Centre, Bangalore - 560 055 This email and any attachments > > > to it may be confidential and are intended solely for the use of the > > > individual to whom it is addressed. Any views or opinions expressed are > > > solely those of the author and do not necessarily represent those of > > > Shape Blue Ltd or related companies. If you are not the intended > > > recipient of this email, you must neither take any action based upon > its > > > contents, nor copy or show it to anyone. Please contact the sender if > > > you believe you have received this email in error. Shape Blue Ltd is a > > > company incorporated in England & Wales. ShapeBlue Services India LLP > is > > > operated under license from Shape Blue Ltd. ShapeBlue is a registered > > > trademark. > > > > > > > > -- Rafael Weingartner > > > > > > > > > > > > Links: > > > ------ > > > [1] > > > mailto:t.a.zi...@gmail.com > > > [2] mailto:shanker.ba...@shapeblue.com > > > [3] > > > http://www.shapeblue.com > > > > > > > > > > > -- > > Rafael Weingartner > > > > > > > -- > Rafael Weingartner > > -- Rafael Weingartner
