I ran each test independently for each block size. iozone -I -i 0 -i 1 -i 2 -r 4k -s 1G
In order: -I to specify direct-IO, -i 0 to specify write/rewrite, -i 1 to specify read/re-read, -i 2 to specify random read/write, -r specify block size, -s to specify file size. The report is pretty thorough. What I put in the email was different, I just took the throughput from each test and put it into a table outside of the full report. ----- Original Message ----- From: "Rafael Weingartner" <rafaelweingart...@gmail.com> To: users@cloudstack.apache.org Sent: Wednesday, September 11, 2013 4:16:23 PM Subject: Re: Cloustack 4.1.0 + GlusterFS I have never used iozone before, How did you get that report? I tried: iozone -s 1024m -t 1 -R But the report was pretty different from yours. 2013/9/11 John Skinner <john.skin...@appcore.com> > I currently have GlusterFS deployed into an 8 node KVM cluster running on > CloudStack 4.1 for primary storage. Gluster is deployed on 28 1TB drives > across 2 separate storage appliances using a distributed-replicated volume > with the replica set to 2. The storage network is 10Gb copper. > > These are the options I have configured for the volume in Gluster, most of > them are from a Red Hat document on configuring Red Hat Enterprise Storage > for VM hosting: > > > > performance.io-thread-count: 32 > performance.cache-size: 1024MB > performance.write-behind-window-size: 5MB > performance.write-behind: on > network.remote-dio: on > cluster.eager-lock: enable > performance.stat-prefetch: off > performance.io-cache: on > performance.read-ahead: on > performance.quick-read: on > > Here are some of the numbers I was getting when benchmarking the storage > from the KVM node directly (not a VM) > > The below table is in KB/s. The test is single stream 1GB file utilizing > Direct I/O (no cache). I used iozone to run the benchmark. > > Write 4k 45729 > Read 4k 10189 > Random Write 4k 31983 > Random Read 4k 9859 > Write 16k 182246 > Read 16k 37146 > Random Write 16k 113026 > Random Read 16k 37237 > Write 64k 420908 > Read 64k 125315 > Random Write 64k 383848 > Random Read 64k 125218 > Write 256k 567501 > Read 256k 218413 > Random Write 256k 508650 > Random Read 256k 229117 > > In the above results, I have the volume mounted to each KVM host as a FUSE > glusterfs file system. They are added to CloudStack as a shared mount > point. In the future it would be great to utilize GlusterFS qemu libvirt > integration with libgfapi so I could bypass fuse altogether. However, that > would require adding that code to CloudStack to support that. > > I maybe have 15 or so VMs running from the storage now and it is still > pretty snappy. Need to do some more testing though and really get it loaded. > > ----- Original Message ----- > > From: "Rafael Weingartner" <rafaelweingart...@gmail.com> > To: users@cloudstack.apache.org > Sent: Wednesday, September 11, 2013 8:48:07 AM > Subject: Re: Cloustack 4.1.0 + GlusterFS > > Right now I can think in three main reasons: > > The first reason, performance (I do not know Gluster and its performance > and if the read and write speed are satisfactory). Please if you can make a > test, post the results. > > Second consistency, I do not know Gluster, but swift that is also a > Distributed File System is not consistency and they make it pretty clear on > their page (http://docs.openstack.org/developer/swift/) > > "Swift is a highly available, distributed, eventually consistent > object/blob store...". > > I would not accept to storage my VMs images on a FS that is eventually > consistent. > > Third, network, I haven't used this kind of FS, but I can image that it > uses a lot of bandwidth to keep synchronizing, managing and securing the > data. So, managing the networking would be a pain. > > > > 2013/9/11 Olivier Mauras <oliv...@core-hosting.net> > > > > > > > Hi, > > > > Those thinking that it's not a good idea, do you mind > > explaining your point of view? > > GlusterFS seems like the only real > > alternative to a highly priced SAN for the primary storage... > > > > > > Thanks, > > Olivier > > > > On 2013-09-11 15:08, Rafael Weingartner wrote: > > > > > I > > totally agree with Tomasz, I do not think that using a distributed FS > > as > > > primary storage is a good idea, but as a secondary it sounds > > interesting. > > > > > > But, off course you can try *;*) > > > > > > 2013/9/11 > > Shanker Balan <shanker.ba...@shapeblue.com> > > > > > >> On 11-Sep-2013, at > > 5:14 PM, Tomasz Zięba <t.a.zi...@gmail.com [1]> wrote: > > >> > > >>> Hi, Some > > time ago I tried to use GlusterFS as storage for cloudstacka but I > > noticed that cloudstack uses the default settings for mount command. By > > default mount command is using the UDP protocol but glusterfs works > > >> > > only > > >> > > >>> using tcp. I think, if cloudstack developers could add "-o > > proto=tcp" to code glusterfs should works. For example: /bin/mount -t > > nfs -o proto=tcp IP:/share /mnt/gluster/ If you are using CitrixXen you > > should mount the share and make it as SR. For cloudstacka is clear > > because you should use the option PreSetup when creating PrimaryStorage. > > Personally, I doubt that using GlusterFS as a primary storage is a good > > solution but for secondary storage it should be very usefull. > > >> And > > maybe as a Swift backend. -- @shankerbalan M: +91 98860 60539 | O: +91 > > (80) 67935867 shanker.ba...@shapeblue.com [2] | www.shapeblue.com [3] | > > Twitter:@shapeblue ShapeBlue Services India LLP, 22nd floor, Unit 2201A, > > World Trade Centre, Bangalore - 560 055 This email and any attachments > > to it may be confidential and are intended solely for the use of the > > individual to whom it is addressed. Any views or opinions expressed are > > solely those of the author and do not necessarily represent those of > > Shape Blue Ltd or related companies. If you are not the intended > > recipient of this email, you must neither take any action based upon its > > contents, nor copy or show it to anyone. Please contact the sender if > > you believe you have received this email in error. Shape Blue Ltd is a > > company incorporated in England & Wales. ShapeBlue Services India LLP is > > operated under license from Shape Blue Ltd. ShapeBlue is a registered > > trademark. > > > > > > -- Rafael Weingartner > > > > > > > > Links: > > ------ > > [1] > > mailto:t.a.zi...@gmail.com > > [2] mailto:shanker.ba...@shapeblue.com > > [3] > > http://www.shapeblue.com > > > > > > -- > Rafael Weingartner > > -- Rafael Weingartner
