[Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-01-30 Thread Mickey Mazarick
Can you tell us a little more about your setup? I'm running many hundreds of vms on our cluster but I found infiniband is necessary if you have any large amount of io (databases, lots of drive access etc). You may simply be saturating your io if you only have a single gigabit interface to your

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-01-31 Thread Gordan Bobic
Traffic shaping with tc and iptables are your friends. ;) Of course, if you are genuinely running out of bandwidth nothing will solve the lack of bandwidth, but if you merely need to make sure it is distributed more fairly and sensibly between the machines, it can be done. Typically I would pu

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-01-31 Thread Ran
You guys are talking about network IO im taking about the gluster server disk IO the idea to shape the trafic does make sence seens the virt machines server do use network to get to the disks(gluster) but what about if there are say 5 KVM servers(with VPS's) all on gluster what do you do then ? its

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-01-31 Thread Gordan Bobic
Ran wrote: You guys are talking about network IO im taking about the gluster server disk IO I also explained how you could use ionice to throttle the entire VM's disk I/O. the idea to shape the trafic does make sence seens the virt machines server do use network to get to the disks(gluster)

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-01 Thread Jeff Darcy
On 01/31/2010 09:06 AM, Ran wrote: > You guys are talking about network IO im taking about the gluster server disk > IO > the idea to shape the trafic does make sence seens the virt machines > server do use network to get to the disks(gluster) > but what about if there are say 5 KVM servers(with V

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-01 Thread Gordan Bobic
Jeff Darcy wrote: On 01/31/2010 09:06 AM, Ran wrote: You guys are talking about network IO im taking about the gluster server disk IO the idea to shape the trafic does make sence seens the virt machines server do use network to get to the disks(gluster) but what about if there are say 5 KVM serv

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-01 Thread Jeff Darcy
On 02/01/2010 10:14 AM, Gordan Bobic wrote: > Optimizing > file systems is a relatively complex thing and a lot of the conventional > wisdom is just plain wrong at times. After approximately fifteen years of doing that kind of tuning, I couldn't agree more. >> Unfortunately, I/O traffic shaping

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-01 Thread Gordan Bobic
Jeff Darcy wrote: Unfortunately, I/O traffic shaping is still in its infancy compared to what's available for networking - or perhaps even "infancy" is too generous. As far as the I/O stack is concerned, all of the traffic is coming from the glusterfsd process(es) without differentiation, so ev

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-02 Thread Ran
> The line that you need to add is the one with "writeback" in it. > If you are running qemu-kvm manually, you'll need to add the > "cache=writeback" > to your list of -drive option parameters. > All of this, of course, doesn't preclude applying ionice to the qemu > container processes. ionice h

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-02 Thread Gordan Bobic
Ran wrote: The line that you need to add is the one with "writeback" in it. If you are running qemu-kvm manually, you'll need to add the "cache=writeback" > to your list of -drive option parameters. All of this, of course, doesn't preclude applying ionice to the qemu container processes. ioni

Re: [Gluster-devel] Re: [Gluster-users] I/O fair share to avoid I/O bottlenecks on small clsuters

2010-02-02 Thread Gordan Bobic
Ran wrote: The line that you need to add is the one with "writeback" in it. If you are running qemu-kvm manually, you'll need to add the "cache=writeback" > to your list of -drive option parameters. All of this, of course, doesn't preclude applying ionice to the qemu container processes. ioni