Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Please add oflag=direct to see how it goes. Any objections on continuing this discussion in thread blkio cgroup where Vivek Goyal, the blkio controller author, replied to my posting? I'm in the middle of answering his questions there. Regards Dominik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Ahh, sure. Dominik Klein wrote: Please add oflag=direct to see how it goes. Any objections on continuing this discussion in thread blkio cgroup where Vivek Goyal, the blkio controller author, replied to my posting? I'm in the middle of answering his questions there. Regards Dominik -- Regards Gui Jianfeng -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Dominik Klein wrote: Hi Gui, thanks for your reply. Actually, this two series doesn't care how blkio cgroup works. They just provide the ability to put Guest into given cgroup and tune the tunables. Currently, You can only control the blkio.weight tunable. For examle, If you'd like to give more I/O bandwidth to a given Guest, you can assign a bigger value to blkio.weight by running blkiotune command. Okay, so then basically, if I understand the current state of the blkio controller correctly (see my original message), this is useless at this point. I don't mean to offend you in any case, I just wondered how you used (or intended to use) this, since I myself did not yet find a good way to actually limit a guests I/O hunger. Hi Dominik, Hmm.. blkio.weight is used to control the Minimal Maximal Bandwidth. If you'd like to control Max bandwidth to let your guests become I/O hungery. blkio.throttle.* should helps. But these tunables aren't supported by blkiotune for the time being. I'm considering to implement them in the future. Thanks, Gui Regards Dominik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
On Fri, Feb 18, 2011 at 04:12:28PM +0800, Gui Jianfeng wrote: blkio.weight is used to control the Minimal Maximal Bandwidth. If you'd like to control Max bandwidth to let your guests become I/O hungery. blkio.throttle.* should helps. But these tunables aren't supported by blkiotune for the time being. I'm considering to implement them in the future. I'm still a bit confused. The weight controls the 'percentage' of resources the VM gets compared to other VMs (or other processes participating in the cgroup)? If I have 5 processes running on the system, and 4 of them are VMs using cgroups, then the VMs get 4/5s of all CPU time for IO tasks, and each VM gets an amount of that 4/5th proportional to it's weight? Yes? Then, blkio.throttle can be used to set actual hard limits on IO consumption -- like, no more than 10Mbps for this VM. I can already do this with blkio cgroups, but just not from libvirt -- yes? Like many people using libvirt, I only use my hosts for running VMs, and I am most interested in limiting IO operations per host to make sure that no host monopolizes our SAN. Is there a way for me to just set a hard IO throttle limit for libvirt as a whole which would apply to all child VMs created by libvirt? I saw a discussion on this list regarding group hierarchy, but it went over my head. --Igor signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Dominik Klein wrote: Hmm.. blkio.weight is used to control the Minimal Maximal Bandwidth. If you'd like to control Max bandwidth to let your guests become I/O hungery. blkio.throttle.* should helps. But these tunables aren't supported by blkiotune for the time being. I'm considering to implement them in the future. Hm, maybe I was not clear then. Let me try to rephrase. As far as I understand, and as far as I saw during my tests, the blkio controller only works for sync I/O requests (eg dd with oflag=direct). Buffered I/O is not part of the control. Yes, you are right. And since a VM's I/O is most likely buffered in some fashion, this does not have any effect. That's what my tests showed. how about the start Guest with option cache=none to bypass pagecache? This should help i think. Did your tests show something different? Maybe I did things wrong then. I think blkio cgroup should work if cache=none is set. But I didn't try it, Will try it later. Thanks Gui Regards Dominik -- Regards Gui Jianfeng -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
As far as I understand, and as far as I saw during my tests, the blkio controller only works for sync I/O requests (eg dd with oflag=direct). Buffered I/O is not part of the control. Yes, you are right. whew :) And since a VM's I/O is most likely buffered in some fashion, this does not have any effect. That's what my tests showed. how about the start Guest with option cache=none to bypass pagecache? This should help i think. I will read up on where to set that and give it a try. Thanks for the hint. Did your tests show something different? Maybe I did things wrong then. I think blkio cgroup should work if cache=none is set. But I didn't try it, Will try it later. Please let us know what you found out during your tests. Regards Dominik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Igor Serebryany wrote: On Fri, Feb 18, 2011 at 04:12:28PM +0800, Gui Jianfeng wrote: blkio.weight is used to control the Minimal Maximal Bandwidth. If you'd like to control Max bandwidth to let your guests become I/O hungery. blkio.throttle.* should helps. But these tunables aren't supported by blkiotune for the time being. I'm considering to implement them in the future. I'm still a bit confused. The weight controls the 'percentage' of resources the VM gets compared to other VMs (or other processes participating in the cgroup)? If I have 5 processes running on the system, and 4 of them are VMs using cgroups, then the VMs get 4/5s of all CPU time for IO tasks, and each VM gets an amount of that 4/5th proportional to it's weight? Yes? Yes, if all weight are the same. Then, blkio.throttle can be used to set actual hard limits on IO consumption -- like, no more than 10Mbps for this VM. I can already do this with blkio cgroups, but just not from libvirt -- yes? yes Like many people using libvirt, I only use my hosts for running VMs, and I am most interested in limiting IO operations per host to make sure that no host monopolizes our SAN. Is there a way for me to just set a hard IO throttle limit for libvirt as a whole which would apply to all child VMs created by libvirt? I saw a discussion on this list regarding group hierarchy, but it went over my head. There's no way to set this limit by libvirt. But, I think you can set the limit by hand in cgroup directory. ;) Thanks, Gui --Igor -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Hi I can only comment on some of your questions as I also can't say to understand this completely. Then, blkio.throttle can be used to set actual hard limits on IO consumption -- like, no more than 10Mbps for this VM. I can already do this with blkio cgroups, but just not from libvirt -- yes? Yes. But bear in mind that this only works for sync I/O, not for buffered I/O (read my discussion with Gui). Like many people using libvirt, I only use my hosts for running VMs, and I am most interested in limiting IO operations per host to make sure that no host monopolizes our SAN. Is there a way for me to just set a hard IO throttle limit for libvirt as a whole which would apply to all child VMs created by libvirt? I saw a discussion on this list regarding group hierarchy, but it went over my head. I did not read the other discussion, but this comes to my mind when thinking about your question: You could try to put the libvirtd process into a cgroup that has certain limits configured. Every spawned child of this process will inherit those limits. Not sure if a VM is actually a libvirtd-child though, since at least ps shows them as init's children on my systems. If you try this, please report back what you did and how it worked. Regards Dominik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Gui Jianfeng wrote: Igor Serebryany wrote: On Fri, Feb 18, 2011 at 04:12:28PM +0800, Gui Jianfeng wrote: blkio.weight is used to control the Minimal Maximal Bandwidth. If you'd like to control Max bandwidth to let your guests become I/O hungery. blkio.throttle.* should helps. But these tunables aren't supported by blkiotune for the time being. I'm considering to implement them in the future. I'm still a bit confused. The weight controls the 'percentage' of resources the VM gets compared to other VMs (or other processes participating in the cgroup)? If I have 5 processes running on the system, and 4 of them are VMs using cgroups, then the VMs get 4/5s of all CPU time for IO tasks, and each VM gets an amount of that 4/5th proportional to it's weight? Yes? Yes, if all weight are the same. I think here will be misleading. I means if VMs will get 4/5 bandwidth, the ratio of total weight of VMs and the group holds the other task should be 4:1. Thanks, Gui Then, blkio.throttle can be used to set actual hard limits on IO consumption -- like, no more than 10Mbps for this VM. I can already do this with blkio cgroups, but just not from libvirt -- yes? yes Like many people using libvirt, I only use my hosts for running VMs, and I am most interested in limiting IO operations per host to make sure that no host monopolizes our SAN. Is there a way for me to just set a hard IO throttle limit for libvirt as a whole which would apply to all child VMs created by libvirt? I saw a discussion on this list regarding group hierarchy, but it went over my head. There's no way to set this limit by libvirt. But, I think you can set the limit by hand in cgroup directory. ;) Thanks, Gui --Igor -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Hi not that I tested this patch or the patch that followed (adding blkiotune command) ... but the way I understand, the cgroup blkio controller does only work for sync'd IO/queues. quote blkio-controller.txt from 2.6.37: Currently only sync IO queues are support. All the buffered writes are still system wide and not per group. When I last tested this (using manual cgroups commands), this seemed true. So may I ask how you use this and how this helps in which setup? Regards Dominik On 02/08/2011 07:38 AM, Gui Jianfeng wrote: Hi This patchset adds blkio cgroup support for qemu and lxc. [PATCH 1/6] cgroup: Enable cgroup hierarchy for blkio cgroup [PATCH 2/6 v3] cgroup: Implement blkio.weight tuning API. [PATCH 3/6] Update XML Schema for new entries. [PATCH 4/6 v3] qemu: Implement blkio tunable XML configuration and parsing. [PATCH 5/6 v3] LXC: LXC Blkio weight configuration support. [PATCH 6/6] Add documentation for blkiotune elements. Will post a patchset to implement virsh command blkiotune to tune blkio cgroup parameter later on. v2 - v3 Changes: o Remove an unused local variable o Rename virCgroup(Set/Get)Weight to virCgroup(Set/Get)BlkioWeight o Add documentation in docs/formatdomain.html.in o Update XML Schema for new entries. docs/formatdomain.html.in | 10 ++ docs/schemas/domain.rng | 20 src/conf/domain_conf.c| 13 + src/conf/domain_conf.h|4 src/libvirt_private.syms |2 ++ src/lxc/lxc_controller.c | 10 ++ src/qemu/qemu_cgroup.c| 16 +++- src/qemu/qemu_conf.c |3 ++- src/util/cgroup.c | 41 - src/util/cgroup.h |4 10 files changed, 120 insertions(+), 3 deletions(-) Thanks Gui -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list -- IN-telegence GmbH Oskar-Jäger-Str. 125 50825 Köln Registergericht AG Köln - HRB 34038 USt-ID DE210882245 Geschäftsführende Gesellschafter: Christian Plätke und Holger Jansen -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Dominik Klein wrote: Hi not that I tested this patch or the patch that followed (adding blkiotune command) ... but the way I understand, the cgroup blkio controller does only work for sync'd IO/queues. quote blkio-controller.txt from 2.6.37: Currently only sync IO queues are support. All the buffered writes are still system wide and not per group. When I last tested this (using manual cgroups commands), this seemed true. So may I ask how you use this and how this helps in which setup? Hi, Dominik Actually, this two series doesn't care how blkio cgroup works. They just provide the ability to put Guest into given cgroup and tune the tunables. Currently, You can only control the blkio.weight tunable. For examle, If you'd like to give more I/O bandwidth to a given Guest, you can assign a bigger value to blkio.weight by running blkiotune command. Thanks, Gui Regards Dominik On 02/08/2011 07:38 AM, Gui Jianfeng wrote: Hi This patchset adds blkio cgroup support for qemu and lxc. [PATCH 1/6] cgroup: Enable cgroup hierarchy for blkio cgroup [PATCH 2/6 v3] cgroup: Implement blkio.weight tuning API. [PATCH 3/6] Update XML Schema for new entries. [PATCH 4/6 v3] qemu: Implement blkio tunable XML configuration and parsing. [PATCH 5/6 v3] LXC: LXC Blkio weight configuration support. [PATCH 6/6] Add documentation for blkiotune elements. Will post a patchset to implement virsh command blkiotune to tune blkio cgroup parameter later on. v2 - v3 Changes: o Remove an unused local variable o Rename virCgroup(Set/Get)Weight to virCgroup(Set/Get)BlkioWeight o Add documentation in docs/formatdomain.html.in o Update XML Schema for new entries. docs/formatdomain.html.in | 10 ++ docs/schemas/domain.rng | 20 src/conf/domain_conf.c| 13 + src/conf/domain_conf.h|4 src/libvirt_private.syms |2 ++ src/lxc/lxc_controller.c | 10 ++ src/qemu/qemu_cgroup.c| 16 +++- src/qemu/qemu_conf.c |3 ++- src/util/cgroup.c | 41 - src/util/cgroup.h |4 10 files changed, 120 insertions(+), 3 deletions(-) Thanks Gui -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list -- Regards Gui Jianfeng -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
On 02/07/2011 11:38 PM, Gui Jianfeng wrote: Hi This patchset adds blkio cgroup support for qemu and lxc. [PATCH 1/6] cgroup: Enable cgroup hierarchy for blkio cgroup [PATCH 2/6 v3] cgroup: Implement blkio.weight tuning API. [PATCH 3/6] Update XML Schema for new entries. [PATCH 4/6 v3] qemu: Implement blkio tunable XML configuration and parsing. [PATCH 5/6 v3] LXC: LXC Blkio weight configuration support. [PATCH 6/6] Add documentation for blkiotune elements. Will post a patchset to implement virsh command blkiotune to tune blkio cgroup parameter later on. Please do :) It's not a showstopper (which is why I pushed the series), since 'dumpxml' and 'edit' can do the same thing. But without the new virsh command, getting at blkiotune is awkward, and you already have memtune to copy from. Remember to update virsh.pod in the process. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 0/6 v3] Add blkio cgroup support
Eric Blake wrote: On 02/07/2011 11:38 PM, Gui Jianfeng wrote: Hi This patchset adds blkio cgroup support for qemu and lxc. [PATCH 1/6] cgroup: Enable cgroup hierarchy for blkio cgroup [PATCH 2/6 v3] cgroup: Implement blkio.weight tuning API. [PATCH 3/6] Update XML Schema for new entries. [PATCH 4/6 v3] qemu: Implement blkio tunable XML configuration and parsing. [PATCH 5/6 v3] LXC: LXC Blkio weight configuration support. [PATCH 6/6] Add documentation for blkiotune elements. Will post a patchset to implement virsh command blkiotune to tune blkio cgroup parameter later on. Please do :) It's not a showstopper (which is why I pushed the series), since 'dumpxml' and 'edit' can do the same thing. But without the new virsh command, getting at blkiotune is awkward, and you already have memtune to copy from. Remember to update virsh.pod in the process. Eric, Thanks for reviewing and committing the series, will work on the blkiotune later. Thanks, Gui -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
