Re: [Vserver] Linux Vserver - Feature Question
On Fri, Apr 01, 2005 at 04:25:41PM -0500, Gregory (Grisha) Trubetskoy wrote: > > > I would also keep measurements of CPU ticks used. Since IO requires CPU > cycles - is it possible that a CPU sched_hard indirectly limits IO just as > well? my suggestion (and actually plans for I/O issues) are something like this: - have a 'generic' token bucket for various things (swap pages, I/O, scheduler, forks ...) why?: because! no seriously if we put all into one bucket it might work, but nobody can figure why it works/doesn't work ... - make those token buckets per cpu with (the now possible) cpuaffinity in place for vservers why?: increased performance and better scaling - have some buckets affect the scheduling behaviour (e.g. swap, forks, scheduler of course) why?: to penalize the contexts which do heavy forking or other heavy system activity ... - use some buckets to adjust/influence schedulers (taks, I/O and maybe network scheduler that is) sched_hard does already indirectly limit I/O, because a not running process/context can not submit I/O requests so that is already working, but I take it isn't sufficient for 'normal' use ... best, Herbert PS: keep up the discussion ... > Grisha > > On Fri, 1 Apr 2005, Matthew Nuzum wrote: > > >>On Thu, Mar 31, 2005 at 09:22:10PM -0600, Matthew Nuzum wrote: > >>>I think I can create a test case for this. I have a server that is not > >>>currently running any vserver stuff that will be ok with a reboot now > >>and > >>>then. > >> > >>sounds good, please try to get 1.9.5.5 working there, > >>because it already contains some blkio accounting > >>and it would be very interesting to monitor those > >>values ... (maybe with rrdtools) > >> > >>TIA, > >>Herbert > > > >I'm still doing my month-end backup, but when that's done I'll start > >installing the vserver 1.9.5.5. > > > >Here is the test case that seems most logical to me, but advice on how to > >actually do concrete tests would be useful. > > > >1. Create two vservers (vsa and vsb), start both. > >2. In vsa start some heavily i/o intensive operation > >3. In vsb try to do some tasks and notice how much i/o bandwidth I have > >available. > > > >Alternative plan: > >1. Create 1 vserver and start it > >2. In the vserver, start some heavily i/o intensive operation > >3. In the host server try to do some tasks and notice how much i/o > >bandwidth > >I have available > >4. After step 2 completes, in host server start a heavily i/o intensive > >operation > >5. In vserver, try to do some tasks and notice how much i/o bandwidth I > >have > >available > > > >I have two ideas on heavily i/o intensive operation > >1. I have a database with 35 million records. Doing any aggregate function > >such as max() requires several sequential scans and takes a significant > >amount of time. > >2. Preparing my month end backup requires copying 13 GB of data. > > > >Any other suggestions? > > > >Question: > >I have only subjectively noticed a dramatic decrease in server performance > >when a vserver is performing i/o intensive tasks. How can I objectively > >measure and produce concrete numbers? > > > >-- > >Matthew Nuzum <[EMAIL PROTECTED]> > >www.followers.net - Makers of "Elite Content Management System" > >View samples of Elite CMS in action by visiting > >http://www.followers.net/portfolio/ > > > >___ > >Vserver mailing list > >Vserver@list.linux-vserver.org > >http://list.linux-vserver.org/mailman/listinfo/vserver > > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
RE: [Vserver] Linux Vserver - Feature Question
The CPU ticks are in /proc/virtual//sched Grisha On Fri, 1 Apr 2005, Matthew Nuzum wrote: I would also keep measurements of CPU ticks used. Since IO requires CPU cycles - is it possible that a CPU sched_hard indirectly limits IO just as well? Grisha How do you do that? P.S. I'm still compiling the vanilla kernel (I haven't even applied the vserver patch yet). At this rate I'll probably get back to you on Sunday. FYI Pii 350 MHz, 128 MB RAM, Ubuntu 4.1, 120 GB UDMA 133 hard drive. -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.followers.net/portfolio/ ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
RE: [Vserver] Linux Vserver - Feature Question
> I would also keep measurements of CPU ticks used. Since IO requires CPU > cycles - is it possible that a CPU sched_hard indirectly limits IO just as > well? > > Grisha How do you do that? P.S. I'm still compiling the vanilla kernel (I haven't even applied the vserver patch yet). At this rate I'll probably get back to you on Sunday. FYI Pii 350 MHz, 128 MB RAM, Ubuntu 4.1, 120 GB UDMA 133 hard drive. -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.followers.net/portfolio/ ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] Linux Vserver - Feature Question
I would also keep measurements of CPU ticks used. Since IO requires CPU cycles - is it possible that a CPU sched_hard indirectly limits IO just as well? Grisha On Fri, 1 Apr 2005, Matthew Nuzum wrote: On Thu, Mar 31, 2005 at 09:22:10PM -0600, Matthew Nuzum wrote: I think I can create a test case for this. I have a server that is not currently running any vserver stuff that will be ok with a reboot now and then. sounds good, please try to get 1.9.5.5 working there, because it already contains some blkio accounting and it would be very interesting to monitor those values ... (maybe with rrdtools) TIA, Herbert I'm still doing my month-end backup, but when that's done I'll start installing the vserver 1.9.5.5. Here is the test case that seems most logical to me, but advice on how to actually do concrete tests would be useful. 1. Create two vservers (vsa and vsb), start both. 2. In vsa start some heavily i/o intensive operation 3. In vsb try to do some tasks and notice how much i/o bandwidth I have available. Alternative plan: 1. Create 1 vserver and start it 2. In the vserver, start some heavily i/o intensive operation 3. In the host server try to do some tasks and notice how much i/o bandwidth I have available 4. After step 2 completes, in host server start a heavily i/o intensive operation 5. In vserver, try to do some tasks and notice how much i/o bandwidth I have available I have two ideas on heavily i/o intensive operation 1. I have a database with 35 million records. Doing any aggregate function such as max() requires several sequential scans and takes a significant amount of time. 2. Preparing my month end backup requires copying 13 GB of data. Any other suggestions? Question: I have only subjectively noticed a dramatic decrease in server performance when a vserver is performing i/o intensive tasks. How can I objectively measure and produce concrete numbers? -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.followers.net/portfolio/ ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] Linux Vserver - Feature Question
On Fri, Apr 01, 2005 at 12:23:00PM -0600, Matthew Nuzum wrote: > > On Thu, Mar 31, 2005 at 09:22:10PM -0600, Matthew Nuzum wrote: > > > I think I can create a test case for this. I have a server that is not > > > currently running any vserver stuff that will be ok with a reboot now > > and > > > then. > > > > sounds good, please try to get 1.9.5.5 working there, > > because it already contains some blkio accounting > > and it would be very interesting to monitor those > > values ... (maybe with rrdtools) > > > > TIA, > > Herbert > > I'm still doing my month-end backup, but when that's done I'll start > installing the vserver 1.9.5.5. > > Here is the test case that seems most logical to me, but advice on how to > actually do concrete tests would be useful. > > 1. Create two vservers (vsa and vsb), start both. > 2. In vsa start some heavily i/o intensive operation > 3. In vsb try to do some tasks and notice how much i/o bandwidth I have > available. > > Alternative plan: > 1. Create 1 vserver and start it > 2. In the vserver, start some heavily i/o intensive operation > 3. In the host server try to do some tasks and notice how much i/o bandwidth > I have available > 4. After step 2 completes, in host server start a heavily i/o intensive > operation > 5. In vserver, try to do some tasks and notice how much i/o bandwidth I have > available > > I have two ideas on heavily i/o intensive operation > 1. I have a database with 35 million records. Doing any aggregate function > such as max() requires several sequential scans and takes a significant > amount of time. > 2. Preparing my month end backup requires copying 13 GB of data. > > Any other suggestions? > > Question: > I have only subjectively noticed a dramatic decrease in server performance > when a vserver is performing i/o intensive tasks. How can I objectively > measure and produce concrete numbers? there are two 'aspects' of what you 'experience' as performance here. first the increased latency when doing I/O (which is the result of several I/O transactions already going on when you do whatever you do), and the decreased throughput (which IMHO is not really the issue here, just think 40MB/s transfer with UDMA and 4MB/s without ...) I would suggest to 'test' with different I/O schedulers activated, because I think that the default I/O scheduler might be sub-optimal for vserver-type I/O loads anyways ... TIA (for testing) Herbert > -- > Matthew Nuzum <[EMAIL PROTECTED]> > www.followers.net - Makers of "Elite Content Management System" > View samples of Elite CMS in action by visiting > http://www.followers.net/portfolio/ > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] Linux Vserver - Feature Question
> On Thu, Mar 31, 2005 at 09:22:10PM -0600, Matthew Nuzum wrote: > > I think I can create a test case for this. I have a server that is not > > currently running any vserver stuff that will be ok with a reboot now > and > > then. > > sounds good, please try to get 1.9.5.5 working there, > because it already contains some blkio accounting > and it would be very interesting to monitor those > values ... (maybe with rrdtools) > > TIA, > Herbert I'm still doing my month-end backup, but when that's done I'll start installing the vserver 1.9.5.5. Here is the test case that seems most logical to me, but advice on how to actually do concrete tests would be useful. 1. Create two vservers (vsa and vsb), start both. 2. In vsa start some heavily i/o intensive operation 3. In vsb try to do some tasks and notice how much i/o bandwidth I have available. Alternative plan: 1. Create 1 vserver and start it 2. In the vserver, start some heavily i/o intensive operation 3. In the host server try to do some tasks and notice how much i/o bandwidth I have available 4. After step 2 completes, in host server start a heavily i/o intensive operation 5. In vserver, try to do some tasks and notice how much i/o bandwidth I have available I have two ideas on heavily i/o intensive operation 1. I have a database with 35 million records. Doing any aggregate function such as max() requires several sequential scans and takes a significant amount of time. 2. Preparing my month end backup requires copying 13 GB of data. Any other suggestions? Question: I have only subjectively noticed a dramatic decrease in server performance when a vserver is performing i/o intensive tasks. How can I objectively measure and produce concrete numbers? -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.followers.net/portfolio/ ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [RE:] Re: [Vserver] Linux Vserver - Feature Question
On Thu, Mar 31, 2005 at 09:22:10PM -0600, Matthew Nuzum wrote: > > > > > > I do think that this would be an interesting feature. > > > > http://www.theshore.net/~caker/uml/patches/ > > (seems to have updated patches regarding this stuff) > > http://www.theshore.net/~caker/uml/patches/token-limiter.README > > > > will look into it .. anybody volunteering to test > > such stuff? > > > > TIA, > > Herbert > > I think I can create a test case for this. I have a server that is not > currently running any vserver stuff that will be ok with a reboot now and > then. sounds good, please try to get 1.9.5.5 working there, because it already contains some blkio accounting and it would be very interesting to monitor those values ... (maybe with rrdtools) TIA, Herbert > I don't monitor this list that closely these days, so jog me when there's > something ready for me to test. > > -- > Matthew Nuzum <[EMAIL PROTECTED]> > www.followers.net - Makers of "Elite Content Management System" > View samples of Elite CMS in action by visiting > http://www.followers.net/portfolio/ > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
RE: [RE:] Re: [Vserver] Linux Vserver - Feature Question
> > > > I do think that this would be an interesting feature. > > http://www.theshore.net/~caker/uml/patches/ > (seems to have updated patches regarding this stuff) > http://www.theshore.net/~caker/uml/patches/token-limiter.README > > will look into it .. anybody volunteering to test > such stuff? > > TIA, > Herbert I think I can create a test case for this. I have a server that is not currently running any vserver stuff that will be ok with a reboot now and then. I don't monitor this list that closely these days, so jog me when there's something ready for me to test. -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.followers.net/portfolio/ ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [RE:] Re: [Vserver] Linux Vserver - Feature Question
On Thu, Mar 31, 2005 at 04:06:46PM -0500, Gregory (Grisha) Trubetskoy wrote: > > There is something like this in a patch to UML developed by the Linode > folks: > > http://www.linode.com/forums/archive/o_t/t_790/linode.com_status_update_04_06_04.html > > Looks like a token bucket, only for IO. > > It may be easier to do something like this in UML because their IO driver > is a constant (UBD) whereas in VServer things aren't so simple since the > driver could be anything? Herbert can probably comment on this better :-) > > I do think that this would be an interesting feature. http://www.theshore.net/~caker/uml/patches/ (seems to have updated patches regarding this stuff) http://www.theshore.net/~caker/uml/patches/token-limiter.README will look into it .. anybody volunteering to test such stuff? TIA, Herbert > Grisha > > > On Thu, 31 Mar 2005, Bodo Eggert wrote: > > >On Thu, 31 Mar 2005, Herbert Poetzl wrote: > >>On Thu, Mar 31, 2005 at 09:26:31AM +0200, [EMAIL PROTECTED] wrote: > > > >>hmm, so you would like to artificially slow down the > >>I/O transfer of a vserver, and make the transaction > >>somewhat longer than necessary? > > > >I guess more like not slowing down the host or other vservers. > > > >-- > >Funny quotes: > >19. Quantum mechanics: The dreams stuff is made of. > >___ > >Vserver mailing list > >Vserver@list.linux-vserver.org > >http://list.linux-vserver.org/mailman/listinfo/vserver > > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [RE:] Re: [Vserver] Linux Vserver - Feature Question
There is something like this in a patch to UML developed by the Linode folks: http://www.linode.com/forums/archive/o_t/t_790/linode.com_status_update_04_06_04.html Looks like a token bucket, only for IO. It may be easier to do something like this in UML because their IO driver is a constant (UBD) whereas in VServer things aren't so simple since the driver could be anything? Herbert can probably comment on this better :-) I do think that this would be an interesting feature. Grisha On Thu, 31 Mar 2005, Bodo Eggert wrote: On Thu, 31 Mar 2005, Herbert Poetzl wrote: On Thu, Mar 31, 2005 at 09:26:31AM +0200, [EMAIL PROTECTED] wrote: hmm, so you would like to artificially slow down the I/O transfer of a vserver, and make the transaction somewhat longer than necessary? I guess more like not slowing down the host or other vservers. -- Funny quotes: 19. Quantum mechanics: The dreams stuff is made of. ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [RE:] Re: [Vserver] Linux Vserver - Feature Question
On Thu, 31 Mar 2005, Herbert Poetzl wrote: > On Thu, Mar 31, 2005 at 09:26:31AM +0200, [EMAIL PROTECTED] wrote: > hmm, so you would like to artificially slow down the > I/O transfer of a vserver, and make the transaction > somewhat longer than necessary? I guess more like not slowing down the host or other vservers. -- Funny quotes: 19. Quantum mechanics: The dreams stuff is made of. ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
RE: [RE:] Re: [Vserver] Linux Vserver - Feature Question
> > > >would it be possilble to limit write/read speed for single vservers > to > >split > > > >the speed of the disc? For vservers with many copy,gzip or in > special > >backup jobs it would be quite usefull. > > hmm, so you would like to artificially slow down the > I/O transfer of a vserver, and make the transaction > somewhat longer than necessary? > > best, > Herbert Actually, it's very easy for a long running PostgreSQL query to consume all of the i/o bandwidth on a server. I've had this happen where a seq scan on a database with many millions of records on an IDE disk has effectively hosed the server for an hour or more. Of course, I am the administrator for all of my virtual servers so I plan around these types of batch jobs, but if I were setting up a hosting environment and gave vserver root access to others I'd be concerned. This would be very easy to duplicate by creating a db table with 20 million random values and then doing a query like this: select min(rand_val) as min, max(rand_val) as max from table1; Just make sure there are no indexes on the table of course. Also make sure the size of the database table is bigger than the amount of RAM the OS can use for caching. -- Matthew Nuzum <[EMAIL PROTECTED]> www.followers.net - Makers of "Elite Content Management System" View samples of Elite CMS in action by visiting http://www.elitecms.com ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [RE:] Re: [Vserver] Linux Vserver - Feature Question
On Thu, Mar 31, 2005 at 09:26:31AM +0200, [EMAIL PROTECTED] wrote: >Thank you for your fast answers. > >I/O Limit for virtual machines. E.g. when you do a gzip job in a >virtual machine, the I/O Load and cpu Load will > >be quite high. To limit the cpu load is implemented i know, but the >i/o load? > > >in detail: > >- maximum write/read speed of a common sata disc 60/60 MB/s >- all of the vservers are installed on that disc > > >would it be possilble to limit write/read speed for single vservers to >split > >the speed of the disc? For vservers with many copy,gzip or in special >backup jobs it would be quite usefull. hmm, so you would like to artificially slow down the I/O transfer of a vserver, and make the transaction somewhat longer than necessary? best, Herbert >Regards > >Tom Eschler > > > >Ursprüngliche Nachricht >Absender: >Herbert Poetzl [1]mailto:[EMAIL PROTECTED] >Gesendet am: > 31.03.2005 08:45:40 >Betreff: >Re: [Vserver] Linux Vserver - Feature Question >Empfänger: >[EMAIL PROTECTED] [3]mailto:[EMAIL PROTECTED] >Cc: >[EMAIL PROTECTED] >[5]mailto:vserver@list.linux-vserver.org > > > > > >On Wed, Mar 30, 2005 at 10:27:50PM +0200, [EMAIL PROTECTED] wrote: > >>Will there ever be an io-load limit option? > > > >what should the I/O load limit option limit? > > > >please elaborate and give some details/examples > >what happens and how it should be limited ... > > > >TIA, > >Herbert > > > >>Regards > >> > >>Tom Eschler > > > >> ___ > >> Vserver mailing list > >> [EMAIL PROTECTED] > >> [8]http://list.linux-vserver.org/mailman/listinfo/vserver > > References > >1. mailto:[EMAIL PROTECTED] >2. file://localhost/home/herbert/tmp/[EMAIL PROTECTED] >3. mailto:[EMAIL PROTECTED] >4. file://localhost/home/herbert/tmp/vserver@list.linux-vserver.org >5. mailto:vserver@list.linux-vserver.org >6. file://localhost/home/herbert/tmp/[EMAIL PROTECTED] >7. file://localhost/home/herbert/tmp/Vserver@list.linux-vserver.org >8. http://list.linux-vserver.org/mailman/listinfo/vserver > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
[RE:] Re: [Vserver] Linux Vserver - Feature Question
Thank you for your fast answers. I/O Limit for virtual machines. E.g. when you do a gzip job in a virtual machine, the I/O Load and cpu Load will be quite high. To limit the cpu load is implemented i know, but the i/o load? in detail: - maximum write/read speed of a common sata disc 60/60 MB/s - all of the vservers are installed on that disc would it be possilble to limit write/read speed for single vservers to split the speed of the disc? For vservers with many copy,gzip or in special backup jobs it would be quite usefull. Regards Tom Eschler Ursprüngliche Nachricht Absender: Herbert Poetzl mailto:[EMAIL PROTECTED] Gesendet am: 31.03.2005 08:45:40 Betreff: Re: [Vserver] Linux Vserver - Feature Question Empfänger: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Cc: vserver@list.linux-vserver.org mailto:vserver@list.linux-vserver.org On Wed, Mar 30, 2005 at 10:27:50PM +0200, [EMAIL PROTECTED] wrote: > Will there ever be an io-load limit option? what should the I/O load limit option limit? please elaborate and give some details/examples what happens and how it should be limited ... TIA, Herbert > Regards > > Tom Eschler > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] Linux Vserver - Feature Question
On Wed, Mar 30, 2005 at 10:27:50PM +0200, [EMAIL PROTECTED] wrote: >Will there ever be an io-load limit option? what should the I/O load limit option limit? please elaborate and give some details/examples what happens and how it should be limited ... TIA, Herbert >Regards > >Tom Eschler > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
