Re: VM Requirement Document - v0.0

2001-07-08 Thread Pavel Machek
Hi! > Here's my first pass at a VM requirements document, > for the embedded, desktop, and server cases. At the end is > a summary of general rules that should take care of all of > these cases. > > Bandwidth Descriptions: > > immediate: RAM, on-chip cache, etc. > fast:

Re: VM Requirement Document - v0.0

2001-07-08 Thread Pavel Machek
Hi! Here's my first pass at a VM requirements document, for the embedded, desktop, and server cases. At the end is a summary of general rules that should take care of all of these cases. Bandwidth Descriptions: immediate: RAM, on-chip cache, etc. fast: Flash

Re: VM Requirement Document - v0.0

2001-07-06 Thread Daniel Phillips
On Friday 06 July 2001 21:09, Rik van Riel wrote: > On Thu, 5 Jul 2001, Daniel Phillips wrote: > > Let me comment on this again, having spent a couple of minutes > > more thinking about it. Would you be happy paying 1% of your > > battery life to get 80% less sluggish response after a memory > >

Re: VM Requirement Document - v0.0

2001-07-06 Thread Rik van Riel
On Thu, 5 Jul 2001, Daniel Phillips wrote: > Let me comment on this again, having spent a couple of minutes > more thinking about it. Would you be happy paying 1% of your > battery life to get 80% less sluggish response after a memory > pig exits? Just to pull a few random numbers out of my

Re: VM Requirement Document - v0.0

2001-07-06 Thread Rik van Riel
On Thu, 5 Jul 2001, Daniel Phillips wrote: Let me comment on this again, having spent a couple of minutes more thinking about it. Would you be happy paying 1% of your battery life to get 80% less sluggish response after a memory pig exits? Just to pull a few random numbers out of my ass

Re: VM Requirement Document - v0.0

2001-07-06 Thread Daniel Phillips
On Friday 06 July 2001 21:09, Rik van Riel wrote: On Thu, 5 Jul 2001, Daniel Phillips wrote: Let me comment on this again, having spent a couple of minutes more thinking about it. Would you be happy paying 1% of your battery life to get 80% less sluggish response after a memory pig

Re: VM Requirement Document - v0.0

2001-07-05 Thread mike_phillips
> Well, on a laptop memory and disk bandwith are rarely wasted - they cost > battery life. I've been playing around with different scenarios to see the differences in performance. A good way to trigger the cache problem is to untar a couple of kernel source trees or other large amounts of

Re: VM Requirement Document - v0.0

2001-07-05 Thread Alan Shutko
Daniel Phillips <[EMAIL PROTECTED]> writes: > Also, notice that the scenario we were originally discussing, the off-hours > updatedb, doesn't normally happen on laptops because they tend to be > suspended at that time. No, even worse, it happens when you open the laptop for the first time in

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 17:00, Xavier Bestel wrote: > On 05 Jul 2001 17:04:00 +0200, Daniel Phillips wrote: > > Also, notice that the scenario we were originally discussing, the > > off-hours updatedb, doesn't normally happen on laptops because they tend > > to be suspended at that time. > >

Re: VM Requirement Document - v0.0

2001-07-05 Thread Xavier Bestel
On 05 Jul 2001 17:04:00 +0200, Daniel Phillips wrote: > > Well, on a laptop memory and disk bandwith are rarely wasted - they cost > > battery life. > > Let me comment on this again, having spent a couple of minutes more > thinking about it. Would you be happy paying 1% of your battery life to

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 16:00, Xavier Bestel wrote: > On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: > > Here's an idea I just came up with while I was composing this... along > > the lines of using unused bandwidth for something that at least has a > > chance of being useful. Suppose we

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 16:00, Xavier Bestel wrote: > On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: > > Here's an idea I just came up with while I was composing this... along > > the lines of using unused bandwidth for something that at least has a > > chance of being useful. Suppose we

Re: VM Requirement Document - v0.0

2001-07-05 Thread Xavier Bestel
On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: > Here's an idea I just came up with while I was composing this... along the > lines of using unused bandwidth for something that at least has a chance of > being useful. Suppose we come to the end of a period of activity, the > general

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 03:49, you wrote: > > Getting the user's "interactive" programs loaded back > > in afterwards is a separate, much more difficult problem > > IMHO, but no doubt still has a reasonable solution. > > Possibly stupid suggestion... Maybe the interactive/GUI programs should >

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 03:49, you wrote: Getting the user's interactive programs loaded back in afterwards is a separate, much more difficult problem IMHO, but no doubt still has a reasonable solution. Possibly stupid suggestion... Maybe the interactive/GUI programs should wake up once

Re: VM Requirement Document - v0.0

2001-07-05 Thread Xavier Bestel
On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: Here's an idea I just came up with while I was composing this... along the lines of using unused bandwidth for something that at least has a chance of being useful. Suppose we come to the end of a period of activity, the general

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 16:00, Xavier Bestel wrote: On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: Here's an idea I just came up with while I was composing this... along the lines of using unused bandwidth for something that at least has a chance of being useful. Suppose we come

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 16:00, Xavier Bestel wrote: On 05 Jul 2001 15:02:51 +0200, Daniel Phillips wrote: Here's an idea I just came up with while I was composing this... along the lines of using unused bandwidth for something that at least has a chance of being useful. Suppose we come

Re: VM Requirement Document - v0.0

2001-07-05 Thread Xavier Bestel
On 05 Jul 2001 17:04:00 +0200, Daniel Phillips wrote: Well, on a laptop memory and disk bandwith are rarely wasted - they cost battery life. Let me comment on this again, having spent a couple of minutes more thinking about it. Would you be happy paying 1% of your battery life to get

Re: VM Requirement Document - v0.0

2001-07-05 Thread Daniel Phillips
On Thursday 05 July 2001 17:00, Xavier Bestel wrote: On 05 Jul 2001 17:04:00 +0200, Daniel Phillips wrote: Also, notice that the scenario we were originally discussing, the off-hours updatedb, doesn't normally happen on laptops because they tend to be suspended at that time. Suspended !=

Re: VM Requirement Document - v0.0

2001-07-05 Thread Alan Shutko
Daniel Phillips [EMAIL PROTECTED] writes: Also, notice that the scenario we were originally discussing, the off-hours updatedb, doesn't normally happen on laptops because they tend to be suspended at that time. No, even worse, it happens when you open the laptop for the first time in the

Re: VM Requirement Document - v0.0

2001-07-05 Thread mike_phillips
Well, on a laptop memory and disk bandwith are rarely wasted - they cost battery life. I've been playing around with different scenarios to see the differences in performance. A good way to trigger the cache problem is to untar a couple of kernel source trees or other large amounts of files,

Re: VM Requirement Document - v0.0

2001-07-04 Thread Dan Maas
> Getting the user's "interactive" programs loaded back > in afterwards is a separate, much more difficult problem > IMHO, but no doubt still has a reasonable solution. Possibly stupid suggestion... Maybe the interactive/GUI programs should wake up once in a while and touch a couple of their

Re: VM Requirement Document - v0.0

2001-07-04 Thread mike_phillips
> Remember that the first message was about a laptop. At 4:00AM there's > no activity but the updatedb one (and the other cron jobs). Simply, > there's no 'accessed-often' data. Moreover, I'd bet that 90% of the > metadata touched by updatedb won't be accessed at all in the future. > Laptop

Re: VM Requirement Document - v0.0

2001-07-04 Thread Daniel Phillips
On Wednesday 04 July 2001 11:41, Marco Colombo wrote: > On Tue, 3 Jul 2001, Daniel Phillips wrote: > > On Tuesday 03 July 2001 12:33, Marco Colombo wrote: > > > Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only > > > when background aging, maybe it's not enough to keep processes

Re: VM Requirement Document - v0.0

2001-07-04 Thread Daniel Phillips
On Wednesday 04 July 2001 10:32, Marco Colombo wrote: > On Tue, 3 Jul 2001, Daniel Phillips wrote: > > On Monday 02 July 2001 20:42, Rik van Riel wrote: > > > On Thu, 28 Jun 2001, Marco Colombo wrote: > > > > I'm not sure that, in general, recent pages with only one access are > > > > still

Re: VM Requirement Document - v0.0

2001-07-04 Thread Marco Colombo
On Tue, 3 Jul 2001, Daniel Phillips wrote: > On Tuesday 03 July 2001 12:33, Marco Colombo wrote: > > Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only > > when background aging, maybe it's not enough to keep processes like > > updatedb from causing interactive pages to be

Re: VM Requirement Document - v0.0

2001-07-04 Thread Marco Colombo
On Tue, 3 Jul 2001, Daniel Phillips wrote: > On Monday 02 July 2001 20:42, Rik van Riel wrote: > > On Thu, 28 Jun 2001, Marco Colombo wrote: > > > I'm not sure that, in general, recent pages with only one access are > > > still better eviction candidates compared to 8 hours old pages. Here > > >

Re: VM Requirement Document - v0.0

2001-07-04 Thread Ari Heitner
On Tue, 3 Jul 2001, Daniel Phillips wrote: > And by the way, this is just brainstorming, it hasn't reached the 'proposal' > stage yet. So while we're here, an idea someone proposed in #debian while discussion this thread ([EMAIL PROTECTED], you know who you are): QoS for application paging on

Re: VM Requirement Document - v0.0

2001-07-04 Thread Ari Heitner
On Tue, 3 Jul 2001, Daniel Phillips wrote: And by the way, this is just brainstorming, it hasn't reached the 'proposal' stage yet. So while we're here, an idea someone proposed in #debian while discussion this thread ([EMAIL PROTECTED], you know who you are): QoS for application paging on

Re: VM Requirement Document - v0.0

2001-07-04 Thread Marco Colombo
On Tue, 3 Jul 2001, Daniel Phillips wrote: On Monday 02 July 2001 20:42, Rik van Riel wrote: On Thu, 28 Jun 2001, Marco Colombo wrote: I'm not sure that, in general, recent pages with only one access are still better eviction candidates compared to 8 hours old pages. Here we need

Re: VM Requirement Document - v0.0

2001-07-04 Thread Marco Colombo
On Tue, 3 Jul 2001, Daniel Phillips wrote: On Tuesday 03 July 2001 12:33, Marco Colombo wrote: Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only when background aging, maybe it's not enough to keep processes like updatedb from causing interactive pages to be evicted.

Re: VM Requirement Document - v0.0

2001-07-04 Thread Daniel Phillips
On Wednesday 04 July 2001 10:32, Marco Colombo wrote: On Tue, 3 Jul 2001, Daniel Phillips wrote: On Monday 02 July 2001 20:42, Rik van Riel wrote: On Thu, 28 Jun 2001, Marco Colombo wrote: I'm not sure that, in general, recent pages with only one access are still better eviction

Re: VM Requirement Document - v0.0

2001-07-04 Thread Daniel Phillips
On Wednesday 04 July 2001 11:41, Marco Colombo wrote: On Tue, 3 Jul 2001, Daniel Phillips wrote: On Tuesday 03 July 2001 12:33, Marco Colombo wrote: Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only when background aging, maybe it's not enough to keep processes like

Re: VM Requirement Document - v0.0

2001-07-04 Thread mike_phillips
Remember that the first message was about a laptop. At 4:00AM there's no activity but the updatedb one (and the other cron jobs). Simply, there's no 'accessed-often' data. Moreover, I'd bet that 90% of the metadata touched by updatedb won't be accessed at all in the future. Laptop users

Re: VM Requirement Document - v0.0

2001-07-04 Thread Dan Maas
Getting the user's interactive programs loaded back in afterwards is a separate, much more difficult problem IMHO, but no doubt still has a reasonable solution. Possibly stupid suggestion... Maybe the interactive/GUI programs should wake up once in a while and touch a couple of their pages?

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
On Monday 02 July 2001 20:42, Rik van Riel wrote: > On Thu, 28 Jun 2001, Marco Colombo wrote: > > I'm not sure that, in general, recent pages with only one access are > > still better eviction candidates compared to 8 hours old pages. Here > > we need either another way to detect one-shot

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
An ammendment to my previous post... > I see three page priority levels: > > 0 - accessed-never/aged to zero > 1 - accessed-once/just loaded > 2 - accessed-often > > with these transitions: > > 0 -> 1, if a page is accessed > 1 -> 2, if a page is accessed a second time > 1 -> 0, if a

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
On Tuesday 03 July 2001 12:33, Marco Colombo wrote: > Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only > when background aging, maybe it's not enough to keep processes like > updatedb from causing interactive pages to be evicted. > That's why I said we should have another way

Re: VM Requirement Document - v0.0

2001-07-03 Thread Marco Colombo
On Mon, 2 Jul 2001, Rik van Riel wrote: > On Thu, 28 Jun 2001, Marco Colombo wrote: > > > I'm not sure that, in general, recent pages with only one access are > > still better eviction candidates compared to 8 hours old pages. Here > > we need either another way to detect one-shot activity (like

Re: VM Requirement Document - v0.0

2001-07-03 Thread Marco Colombo
On Mon, 2 Jul 2001, Rik van Riel wrote: On Thu, 28 Jun 2001, Marco Colombo wrote: I'm not sure that, in general, recent pages with only one access are still better eviction candidates compared to 8 hours old pages. Here we need either another way to detect one-shot activity (like the one

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
On Tuesday 03 July 2001 12:33, Marco Colombo wrote: Oh, yes, since that PAGE_AGE_BG_INTERACTIVE_MINIMUM is applied only when background aging, maybe it's not enough to keep processes like updatedb from causing interactive pages to be evicted. That's why I said we should have another way to

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
An ammendment to my previous post... I see three page priority levels: 0 - accessed-never/aged to zero 1 - accessed-once/just loaded 2 - accessed-often with these transitions: 0 - 1, if a page is accessed 1 - 2, if a page is accessed a second time 1 - 0, if a page gets old

Re: VM Requirement Document - v0.0

2001-07-03 Thread Daniel Phillips
On Monday 02 July 2001 20:42, Rik van Riel wrote: On Thu, 28 Jun 2001, Marco Colombo wrote: I'm not sure that, in general, recent pages with only one access are still better eviction candidates compared to 8 hours old pages. Here we need either another way to detect one-shot activity (like

Re: VM Requirement Document - v0.0

2001-07-02 Thread Rik van Riel
On Thu, 28 Jun 2001, Marco Colombo wrote: > I'm not sure that, in general, recent pages with only one access are > still better eviction candidates compared to 8 hours old pages. Here > we need either another way to detect one-shot activity (like the one > performed by updatedb), Fully agreed,

AW: VM Requirement Document - v0.0

2001-07-02 Thread Jens Hoffmann
Hi, > My laptop has 256mb of ram, > but every day > it runs the updatedb for locate. The remedy here is simple: Do not run updatedb from cron, but only when you made changes. Greetings, Jens - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a

AW: VM Requirement Document - v0.0

2001-07-02 Thread Jens Hoffmann
Hi, My laptop has 256mb of ram, but every day it runs the updatedb for locate. The remedy here is simple: Do not run updatedb from cron, but only when you made changes. Greetings, Jens - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to

Re: VM Requirement Document - v0.0

2001-07-02 Thread Rik van Riel
On Thu, 28 Jun 2001, Marco Colombo wrote: I'm not sure that, in general, recent pages with only one access are still better eviction candidates compared to 8 hours old pages. Here we need either another way to detect one-shot activity (like the one performed by updatedb), Fully agreed, but

Re: VM Requirement Document - v0.0

2001-06-28 Thread John Fremlin
[...] > immediate: RAM, on-chip cache, etc. > fast: Flash reads, ROMs, etc. > medium:Hard drives, CD-ROMs, 100Mb ethernet, etc. > slow: Flash writes, floppy disks, CD-WR burners > packeted: Reads/write should be in as large a packet as possible >

Re: VM Requirement Document - v0.0

2001-06-28 Thread Marco Colombo
On Thu, 28 Jun 2001, Daniel Phillips wrote: > On Thursday 28 June 2001 14:20, [EMAIL PROTECTED] wrote: > > > If individual pages could be classified as code (text segments), > > > data, file cache, and so on, I would specify costs to the paging > > > of such pages in or out. This way I can make

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 17:21, Jonathan Morton wrote: > >There is a simple change in strategy that will fix up the updatedb case > > quite nicely, it goes something like this: a single access to a page > > (e.g., reading it) isn't enough to bring it to the front of the LRU > > queue, but

Re: VM Requirement Document - v0.0

2001-06-28 Thread Jonathan Morton
>There is a simple change in strategy that will fix up the updatedb case quite >nicely, it goes something like this: a single access to a page (e.g., reading >it) isn't enough to bring it to the front of the LRU queue, but accessing it >twice or more is. This is being looked at. Say, when a

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 15:37, Alan Cox wrote: > > The problem with updatedb is that it pushes all applications to the swap, > > and when you get back in the morning, everything has to be paged back > > from swap just because the (stupid) OS is prepared for yet another > > updatedb run. > >

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 14:20, [EMAIL PROTECTED] wrote: > > If individual pages could be classified as code (text segments), > > data, file cache, and so on, I would specify costs to the paging > > of such pages in or out. This way I can make the system perfer > > to drop a file cache page that

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
> > Updatedb is a bit odd in that it mostly sucks in metadata and the buffer to > > page cache balancing is a bit suspect IMHO. > > In 2.4.6-pre, the buffer cache is no longer used for metata, right? For ext2 directory blocks the page cache is now used - To unsubscribe from this list: send the

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Alan Cox wrote: > > > That isnt really down to labelling pages, what you are talking qbout is what > > > you get for free when page aging works right (eg 2.0.39) but don't get in > > > 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre. > > > > Correct,

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
> > That isnt really down to labelling pages, what you are talking qbout is what > > you get for free when page aging works right (eg 2.0.39) but don't get in > > 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre. > > Correct, but all pages are not equal. That is the whole

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Alan Cox wrote: > > This would be extremely useful. My laptop has 256mb of ram, but every day > > it runs the updatedb for locate. This fills the memory with the file > > cache. Interactivity is then terrible, and swap is unnecessarily used. On > > the laptop all this hard

Re: VM Requirement Document - v0.0

2001-06-28 Thread John Fremlin
Stefan Hoffmeister <[EMAIL PROTECTED]> writes: [...] > Windows NT/2000 has flags that can be for each CreateFile operation > ("open" in Unix terms), for instance > > FILE_ATTRIBUTE_TEMPORARY > > FILE_FLAG_WRITE_THROUGH > FILE_FLAG_NO_BUFFERING > FILE_FLAG_RANDOM_ACCESS >

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On 28 Jun 2001, Xavier Bestel wrote: > On 28 Jun 2001 14:02:09 +0200, Tobias Ringstrom wrote: > > > This would be very useful, I think. Would it be very hard to classify > > pages like this (text/data/cache/...)? > > How would you classify a page of perl code ? I do know how the Perl

Re: VM Requirement Document - v0.0

2001-06-28 Thread Xavier Bestel
On 28 Jun 2001 14:02:09 +0200, Tobias Ringstrom wrote: > This would be very useful, I think. Would it be very hard to classify > pages like this (text/data/cache/...)? How would you classify a page of perl code ? Xav - To unsubscribe from this list: send the line "unsubscribe linux-kernel"

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
> This would be extremely useful. My laptop has 256mb of ram, but every day > it runs the updatedb for locate. This fills the memory with the file > cache. Interactivity is then terrible, and swap is unnecessarily used. On > the laptop all this hard drive thrashing is bad news for battery life

Re: VM Requirement Document - v0.0

2001-06-28 Thread mike_phillips
> If individual pages could be classified as code (text segments), > data, file cache, and so on, I would specify costs to the paging > of such pages in or out. This way I can make the system perfer > to drop a file cache page that has not been accessed for five > minutes, over a program

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Helge Hafting wrote: > Preventing swap-trashing at all cost doesn't help if the > machine loose to io-trashing instead. Performance will be > just as much down, although perhaps more satisfying because > people aren't that surprised if explicit file operations > take a long

Re: VM Requirement Document - v0.0

2001-06-28 Thread Martin Knoblauch
Helge Hafting wrote: > > Martin Knoblauch wrote: > > > > > maybe more specific: If the hit-rate is low and the cache is already > > 70+% of the systems memory, the chances maybe slim that more cache is > > going to improve the hit-rate. > > > Oh, but this is posible. You can get into

Re: VM Requirement Document - v0.0

2001-06-28 Thread Helge Hafting
Martin Knoblauch wrote: > > maybe more specific: If the hit-rate is low and the cache is already > 70+% of the systems memory, the chances maybe slim that more cache is > going to improve the hit-rate. > Oh, but this is posible. You can get into situations where the (file cache) working set

Re: VM Requirement Document - v0.0

2001-06-28 Thread Martin Knoblauch
Rik van Riel wrote: > > On Wed, 27 Jun 2001, Martin Knoblauch wrote: > > > I do not care much whether the cache is using 99% of the systems memory > > or 50%. As long as there is free memory, using it for cache is great. I > > care a lot if the cache takes down interactivity, because it pushes

Re: VM Requirement Document - v0.0

2001-06-28 Thread Martin Knoblauch
Rik van Riel wrote: On Wed, 27 Jun 2001, Martin Knoblauch wrote: I do not care much whether the cache is using 99% of the systems memory or 50%. As long as there is free memory, using it for cache is great. I care a lot if the cache takes down interactivity, because it pushes out

Re: VM Requirement Document - v0.0

2001-06-28 Thread Helge Hafting
Martin Knoblauch wrote: maybe more specific: If the hit-rate is low and the cache is already 70+% of the systems memory, the chances maybe slim that more cache is going to improve the hit-rate. Oh, but this is posible. You can get into situations where the (file cache) working set needs

Re: VM Requirement Document - v0.0

2001-06-28 Thread Martin Knoblauch
Helge Hafting wrote: Martin Knoblauch wrote: maybe more specific: If the hit-rate is low and the cache is already 70+% of the systems memory, the chances maybe slim that more cache is going to improve the hit-rate. Oh, but this is posible. You can get into situations where the

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Helge Hafting wrote: Preventing swap-trashing at all cost doesn't help if the machine loose to io-trashing instead. Performance will be just as much down, although perhaps more satisfying because people aren't that surprised if explicit file operations take a long time.

Re: VM Requirement Document - v0.0

2001-06-28 Thread mike_phillips
If individual pages could be classified as code (text segments), data, file cache, and so on, I would specify costs to the paging of such pages in or out. This way I can make the system perfer to drop a file cache page that has not been accessed for five minutes, over a program text

Re: VM Requirement Document - v0.0

2001-06-28 Thread Xavier Bestel
On 28 Jun 2001 14:02:09 +0200, Tobias Ringstrom wrote: This would be very useful, I think. Would it be very hard to classify pages like this (text/data/cache/...)? How would you classify a page of perl code ? Xav - To unsubscribe from this list: send the line unsubscribe linux-kernel in

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On 28 Jun 2001, Xavier Bestel wrote: On 28 Jun 2001 14:02:09 +0200, Tobias Ringstrom wrote: This would be very useful, I think. Would it be very hard to classify pages like this (text/data/cache/...)? How would you classify a page of perl code ? I do know how the Perl interpreter

Re: VM Requirement Document - v0.0

2001-06-28 Thread John Fremlin
Stefan Hoffmeister [EMAIL PROTECTED] writes: [...] Windows NT/2000 has flags that can be for each CreateFile operation (open in Unix terms), for instance FILE_ATTRIBUTE_TEMPORARY FILE_FLAG_WRITE_THROUGH FILE_FLAG_NO_BUFFERING FILE_FLAG_RANDOM_ACCESS

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 14:20, [EMAIL PROTECTED] wrote: If individual pages could be classified as code (text segments), data, file cache, and so on, I would specify costs to the paging of such pages in or out. This way I can make the system perfer to drop a file cache page that has not

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 15:37, Alan Cox wrote: The problem with updatedb is that it pushes all applications to the swap, and when you get back in the morning, everything has to be paged back from swap just because the (stupid) OS is prepared for yet another updatedb run. Updatedb is a

Re: VM Requirement Document - v0.0

2001-06-28 Thread Jonathan Morton
There is a simple change in strategy that will fix up the updatedb case quite nicely, it goes something like this: a single access to a page (e.g., reading it) isn't enough to bring it to the front of the LRU queue, but accessing it twice or more is. This is being looked at. Say, when a page is

Re: VM Requirement Document - v0.0

2001-06-28 Thread Daniel Phillips
On Thursday 28 June 2001 17:21, Jonathan Morton wrote: There is a simple change in strategy that will fix up the updatedb case quite nicely, it goes something like this: a single access to a page (e.g., reading it) isn't enough to bring it to the front of the LRU queue, but accessing it

Re: VM Requirement Document - v0.0

2001-06-28 Thread Marco Colombo
On Thu, 28 Jun 2001, Daniel Phillips wrote: On Thursday 28 June 2001 14:20, [EMAIL PROTECTED] wrote: If individual pages could be classified as code (text segments), data, file cache, and so on, I would specify costs to the paging of such pages in or out. This way I can make the

Re: VM Requirement Document - v0.0

2001-06-28 Thread John Fremlin
[...] immediate: RAM, on-chip cache, etc. fast: Flash reads, ROMs, etc. medium:Hard drives, CD-ROMs, 100Mb ethernet, etc. slow: Flash writes, floppy disks, CD-WR burners packeted: Reads/write should be in as large a packet as possible

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
Updatedb is a bit odd in that it mostly sucks in metadata and the buffer to page cache balancing is a bit suspect IMHO. In 2.4.6-pre, the buffer cache is no longer used for metata, right? For ext2 directory blocks the page cache is now used - To unsubscribe from this list: send the line

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Alan Cox wrote: This would be extremely useful. My laptop has 256mb of ram, but every day it runs the updatedb for locate. This fills the memory with the file cache. Interactivity is then terrible, and swap is unnecessarily used. On the laptop all this hard drive

Re: VM Requirement Document - v0.0

2001-06-28 Thread Tobias Ringstrom
On Thu, 28 Jun 2001, Alan Cox wrote: That isnt really down to labelling pages, what you are talking qbout is what you get for free when page aging works right (eg 2.0.39) but don't get in 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre. Correct, but all pages

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
That isnt really down to labelling pages, what you are talking qbout is what you get for free when page aging works right (eg 2.0.39) but don't get in 2.2 - and don't yet (although its coming) quite get right in 2.4.6pre. Correct, but all pages are not equal. That is the whole point of

Re: VM Requirement Document - v0.0

2001-06-28 Thread Alan Cox
This would be extremely useful. My laptop has 256mb of ram, but every day it runs the updatedb for locate. This fills the memory with the file cache. Interactivity is then terrible, and swap is unnecessarily used. On the laptop all this hard drive thrashing is bad news for battery life

Re: VM Requirement Document - v0.0

2001-06-27 Thread Rik van Riel
On Wed, 27 Jun 2001, Martin Knoblauch wrote: > I do not care much whether the cache is using 99% of the systems memory > or 50%. As long as there is free memory, using it for cache is great. I > care a lot if the cache takes down interactivity, because it pushes out > processes that it thinks

Re: VM Requirement Document - v0.0

2001-06-27 Thread Pozsar Balazs
> Rik> ... but I fail to see this one. If we get a low cache hit rate, > Rik> couldn't that just mean we allocated too little memory for the > Rik> cache ? > Or that we're doing big sequential reads of file(s) which are larger > than memory, in which case expanding the cache size buys us

Re: VM Requirement Document - v0.0

2001-06-27 Thread Marco Colombo
On Tue, 26 Jun 2001, Rik van Riel wrote: > On Tue, 26 Jun 2001, John Stoffel wrote: > > > >> * If we're getting low cache hit rates, don't flush > > >> processes to swap. > > >> * If we're getting good cache hit rates, flush old, idle > > >> processes to swap. > > > > Rik> ... but I fail to see

Re: VM Requirement Document - v0.0

2001-06-27 Thread Xavier Bestel
On 26 Jun 2001 20:43:33 -0400, Dan Maas wrote: > > Windows NT/2000 has flags that can be for each CreateFile operation > > ("open" in Unix terms), for instance > > > > FILE_ATTRIBUTE_TEMPORARY > > FILE_FLAG_WRITE_THROUGH > > FILE_FLAG_NO_BUFFERING > > FILE_FLAG_RANDOM_ACCESS > >

Re: VM Requirement Document - v0.0

2001-06-27 Thread Martin Knoblauch
>> * If we're getting low cache hit rates, don't flush >> processes to swap. >> * If we're getting good cache hit rates, flush old, idle >> processes to swap. Rik> ... but I fail to see this one. If we get a low cache hit rate, Rik> couldn't that just mean we allocated too little memory for

Re: VM Requirement Document - v0.0

2001-06-27 Thread Martin Knoblauch
* If we're getting low cache hit rates, don't flush processes to swap. * If we're getting good cache hit rates, flush old, idle processes to swap. Rik ... but I fail to see this one. If we get a low cache hit rate, Rik couldn't that just mean we allocated too little memory for the Rik

Re: VM Requirement Document - v0.0

2001-06-27 Thread Xavier Bestel
On 26 Jun 2001 20:43:33 -0400, Dan Maas wrote: Windows NT/2000 has flags that can be for each CreateFile operation (open in Unix terms), for instance FILE_ATTRIBUTE_TEMPORARY FILE_FLAG_WRITE_THROUGH FILE_FLAG_NO_BUFFERING FILE_FLAG_RANDOM_ACCESS FILE_FLAG_SEQUENTIAL_SCAN

Re: VM Requirement Document - v0.0

2001-06-27 Thread Marco Colombo
On Tue, 26 Jun 2001, Rik van Riel wrote: On Tue, 26 Jun 2001, John Stoffel wrote: * If we're getting low cache hit rates, don't flush processes to swap. * If we're getting good cache hit rates, flush old, idle processes to swap. Rik ... but I fail to see this one. If we get a

Re: VM Requirement Document - v0.0

2001-06-27 Thread Pozsar Balazs
Rik ... but I fail to see this one. If we get a low cache hit rate, Rik couldn't that just mean we allocated too little memory for the Rik cache ? Or that we're doing big sequential reads of file(s) which are larger than memory, in which case expanding the cache size buys us nothing, and

Re: VM Requirement Document - v0.0

2001-06-27 Thread Rik van Riel
On Wed, 27 Jun 2001, Martin Knoblauch wrote: I do not care much whether the cache is using 99% of the systems memory or 50%. As long as there is free memory, using it for cache is great. I care a lot if the cache takes down interactivity, because it pushes out processes that it thinks idle,

Re: VM Requirement Document - v0.0

2001-06-26 Thread Daniel Phillips
> I personally don't feel that the cache should be allowed to grow over > 50% of the system's memory at all, we've got so much in the cache at > that point, that we're probably not hitting it all that much. That depends very much on what you're using the system for. Suppose you're running a

Re: VM Requirement Document - v0.0

2001-06-26 Thread Mike Castle
On Tue, Jun 26, 2001 at 08:43:33PM -0400, Dan Maas wrote: > (hrm, maybe I could hack up my own manual read-ahead/drop-behind with mmap() > and memory locking...) Just to argue portability for a moment (portability on the expected results, that is, vs APIs). Would this technique work across a

Re: VM Requirement Document - v0.0

2001-06-26 Thread Dan Maas
> Windows NT/2000 has flags that can be for each CreateFile operation > ("open" in Unix terms), for instance > > FILE_ATTRIBUTE_TEMPORARY > FILE_FLAG_WRITE_THROUGH > FILE_FLAG_NO_BUFFERING > FILE_FLAG_RANDOM_ACCESS > FILE_FLAG_SEQUENTIAL_SCAN > There is a BSD-originated convention for

Re: VM Requirement Document - v0.0

2001-06-26 Thread Mike Castle
On Tue, Jun 26, 2001 at 03:48:09PM -0700, Jeffrey W. Baker wrote: > These flags would be really handy. We already have the raw device for > sequential reading of e.g. CDROM and DVD devices. Not going to help 99% of the applications out there. mrc -- Mike Castle [EMAIL PROTECTED]

  1   2   >