Re: [PATCH] Prevent OOM from killing init

[Anthony de Boer - USEnet]

Stephen Clouse wrote:
> We run Oracle on a development box here, and it's always the first to get the
> axe (non-root process using 70-80 MB VM).  ...
> It would be nice to give immunity to certain uids, ...

It would seem to me that the new capabilities stuff _could_ be the answer.

Basically, all "am I root?" checks in the kernel should be becoming cap
flags, the OOM killer already avoids killing root processes, it's already
a tenet that yes you can hose your system doing insane things as root but
that nonroot users should NOT be able to hose a system, so being able to
eg. grant this capability to Oracle or ungrant it from sendmail could let
a sysadmin tell the kernel what must be preserved regardless of its UID.

As a baseline I'd want to see all user processes die before any UID 0
stuff, but being able to retune this would be extremely good.

-- 
Anthony de Boer -- as seen at http://www.leftmind.net/~adb/ -- BOFH, eh?
 / "Just when you think you've got a handle on herding cats, \
 \ along comes a three-legged cat on amphetamines."  -- Skud /
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Roger Gammans]

On Sat, Mar 24, 2001 at 02:54:55AM -0300, Rik van Riel wrote:
> On Fri, 23 Mar 2001, Szabolcs Szakacsits wrote:
> >  - trying to kill a task that is permanently in TASK_UNINTERRUPTIBLE
> >will probably deadlock the machine [or the random OOM killer will
> >kill the box].
> 
> This could indeed be a problem, though I cannot really see any
> case where a task would be in TASK_UNINTERRUPTIBLE permanently.

I've seen this with 'mt rewind' jamming on ide-tape. I'm
not sure of the exact pathology , but ISTR that it
was related issuing that command while the hardware was busy.

In any case the point is that a badly written driver or faulty
h/w even in a subsiduary system can cause this.

In an ideal world of course these wouldn't happen, but OTOH
is this an issue in failing a box which is going to fail
anyway if we don't kill the process. If we could ensure
a graceful failure so much the better.

TTFN
-- 
Roger
 Think of the mess on the carpet. Sensible people do all their
 demon-summoning in the garage, which you can just hose down afterwards.
-- [EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Roger Gammans]

On Sat, Mar 24, 2001 at 02:54:55AM -0300, Rik van Riel wrote:
 On Fri, 23 Mar 2001, Szabolcs Szakacsits wrote:
   - trying to kill a task that is permanently in TASK_UNINTERRUPTIBLE
 will probably deadlock the machine [or the random OOM killer will
 kill the box].
 
 This could indeed be a problem, though I cannot really see any
 case where a task would be in TASK_UNINTERRUPTIBLE permanently.

I've seen this with 'mt rewind' jamming on ide-tape. I'm
not sure of the exact pathology , but ISTR that it
was related issuing that command while the hardware was busy.

In any case the point is that a badly written driver or faulty
h/w even in a subsiduary system can cause this.

In an ideal world of course these wouldn't happen, but OTOH
is this an issue in failing a box which is going to fail
anyway if we don't kill the process. If we could ensure
a graceful failure so much the better.

TTFN
-- 
Roger
 Think of the mess on the carpet. Sensible people do all their
 demon-summoning in the garage, which you can just hose down afterwards.
-- [EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Anthony de Boer - USEnet]

Stephen Clouse wrote:
 We run Oracle on a development box here, and it's always the first to get the
 axe (non-root process using 70-80 MB VM).  ...
 It would be nice to give immunity to certain uids, ...

It would seem to me that the new capabilities stuff _could_ be the answer.

Basically, all "am I root?" checks in the kernel should be becoming cap
flags, the OOM killer already avoids killing root processes, it's already
a tenet that yes you can hose your system doing insane things as root but
that nonroot users should NOT be able to hose a system, so being able to
eg. grant this capability to Oracle or ungrant it from sendmail could let
a sysadmin tell the kernel what must be preserved regardless of its UID.

As a baseline I'd want to see all user processes die before any UID 0
stuff, but being able to retune this would be extremely good.

-- 
Anthony de Boer -- as seen at http://www.leftmind.net/~adb/ -- BOFH, eh?
 / "Just when you think you've got a handle on herding cats, \
 \ along comes a three-legged cat on amphetamines."  -- Skud /
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Helge Hafting]

Alan Cox wrote:
> 
> > >How do you return an out of memory error to a C program that is out of memory
> > >due to a stack growth fault. There is actually not a language construct for it
> > SIGSEGV.
> > Stack overflow for a language like C using standard implementation techniques
> > is the same as a page fault while accessing a page for which there is no backing
> > store. SIGSEGV is the logical choice, and the one I'd expect on other Unices.
> 
> Guess again. You are expanding the stack because you have no room left on it.
> You take a fault. You want to report a SIGSEGV. Now where are you
> going to put the stack frame ?
> 
> SIGSEGV in combination with a preallocated alternate stack maybe, but then you
> still need to recover. C++ you can maybe do it with exception handling but
> C doesnt really have the structure and longjmp just doesnt cut it.

Seems to me a guard page would do the trick.  Make the last page of the
stack
non-overcommitable and marked not present.  Maybe non-swappable too in
case
nothing else can be swapped out for some reason.
(Yes, that wastes a page per process)
Whenever we hit the guard page, try expanding the stack.  
If it works - fine.  If not - make the guard page present _and_ deliver
the SIGSEGV using this last page of stack.  No complicated alternate
stack construct, just report OOM one page in advance.

OOM is still possible if the program don't handle SIGSEGV well.
But a smart program now have the option of doing emergency deallocations
and/or dump its precious intermediate results to file.

Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On 26 Mar 2001, James Antill wrote:

>  If you want overcommit great, and I think it's a valid default
> ... but it'd be nice if I could say I don't want it for apps that
> aren't written using glib etc.

Agreed.  Jonathan Morton seems to be making progress in testing
and debugging the non-overcommit patch from some time ago. If
things turn out to be trivial enough I wouldn't be surprised if
we got to see the option of non-overcommit somewhere in future
2.4 and 2.5 kernels...

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Mon, 26 Mar 2001, Jonathan Morton wrote:

> I have, however, found a bug in the non-overcommit patch - it seems to
> be capable of double-freeing (and then some) - starting 4 Java VMs and
> then closing them causes VMReserved to go negative on my system.

*grin*

It's nice to see the non-overcommit code being tested and
fixed like this. If there turns out to be a demand for this
patch, I guess we'll even want to integrate this into the
kernel ... possibly even the 2.4 kernel, if the code changes
are small/managable enough.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>> I'm currently investigating the old non-overcommit patch, which (apart from
>> needing manual applying to recent kernels) appears to be rather broken in a
>> trivial way.  It prevents allocation if total reserved memory is greater
>> than the total unallocated memory.  Let me say that again, a different way
>> - it prevents memory usage from exceeding 50%...
>
>Think fork(2).

fork() is allowed to return a failure value, and it already does so if
there isn't enough memory (at least with the limited tests I've come up
with).  Guess again.

I have, however, found a bug in the non-overcommit patch - it seems to be
capable of double-freeing (and then some) - starting 4 Java VMs and then
closing them causes VMReserved to go negative on my system.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Horst von Brand]

Jonathan Morton <[EMAIL PROTECTED]> said:
> I'm currently investigating the old non-overcommit patch, which (apart from
> needing manual applying to recent kernels) appears to be rather broken in a
> trivial way.  It prevents allocation if total reserved memory is greater
> than the total unallocated memory.  Let me say that again, a different way
> - it prevents memory usage from exceeding 50%...

Think fork(2).
-- 
Horst von Brand [EMAIL PROTECTED]
Casilla 9G, Vin~a del Mar, Chile   +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Horst von Brand]

Jonathan Morton [EMAIL PROTECTED] said:
 I'm currently investigating the old non-overcommit patch, which (apart from
 needing manual applying to recent kernels) appears to be rather broken in a
 trivial way.  It prevents allocation if total reserved memory is greater
 than the total unallocated memory.  Let me say that again, a different way
 - it prevents memory usage from exceeding 50%...

Think fork(2).
-- 
Horst von Brand [EMAIL PROTECTED]
Casilla 9G, Vin~a del Mar, Chile   +56 32 672616
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

 I'm currently investigating the old non-overcommit patch, which (apart from
 needing manual applying to recent kernels) appears to be rather broken in a
 trivial way.  It prevents allocation if total reserved memory is greater
 than the total unallocated memory.  Let me say that again, a different way
 - it prevents memory usage from exceeding 50%...

Think fork(2).

fork() is allowed to return a failure value, and it already does so if
there isn't enough memory (at least with the limited tests I've come up
with).  Guess again.

I have, however, found a bug in the non-overcommit patch - it seems to be
capable of double-freeing (and then some) - starting 4 Java VMs and then
closing them causes VMReserved to go negative on my system.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Mon, 26 Mar 2001, Jonathan Morton wrote:

 I have, however, found a bug in the non-overcommit patch - it seems to
 be capable of double-freeing (and then some) - starting 4 Java VMs and
 then closing them causes VMReserved to go negative on my system.

*grin*

It's nice to see the non-overcommit code being tested and
fixed like this. If there turns out to be a demand for this
patch, I guess we'll even want to integrate this into the
kernel ... possibly even the 2.4 kernel, if the code changes
are small/managable enough.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On 26 Mar 2001, James Antill wrote:

  If you want overcommit great, and I think it's a valid default
 ... but it'd be nice if I could say I don't want it for apps that
 aren't written using glib etc.

Agreed.  Jonathan Morton seems to be making progress in testing
and debugging the non-overcommit patch from some time ago. If
things turn out to be trivial enough I wouldn't be surprised if
we got to see the option of non-overcommit somewhere in future
2.4 and 2.5 kernels...

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Helge Hafting]

Alan Cox wrote:
 
  How do you return an out of memory error to a C program that is out of memory
  due to a stack growth fault. There is actually not a language construct for it
  SIGSEGV.
  Stack overflow for a language like C using standard implementation techniques
  is the same as a page fault while accessing a page for which there is no backing
  store. SIGSEGV is the logical choice, and the one I'd expect on other Unices.
 
 Guess again. You are expanding the stack because you have no room left on it.
 You take a fault. You want to report a SIGSEGV. Now where are you
 going to put the stack frame ?
 
 SIGSEGV in combination with a preallocated alternate stack maybe, but then you
 still need to recover. C++ you can maybe do it with exception handling but
 C doesnt really have the structure and longjmp just doesnt cut it.

Seems to me a guard page would do the trick.  Make the last page of the
stack
non-overcommitable and marked not present.  Maybe non-swappable too in
case
nothing else can be swapped out for some reason.
(Yes, that wastes a page per process)
Whenever we hit the guard page, try expanding the stack.  
If it works - fine.  If not - make the guard page present _and_ deliver
the SIGSEGV using this last page of stack.  No complicated alternate
stack construct, just report OOM one page in advance.

OOM is still possible if the program don't handle SIGSEGV well.
But a smart program now have the option of doing emergency deallocations
and/or dump its precious intermediate results to file.

Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sun, 25 Mar 2001, Jonathan Morton wrote:

> >> My patch already fixes OOM problems caused by overgrown caches/buffers, by
> >> making sure OOM is not triggered until these buffers have been cannibalised
> >> down to freepages.high.  If balancing problems still exist, then they
> >> should be retuned with my patch (or something very like it) in hand, to
> >> separate one problem from the other.  AFAIK, balancing should now be a
> >> performance issue rather than a stability issue.
> >
> >Great.  I haven't seen your patch yet as my gateway ate it's very last
> >disk.  I look forward to reading it.
>
> I'm currently investigating the old non-overcommit patch, which (apart from
> needing manual applying to recent kernels) appears to be rather broken in a
> trivial way.  It prevents allocation if total reserved memory is greater
> than the total unallocated memory.  Let me say that again, a different way
> - it prevents memory usage from exceeding 50%...
>
> Is there a fast way of getting total VM size?  Eg. equivalent to the
> following code:
>
> si_meminfo();
> si_swapinfo();
> free = i.totalram + i.totalswap;

Other than using their components?.. don't know.

> If not, I have to do some jiggery to keep good performance along with true
> non-overcommittance.

(thinking about mlock and what that could do to any saved state.. and
how long allocations can block and where.. egad.  then there's zones)

I'm no VM expert, but I wonder if the overhead of obtaining this info
will be the worst you have to deal with.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Stephen Satchell]

At 05:30 PM 3/25/01 +0200, you wrote:
> > Ultra reliable systems dont contain memory allocators. There are good 
> reasons
> > for this but the design trade offs are rather hard to make in a real world
> > environment
>
>I esp. they run on CPU's without a stack or what?

No dynamic memory allocation AT ALL.  That includes the prohibition of a 
stack.  I've seen avionics-loop systems that abstract a stack but the 
"allocators" are part of the application and are designed to fall over 
gracefully when they become full -- but getting this past a project manager 
is hard, as it should be.

Then there are those systems with rather interesting watchdog timers.  If 
you don't tickle them just right, they fire and force a restart.  The 
nastiest of these required that you send four specific values to a specific 
I/O port, and the hardware looked to see if the values violated certain 
timing guidelines.  If you sent the code too early or too late, or if the 
value in the sequence was incorrect, BAM.  The hardware was designed by a 
guy with some rather interesting experiences with software "engineers" 
dealing with watchdog timers...

Satch
   

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>> My patch already fixes OOM problems caused by overgrown caches/buffers, by
>> making sure OOM is not triggered until these buffers have been cannibalised
>> down to freepages.high.  If balancing problems still exist, then they
>> should be retuned with my patch (or something very like it) in hand, to
>> separate one problem from the other.  AFAIK, balancing should now be a
>> performance issue rather than a stability issue.
>
>Great.  I haven't seen your patch yet as my gateway ate it's very last
>disk.  I look forward to reading it.

I'm currently investigating the old non-overcommit patch, which (apart from
needing manual applying to recent kernels) appears to be rather broken in a
trivial way.  It prevents allocation if total reserved memory is greater
than the total unallocated memory.  Let me say that again, a different way
- it prevents memory usage from exceeding 50%...

Is there a fast way of getting total VM size?  Eg. equivalent to the
following code:

si_meminfo();
si_swapinfo();
free = i.totalram + i.totalswap;

If not, I have to do some jiggery to keep good performance along with true
non-overcommittance.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Guest section DW]

On Sun, Mar 25, 2001 at 01:32:42AM +0100, Kurt Garloff wrote:
> On Fri, Mar 23, 2001 at 05:26:22PM +, James A. Sutherland wrote:
> > If SuSE's install program needs more than a quarter Gb of RAM, you need a
> > better distro.
> 
> Well, it's rpm ...

Yes. I investigated and found rpm's data base corrupted, and rpm cannot handle
that. Since I have several occurrences of rpm being killed by the oom killer
in my logs it is entirely possible that the data base got corrupted because
rpm was killed while in the process of updating it.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

> >> While my post didn't give an exact formula, I was quite clear on the
> >>fact that
> >> the system is allowing the caches to overrun memory and cause oom problems.
> >
> >Yes.  A testcase would be good.  It's not happening to everybody nor is
> >it happening under all loads.  (if it were, it'd be long dead)
> >
> >> I'm more than happy to test patches, and I would even be willing to suggest
> >> some algorithms that might help, but I don't know where to stick them in the
> >> code.  Most of the people who have been griping are in a similar position.
> >
> >First step toward killing the critter is to lure him onto open ground.
> >Once there.. well, I've seen some pretty fancy shooting on this list.
>
> My patch already fixes OOM problems caused by overgrown caches/buffers, by
> making sure OOM is not triggered until these buffers have been cannibalised
> down to freepages.high.  If balancing problems still exist, then they
> should be retuned with my patch (or something very like it) in hand, to
> separate one problem from the other.  AFAIK, balancing should now be a
> performance issue rather than a stability issue.

Great.  I haven't seen your patch yet as my gateway ate it's very last
disk.  I look forward to reading it.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>[  about non-overcommit  ]
>> > Nobody feels its very important because nobody has implemented it.
>
>Enterprises use other systems because they have much better resource
>management than Linux -- adding non-overcommit wouldn't help them much.
>Desktop users, Linux newbies don't understand what's
>eager/early/non-overcommit vs lazy/late/overcommit memory management
>[just see these threads here if you aren't bored already enough ;)] and
>even if they do at last they don't have the ability to implement it. And
>between them, people are mostly fine with ulimit.
>
>> Small correction - It was implemented, just not included in the standard
>> kernel.
>
>Please note, adding optional non-overcommit also wouldn't help much
>without guaranteed/reserved resources [e.g. you are OOM -> appps, users
>complain, admin login in and BANG OOM killer just killed one of the
>jobs]. This was one of the reasons I made the reserved root memory
>patch [this is also the way other OS'es do]. Now just the different
>patches should be merged and write an OOM FAQ for users how to avoid,
>control, etc it].

I'm currently trying to apply the 2.3.99.whatever non-overcommit patch to
2.4.1 - decidedly nontrivial, lots of failed hunks, parts of the kernel
have changed significantly even in this (fairly short) time.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Guest section DW]

On Sat, Mar 24, 2001 at 02:57:27AM -0300, Rik van Riel wrote:
> On Fri, 23 Mar 2001, Guest section DW wrote:
> > On Fri, Mar 23, 2001 at 11:56:23AM -0300, Rik van Riel wrote:
> > > On Fri, 23 Mar 2001, Martin Dalecki wrote:
> > 
> > > > > Feel free to write better-working code.
> > > > 
> > > > I don't get paid for it and I'm not idling through my days...
> > > 
> > >   
> > 
> > No lies please.
> 
> You mean that you ARE willing to implement what you've been
> arguing for?

There had not been any such response by me -
thus you should not ascribe to me such a response.

Concerning overcommit: people tell me that Eduardo Horvath
in his patch submitted to l-k on 2000-03-31 already solved
the problem (entirely or to a large extent).

: This patch will prevent the linux kernel from allowing VM overcommit.

I have not yet read the code.

Andries
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Szabolcs Szakacsits]

On Sat, 24 Mar 2001, Jesse Pollard wrote:
> On Fri, 23 Mar 2001, Alan Cox wrote:
[  about non-overcommit  ]
> > Nobody feels its very important because nobody has implemented it.

Enterprises use other systems because they have much better resource
management than Linux -- adding non-overcommit wouldn't help them much.
Desktop users, Linux newbies don't understand what's
eager/early/non-overcommit vs lazy/late/overcommit memory management
[just see these threads here if you aren't bored already enough ;)] and
even if they do at last they don't have the ability to implement it. And
between them, people are mostly fine with ulimit.

> Small correction - It was implemented, just not included in the standard
> kernel.

Please note, adding optional non-overcommit also wouldn't help much
without guaranteed/reserved resources [e.g. you are OOM -> appps, users
complain, admin login in and BANG OOM killer just killed one of the
jobs]. This was one of the reasons I made the reserved root memory
patch [this is also the way other OS'es do]. Now just the different
patches should be merged and write an OOM FAQ for users how to avoid,
control, etc it].

Szaka

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>> >start your app, wait for malloc to fail, hit enter for the other app and
>> >watch you app to be OOM killed ;)
>>
>> That would only happen if memory_overcommit was turned on, in which case my
>> modification would have zero effect anyway (the overcommit test happens
>> before my code).
>
>Thanks for listening and trying out the above trivial code instead of
>wrong theoretical arguments ;)
>
>So again, Linux *always* overcommit memory, the
>/proc/sys/vm/overcommit_memory controls total overcommit or
>quasi-overcommit [ehen you make your check in vm_enough() the memory is
>already overcommitted].

OK, looks like I got mixed up between *reservation* (malloc) and
*allocation* (access), and we're checking allocated memory when we should
really be checking reserved.  Be patient - I haven't done much of this type
of thing...  but your argument turns out to be correct, and I eventually
figured it out for myself.  I certainly agree that the default should be to
assume that all reserved memory will be used.  Maybe even do little nasty
things like printk(KERN_WARN "root is overcommitting memory!\n"); in
appropriate places, to discourage overcommitting.

>The solution is something like,
>add optional non-overcommit support,
>   http://lwn.net/2000/0406/a/no-overcommit.html

This sounds like a good solution.  Saw the size of the patch, it's big and
touches lots of bits of VM code, but it looks as though parts of my ideas
will also fit in there and be helpful.

Hmm... so we get an adjusted or replaced OOM-kill-selection algorithm, my
out_of_memory() fix and runaway process clamp, and this big(ish)
memory-accounting patch.  Sounds like a good combination to me, fixing all
the problems I've heard about recently.

There are some unrelated performance problems I've encountered during my
testing (eg. kswapd gets incredibly inefficient when swap usage grows
beyond about 500Mb on my 256Mb physical machine, causing swap bandwidth to
fall way below the HDs' capabilities), which I'm going to ignore for now.
Probably whoever takes on the VM balancing problem can look into that, as
it's probably related to that rather than this...

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Alan Cox wrote:
> 
> > That depends what you mean by "must not". If it's your missile guidance
> > system, aircraft autopilot or life support system, the system must not run
> > out of memory in the first place. If the system breaks down badly, killing
> > init and thus panicking (hence rebooting, if the system is set up that
> > way) seems the best approach.
> 
> Ultra reliable systems dont contain memory allocators. There are good reasons
> for this but the design trade offs are rather hard to make in a real world
> environment

I esp. they run on CPU's without a stack or what?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Sandy Harris]

Kurt Garloff wrote:

> Kernel related questions IMHO are:
> (1) Why do we get into OOM?

There was a long thread about this a few months back. We get into OOM because
malloc(), calloc() etc. can allocate more memory than is actually available.

e.g. Say you have machine with 64 RAM + 64 swap = 128 megs with 40 megs in use,
so 88 free. Now two processes each malloc() 80 megs. Both succeed. If both
processes then use that memory, someone is likely to fail later.

> Can we avoid it?

The obvious solution is to consider the above behaviour a bug and fix it.
The second malloc() should fail. The process making that call can then look
at the return value and decide what to do about the failure.

However, this was extensively discussed here last year, and that solution was
quite firmly rejected. I never understood the reasons. See the archives.

Someone did announce they were working on patches implementing a sane malloc().
What happened to that project?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Marco Colombo]

On Fri, 23 Mar 2001, Jonathan Morton wrote:

> >The main point is letting malloc fail when the memory cannot be
> >guaranteed.
> 
> If I read various things correctly, malloc() is supposed to fail as you
> would expect if /proc/sys/vm/overcommit_memory is 0.  This is the case on
> my RH 6.2 box, dunno about yours.  I can write a simple test program which
> simply allocates tons of memory if you like...
> 
> ...and I did.  It filled up my physical and swap memory, and got killed by
> the OOM handler before malloc() failed, even though overcommit_memory was
> set to 0.
> 
> *BAD!*

Please search list archives, there are plenty of threads about
overcommitment.

Have a look at the sources, that part is easy to read and you'll
realize that /proc/sys/vm/overcommit_memory does not really enable
/ disable memory overcommitment: its closer to a sanity check to
disallow absurdly sized requests, IIRC.

.TM.
-- 
  /  /   /
 /  /   /   Marco Colombo
___/  ___  /   /  Technical Manager
   /  /   /  ESI s.r.l.
 _/ _/  _/ [EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Stephen Satchell wrote:
> 
> At 12:41 AM 3/25/01 +0100, you wrote:
> >If your box is running for example a mail server, and it appears that
> >another process is juste eating the free memory, do you really want to kill
> >the mail server, just because it's the main process and consuming more
> >memory and CPU than others?
> >
> >Well, fine, your OS is up, but your application is not here anymore.
> 
> If you have a mission-critical application running on your box, add it to
> the inittab file with the RESPAWN attribute.  That way, OOM killer kills
> it, init notices it, and init restarts your server.

That makes me actually rolling on by back... Just try to add oracle to
inittab
crash it and watch it grabefully restarting by repawn!

> By the way, are the people working on the OOM-killer also working to avoid
> killing task 1?

Already done.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Jonathan Morton wrote:
> 
> >Right now my best approximation is to make the OOM test be as optimistic as
> >it is safe to be, and the vm_enough_memory() test as pessimistic as
> >sensible.  Expect a test patch to appear on this list soon.
> 
> ...and here it is!
> 
> This fixes a number of small but linked problems:
> 
> - malloc() never returned 0 when the system ran out of memory, instead the OOM 
>killer was triggered.  Now, malloc() will return 0 if the calling process is more 
>than 4 times the size of the amount of free memory.  As a speedup, available swap 
>space is not considered unless physical memory is not sufficient to contain the 
>process.  Note that if overcommit_memory is switched on, malloc() will never return 0 
>anyway.
> 
> - OOM killer was triggered too early - now takes account of buffer and cache memory, 
>which can be cannibalised before the system has completely run out.
> 
> - OOM killer badness() factors readjusted in favour of Oracle-like processes 
>(consuming 10's of MB of RAM but up for 3 days or so and with a low-order UID?  Now 
>less likely to be killed...)
> 
> --- begin oom-patch.diff ---
> diff -u linux-2.4.1.orig/mm/mmap.c linux/mm/mmap.c
> --- linux-2.4.1.orig/mm/mmap.c  Mon Jan 29 16:10:41 2001
> +++ linux/mm/mmap.c Sat Mar 24 19:29:51 2001
> @@ -54,6 +54,7 @@
>  */
> 
> long free;
> +   struct sysinfo swp_info;
> 
>  /* Sometimes we want to use more memory than we have. */
> if (sysctl_overcommit_memory)
> @@ -62,8 +63,32 @@
> free = atomic_read(_pages);
> free += atomic_read(_cache_size);
> free += nr_free_pages();
> -   free += nr_swap_pages;
> -   return free > pages;
> +
> +   /* Attempt to curtail memory allocations before hard OOM occurs.
> +* Based on current process size, which is hopefully a good and fast 
>heuristic.
> +* Also fix bug where the real OOM limit of (free == freepages.min) is not 
>taken into account.
> +* In fact, we use freepages.high as the threshold to make sure there's 
>still room for buffers+cache.
> +*
> +* -- Jonathan "Chromatix" Morton, 24th March 2001
> +*/
> +
> +   if(current && current->mm)
> + free -= (current->mm->total_vm / 4);
> +
> +   free -= freepages.high;
> +
> +   /* Since getting swap info is expensive, see if our allocation can happen in 
>physical RAM */
> +   if(free > pages)
> + return 1;
> +
> +   /* Use the number of FREE swap pages, not the total */
> +   si_swapinfo(_info);
> +   free += swp_info.freeswap;
> +
> +   if(free > pages)
> + return 1;
> +
> +   return 0;
>  }
> 
>  /* Remove one vm structure from the inode's i_mapping address space. */
> Only in linux/mm/: mmap.c~
> diff -u linux-2.4.1.orig/mm/oom_kill.c linux/mm/oom_kill.c
> --- linux-2.4.1.orig/mm/oom_kill.c  Tue Nov 14 18:56:46 2000
> +++ linux/mm/oom_kill.c Sat Mar 24 20:35:20 2001
> @@ -76,7 +76,9 @@
> run_time = (jiffies - p->start_time) >> (SHIFT_HZ + 10);
> 
> points /= int_sqrt(cpu_time);
> -   points /= int_sqrt(int_sqrt(run_time));
> +
> +   /* Long-running processes are *very* important, so don't take the 4th root */
> +   points /= run_time;
> 
> /*
>  * Niced processes are most likely less important, so double
> @@ -93,6 +95,10 @@
> p->uid == 0 || p->euid == 0)
> points /= 4;
> 
> +   /* Much the same goes for processes with low UIDs */
> +   if(p->uid < 100 || p->euid < 100)
> + points /= 2;
> +
> /*
>  * We don't want to kill a process with direct hardware access.
>  * Not only could that mess up the hardware, but usually users
> @@ -192,12 +198,20 @@
>  int out_of_memory(void)
>  {
> struct sysinfo swp_info;
> +   long free;
> 
> /* Enough free memory?  Not OOM. */
> -   if (nr_free_pages() > freepages.min)
> +   free = nr_free_pages();
> +   if (free > freepages.min)
> +   return 0;
> +
> +   if (free + nr_inactive_clean_pages() > freepages.low)
> return 0;
> 
> -   if (nr_free_pages() + nr_inactive_clean_pages() > freepages.low)
> +   /* Buffers and caches can be freed up (Jonathan "Chromatix" Morton) */
> +   free += atomic_read(_pages);
> +   free += atomic_read(_cache_size);
> +   if (free > freepages.low)
> return 0;

Ahh this will make the oom killer robust against misbalanced
MM. I will assimiliate this idea.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Benoit Garnier wrote:
> 
> Szabolcs Szakacsits wrote :
> 
> > But if you start
> > to think you get the conclusion that process killing can't be avoided if
> > you want the system keep running.
> 
> What's the point in keeping the OS running if the applications are silently
> killed?
> 
> If your box is running for example a mail server, and it appears that
> another process is juste eating the free memory, do you really want to kill
> the mail server, just because it's the main process and consuming more
> memory and CPU than others?

Yes bloody dumn, becouse I can then go no to the box, blacklist
the smapper causing this with ipchains (or whatever it's called)
and restart sendmail - WITHOUT DRIVING 1900km.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Mike Galbraith wrote:
> 
> On Sat, 24 Mar 2001, Doug Ledford wrote:
> 
> [snip list of naughty behavior]
> 
> > What was that you were saying about "should *never* happen"?  Oh, and let's
> Get off your lazy butts and do something about it.  Don't work on the
> oom-killer though.. that's only a symptom.  Work on the problem instead.

You are absolutely right about the fact that there are serious
memmory balancing problems out there as well. But ther oom_killer.c
needs to be changed as well - becouse in it's current state it's
buggy as hell as well. You propably know that you earn stability
in SW systems by making them survive the borderline conditions...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Doug Ledford wrote:
> 
> Horst von Brand wrote:
> >
> > "Christian Bodmer" <[EMAIL PROTECTED]> said:
> >
> > > I can't say I understand the whole MM system, however the random killing
> > > of processes seems like a rather unfortunate solution to the problem. If
> > > someone has a spare minute, maybe they could explain to me why running
> > > out of free memory in kswapd results in a deadlock situation.
> >
> > OOM is not "normal operations", it is a machine under very extreme stress,
> > and should *never* happen. To complicate (or even worse, slow down or
> > otherwise use up resources like memory) normal operations for "better
> > handling of OOM" is total nonsense.
> 
> Puh-Leeze.   Let's inject some reality into this conversation:
> 
> [dledford@aic-cvs dledford]$ more kill-list
> Mar 10 22:02:34 monster kernel: Out of Memory: Killed process 475 (identd).
> Mar 10 22:03:25 monster kernel: Out of Memory: Killed process 660 (xfs).
...
> Mar 22 15:45:54 monster kernel: Out of Memory: Killed process 504 (atd).
> Mar 22 16:12:13 monster kernel: Out of Memory: Killed process 524 (sshd).
> [dledford@aic-cvs dledford]$
> 
> What was that you were saying about "should *never* happen"?  Oh, and let's
> not overlook the fact that it killed off mostly system daemons to start off
> with while leaving the real culprits alone.  Once it did get around to the
> real culprits (diff and tar), it wasn't even killing them because they were
> overly large, it was killing them because it wasn't reclaiming space from the
> buffer cache and page cache.  All of the programs running on this machine were
> never more than roughly 256MB of program code, and this is a 1GB machine.

This is due to the fact that Riks killer doesn't normalize the
resource units it's using for measure. Basically the current
penatly calculations are a good random number generator.

> This behavior is totally unacceptable and, as Alan put it, is a bug in the
> code.  It should never trigger the oom killer with 750+MB of cache sitting
> around, but it does.  If you want people to buy into the value of the oom
> killer, you've at least got to get it to quit killing shit when it absolutely
> doesn't need to.
> 
> To those people that would suggest I send in code I only have this to say.
> Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
> the cache reclaim functions and never kill anything.  That would at least fix
> the bug you see above.

Please just apply it to the patch I have recently send... It will help
more :-).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Doug Ledford wrote:
 
 Horst von Brand wrote:
 
  "Christian Bodmer" [EMAIL PROTECTED] said:
 
   I can't say I understand the whole MM system, however the random killing
   of processes seems like a rather unfortunate solution to the problem. If
   someone has a spare minute, maybe they could explain to me why running
   out of free memory in kswapd results in a deadlock situation.
 
  OOM is not "normal operations", it is a machine under very extreme stress,
  and should *never* happen. To complicate (or even worse, slow down or
  otherwise use up resources like memory) normal operations for "better
  handling of OOM" is total nonsense.
 
 Puh-Leeze.   Let's inject some reality into this conversation:
 
 [dledford@aic-cvs dledford]$ more kill-list
 Mar 10 22:02:34 monster kernel: Out of Memory: Killed process 475 (identd).
 Mar 10 22:03:25 monster kernel: Out of Memory: Killed process 660 (xfs).
...
 Mar 22 15:45:54 monster kernel: Out of Memory: Killed process 504 (atd).
 Mar 22 16:12:13 monster kernel: Out of Memory: Killed process 524 (sshd).
 [dledford@aic-cvs dledford]$
 
 What was that you were saying about "should *never* happen"?  Oh, and let's
 not overlook the fact that it killed off mostly system daemons to start off
 with while leaving the real culprits alone.  Once it did get around to the
 real culprits (diff and tar), it wasn't even killing them because they were
 overly large, it was killing them because it wasn't reclaiming space from the
 buffer cache and page cache.  All of the programs running on this machine were
 never more than roughly 256MB of program code, and this is a 1GB machine.

This is due to the fact that Riks killer doesn't normalize the
resource units it's using for measure. Basically the current
penatly calculations are a good random number generator.

 This behavior is totally unacceptable and, as Alan put it, is a bug in the
 code.  It should never trigger the oom killer with 750+MB of cache sitting
 around, but it does.  If you want people to buy into the value of the oom
 killer, you've at least got to get it to quit killing shit when it absolutely
 doesn't need to.
 
 To those people that would suggest I send in code I only have this to say.
 Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
 the cache reclaim functions and never kill anything.  That would at least fix
 the bug you see above.

Please just apply it to the patch I have recently send... It will help
more :-).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Mike Galbraith wrote:
 
 On Sat, 24 Mar 2001, Doug Ledford wrote:
 
 [snip list of naughty behavior]
 
  What was that you were saying about "should *never* happen"?  Oh, and let's
 Get off your lazy butts and do something about it.  Don't work on the
 oom-killer though.. that's only a symptom.  Work on the problem instead.

You are absolutely right about the fact that there are serious
memmory balancing problems out there as well. But ther oom_killer.c
needs to be changed as well - becouse in it's current state it's
buggy as hell as well. You propably know that you earn stability
in SW systems by making them survive the borderline conditions...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Benoit Garnier wrote:
 
 Szabolcs Szakacsits wrote :
 
  But if you start
  to think you get the conclusion that process killing can't be avoided if
  you want the system keep running.
 
 What's the point in keeping the OS running if the applications are silently
 killed?
 
 If your box is running for example a mail server, and it appears that
 another process is juste eating the free memory, do you really want to kill
 the mail server, just because it's the main process and consuming more
 memory and CPU than others?

Yes bloody dumn, becouse I can then go no to the box, blacklist
the smapper causing this with ipchains (or whatever it's called)
and restart sendmail - WITHOUT DRIVING 1900km.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Jonathan Morton wrote:
 
 Right now my best approximation is to make the OOM test be as optimistic as
 it is safe to be, and the vm_enough_memory() test as pessimistic as
 sensible.  Expect a test patch to appear on this list soon.
 
 ...and here it is!
 
 This fixes a number of small but linked problems:
 
 - malloc() never returned 0 when the system ran out of memory, instead the OOM 
killer was triggered.  Now, malloc() will return 0 if the calling process is more 
than 4 times the size of the amount of free memory.  As a speedup, available swap 
space is not considered unless physical memory is not sufficient to contain the 
process.  Note that if overcommit_memory is switched on, malloc() will never return 0 
anyway.
 
 - OOM killer was triggered too early - now takes account of buffer and cache memory, 
which can be cannibalised before the system has completely run out.
 
 - OOM killer badness() factors readjusted in favour of Oracle-like processes 
(consuming 10's of MB of RAM but up for 3 days or so and with a low-order UID?  Now 
less likely to be killed...)
 
 --- begin oom-patch.diff ---
 diff -u linux-2.4.1.orig/mm/mmap.c linux/mm/mmap.c
 --- linux-2.4.1.orig/mm/mmap.c  Mon Jan 29 16:10:41 2001
 +++ linux/mm/mmap.c Sat Mar 24 19:29:51 2001
 @@ -54,6 +54,7 @@
  */
 
 long free;
 +   struct sysinfo swp_info;
 
  /* Sometimes we want to use more memory than we have. */
 if (sysctl_overcommit_memory)
 @@ -62,8 +63,32 @@
 free = atomic_read(buffermem_pages);
 free += atomic_read(page_cache_size);
 free += nr_free_pages();
 -   free += nr_swap_pages;
 -   return free  pages;
 +
 +   /* Attempt to curtail memory allocations before hard OOM occurs.
 +* Based on current process size, which is hopefully a good and fast 
heuristic.
 +* Also fix bug where the real OOM limit of (free == freepages.min) is not 
taken into account.
 +* In fact, we use freepages.high as the threshold to make sure there's 
still room for buffers+cache.
 +*
 +* -- Jonathan "Chromatix" Morton, 24th March 2001
 +*/
 +
 +   if(current  current-mm)
 + free -= (current-mm-total_vm / 4);
 +
 +   free -= freepages.high;
 +
 +   /* Since getting swap info is expensive, see if our allocation can happen in 
physical RAM */
 +   if(free  pages)
 + return 1;
 +
 +   /* Use the number of FREE swap pages, not the total */
 +   si_swapinfo(swp_info);
 +   free += swp_info.freeswap;
 +
 +   if(free  pages)
 + return 1;
 +
 +   return 0;
  }
 
  /* Remove one vm structure from the inode's i_mapping address space. */
 Only in linux/mm/: mmap.c~
 diff -u linux-2.4.1.orig/mm/oom_kill.c linux/mm/oom_kill.c
 --- linux-2.4.1.orig/mm/oom_kill.c  Tue Nov 14 18:56:46 2000
 +++ linux/mm/oom_kill.c Sat Mar 24 20:35:20 2001
 @@ -76,7 +76,9 @@
 run_time = (jiffies - p-start_time)  (SHIFT_HZ + 10);
 
 points /= int_sqrt(cpu_time);
 -   points /= int_sqrt(int_sqrt(run_time));
 +
 +   /* Long-running processes are *very* important, so don't take the 4th root */
 +   points /= run_time;
 
 /*
  * Niced processes are most likely less important, so double
 @@ -93,6 +95,10 @@
 p-uid == 0 || p-euid == 0)
 points /= 4;
 
 +   /* Much the same goes for processes with low UIDs */
 +   if(p-uid  100 || p-euid  100)
 + points /= 2;
 +
 /*
  * We don't want to kill a process with direct hardware access.
  * Not only could that mess up the hardware, but usually users
 @@ -192,12 +198,20 @@
  int out_of_memory(void)
  {
 struct sysinfo swp_info;
 +   long free;
 
 /* Enough free memory?  Not OOM. */
 -   if (nr_free_pages()  freepages.min)
 +   free = nr_free_pages();
 +   if (free  freepages.min)
 +   return 0;
 +
 +   if (free + nr_inactive_clean_pages()  freepages.low)
 return 0;
 
 -   if (nr_free_pages() + nr_inactive_clean_pages()  freepages.low)
 +   /* Buffers and caches can be freed up (Jonathan "Chromatix" Morton) */
 +   free += atomic_read(buffermem_pages);
 +   free += atomic_read(page_cache_size);
 +   if (free  freepages.low)
 return 0;

Ahh this will make the oom killer robust against misbalanced
MM. I will assimiliate this idea.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Stephen Satchell wrote:
 
 At 12:41 AM 3/25/01 +0100, you wrote:
 If your box is running for example a mail server, and it appears that
 another process is juste eating the free memory, do you really want to kill
 the mail server, just because it's the main process and consuming more
 memory and CPU than others?
 
 Well, fine, your OS is up, but your application is not here anymore.
 
 If you have a mission-critical application running on your box, add it to
 the inittab file with the RESPAWN attribute.  That way, OOM killer kills
 it, init notices it, and init restarts your server.

That makes me actually rolling on by back... Just try to add oracle to
inittab
crash it and watch it grabefully restarting by repawn!

 By the way, are the people working on the OOM-killer also working to avoid
 killing task 1?

Already done.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Marco Colombo]

On Fri, 23 Mar 2001, Jonathan Morton wrote:

 The main point is letting malloc fail when the memory cannot be
 guaranteed.
 
 If I read various things correctly, malloc() is supposed to fail as you
 would expect if /proc/sys/vm/overcommit_memory is 0.  This is the case on
 my RH 6.2 box, dunno about yours.  I can write a simple test program which
 simply allocates tons of memory if you like...
 
 ...and I did.  It filled up my physical and swap memory, and got killed by
 the OOM handler before malloc() failed, even though overcommit_memory was
 set to 0.
 
 *BAD!*

Please search list archives, there are plenty of threads about
overcommitment.

Have a look at the sources, that part is easy to read and you'll
realize that /proc/sys/vm/overcommit_memory does not really enable
/ disable memory overcommitment: its closer to a sanity check to
disallow absurdly sized requests, IIRC.

.TM.
-- 
  /  /   /
 /  /   /   Marco Colombo
___/  ___  /   /  Technical Manager
   /  /   /  ESI s.r.l.
 _/ _/  _/ [EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Sandy Harris]

Kurt Garloff wrote:

 Kernel related questions IMHO are:
 (1) Why do we get into OOM?

There was a long thread about this a few months back. We get into OOM because
malloc(), calloc() etc. can allocate more memory than is actually available.

e.g. Say you have machine with 64 RAM + 64 swap = 128 megs with 40 megs in use,
so 88 free. Now two processes each malloc() 80 megs. Both succeed. If both
processes then use that memory, someone is likely to fail later.

 Can we avoid it?

The obvious solution is to consider the above behaviour a bug and fix it.
The second malloc() should fail. The process making that call can then look
at the return value and decide what to do about the failure.

However, this was extensively discussed here last year, and that solution was
quite firmly rejected. I never understood the reasons. See the archives.

Someone did announce they were working on patches implementing a sane malloc().
What happened to that project?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Martin Dalecki]

Alan Cox wrote:
 
  That depends what you mean by "must not". If it's your missile guidance
  system, aircraft autopilot or life support system, the system must not run
  out of memory in the first place. If the system breaks down badly, killing
  init and thus panicking (hence rebooting, if the system is set up that
  way) seems the best approach.
 
 Ultra reliable systems dont contain memory allocators. There are good reasons
 for this but the design trade offs are rather hard to make in a real world
 environment

I esp. they run on CPU's without a stack or what?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

 start your app, wait for malloc to fail, hit enter for the other app and
 watch you app to be OOM killed ;)

 That would only happen if memory_overcommit was turned on, in which case my
 modification would have zero effect anyway (the overcommit test happens
 before my code).

Thanks for listening and trying out the above trivial code instead of
wrong theoretical arguments ;)

So again, Linux *always* overcommit memory, the
/proc/sys/vm/overcommit_memory controls total overcommit or
quasi-overcommit [ehen you make your check in vm_enough() the memory is
already overcommitted].

OK, looks like I got mixed up between *reservation* (malloc) and
*allocation* (access), and we're checking allocated memory when we should
really be checking reserved.  Be patient - I haven't done much of this type
of thing...  but your argument turns out to be correct, and I eventually
figured it out for myself.  I certainly agree that the default should be to
assume that all reserved memory will be used.  Maybe even do little nasty
things like printk(KERN_WARN "root is overcommitting memory!\n"); in
appropriate places, to discourage overcommitting.

The solution is something like,
add optional non-overcommit support,
   http://lwn.net/2000/0406/a/no-overcommit.html

This sounds like a good solution.  Saw the size of the patch, it's big and
touches lots of bits of VM code, but it looks as though parts of my ideas
will also fit in there and be helpful.

Hmm... so we get an adjusted or replaced OOM-kill-selection algorithm, my
out_of_memory() fix and runaway process clamp, and this big(ish)
memory-accounting patch.  Sounds like a good combination to me, fixing all
the problems I've heard about recently.

There are some unrelated performance problems I've encountered during my
testing (eg. kswapd gets incredibly inefficient when swap usage grows
beyond about 500Mb on my 256Mb physical machine, causing swap bandwidth to
fall way below the HDs' capabilities), which I'm going to ignore for now.
Probably whoever takes on the VM balancing problem can look into that, as
it's probably related to that rather than this...

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Szabolcs Szakacsits]

On Sat, 24 Mar 2001, Jesse Pollard wrote:
 On Fri, 23 Mar 2001, Alan Cox wrote:
[  about non-overcommit  ]
  Nobody feels its very important because nobody has implemented it.

Enterprises use other systems because they have much better resource
management than Linux -- adding non-overcommit wouldn't help them much.
Desktop users, Linux newbies don't understand what's
eager/early/non-overcommit vs lazy/late/overcommit memory management
[just see these threads here if you aren't bored already enough ;)] and
even if they do at last they don't have the ability to implement it. And
between them, people are mostly fine with ulimit.

 Small correction - It was implemented, just not included in the standard
 kernel.

Please note, adding optional non-overcommit also wouldn't help much
without guaranteed/reserved resources [e.g. you are OOM - appps, users
complain, admin login in and BANG OOM killer just killed one of the
jobs]. This was one of the reasons I made the reserved root memory
patch [this is also the way other OS'es do]. Now just the different
patches should be merged and write an OOM FAQ for users how to avoid,
control, etc it].

Szaka

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Guest section DW]

On Sat, Mar 24, 2001 at 02:57:27AM -0300, Rik van Riel wrote:
 On Fri, 23 Mar 2001, Guest section DW wrote:
  On Fri, Mar 23, 2001 at 11:56:23AM -0300, Rik van Riel wrote:
   On Fri, 23 Mar 2001, Martin Dalecki wrote:
  
 Feel free to write better-working code.

I don't get paid for it and I'm not idling through my days...
   
 similar response from Andries
  
  No lies please.
 
 You mean that you ARE willing to implement what you've been
 arguing for?

There had not been any such response by me -
thus you should not ascribe to me such a response.

Concerning overcommit: people tell me that Eduardo Horvath
in his patch submitted to l-k on 2000-03-31 already solved
the problem (entirely or to a large extent).

: This patch will prevent the linux kernel from allowing VM overcommit.

I have not yet read the code.

Andries
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

[  about non-overcommit  ]
  Nobody feels its very important because nobody has implemented it.

Enterprises use other systems because they have much better resource
management than Linux -- adding non-overcommit wouldn't help them much.
Desktop users, Linux newbies don't understand what's
eager/early/non-overcommit vs lazy/late/overcommit memory management
[just see these threads here if you aren't bored already enough ;)] and
even if they do at last they don't have the ability to implement it. And
between them, people are mostly fine with ulimit.

 Small correction - It was implemented, just not included in the standard
 kernel.

Please note, adding optional non-overcommit also wouldn't help much
without guaranteed/reserved resources [e.g. you are OOM - appps, users
complain, admin login in and BANG OOM killer just killed one of the
jobs]. This was one of the reasons I made the reserved root memory
patch [this is also the way other OS'es do]. Now just the different
patches should be merged and write an OOM FAQ for users how to avoid,
control, etc it].

I'm currently trying to apply the 2.3.99.whatever non-overcommit patch to
2.4.1 - decidedly nontrivial, lots of failed hunks, parts of the kernel
have changed significantly even in this (fairly short) time.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

  While my post didn't give an exact formula, I was quite clear on the
 fact that
  the system is allowing the caches to overrun memory and cause oom problems.
 
 Yes.  A testcase would be good.  It's not happening to everybody nor is
 it happening under all loads.  (if it were, it'd be long dead)
 
  I'm more than happy to test patches, and I would even be willing to suggest
  some algorithms that might help, but I don't know where to stick them in the
  code.  Most of the people who have been griping are in a similar position.
 
 First step toward killing the critter is to lure him onto open ground.
 Once there.. well, I've seen some pretty fancy shooting on this list.

 My patch already fixes OOM problems caused by overgrown caches/buffers, by
 making sure OOM is not triggered until these buffers have been cannibalised
 down to freepages.high.  If balancing problems still exist, then they
 should be retuned with my patch (or something very like it) in hand, to
 separate one problem from the other.  AFAIK, balancing should now be a
 performance issue rather than a stability issue.

Great.  I haven't seen your patch yet as my gateway ate it's very last
disk.  I look forward to reading it.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Guest section DW]

On Sun, Mar 25, 2001 at 01:32:42AM +0100, Kurt Garloff wrote:
 On Fri, Mar 23, 2001 at 05:26:22PM +, James A. Sutherland wrote:
  If SuSE's install program needs more than a quarter Gb of RAM, you need a
  better distro.
 
 Well, it's rpm ...

Yes. I investigated and found rpm's data base corrupted, and rpm cannot handle
that. Since I have several occurrences of rpm being killed by the oom killer
in my logs it is entirely possible that the data base got corrupted because
rpm was killed while in the process of updating it.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

 My patch already fixes OOM problems caused by overgrown caches/buffers, by
 making sure OOM is not triggered until these buffers have been cannibalised
 down to freepages.high.  If balancing problems still exist, then they
 should be retuned with my patch (or something very like it) in hand, to
 separate one problem from the other.  AFAIK, balancing should now be a
 performance issue rather than a stability issue.

Great.  I haven't seen your patch yet as my gateway ate it's very last
disk.  I look forward to reading it.

I'm currently investigating the old non-overcommit patch, which (apart from
needing manual applying to recent kernels) appears to be rather broken in a
trivial way.  It prevents allocation if total reserved memory is greater
than the total unallocated memory.  Let me say that again, a different way
- it prevents memory usage from exceeding 50%...

Is there a fast way of getting total VM size?  Eg. equivalent to the
following code:

si_meminfo(i);
si_swapinfo(i);
free = i.totalram + i.totalswap;

If not, I have to do some jiggery to keep good performance along with true
non-overcommittance.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Stephen Satchell]

At 05:30 PM 3/25/01 +0200, you wrote:
  Ultra reliable systems dont contain memory allocators. There are good 
 reasons
  for this but the design trade offs are rather hard to make in a real world
  environment

I esp. they run on CPU's without a stack or what?

No dynamic memory allocation AT ALL.  That includes the prohibition of a 
stack.  I've seen avionics-loop systems that abstract a stack but the 
"allocators" are part of the application and are designed to fall over 
gracefully when they become full -- but getting this past a project manager 
is hard, as it should be.

Then there are those systems with rather interesting watchdog timers.  If 
you don't tickle them just right, they fire and force a restart.  The 
nastiest of these required that you send four specific values to a specific 
I/O port, and the hardware looked to see if the values violated certain 
timing guidelines.  If you sent the code too early or too late, or if the 
value in the sequence was incorrect, BAM.  The hardware was designed by a 
guy with some rather interesting experiences with software "engineers" 
dealing with watchdog timers...

Satch
   

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sun, 25 Mar 2001, Jonathan Morton wrote:

  My patch already fixes OOM problems caused by overgrown caches/buffers, by
  making sure OOM is not triggered until these buffers have been cannibalised
  down to freepages.high.  If balancing problems still exist, then they
  should be retuned with my patch (or something very like it) in hand, to
  separate one problem from the other.  AFAIK, balancing should now be a
  performance issue rather than a stability issue.
 
 Great.  I haven't seen your patch yet as my gateway ate it's very last
 disk.  I look forward to reading it.

 I'm currently investigating the old non-overcommit patch, which (apart from
 needing manual applying to recent kernels) appears to be rather broken in a
 trivial way.  It prevents allocation if total reserved memory is greater
 than the total unallocated memory.  Let me say that again, a different way
 - it prevents memory usage from exceeding 50%...

 Is there a fast way of getting total VM size?  Eg. equivalent to the
 following code:

 si_meminfo(i);
 si_swapinfo(i);
 free = i.totalram + i.totalswap;

Other than using their components?.. don't know.

 If not, I have to do some jiggery to keep good performance along with true
 non-overcommittance.

(thinking about mlock and what that could do to any saved state.. and
how long allocations can block and where.. egad.  then there's zones)

I'm no VM expert, but I wonder if the overhead of obtaining this info
will be the worst you have to deal with.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Stephen Clouse]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sat, Mar 24, 2001 at 09:45:01PM -0800, Stephen Satchell wrote:
> If you have a mission-critical application running on your box, add it to 
> the inittab file with the RESPAWN attribute.  That way, OOM killer kills 
> it, init notices it, and init restarts your server.

Ah, that's great for simple daemons.  Now tell me how to help an app like this 
(Oracle exampled here):

oracle  89  0.0  0.4 41076 1776 ?SMar22   0:00 ora_pmon_slash
oracle  91  0.0  0.6 40676 2620 ?SMar22   0:00 ora_dbw0_slash
oracle  93  0.0  0.4 40544 1788 ?SMar22   0:00 ora_lgwr_slash
oracle  95  0.0  0.4 40544 1744 ?SMar22   0:00 ora_ckpt_slash
oracle  97  0.0  1.1 40556 4404 ?SMar22   0:00 ora_smon_slash
oracle  99  0.0  0.5 40536 2188 ?SMar22   0:00 ora_reco_slash
oracle 101  0.0  0.4 40656 1756 ?SMar22   0:00 ora_arc0_slash

In this example, when oom_kill reaps one of these autonomous threads, Oracle 
opts to crash and burn.  Database corruption is almost guaranteed.

In all reality, I'm sure any daemon (threads or no) that works heavily with disk
files is likely to screw itself and its data if it gets sigkilled for no
reason.  And in our environment, there is no reason for it to get sigkilled.

I'm going to severely hurt the first person that says such a program should be
*expecting* random untrappable annihilation of its threads.  (And what happens
when the master process *is* the target?)

- -- 
Stephen Clouse <[EMAIL PROTECTED]>
Senior Programmer, IQ Coordinator Project Lead
The IQ Group, Inc. 

-BEGIN PGP SIGNATURE-
Version: PGP 6.5.8

iQA/AwUBOr2XDgOGqGs0PadnEQK0rACfQELDid11+m90bS/DrGyrsHW45ZEAn19G
mL3fSCdi2TeHDxGLA8uXT8l5
=oQPV
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Stephen Satchell]

At 12:41 AM 3/25/01 +0100, you wrote:
>If your box is running for example a mail server, and it appears that
>another process is juste eating the free memory, do you really want to kill
>the mail server, just because it's the main process and consuming more
>memory and CPU than others?
>
>Well, fine, your OS is up, but your application is not here anymore.

If you have a mission-critical application running on your box, add it to 
the inittab file with the RESPAWN attribute.  That way, OOM killer kills 
it, init notices it, and init restarts your server.

By the way, are the people working on the OOM-killer also working to avoid 
killing task 1?

Satch

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Kurt Garloff]

On Fri, Mar 23, 2001 at 05:26:22PM +, James A. Sutherland wrote:
> If SuSE's install program needs more than a quarter Gb of RAM, you need a
> better distro.

Well, it's rpm ...
I guess the Debian packager is more friendly.
But if you choose to install a huge number of packages, the job to do for
the package manager (dependencies ...) is no trivial to do with few resources.

But that's not the point of the discussion.

Kernel related questions IMHO are:
(1) Why do we get into OOM? Can we avoid it?
(2) Is OOM sometimes misdetected (or triggered too early) and why?
(3) Does the OOM killer choose the right processes?

Regards,
-- 
Kurt Garloff  <[EMAIL PROTECTED]>  Eindhoven, NL
GPG key: See mail header, key servers Linux kernel development
SuSE GmbH, Nuernberg, FRG   SCSI, Security

 PGP signature

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>> While my post didn't give an exact formula, I was quite clear on the
>>fact that
>> the system is allowing the caches to overrun memory and cause oom problems.
>
>Yes.  A testcase would be good.  It's not happening to everybody nor is
>it happening under all loads.  (if it were, it'd be long dead)
>
>> I'm more than happy to test patches, and I would even be willing to suggest
>> some algorithms that might help, but I don't know where to stick them in the
>> code.  Most of the people who have been griping are in a similar position.
>
>First step toward killing the critter is to lure him onto open ground.
>Once there.. well, I've seen some pretty fancy shooting on this list.

My patch already fixes OOM problems caused by overgrown caches/buffers, by
making sure OOM is not triggered until these buffers have been cannibalised
down to freepages.high.  If balancing problems still exist, then they
should be retuned with my patch (or something very like it) in hand, to
separate one problem from the other.  AFAIK, balancing should now be a
performance issue rather than a stability issue.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>> free = atomic_read(_pages);
>> free += atomic_read(_cache_size);
>> free += nr_free_pages();
>> -   free += nr_swap_pages;
>
>> +   /* Since getting swap info is expensive, see if our allocation
>>can happen in physical RAM */
>
>Actually, getting swap info is as cheap as reading the variable
>nr_swap_pages. I should fix this in the OOM killer ;)

Just fixed that for myself (in both places) and about to test.  I'm almost
sure I actually encountered an error related to this, but I'll retest and
make sure...

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Benoit Garnier]

Szabolcs Szakacsits wrote :

> But if you start
> to think you get the conclusion that process killing can't be avoided if
> you want the system keep running.

What's the point in keeping the OS running if the applications are silently
killed?

If your box is running for example a mail server, and it appears that
another process is juste eating the free memory, do you really want to kill
the mail server, just because it's the main process and consuming more
memory and CPU than others?

Well, fine, your OS is up, but your application is not here anymore.

I just think there's no general solution, users must have the chance to
choose processes not to be killed, or malloc() returning errors.


Benoît GARNIER


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Doug Ledford wrote:

> Mike Galbraith wrote:
> >
> > General thread comment:
> > To those who are griping, and obviously rightfully so, Rik has twice
> > stated on this list that he could use some help with VM auto-balancing.
> > The responses (visible on this list at least) was rather underwhelming.
> > I noted no public exchange of ideas.. nada in fact.
>
> While my post didn't give an exact formula, I was quite clear on the fact that
> the system is allowing the caches to overrun memory and cause oom problems.

Yes.  A testcase would be good.  It's not happening to everybody nor is
it happening under all loads.  (if it were, it'd be long dead)

> I'm more than happy to test patches, and I would even be willing to suggest
> some algorithms that might help, but I don't know where to stick them in the
> code.  Most of the people who have been griping are in a similar position.

First step toward killing the critter is to lure him onto open ground.
Once there.. well, I've seen some pretty fancy shooting on this list.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

> free = atomic_read(_pages);
> free += atomic_read(_cache_size);
> free += nr_free_pages();
> -   free += nr_swap_pages;

> +   /* Since getting swap info is expensive, see if our allocation can happen in 
>physical RAM */

Actually, getting swap info is as cheap as reading the variable
nr_swap_pages. I should fix this in the OOM killer ;)

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>Right now my best approximation is to make the OOM test be as optimistic as
>it is safe to be, and the vm_enough_memory() test as pessimistic as
>sensible.  Expect a test patch to appear on this list soon.

...and here it is!

This fixes a number of small but linked problems:

- malloc() never returned 0 when the system ran out of memory, instead the OOM killer 
was triggered.  Now, malloc() will return 0 if the calling process is more than 4 
times the size of the amount of free memory.  As a speedup, available swap space is 
not considered unless physical memory is not sufficient to contain the process.  Note 
that if overcommit_memory is switched on, malloc() will never return 0 anyway.

- OOM killer was triggered too early - now takes account of buffer and cache memory, 
which can be cannibalised before the system has completely run out.

- OOM killer badness() factors readjusted in favour of Oracle-like processes 
(consuming 10's of MB of RAM but up for 3 days or so and with a low-order UID?  Now 
less likely to be killed...)

--- begin oom-patch.diff ---
diff -u linux-2.4.1.orig/mm/mmap.c linux/mm/mmap.c
--- linux-2.4.1.orig/mm/mmap.c  Mon Jan 29 16:10:41 2001
+++ linux/mm/mmap.c Sat Mar 24 19:29:51 2001
@@ -54,6 +54,7 @@
 */

long free;
+   struct sysinfo swp_info;

 /* Sometimes we want to use more memory than we have. */
if (sysctl_overcommit_memory)
@@ -62,8 +63,32 @@
free = atomic_read(_pages);
free += atomic_read(_cache_size);
free += nr_free_pages();
-   free += nr_swap_pages;
-   return free > pages;
+
+   /* Attempt to curtail memory allocations before hard OOM occurs.
+* Based on current process size, which is hopefully a good and fast heuristic.
+* Also fix bug where the real OOM limit of (free == freepages.min) is not 
+taken into account.
+* In fact, we use freepages.high as the threshold to make sure there's still 
+room for buffers+cache.
+*
+* -- Jonathan "Chromatix" Morton, 24th March 2001
+*/
+
+   if(current && current->mm)
+ free -= (current->mm->total_vm / 4);
+
+   free -= freepages.high;
+
+   /* Since getting swap info is expensive, see if our allocation can happen in 
+physical RAM */
+   if(free > pages)
+ return 1;
+
+   /* Use the number of FREE swap pages, not the total */
+   si_swapinfo(_info);
+   free += swp_info.freeswap;
+
+   if(free > pages)
+ return 1;
+
+   return 0;
 }

 /* Remove one vm structure from the inode's i_mapping address space. */
Only in linux/mm/: mmap.c~
diff -u linux-2.4.1.orig/mm/oom_kill.c linux/mm/oom_kill.c
--- linux-2.4.1.orig/mm/oom_kill.c  Tue Nov 14 18:56:46 2000
+++ linux/mm/oom_kill.c Sat Mar 24 20:35:20 2001
@@ -76,7 +76,9 @@
run_time = (jiffies - p->start_time) >> (SHIFT_HZ + 10);

points /= int_sqrt(cpu_time);
-   points /= int_sqrt(int_sqrt(run_time));
+
+   /* Long-running processes are *very* important, so don't take the 4th root */
+   points /= run_time;

/*
 * Niced processes are most likely less important, so double
@@ -93,6 +95,10 @@
p->uid == 0 || p->euid == 0)
points /= 4;

+   /* Much the same goes for processes with low UIDs */
+   if(p->uid < 100 || p->euid < 100)
+ points /= 2;
+
/*
 * We don't want to kill a process with direct hardware access.
 * Not only could that mess up the hardware, but usually users
@@ -192,12 +198,20 @@
 int out_of_memory(void)
 {
struct sysinfo swp_info;
+   long free;

/* Enough free memory?  Not OOM. */
-   if (nr_free_pages() > freepages.min)
+   free = nr_free_pages();
+   if (free > freepages.min)
+   return 0;
+
+   if (free + nr_inactive_clean_pages() > freepages.low)
return 0;

-   if (nr_free_pages() + nr_inactive_clean_pages() > freepages.low)
+   /* Buffers and caches can be freed up (Jonathan "Chromatix" Morton) */
+   free += atomic_read(_pages);
+   free += atomic_read(_cache_size);
+   if (free > freepages.low)
return 0;

/* Enough swap space left?  Not OOM. */
Only in linux/mm/: oom_kill.c~
--- end oom-patch.diff ---

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ 
PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a 

Re: [PATCH] Prevent OOM from killing init

[Jesse Pollard]

On Fri, 23 Mar 2001, Paul Jakma wrote:
>On Fri, 23 Mar 2001, Guest section DW wrote:
>
>> But yes, I am complaining because Linux by default is unreliable.
>
>no, your distribution is unreliable by default.
>
>> I strongly prefer a system that is reliable by default,
>> and I'll leave it to others to run it in an unreliable mode.
>
>currently, setting sensible user limits on my machines means i never
>get a hosed machine due to OOM. These limits are easy to set via
>pam_limits. (not perfect though, i think its session specific..)

Process specific. Each forked process gets the same limits. You get OOM
as soon as all processes together use more than the system capacity.

>granted, if the machine hasn't been setup with user limits, then linux
>doesn't deal at all well with OOM, so this should be fixed. but it can
>easily be argued that admin error in not configuring limits is the
>main cause for OOM.

Admin has no real control is the problem. Limits are only good for one
process. As soon as that process forks one other process then the
useage limit is twice the limit established.

>> Andries
>
>regards,
>
>--paulj

-- 
-
Jesse I Pollard, II
Email: [EMAIL PROTECTED]

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>While my post didn't give an exact formula, I was quite clear on the fact that
>the system is allowing the caches to overrun memory and cause oom problems.
>I'm more than happy to test patches, and I would even be willing to suggest
>some algorithms that might help, but I don't know where to stick them in the
>code.  Most of the people who have been griping are in a similar position.

Meanwhile, I'm looking *very* hard at the VM system and trying to figure
out how it works.  So far I've got an "improved" system under test which
requires a little stress to cause an OOM-before-malloc-failure.  Right now
I'm working on making the OOM happen only when it *really* needs to -
previously, as some pointed out, it could trigger far too early, for
example when there was lots of buffer and cache memory that could
potentially be cannibalised.

Right now my best approximation is to make the OOM test be as optimistic as
it is safe to be, and the vm_enough_memory() test as pessimistic as
sensible.  Expect a test patch to appear on this list soon.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Doug Ledford]

Mike Galbraith wrote:
> 
> On Sat, 24 Mar 2001, Doug Ledford wrote:
> > To those people that would suggest I send in code I only have this to say.
> > Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
> > the cache reclaim functions and never kill anything.  That would at least fix
> > the bug you see above.
> 
> That won't fix the problem, but merely paper it over.  The problem is
> in the balancing code that lets swap be exausted while at the same time
> allowing cache to become obscenely obese in the first place.  I can't
> trigger that behavior here, but it obviously exists for some workloads.

I would be more than happy to fix the problem properly if I knew the first
thing about the vm subsystem, but I don't.

> General thread comment:
> To those who are griping, and obviously rightfully so, Rik has twice
> stated on this list that he could use some help with VM auto-balancing.
> The responses (visible on this list at least) was rather underwhelming.
> I noted no public exchange of ideas.. nada in fact.

While my post didn't give an exact formula, I was quite clear on the fact that
the system is allowing the caches to overrun memory and cause oom problems. 
I'm more than happy to test patches, and I would even be willing to suggest
some algorithms that might help, but I don't know where to stick them in the
code.  Most of the people who have been griping are in a similar position.

> Get off your lazy butts and do something about it.  Don't work on the
> oom-killer though.. that's only a symptom.  Work on the problem instead.
> 
> -Mike  (who doesn't give a rats ass if he gets flamed;-)
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 

 Doug Ledford <[EMAIL PROTECTED]>  http://people.redhat.com/dledford
  Please check my web site for aic7xxx updates/answers before
  e-mailing me about problems
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jesse Pollard]

On Fri, 23 Mar 2001, Alan Cox wrote:
>> infinite storage. After all, earlier Unix flavours did not need
>> an OOM killer either, and my editor was not killed under Unix V6
>> on 64k when I started some other process.
>
>You were lucky. Its quite possible for V6 to kill processes when you run out
>of swap

Not lucky. I've used V6 - It would not start a process if the resources
werent available (no overcommit). It was also a swap based system and
not a page based system (PDP-11/45 1123+... supported both, but UNIX
only used swapping because it was easy to swap a 64Kbyte process).

>> The old Unix guarantee that a program only crashes because of
>> its own behaviour is lost. That is very sad.
>
>No such guarantee ever existed. There are systems that had stuff like per
>user memory quotas but those were mostly much more mainframe oriented

Only the swapping based systems gave this guarantee. Even AT System V
release 2 was swap based (M68020 systems).

>> 200 MB then the rest of that memory is not wasted. But it can
>> only be used for things that can be freed when needed, like
>> inode and buffer cache.
>
>No. You cannot free the inode and buffer cache arbitarily. You only have a
>probability - that puts you back at square 1.
>
>> But inefficient or not, I much prefer a system with guarantees,
>> something that is reliable by default, above something that
>> works well if you are lucky and fails at unpredictable moments.
>
>malloc is merely an accounting exercise (actually its mostly mmap 
>accounting). ptrace is the only quirk. Nobody feels its very important because
>nobody has implemented it.

Small correction - It was implemented, just not included in the standard
kernel.

Check mailing lists around March-April of 2000. The patch was generated
by Eduardo Horvath <[EMAIL PROTECTED]> for  2.3.99-pre3 and allowed the
administrator to:

"Available virtual memory is calculated as the sum of all swap space as
 well as free and reclaimable RAM, essentially the same value as used
 before.  The kernel will now operate in 4 different modes depending on the
 value of sysctl_overcommit_memory:
 
 1   Do accounting but do not prevent any allocations (old behavior)
 
 0   Do accounting but only prevent individual allocations that exceed
total VM (old behavior)
 
 -1  Do accounting and prevent a user from making the amount of
reserved memory exceed the total virtual memory.
 
 -2  Same as above but also for root.
 
 The default is set to -1 to allow root to essentially do whatever it
 wants.  But then if someone's broken root you're in trouble anyway.
 
 If the kernel itself requires memory it can allocate as much as it wants
 and can bring the system into an unsafe state (reserved > total).
 
 Memory segments that are not COW, ZFOD or otherwise swap backed do not
 require reservation."

It was a limited implementation, but worked quite well in testing.

-- 
-
Jesse I Pollard, II
Email: [EMAIL PROTECTED]

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

> >General thread comment:
> >To those who are griping, and obviously rightfully so, Rik has twice
> >stated on this list that he could use some help with VM auto-balancing.
> >The responses (visible on this list at least) was rather underwhelming.
> >I noted no public exchange of ideas.. nada in fact.
> >
> >Get off your lazy butts and do something about it.  Don't work on the
> >oom-killer though.. that's only a symptom.  Work on the problem instead.
>
> Since I'm hacking around in this area anyway (warning: kernel newbie
> alert!), I'd be happy to help examine the balancing code from a fresh
> perspective.  Where should I be looking?

Everything in mm plus fs/buffer.c at least. (plus includes)  A good
place to start is with __alloc_pages().. that will drag you through
a lot of the balancing code.  Following entry points (sys_brk, sys_mmap
etc) is highly recommended.  Be prepared for dizzy spells if you've
never toured mm-land before :)

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>I thought of some things which could break it, which I want to try and deal
>with before releasing a patch.  Specifically, I want to make freepages.min
>sacrosanct, so that malloc() *never* tries to use it.  This should be
>fairly easy to implement - simply subtract freepages.min from the freemem
>part.  An even nicer way would be to subtract freepages.low (or some
>similar value) instead of freepages.min for non-root or non-privileged
>processes.

Hmm, interesting.  Even with my modification - which means that
vm_enough_memory() will always return false if the allocation would clobber
freepages.min - I can still trigger OOM quite easily.  Even with no swap on
my box, there's a lot of disk activity, probably due to there being
virtually no disk cache left - could the generation of disk buffer and
cache pages be bypassing vm_enough_memory()?  If so, would using
freepages.low as the threshold rather than freepages.min help at all? (or
have I got everything muddled...)

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Gérard Roudier]

On Fri, 23 Mar 2001, Stephen E. Clark wrote:

> Alan Cox wrote:
> > 
> > > You don't beleve me if I tell you: DOS extender and JVM (Java Virtual
> > > Machine)
> > 
> > The JVM doesnt actually. The JVM will itself spontaenously explode in real
> > life when out of memory. Maybe the JVM on a DOS extender 8)
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [EMAIL PROTECTED]
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> 
> Back in the early nineties I was working with 18 developers on a Data
> General Aviion running DGUX. The system had only 16mb of memory and
> 600mb of disk. We were all continuously going thru the edit, compile,
> debug steps developing as large Computer Aided Dispatch System. Never
> did this system with its limited resources crash, or randomly start
> killing user or system processes.

What about the following (it is an estimate):

early nineties  -->  early eighties
18 developers   -->  18 developers
16mb of memory  -->   1 mb of memory
600 mb of disk  -->  70 mb of disk

Most current applications are so huge BLOATAGE that they should not 
deserve to be run just once. :-)
The kernel must try to cope with that and also with its own BLOATAGE.

Human nature is to eat what can be eaten, regardless if it is useful or
not.

> My $.02.

What about 'My M$.02' in some decades. :)

Btw, 'decade' comes from Latin 'deca'=10 and dies=days (not sure for
dies). As a result, it should have meant a period of 10 days instead of 10
years. It means a period of 10 days in French.

May-be, a knowledgeable person at this list has an explanation for this
misinterpretation. Could it be due to the word 'decadent' that has a 
very different ethymology.

10 days is too short for getting decadent, but 10 years should be enough,
no ? :-)

> Steve

  Gérard.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

At 6:58 am + 24/3/2001, Rik van Riel wrote:
>On Sat, 24 Mar 2001, Jonathan Morton wrote:
>
>> Hmm...  "if ( freemem < (size_of_mallocing_process / 20) )
>>fail_to_allocate;"
>>
>> Seems like a reasonable soft limit - processes which have already got
>> lots of RAM can probably stand not to have that little bit more and
>> can be curbed more quickly.
>
>This looks like it could nicely in preventing a single process
>from getting out of hand and gobbling up all memory.
>
>It won't prevent the system from a mongolian horde of processes,
>but nobody should expect your one-liner to fix world piece ;)
>
>I like it, now lets test it ;)

I thought of some things which could break it, which I want to try and deal
with before releasing a patch.  Specifically, I want to make freepages.min
sacrosanct, so that malloc() *never* tries to use it.  This should be
fairly easy to implement - simply subtract freepages.min from the freemem
part.  An even nicer way would be to subtract freepages.low (or some
similar value) instead of freepages.min for non-root or non-privileged
processes.

BTW, is the 'current' pointer always valid when vm_enough_memory() is
called?  If so, I can remove one redundant check.

My NetBSD friend appears to have found code in the BSD kernel which sets up
ulimit values sensibly by default - eg. it's not handled by the boot
scripts.  Presumably a root process is capable of changing the limits, but
I'm guessing that sensible defaults in the kernel have to be a Good Thing.
Comments?

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>General thread comment:
>To those who are griping, and obviously rightfully so, Rik has twice
>stated on this list that he could use some help with VM auto-balancing.
>The responses (visible on this list at least) was rather underwhelming.
>I noted no public exchange of ideas.. nada in fact.
>
>Get off your lazy butts and do something about it.  Don't work on the
>oom-killer though.. that's only a symptom.  Work on the problem instead.

Since I'm hacking around in this area anyway (warning: kernel newbie
alert!), I'd be happy to help examine the balancing code from a fresh
perspective.  Where should I be looking?

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Andries . Brouwer]

From [EMAIL PROTECTED] Sat Mar 24 03:00:17 2001

> No, ulimit does not work. (But it helps a little.)

no, not perfect, i very much agree. but in daily usage it reduces
chance of OOM to close to 0.

No. How would you use it? Compute individual limits for
each process? One typically has a few very large processes
that may easily take most of memory, and lots of small processes.
With a low ulimit these large processes do not run.
With a large ulimit it does not help against OOM.
The job of accounting what is available belongs to the system,
not the user.

Note that ulimit does not limit the sum of your processes,
it limits each individual process.

Andries
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Doug Ledford wrote:

[snip list of naughty behavior]

> What was that you were saying about "should *never* happen"?  Oh, and let's
> not overlook the fact that it killed off mostly system daemons to start off
> with while leaving the real culprits alone.  Once it did get around to the
> real culprits (diff and tar), it wasn't even killing them because they were
> overly large, it was killing them because it wasn't reclaiming space from the
> buffer cache and page cache.  All of the programs running on this machine were
> never more than roughly 256MB of program code, and this is a 1GB machine.
> This behavior is totally unacceptable and, as Alan put it, is a bug in the
> code.  It should never trigger the oom killer with 750+MB of cache sitting
> around, but it does.  If you want people to buy into the value of the oom
> killer, you've at least got to get it to quit killing shit when it absolutely
> doesn't need to.
>
> To those people that would suggest I send in code I only have this to say.
> Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
> the cache reclaim functions and never kill anything.  That would at least fix
> the bug you see above.

That won't fix the problem, but merely paper it over.  The problem is
in the balancing code that lets swap be exausted while at the same time
allowing cache to become obscenely obese in the first place.  I can't
trigger that behavior here, but it obviously exists for some workloads.

General thread comment:
To those who are griping, and obviously rightfully so, Rik has twice
stated on this list that he could use some help with VM auto-balancing.
The responses (visible on this list at least) was rather underwhelming.
I noted no public exchange of ideas.. nada in fact.

Get off your lazy butts and do something about it.  Don't work on the
oom-killer though.. that's only a symptom.  Work on the problem instead.

-Mike  (who doesn't give a rats ass if he gets flamed;-)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Doug Ledford wrote:

[snip list of naughty behavior]

 What was that you were saying about "should *never* happen"?  Oh, and let's
 not overlook the fact that it killed off mostly system daemons to start off
 with while leaving the real culprits alone.  Once it did get around to the
 real culprits (diff and tar), it wasn't even killing them because they were
 overly large, it was killing them because it wasn't reclaiming space from the
 buffer cache and page cache.  All of the programs running on this machine were
 never more than roughly 256MB of program code, and this is a 1GB machine.
 This behavior is totally unacceptable and, as Alan put it, is a bug in the
 code.  It should never trigger the oom killer with 750+MB of cache sitting
 around, but it does.  If you want people to buy into the value of the oom
 killer, you've at least got to get it to quit killing shit when it absolutely
 doesn't need to.

 To those people that would suggest I send in code I only have this to say.
 Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
 the cache reclaim functions and never kill anything.  That would at least fix
 the bug you see above.

That won't fix the problem, but merely paper it over.  The problem is
in the balancing code that lets swap be exausted while at the same time
allowing cache to become obscenely obese in the first place.  I can't
trigger that behavior here, but it obviously exists for some workloads.

General thread comment:
To those who are griping, and obviously rightfully so, Rik has twice
stated on this list that he could use some help with VM auto-balancing.
The responses (visible on this list at least) was rather underwhelming.
I noted no public exchange of ideas.. nada in fact.

Get off your lazy butts and do something about it.  Don't work on the
oom-killer though.. that's only a symptom.  Work on the problem instead.

-Mike  (who doesn't give a rats ass if he gets flamed;-)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Andries . Brouwer]

From [EMAIL PROTECTED] Sat Mar 24 03:00:17 2001

 No, ulimit does not work. (But it helps a little.)

no, not perfect, i very much agree. but in daily usage it reduces
chance of OOM to close to 0.

No. How would you use it? Compute individual limits for
each process? One typically has a few very large processes
that may easily take most of memory, and lots of small processes.
With a low ulimit these large processes do not run.
With a large ulimit it does not help against OOM.
The job of accounting what is available belongs to the system,
not the user.

Note that ulimit does not limit the sum of your processes,
it limits each individual process.

Andries
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

General thread comment:
To those who are griping, and obviously rightfully so, Rik has twice
stated on this list that he could use some help with VM auto-balancing.
The responses (visible on this list at least) was rather underwhelming.
I noted no public exchange of ideas.. nada in fact.

Get off your lazy butts and do something about it.  Don't work on the
oom-killer though.. that's only a symptom.  Work on the problem instead.

Since I'm hacking around in this area anyway (warning: kernel newbie
alert!), I'd be happy to help examine the balancing code from a fresh
perspective.  Where should I be looking?

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

At 6:58 am + 24/3/2001, Rik van Riel wrote:
On Sat, 24 Mar 2001, Jonathan Morton wrote:

 Hmm...  "if ( freemem  (size_of_mallocing_process / 20) )
fail_to_allocate;"

 Seems like a reasonable soft limit - processes which have already got
 lots of RAM can probably stand not to have that little bit more and
 can be curbed more quickly.

This looks like it could nicely in preventing a single process
from getting out of hand and gobbling up all memory.

It won't prevent the system from a mongolian horde of processes,
but nobody should expect your one-liner to fix world piece ;)

I like it, now lets test it ;)

I thought of some things which could break it, which I want to try and deal
with before releasing a patch.  Specifically, I want to make freepages.min
sacrosanct, so that malloc() *never* tries to use it.  This should be
fairly easy to implement - simply subtract freepages.min from the freemem
part.  An even nicer way would be to subtract freepages.low (or some
similar value) instead of freepages.min for non-root or non-privileged
processes.

BTW, is the 'current' pointer always valid when vm_enough_memory() is
called?  If so, I can remove one redundant check.

My NetBSD friend appears to have found code in the BSD kernel which sets up
ulimit values sensibly by default - eg. it's not handled by the boot
scripts.  Presumably a root process is capable of changing the limits, but
I'm guessing that sensible defaults in the kernel have to be a Good Thing.
Comments?

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Gérard Roudier]

On Fri, 23 Mar 2001, Stephen E. Clark wrote:

 Alan Cox wrote:
  
   You don't beleve me if I tell you: DOS extender and JVM (Java Virtual
   Machine)
  
  The JVM doesnt actually. The JVM will itself spontaenously explode in real
  life when out of memory. Maybe the JVM on a DOS extender 8)
  
  -
  To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
  the body of a message to [EMAIL PROTECTED]
  More majordomo info at  http://vger.kernel.org/majordomo-info.html
  Please read the FAQ at  http://www.tux.org/lkml/
 
 Back in the early nineties I was working with 18 developers on a Data
 General Aviion running DGUX. The system had only 16mb of memory and
 600mb of disk. We were all continuously going thru the edit, compile,
 debug steps developing as large Computer Aided Dispatch System. Never
 did this system with its limited resources crash, or randomly start
 killing user or system processes.

What about the following (it is an estimate):

early nineties  --  early eighties
18 developers   --  18 developers
16mb of memory  --   1 mb of memory
600 mb of disk  --  70 mb of disk

Most current applications are so huge BLOATAGE that they should not 
deserve to be run just once. :-)
The kernel must try to cope with that and also with its own BLOATAGE.

Human nature is to eat what can be eaten, regardless if it is useful or
not.

 My $.02.

What about 'My M$.02' in some decades. :)

Btw, 'decade' comes from Latin 'deca'=10 and dies=days (not sure for
dies). As a result, it should have meant a period of 10 days instead of 10
years. It means a period of 10 days in French.

May-be, a knowledgeable person at this list has an explanation for this
misinterpretation. Could it be due to the word 'decadent' that has a 
very different ethymology.

10 days is too short for getting decadent, but 10 years should be enough,
no ? :-)

 Steve

  Grard.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

I thought of some things which could break it, which I want to try and deal
with before releasing a patch.  Specifically, I want to make freepages.min
sacrosanct, so that malloc() *never* tries to use it.  This should be
fairly easy to implement - simply subtract freepages.min from the freemem
part.  An even nicer way would be to subtract freepages.low (or some
similar value) instead of freepages.min for non-root or non-privileged
processes.

Hmm, interesting.  Even with my modification - which means that
vm_enough_memory() will always return false if the allocation would clobber
freepages.min - I can still trigger OOM quite easily.  Even with no swap on
my box, there's a lot of disk activity, probably due to there being
virtually no disk cache left - could the generation of disk buffer and
cache pages be bypassing vm_enough_memory()?  If so, would using
freepages.low as the threshold rather than freepages.min help at all? (or
have I got everything muddled...)

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

 General thread comment:
 To those who are griping, and obviously rightfully so, Rik has twice
 stated on this list that he could use some help with VM auto-balancing.
 The responses (visible on this list at least) was rather underwhelming.
 I noted no public exchange of ideas.. nada in fact.
 
 Get off your lazy butts and do something about it.  Don't work on the
 oom-killer though.. that's only a symptom.  Work on the problem instead.

 Since I'm hacking around in this area anyway (warning: kernel newbie
 alert!), I'd be happy to help examine the balancing code from a fresh
 perspective.  Where should I be looking?

Everything in mm plus fs/buffer.c at least. (plus includes)  A good
place to start is with __alloc_pages().. that will drag you through
a lot of the balancing code.  Following entry points (sys_brk, sys_mmap
etc) is highly recommended.  Be prepared for dizzy spells if you've
never toured mm-land before :)

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jesse Pollard]

On Fri, 23 Mar 2001, Alan Cox wrote:
 infinite storage. After all, earlier Unix flavours did not need
 an OOM killer either, and my editor was not killed under Unix V6
 on 64k when I started some other process.

You were lucky. Its quite possible for V6 to kill processes when you run out
of swap

Not lucky. I've used V6 - It would not start a process if the resources
werent available (no overcommit). It was also a swap based system and
not a page based system (PDP-11/45 1123+... supported both, but UNIX
only used swapping because it was easy to swap a 64Kbyte process).

 The old Unix guarantee that a program only crashes because of
 its own behaviour is lost. That is very sad.

No such guarantee ever existed. There are systems that had stuff like per
user memory quotas but those were mostly much more mainframe oriented

Only the swapping based systems gave this guarantee. Even ATT System V
release 2 was swap based (M68020 systems).

 200 MB then the rest of that memory is not wasted. But it can
 only be used for things that can be freed when needed, like
 inode and buffer cache.

No. You cannot free the inode and buffer cache arbitarily. You only have a
probability - that puts you back at square 1.

 But inefficient or not, I much prefer a system with guarantees,
 something that is reliable by default, above something that
 works well if you are lucky and fails at unpredictable moments.

malloc is merely an accounting exercise (actually its mostly mmap 
accounting). ptrace is the only quirk. Nobody feels its very important because
nobody has implemented it.

Small correction - It was implemented, just not included in the standard
kernel.

Check mailing lists around March-April of 2000. The patch was generated
by Eduardo Horvath [EMAIL PROTECTED] for  2.3.99-pre3 and allowed the
administrator to:

"Available virtual memory is calculated as the sum of all swap space as
 well as free and reclaimable RAM, essentially the same value as used
 before.  The kernel will now operate in 4 different modes depending on the
 value of sysctl_overcommit_memory:
 
 1   Do accounting but do not prevent any allocations (old behavior)
 
 0   Do accounting but only prevent individual allocations that exceed
total VM (old behavior)
 
 -1  Do accounting and prevent a user from making the amount of
reserved memory exceed the total virtual memory.
 
 -2  Same as above but also for root.
 
 The default is set to -1 to allow root to essentially do whatever it
 wants.  But then if someone's broken root you're in trouble anyway.
 
 If the kernel itself requires memory it can allocate as much as it wants
 and can bring the system into an unsafe state (reserved  total).
 
 Memory segments that are not COW, ZFOD or otherwise swap backed do not
 require reservation."

It was a limited implementation, but worked quite well in testing.

-- 
-
Jesse I Pollard, II
Email: [EMAIL PROTECTED]

Any opinions expressed are solely my own.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Doug Ledford]

Mike Galbraith wrote:
 
 On Sat, 24 Mar 2001, Doug Ledford wrote:
  To those people that would suggest I send in code I only have this to say.
  Fine, I'll send in a patch to fix this bug.  It will make the oom killer call
  the cache reclaim functions and never kill anything.  That would at least fix
  the bug you see above.
 
 That won't fix the problem, but merely paper it over.  The problem is
 in the balancing code that lets swap be exausted while at the same time
 allowing cache to become obscenely obese in the first place.  I can't
 trigger that behavior here, but it obviously exists for some workloads.

I would be more than happy to fix the problem properly if I knew the first
thing about the vm subsystem, but I don't.

 General thread comment:
 To those who are griping, and obviously rightfully so, Rik has twice
 stated on this list that he could use some help with VM auto-balancing.
 The responses (visible on this list at least) was rather underwhelming.
 I noted no public exchange of ideas.. nada in fact.

While my post didn't give an exact formula, I was quite clear on the fact that
the system is allowing the caches to overrun memory and cause oom problems. 
I'm more than happy to test patches, and I would even be willing to suggest
some algorithms that might help, but I don't know where to stick them in the
code.  Most of the people who have been griping are in a similar position.

 Get off your lazy butts and do something about it.  Don't work on the
 oom-killer though.. that's only a symptom.  Work on the problem instead.
 
 -Mike  (who doesn't give a rats ass if he gets flamed;-)
 
 -
 To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/

-- 

 Doug Ledford [EMAIL PROTECTED]  http://people.redhat.com/dledford
  Please check my web site for aic7xxx updates/answers before
  e-mailing me about problems
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

While my post didn't give an exact formula, I was quite clear on the fact that
the system is allowing the caches to overrun memory and cause oom problems.
I'm more than happy to test patches, and I would even be willing to suggest
some algorithms that might help, but I don't know where to stick them in the
code.  Most of the people who have been griping are in a similar position.

Meanwhile, I'm looking *very* hard at the VM system and trying to figure
out how it works.  So far I've got an "improved" system under test which
requires a little stress to cause an OOM-before-malloc-failure.  Right now
I'm working on making the OOM happen only when it *really* needs to -
previously, as some pointed out, it could trigger far too early, for
example when there was lots of buffer and cache memory that could
potentially be cannibalised.

Right now my best approximation is to make the OOM test be as optimistic as
it is safe to be, and the vm_enough_memory() test as pessimistic as
sensible.  Expect a test patch to appear on this list soon.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

 free = atomic_read(buffermem_pages);
 free += atomic_read(page_cache_size);
 free += nr_free_pages();
 -   free += nr_swap_pages;

 +   /* Since getting swap info is expensive, see if our allocation can happen in 
physical RAM */

Actually, getting swap info is as cheap as reading the variable
nr_swap_pages. I should fix this in the OOM killer ;)

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Mike Galbraith]

On Sat, 24 Mar 2001, Doug Ledford wrote:

 Mike Galbraith wrote:
 
  General thread comment:
  To those who are griping, and obviously rightfully so, Rik has twice
  stated on this list that he could use some help with VM auto-balancing.
  The responses (visible on this list at least) was rather underwhelming.
  I noted no public exchange of ideas.. nada in fact.

 While my post didn't give an exact formula, I was quite clear on the fact that
 the system is allowing the caches to overrun memory and cause oom problems.

Yes.  A testcase would be good.  It's not happening to everybody nor is
it happening under all loads.  (if it were, it'd be long dead)

 I'm more than happy to test patches, and I would even be willing to suggest
 some algorithms that might help, but I don't know where to stick them in the
 code.  Most of the people who have been griping are in a similar position.

First step toward killing the critter is to lure him onto open ground.
Once there.. well, I've seen some pretty fancy shooting on this list.

-Mike

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

 free = atomic_read(buffermem_pages);
 free += atomic_read(page_cache_size);
 free += nr_free_pages();
 -   free += nr_swap_pages;

 +   /* Since getting swap info is expensive, see if our allocation
can happen in physical RAM */

Actually, getting swap info is as cheap as reading the variable
nr_swap_pages. I should fix this in the OOM killer ;)

Just fixed that for myself (in both places) and about to test.  I'm almost
sure I actually encountered an error related to this, but I'll retest and
make sure...

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Benoit Garnier]

Szabolcs Szakacsits wrote :

 But if you start
 to think you get the conclusion that process killing can't be avoided if
 you want the system keep running.

What's the point in keeping the OS running if the applications are silently
killed?

If your box is running for example a mail server, and it appears that
another process is juste eating the free memory, do you really want to kill
the mail server, just because it's the main process and consuming more
memory and CPU than others?

Well, fine, your OS is up, but your application is not here anymore.

I just think there's no general solution, users must have the chance to
choose processes not to be killed, or malloc() returning errors.


Benot GARNIER


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

 While my post didn't give an exact formula, I was quite clear on the
fact that
 the system is allowing the caches to overrun memory and cause oom problems.

Yes.  A testcase would be good.  It's not happening to everybody nor is
it happening under all loads.  (if it were, it'd be long dead)

 I'm more than happy to test patches, and I would even be willing to suggest
 some algorithms that might help, but I don't know where to stick them in the
 code.  Most of the people who have been griping are in a similar position.

First step toward killing the critter is to lure him onto open ground.
Once there.. well, I've seen some pretty fancy shooting on this list.

My patch already fixes OOM problems caused by overgrown caches/buffers, by
making sure OOM is not triggered until these buffers have been cannibalised
down to freepages.high.  If balancing problems still exist, then they
should be retuned with my patch (or something very like it) in hand, to
separate one problem from the other.  AFAIK, balancing should now be a
performance issue rather than a stability issue.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Kurt Garloff]

On Fri, Mar 23, 2001 at 05:26:22PM +, James A. Sutherland wrote:
 If SuSE's install program needs more than a quarter Gb of RAM, you need a
 better distro.

Well, it's rpm ...
I guess the Debian packager is more friendly.
But if you choose to install a huge number of packages, the job to do for
the package manager (dependencies ...) is no trivial to do with few resources.

But that's not the point of the discussion.

Kernel related questions IMHO are:
(1) Why do we get into OOM? Can we avoid it?
(2) Is OOM sometimes misdetected (or triggered too early) and why?
(3) Does the OOM killer choose the right processes?

Regards,
-- 
Kurt Garloff  [EMAIL PROTECTED]  Eindhoven, NL
GPG key: See mail header, key servers Linux kernel development
SuSE GmbH, Nuernberg, FRG   SCSI, Security

 PGP signature

Re: [PATCH] Prevent OOM from killing init

[Stephen Satchell]

At 12:41 AM 3/25/01 +0100, you wrote:
If your box is running for example a mail server, and it appears that
another process is juste eating the free memory, do you really want to kill
the mail server, just because it's the main process and consuming more
memory and CPU than others?

Well, fine, your OS is up, but your application is not here anymore.

If you have a mission-critical application running on your box, add it to 
the inittab file with the RESPAWN attribute.  That way, OOM killer kills 
it, init notices it, and init restarts your server.

By the way, are the people working on the OOM-killer also working to avoid 
killing task 1?

Satch

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Stephen Clouse]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sat, Mar 24, 2001 at 09:45:01PM -0800, Stephen Satchell wrote:
 If you have a mission-critical application running on your box, add it to 
 the inittab file with the RESPAWN attribute.  That way, OOM killer kills 
 it, init notices it, and init restarts your server.

Ah, that's great for simple daemons.  Now tell me how to help an app like this 
(Oracle exampled here):

oracle  89  0.0  0.4 41076 1776 ?SMar22   0:00 ora_pmon_slash
oracle  91  0.0  0.6 40676 2620 ?SMar22   0:00 ora_dbw0_slash
oracle  93  0.0  0.4 40544 1788 ?SMar22   0:00 ora_lgwr_slash
oracle  95  0.0  0.4 40544 1744 ?SMar22   0:00 ora_ckpt_slash
oracle  97  0.0  1.1 40556 4404 ?SMar22   0:00 ora_smon_slash
oracle  99  0.0  0.5 40536 2188 ?SMar22   0:00 ora_reco_slash
oracle 101  0.0  0.4 40656 1756 ?SMar22   0:00 ora_arc0_slash

In this example, when oom_kill reaps one of these autonomous threads, Oracle 
opts to crash and burn.  Database corruption is almost guaranteed.

In all reality, I'm sure any daemon (threads or no) that works heavily with disk
files is likely to screw itself and its data if it gets sigkilled for no
reason.  And in our environment, there is no reason for it to get sigkilled.

I'm going to severely hurt the first person that says such a program should be
*expecting* random untrappable annihilation of its threads.  (And what happens
when the master process *is* the target?)

- -- 
Stephen Clouse [EMAIL PROTECTED]
Senior Programmer, IQ Coordinator Project Lead
The IQ Group, Inc. http://www.theiqgroup.com/

-BEGIN PGP SIGNATURE-
Version: PGP 6.5.8

iQA/AwUBOr2XDgOGqGs0PadnEQK0rACfQELDid11+m90bS/DrGyrsHW45ZEAn19G
mL3fSCdi2TeHDxGLA8uXT8l5
=oQPV
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Doug Ledford]

"James A. Sutherland" wrote:
> On Thu, 22 Mar 2001, Guest section DW wrote:
> > (I think 2.4.0.)
> >
> > Clearly, Linux cannot be reliable if any process can be killed
> > at any moment.
> 
> What on earth did you expect to happen when the process exceeded the
> machine's capabilities? Using more than all the resources fails. There
> isn't an alternative.

You might be successful in convincing myself or Andries of this as soon as the
oom killer only kills things when the system is really out of memory.  Right
now, it's not really an oom killer, it's more like an "I'm Too Lazy To Free Up
Some More Pages So Now You Die" (ITLTFUSMPSNYD) killer.

-- 

 Doug Ledford <[EMAIL PROTECTED]>  http://people.redhat.com/dledford
  Please check my web site for aic7xxx updates/answers before
  e-mailing me about problems
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Doug Ledford]

Horst von Brand wrote:
> 
> "Christian Bodmer" <[EMAIL PROTECTED]> said:
> 
> > I can't say I understand the whole MM system, however the random killing
> > of processes seems like a rather unfortunate solution to the problem. If
> > someone has a spare minute, maybe they could explain to me why running
> > out of free memory in kswapd results in a deadlock situation.
> 
> OOM is not "normal operations", it is a machine under very extreme stress,
> and should *never* happen. To complicate (or even worse, slow down or
> otherwise use up resources like memory) normal operations for "better
> handling of OOM" is total nonsense.

Puh-Leeze.   Let's inject some reality into this conversation:

[dledford@aic-cvs dledford]$ more kill-list 
Mar 10 22:02:34 monster kernel: Out of Memory: Killed process 475 (identd).
Mar 10 22:03:25 monster kernel: Out of Memory: Killed process 660 (xfs).
Mar 10 23:02:43 monster kernel: Out of Memory: Killed process 415 (rpc.statd).
Mar 11 01:20:31 monster kernel: Out of Memory: Killed process 397 (portmap).
Mar 11 01:37:09 monster kernel: Out of Memory: Killed process 474 (identd).
Mar 11 02:56:54 monster kernel: Out of Memory: Killed process 659 (xfs).
Mar 11 03:01:43 monster kernel: Out of Memory: Killed process 414 (rpc.statd).
Mar 11 03:09:30 monster kernel: Out of Memory: Killed process 396 (portmap).
Mar 11 03:37:30 monster kernel: Out of Memory: Killed process 538 (lpd).
Mar 11 03:49:46 monster kernel: Out of Memory: Killed process 493 (atd).
Mar 11 04:02:15 monster kernel: Out of Memory: Killed process 517 (sshd).
Mar 11 04:05:05 monster kernel: Out of Memory: Killed process 724 (bash).
Mar 11 05:02:40 monster kernel: Out of Memory: Killed process 717 (login).
Mar 11 05:54:04 monster kernel: Out of Memory: Killed process 718 (login).
Mar 11 13:34:25 monster kernel: Out of Memory: Killed process 20357 (bash).
Mar 11 16:04:12 monster kernel: Out of Memory: Killed process 5879 (diff).
Mar 11 16:52:41 monster kernel: Out of Memory: Killed process 7948 (tar).
Mar 11 17:37:09 monster kernel: Out of Memory: Killed process 10072 (tar).
Mar 11 17:42:26 monster kernel: Out of Memory: Killed process 10358 (tar).
Mar 11 18:24:30 monster kernel: Out of Memory: Killed process 11300
(run-parts).
Mar 11 19:23:56 monster kernel: Out of Memory: Killed process 11301
(set-time).
Mar 11 20:28:54 monster kernel: Out of Memory: Killed process 18165 (tar).
Mar 11 20:28:55 monster kernel: Out of Memory: Killed process 18167 (gzip).
Mar 11 21:30:51 monster kernel: Out of Memory: Killed process 21205 (tar).
Mar 11 21:33:09 monster kernel: Out of Memory: Killed process 11303 (rdate).
Mar 11 21:50:36 monster kernel: Out of Memory: Killed process 22195 (tar).
Mar 11 22:07:57 monster kernel: Out of Memory: Killed process 23049 (tar).
Mar 11 22:10:01 monster kernel: Out of Memory: Killed process 22987 (diff).
Mar 11 22:12:28 monster kernel: Out of Memory: Killed process 23233 (diff).
Mar 12 00:25:38 monster kernel: Out of Memory: Killed process 29692 (diff).
Mar 12 00:35:34 monster kernel: Out of Memory: Killed process 30229 (tar).
Mar 12 00:57:42 monster kernel: Out of Memory: Killed process 30796 (diff).
Mar 12 01:49:33 monster kernel: Out of Memory: Killed process 1153 (diff).
Mar 12 02:41:31 monster kernel: Out of Memory: Killed process 3488 (tar).
Mar 12 03:06:00 monster kernel: Out of Memory: Killed process 4257 (diff).
Mar 12 04:55:27 monster kernel: Out of Memory: Killed process 8845 (diff).
Mar 12 05:20:07 monster kernel: Out of Memory: Killed process 9712 (sh).
Mar 12 05:50:47 monster kernel: Out of Memory: Killed process 10475 (diff).
Mar 12 05:51:46 monster kernel: Out of Memory: Killed process 10838 (tar).
Mar 12 05:59:07 monster kernel: Out of Memory: Killed process 11162 (tar).
Mar 12 07:45:19 monster kernel: Out of Memory: Killed process 15489 (diff).
Mar 12 08:08:01 monster kernel: Out of Memory: Killed process 16340 (diff).
Mar 12 09:19:18 monster kernel: Out of Memory: Killed process 20182 (diff).
Mar 12 09:29:41 monster kernel: Out of Memory: Killed process 20237 (diff).
Mar 12 11:17:54 monster kernel: Out of Memory: Killed process 25611 (diff).
Mar 12 11:20:05 monster kernel: Out of Memory: Killed process 26133 (diff).
Mar 12 12:34:51 monster kernel: Out of Memory: Killed process 29826 (tar).
Mar 12 13:24:21 monster kernel: Out of Memory: Killed process 32281 (tar).
Mar 12 13:44:20 monster kernel: Out of Memory: Killed process 819 (tar).
Mar 12 13:49:37 monster kernel: Out of Memory: Killed process 1108 (tar).
Mar 12 14:03:46 monster kernel: Out of Memory: Killed process 1304 (diff).
Mar 12 14:26:29 monster kernel: Out of Memory: Killed process 2933 (tar).
Mar 12 14:29:08 monster kernel: Out of Memory: Killed process 3035 (diff).
Mar 12 14:45:53 monster kernel: Out of Memory: Killed process 3828 (diff).
Mar 12 15:06:05 monster kernel: Out of Memory: Killed process 4832 (tar).
Mar 12 16:03:42 monster kernel: Out of Memory: Killed process 7552 (tar).
Mar 12 17:10:35 monster kernel: 

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Sat, 24 Mar 2001, Jonathan Morton wrote:

> Hmm...  "if ( freemem < (size_of_mallocing_process / 20) ) fail_to_allocate;"
> 
> Seems like a reasonable soft limit - processes which have already got
> lots of RAM can probably stand not to have that little bit more and
> can be curbed more quickly.

This looks like it could nicely in preventing a single process
from getting out of hand and gobbling up all memory.

It won't prevent the system from a mongolian horde of processes,
but nobody should expect your one-liner to fix world piece ;)

I like it, now lets test it ;)

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: [PATCH] Prevent OOM from killing init

[Juha Saarinen]

:: Your ideas sound really good, would you have the time to implement
:: them for 2.4 ?

2.4 or 2.5?

-- Juha
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Fri, 23 Mar 2001, Guest section DW wrote:
> On Fri, Mar 23, 2001 at 11:56:23AM -0300, Rik van Riel wrote:
> > On Fri, 23 Mar 2001, Martin Dalecki wrote:
> 
> > > > Feel free to write better-working code.
> > > 
> > > I don't get paid for it and I'm not idling through my days...
> > 
> >   
> 
> No lies please.

You mean that you ARE willing to implement what you've been
arguing for?

Cool, I can't wait to see your patch.

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Fri, 23 Mar 2001, Szabolcs Szakacsits wrote:

> When I ported your OOM killer to 2.2.x and integrated it into the
> 'reserved root memory' [*] patch, during intensive testing I found two
> cases when init was killed. It happened on low-end machines and when
> OOM killer wasn't triggered so init was killed in the page fault
> handler. The later was also one of the reasons I replaced the "random"
> OOM killer in page fault handler with yours [so there is only one OOM
> killer].

Good idea, we should do this for 2.4.  I cannot remember
reading an email from you about this, it's quite possible
I just missed it and didn't answer because I never read
it ...

> Other things that bothered me,
>  - niced processes are penalized

This can be considered a bug and should be fixed...

>  - trying to kill a task that is permanently in TASK_UNINTERRUPTIBLE
>will probably deadlock the machine [or the random OOM killer will
>kill the box].

This could indeed be a problem, though I cannot really see any
case where a task would be in TASK_UNINTERRUPTIBLE permanently.
OTOH, a 1GB read() will take a (much) too long time to finish.

Your ideas sound really good, would you have the time to implement
them for 2.4 ?

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Rik van Riel]

On Fri, 23 Mar 2001, george anzinger wrote:

> What happens if you just make swap VERY large?  Does the system thrash
> it self to a virtual standstill?

It does.  I need to implement load control code (so we suspend
processes in turn to keep the load low enough so we can avoid
thrashing).

> Is this a possible answer?  Supposedly you could then sneak in and
> blow away the bad guys manually ...

This certainly works.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Fwd: Re: [PATCH] Prevent OOM from killing init

[Ed Tomlinson]

,--- Forwarded message (begin)

 Subject: Re: [PATCH] Prevent OOM from killing init
 From: Jonathan Morton <[EMAIL PROTECTED]>
 Date: Fri, 23 Mar 2001 20:45:43 -0500

 >Hmm...  "if ( freemem < (size_of_mallocing_process / 20) ) fail_to_allocate;"

Not sure this is that reasonable on a 4G box... 800M is a big chunk...

Why not base this on the vm's free goal.  If I remember correctly it tries to keep one
second of pages ready for allocating.  If memory is so tight that a second's worth of
memory does not exist.  Note that I mean memory free in main memory and swap.

Think your malloc patch along with UID weighting (1-99 protected, 100-999 endangered, 
1000+ open season - with poaching expected if there is no choise) will make help oom 
processing.

Ed Tomlinson

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Andreas Franck]

Hi together,

seems like a hot discussion going on, but I couldn't resist and would like to
throw in my $0.02.

Besides misunderstandings and general displeasure, some very interesting
facts have shown up in the discussion (oh, yeah), which I'd like to know more
about, and just extend them with a bit of my latest experience regarding
memory usage.

First one is about buffer/inode cache. What I expect as a medium-skilled
system hacker would be: Before giving up with an OOM-whatever,

a) all non-dirty buffers should be freed, possibly giving tons of memory
b) all dirty buffers should be flushed and freed, alas

I'm not sure if both is tried ATM, but I think enough experts are here to
answer my questions :)

What I saw lately was some general system sluggishness after copying very big
files (ripping a CD image to disk) - it seems the system has paged out most
of its processes (including the calling bash shell) in favor of the copying
task, just for buffers! Up to which degree is this reasonable? It seems to
slow down the system when using swap, so for this task I better had
deactivated it. Not what one "intuitively" expects.

So, what is the second important point? The current system cannot properly
distinguish between memory an application "really" needs and memory an
application "eventually" needs (as internal caches, ...).

A possible solution could be the implementation of something like SIGDANGER,
which would be sent to an application in case of memory overload, so
it should try to free a bit memory if it can. Surely applications would have
to be modified to use that information. How about the C library, does it
maintain any big buffers, for I/O or so? I don't know, changes there could
surely be passed on transparently. Ok, ok, it's the MacOS way of thinking, so
the other possibility. This problems are intimately related to memory
overcommitting, or not doing so, so what might be fatal in overcommitting?

One problem arises if an application gets a huge part of overcommitted memory
and then tries to use it, which spontaneously fails - just because the memory
was committed somewhere else, to the 999 other apps which are already
 running.

The flaw there is that at some time, you can guarantee that the overcommit
would fail, if the memory was really used. At this point, the application
could be halted (so that it does not get the chance to make use of the
overcommit promise), until some more memory is available again - either by
paging, or by waiting for other jobs to terminate. This could lead to
starvation, but it potentially could let the system survive.

A further idea would be to use overcommitted memory only for buffers and
caches, this was already mentioned before. In any situation "near" an OOM,
further memory pressure should be avoided - for example, by letting malloc()
fail. This might also hurt existing processes, so some heuristics could
decide - a malloc() from a freshly started process should fail regardlessly
of its size, while older processes might get some more tolerance, because the
system might trust their behaviour a bit more.

So far from me, this was just a collection of some more or less unrelated
thoughts, which I'd like to know a bit more about, or hear from experts why
all of this is b*llshit (or: already done(TM)!)

Greetings,
Andreas
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Paul Jakma]

On Sat, 24 Mar 2001, Szabolcs Szakacsits wrote:

> Nonsense hodgepodge. See and/or mesaure the impact. I sent numbers in my
> former email. You also missed non-overcommit must be _optional_ [i.e.
> you wouldn't be forced to use it ;)]. Yes, there are users and
> enterprises who require it and would happily pay the 50-100% extra swap
> space for the same workload and extra reliability.

ok.. the last time OOM came up, the main objection to fully
guaranteed vm was the possible huge overhead.

if someone knows how to do it without a huge overhead, i'd love to
see it and try it out.

> At every time you add/delete users, add/delete special apps, etc.

no.. pam_limits knows about groups, and you can specify limit for
that group, one time.

@user ... ... ...

> Rik's killer is quite fine at _default_. But there will be always
> people who won't like it

exactly... so lets try avoid ever needing it. it is a last resort.

> default, use the /proc/sys/vm/oom_killer interface"? As I said
> before there are also such patch by Chris Swiedler and definitely
> not a huge, complex one.

uhmm.. where?

> And these stupid threads could be forgotten for good and all.

:)

>   Szaka

regards,
-- 
Paul Jakma  [EMAIL PROTECTED]   [EMAIL PROTECTED]
PGP5 key: http://www.clubi.ie/jakma/publickey.txt
---
Fortune:
The optimum committee has no members.
-- Norman Augustine

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Paul Jakma]

On Sat, 24 Mar 2001 [EMAIL PROTECTED] wrote:

> No, ulimit does not work. (But it helps a little.)

no, not perfect, i very much agree. but in daily usage it reduces
chance of OOM to close to 0.

> No, /proc/sys/vm/overcommit_memory does not work.

that's because it disables the very rough resource checking that
linux has. it makes OOM even easier to achieve:

mm/mmap.c::vm_enough_memory():

/* Sometimes we want to use more memory than we have. */
if (sysctl_overcommit_memory)
return 1;

it doesn't make linux go into a 'non-overcommit' mode, cause linux
does not have the accounting to cover it...

solution according to more knowledgable folks than i, sysadmin, is
better accounting so that vm_enough_memory can be more accurate
rather than developing an all-seeing oom_killer().

> Andries

regards,
-- 
Paul Jakma  [EMAIL PROTECTED]   [EMAIL PROTECTED]
PGP5 key: http://www.clubi.ie/jakma/publickey.txt
---
Fortune:
"We are on the verge: Today our program proved Fermat's next-to-last theorem."
-- Epigrams in Programming, ACM SIGPLAN Sept. 1982

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Jonathan Morton]

>Hmm...  "if ( freemem < (size_of_mallocing_process / 20) ) fail_to_allocate;"
>
>Seems like a reasonable soft limit - processes which have already got lots
>of RAM can probably stand not to have that little bit more and can be
>curbed more quickly.  Processes with less probably don't deserve to die and
>furthermore are less likely to be engineered to handle malloc() failure, so
>failure only occurs closer to the mark.  In this scenario OOM killing
>(which is, after all, a last resort) should trigger rarely and simple
>malloc() failure (which userspace apps can cope with more easily) is an
>early-warning and prevention system.

Following up my own post with some action, I hacked 2.4.1's
mm/mmap.c::vm_enough_pages() to include something similar to the above
algorithm.  In fact, it triggers malloc() failure when 1/16th of
current->mm->total_vm would be greater than the sum of the free space and
the potentially-allocated area.

My very quick tests show that my test program (the rogue allocator) now in
fact does encounter a failed malloc() at approx. 475M, instead of being
killed by the OOM handler at approx. 490M.  This is pretty much the desired
behaviour.

If someone would like me to post a patch and have it tested, I'd be happy
to do so.

--
from: Jonathan "Chromatix" Morton
mail: [EMAIL PROTECTED]  (not for attachments)
big-mail: [EMAIL PROTECTED]
uni-mail: [EMAIL PROTECTED]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-BEGIN GEEK CODE BLOCK-
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS
PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*)
-END GEEK CODE BLOCK-


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Andries . Brouwer]

> It was actually worse than that. Grab your copy of "Lions", and check lines
> 4375-4377 in function xswap(). A failure to allocate space in the swapmap
> caused a panic. Same problem in xalloc().

[no Lions nearby; somewhere I still have the printout but am
too lazy to search; I also have the tape but nothing to read it with]

yes, you may well be right if you say that my picture
of the distant past is too rosy - maybe I forgot all
this trouble
still - yesterday I lost three edit sessions -
I do not recall any such occurrence in the 25 years before

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] Prevent OOM from killing init

[Tim Wright]

On Fri, Mar 23, 2001 at 06:38:37PM +, Alan Cox wrote:
> > infinite storage. After all, earlier Unix flavours did not need
> > an OOM killer either, and my editor was not killed under Unix V6
> > on 64k when I started some other process.
> 
> You were lucky. Its quite possible for V6 to kill processes when you run out
> of swap
> 

It was actually worse than that. Grab your copy of "Lions", and check lines
4375-4377 in function xswap(). A failure to allocate space in the swapmap
caused a panic. Same problem in xalloc().

Tim

-- 
Tim Wright - [EMAIL PROTECTED] or [EMAIL PROTECTED] or [EMAIL PROTECTED]
IBM Linux Technology Center, Beaverton, Oregon
Interested in Linux scalability ? Look at http://lse.sourceforge.net/
"Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/