Re: Why does Linux crash?
On Fri, 2011-04-22 at 15:35 -0500, Boyd Stephen Smith Jr. wrote: > In <1303435546.3090.3.ca...@zircon.lan.walnut.gen.nz>, Richard Hector > wrote: > >On Wed, 2011-04-20 at 12:04 +0200, Axel Freyn wrote: > >> But the principal problem is: each of those limits/protections > reduces > >> the usability (e.g. if you have 2GB Ram, and you limit eclipse to > 2GB, > >> it will be killed by the Kernel as soon as it tries to use 2GB and > 1 > >> byte from the SWAP > > > >Really? > > > >I'd have thought eclipse's request for more memory (malloc) would > just > >fail at that point - which it may or may not handle appropriately, > and > >may handle by exiting. I see no reason for the kernel to kill it. > > You'd think, right? Since malloc() has nice, documented ways it > fails > gracefully we should use them. > > We do, sometimes. However, in Linux with the default settings, that's > not > entirely true. When over-commit is on, there a minimal checks to see > if a the > memory requested can actually be satisfied. Instead, the map is made > lazily, > when the virtual memory pages are accessed. Unfortunately, it's > possible for > the kernel be unable to satisfy a mapping when it is needed. So, > instead of > failing on "mem = malloc(count)", where the userland process can > handle things > cleanly, the we fail on "*mem = things" were everything has to be > handled in > kernel space. Enter the OOM killer. > > These minimal checks may include what is set by ulimit; I'm not sure. Yes, I was aware of overcommit (though not the precise details). I guess I assumed that a nice specific requirement like ulimit would be honoured at malloc() time - after all, it's likely to remain fixed, while some other process could free a bunch of memory in time for us to use it - but assumptions are dangerous, of course :-) Thanks, Richard -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1303526510.2464.3.ca...@zircon.lan.walnut.gen.nz
Re: Why does Linux crash?
Boyd Stephen Smith Jr. wrote at 2011-04-22 15:35 -0500: > I turn off over-commit on all of my systems. I'd rather have malloc fail and > let the userland handle things (possibly by crashing!) than have the semi- > predictable OOM killer set loose. Yes, I turn off overcommit as well. Before I started doing that, the oom-killer scared me several times with its reckless killing of random processes. I hope it has improved some but it still seems like overcommit is a bad default. signature.asc Description: Digital signature
Re: Why does Linux crash?
In <1303435546.3090.3.ca...@zircon.lan.walnut.gen.nz>, Richard Hector wrote: >On Wed, 2011-04-20 at 12:04 +0200, Axel Freyn wrote: >> But the principal problem is: each of those limits/protections reduces >> the usability (e.g. if you have 2GB Ram, and you limit eclipse to 2GB, >> it will be killed by the Kernel as soon as it tries to use 2GB and 1 >> byte from the SWAP > >Really? > >I'd have thought eclipse's request for more memory (malloc) would just >fail at that point - which it may or may not handle appropriately, and >may handle by exiting. I see no reason for the kernel to kill it. You'd think, right? Since malloc() has nice, documented ways it fails gracefully we should use them. We do, sometimes. However, in Linux with the default settings, that's not entirely true. When over-commit is on, there a minimal checks to see if a the memory requested can actually be satisfied. Instead, the map is made lazily, when the virtual memory pages are accessed. Unfortunately, it's possible for the kernel be unable to satisfy a mapping when it is needed. So, instead of failing on "mem = malloc(count)", where the userland process can handle things cleanly, the we fail on "*mem = things" were everything has to be handled in kernel space. Enter the OOM killer. These minimal checks may include what is set by ulimit; I'm not sure. If they don't, it's possible that a process could request (via malloc) more memory than it's ulimit allows, have the kernel not report at error at that time, and then get killed when it attempts to use all the memory it was promised! I turn off over-commit on all of my systems. I'd rather have malloc fail and let the userland handle things (possibly by crashing!) than have the semi- predictable OOM killer set loose. (I've heard decent arguments in favor of over-commit, but I think it fundamentally undermines the safety of the system, so I am still against it being the default.) -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: Re: Why does Linux crash?
Thank you, Boyd, that was just the sort of answer I was looking for. I tried using Ctrl+Alt+F1 to drop into a terminal but, again, it wasn't responsive. I'll commit your suggestions to memory for the next time the system locks up. You touched on the crux of my complaint: sure, I expect the odd processes to get out of control but keyboard/mouse/touchpad/input should still take precedence and allow someone to recover a badly crippled system. I was frustrated that Linux doesn't enforce this by default - assuming that it can. However, although I summarised my story as 'wait or yank power,' I did try a few keyboard interrupts: Ctrl+Alt+Del (possibly disabled), Ctrl+Alt+Backspace (probably disabled), and the Ctrl+Alt+F# keys. Unless the keyboard interrupts take special precedence, it's safe to assume that my system, indeed, locked up. I fix computers part time and I've seen first-hand what yanking the power can do to file system integrity. The thing to remember is that I'm not as smart as most of the people on this list but I'm not completely stupid either. Computer competence falls on a spectrum and not simply Beginner, Intermediate and Advanced. However, it's feedback like this which helps me improve my skills. With thanks, -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201104221453.38558.j...@bordenrhodes.com
Re: Why does Linux crash?
On Wed, 2011-04-20 at 12:04 +0200, Axel Freyn wrote: > But the principal problem is: each of those limits/protections reduces > the usability (e.g. if you have 2GB Ram, and you limit eclipse to 2GB, > it will be killed by the Kernel as soon as it tries to use 2GB and 1 > byte from the SWAP Really? I'd have thought eclipse's request for more memory (malloc) would just fail at that point - which it may or may not handle appropriately, and may handle by exiting. I see no reason for the kernel to kill it. Richard -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1303435546.3090.3.ca...@zircon.lan.walnut.gen.nz
Re: Why does Linux crash?
In <4dae819c.8020...@biotec.tu-dresden.de>, Alex Mestiashvili wrote: >just a simple forkbomb will make your OS unusable ... It shouldn't. When I first got my current hardware up and running, I fork bombed myself and then recovered just to prove the OS and hardware could handle it. I was able to login from another terminal (/sbin/login uses exec() to start your shell so it doesn't need another pid) and use the /proc file system as well as various shell built-ins to terminate all the fork bomb processes. "Yanking the power" is rarely the correct thing to do. Even if normal keyboard and mouse input appear to be ignored, there should be some attempt to use the "Magic" SysRq to take the keyboard back from X, terminate / kill various processes, sync / remount read-only file systems, produce a crash dump, etc. including doing a warm reboot or a soft power off. If you were actively using lots of swap and have locked up X11, it can take quite a while to switch to a vt and kill eclipse. It will normally be possible though. In cases where that isn't possible, Alt+SysRq+K might help get rid of X and processes that are talking to it. If that fails to give you back the system, Alt+SysRq+E will, but it'll also shut down all the system services, so you'll need to restart them. In short, there's a ocean of possibilities between "wait for Linux/Eclipse to resolve things" and "yank the power" that you don't seem to be exploring. -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: Why does Linux crash?
On 04/19/2011 09:57 PM, Borden Rhodes wrote: Good evening, I do mean this earnestly and, despite my frustration, I am not trying to flame the good people of Debian, GNU or Linux. Nevertheless, I have to ask: why is it that in 2011, the world's greatest operating system lets Eclipse seize control of my computer, eat up 2 GB of RAM, monopolise a 2.2 GHz, dual-core processor and flood my hard drive with I/O? I thought that a computer capable of processing over 4 billion operations a second could sort itself out in 20 minutes but, alas, I had to yank the power. I'd run top(1) in a terminal window, with the window set to "always on top" and watch what happens when you do whatever you do which causes Eclipse to consume all resources. -- "Neither the wisest constitution nor the wisest laws will secure the liberty and happiness of a people whose manners are universally corrupt." Samuel Adams, essay in The Public Advertiser, 1749 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4daf2056.2050...@cox.net
Re: Re: Why does Linux crash?
Thank you for your reply, Axel; perfect answer. Now that I know that these features of Linux exist I can go hunt them down and figure out how to use them and stop this from happening again (like it did after I sent my original e-mail). With thanks again, -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/banlktim+xgp4ltk1hdyp2eacetzxs+i...@mail.gmail.com
Re: Why does Linux crash?
Apropos this discussion: The original poster does have a point: resource allocation and process isolation are core o/s functions. I've never really given a lot of thought to the details - for most of the servers I've set up over the years, everything has pretty much just worked (under Solaris for a while, then Redhat, these days Debian). To the extent that I've tuned resource consumption, I've mostly done it across Xen virtual machines. But now that someone's raised the question: Can somebody suggest a good reference for an overview of the resource management philosophy, architecture, tools, and defaults for both the Linux kernel and for Debian? Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4daed5c5.5020...@meetinghouse.net
Re: Re: Why does Linux crash?
on 01:38 Wed 20 Apr, Borden Rhodes (j...@bordenrhodes.com) wrote: > Thank you for your reply and your consolation that other people are > equally miffed with Eclipse. My question, though, is about Linux > control systems. Is one of the kernel's design goals to manage system > resources to prevent a buggy program from crippling the system and > forcing a hard restart? Generally: via a memory segfault (a process requesting memory it doesn't have access to), yes. However, system resources can still be overcommitted (memory is allocated in excess of system resources, look up OOM killer), there are other resources (open file handles, open sockets, disk I/O, paging rates), which can be exhausted or when occuring at too high a rate will make a system unresponsive. There's no surefire way of preventing this (Google "halting problem"), though there are measures which can be taken to reduce these risks. > If so, the control failed and it needs to be reported... right? Not necessarily. If you can produce and provide specific debugging information, file a bug via the Debian bugtracking system. Given what you've posted so far, I'd suggest you not do so, as you don't understand the problem and would be producing noise in the system. -- Dr. Ed Morbius, Chief Scientist /| Robot Wrangler / Staff Psychologist| When you seek unlimited power Krell Power Systems Unlimited| Go to Krell! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110420120355.gb22...@altaira.krellpowersys.exo
Re: Why does Linux crash?
Hi, On Tue, Apr 19, 2011 at 10:57:19PM -0400, Borden Rhodes wrote: > Good evening, > > I do mean this earnestly and, despite my frustration, I am not trying > to flame the good people of Debian, GNU or Linux. Nevertheless, I > have to ask: why is it that in 2011, the world's greatest operating > system lets Eclipse seize control of my computer, eat up 2 GB of RAM, > monopolise a 2.2 GHz, dual-core processor and flood my hard drive with > I/O? I thought that a computer capable of processing over 4 billion > operations a second could sort itself out in 20 minutes but, alas, I > had to yank the power. > > I thought the Linux kernel was supposed to have controls in place to > prevent programs from getting away with this. Of course, the problems > inherent in Java, and by extension, Eclipse, are a whole other topic. > However, is there a kernel task force working to prevent this from > happening and, if so, what's the best way of giving them feedback when > my system locks up so they can plug up the hole? I hate to think what > a malicious program could do to a web server if Eclipse can do this to > my computer. The central point is: what rights/limits do you give to your programs? on Web servers, such limits are normally activated -- while they are deactivated on private systems. Just a few examples: - you can add disk quota for each user/directory, limiting its maximal size and thus preventing the hard disk of overflowing - you can use "ulimit" to pass runtime-limits to eclipse (how much ram it may use, how much calculation time, ) - you can limit the priority of eclipse compare to other jobs (that is: if the processor is occupied, will he give calculation time to eclipse or to something else?) - you can adjust which processes are killed in which order if you run out of memory (by adjusting /proc/PID/oom_adj) - ... But the principal problem is: each of those limits/protections reduces the usability (e.g. if you have 2GB Ram, and you limit eclipse to 2GB, it will be killed by the Kernel as soon as it tries to use 2GB and 1 byte from the SWAP -- which would not cause a problem at all). and only YOU can decide what you want ;-) May be there are people how give a difficult task to eclipse and accept then to wait 2h until it finisheѕ... That's the reason (I believe) that those features are not activated by default on "private" computers. And e.g. for Debian: It is almost impossible to define "sensible" rules which would apply for ALL computers and ALL users where Debian will be installed -- so I think the solution "don't apply restrictions by default" is the best approach... Axel -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110420100409.GQ7351@axel
Re: Why does Linux crash?
On 04/20/2011 07:38 AM, Borden Rhodes wrote: Thank you for your reply and your consolation that other people are equally miffed with Eclipse. My question, though, is about Linux control systems. Is one of the kernel's design goals to manage system resources to prevent a buggy program from crippling the system and forcing a hard restart? If so, the control failed and it needs to be reported... right? OOM-killer , Grsecurity patches , ulimit , virtualization Run eclipse in a VM . just a simple forkbomb will make your OS unusable ... IMHO If you expect that your application can go crazy you should think in advance how to prevent it from eating up resources . Regards , Alex -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4dae819c.8020...@biotec.tu-dresden.de
Re: Why does Linux crash?
On 20 April 2011 14:16, Paul E Condon wrote: Enormous Probably the second largest one I've done since I've been > back! > > But given your experience with Eclipse, I hope you are developing a healthy > scepticism about proposals to have elections voted and votes counted on the > Internet. Imagine what malicious persons could do with that. > Well there'll be no problem there, because of the new RealID programmes being introduced in order to 'protect us' from ourselves. But, beside that, well said! And true, that man, but they pay no heed to the electorate now. The vote means nothing. The only consideration given is to the 'Back-Pocket Electorate' as I term them. Look at what has been happening to the good people of Wisconsin, Minnesota and Ohio, amongst other locations. Look at the dissatisfied personalities marching through London. But still the poorer classes and now also, the middle classes, are required to lose their homes in order to fund massive pay-outs for Bankers and business executives along with the wars required to gain the raw materials for a manufacturing sector that is destroying the planet we all exist on with its effluent. But that will be quite enough of my rabble-rousing. I should, if truth be known, have been classified a 'terrorist' years ago. Regards, Weaver. -- Religion is regarded by the common people as true, by the wise as false, and by the rulers as useful. — Lucius Annæus Seneca. Terrorism, the new religion.
Re: Re: Why does Linux crash?
Thank you for your reply and your consolation that other people are equally miffed with Eclipse. My question, though, is about Linux control systems. Is one of the kernel's design goals to manage system resources to prevent a buggy program from crippling the system and forcing a hard restart? If so, the control failed and it needs to be reported... right? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/banlktim63ap_tzszybum11eqj+hjvu3...@mail.gmail.com
Re: Why does Linux crash?
On 20110419_225719, Borden Rhodes wrote: > Good evening, > > I do mean this earnestly and, despite my frustration, I am not trying > to flame the good people of Debian, GNU or Linux. Nevertheless, I > have to ask: why is it that in 2011, the world's greatest operating > system lets Eclipse seize control of my computer, eat up 2 GB of RAM, > monopolise a 2.2 GHz, dual-core processor and flood my hard drive with > I/O? I thought that a computer capable of processing over 4 billion > operations a second could sort itself out in 20 minutes but, alas, I > had to yank the power. > > I thought the Linux kernel was supposed to have controls in place to > prevent programs from getting away with this. Of course, the problems > inherent in Java, and by extension, Eclipse, are a whole other topic. > However, is there a kernel task force working to prevent this from > happening and, if so, what's the best way of giving them feedback when > my system locks up so they can plug up the hole? I hate to think what > a malicious program could do to a web server if Eclipse can do this to > my computer. > > With thanks, > > Borden Rhodes There is no software that never fails. I don't use Eclipse, but it appears to be an integrated development platform. In other words software for helping programmers write software better and/or faster. It is for developers, not noobs who have no idea why software fails. Why are you trying to use it? But given your experience with Eclipse, I hope you are developing a healthy scepticism about proposals to have elections voted and votes counted on the Internet. Imagine what malicious persons could do with that. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110420041656.ga6...@big.lan.gnu
Why does Linux crash?
Good evening, I do mean this earnestly and, despite my frustration, I am not trying to flame the good people of Debian, GNU or Linux. Nevertheless, I have to ask: why is it that in 2011, the world's greatest operating system lets Eclipse seize control of my computer, eat up 2 GB of RAM, monopolise a 2.2 GHz, dual-core processor and flood my hard drive with I/O? I thought that a computer capable of processing over 4 billion operations a second could sort itself out in 20 minutes but, alas, I had to yank the power. I thought the Linux kernel was supposed to have controls in place to prevent programs from getting away with this. Of course, the problems inherent in Java, and by extension, Eclipse, are a whole other topic. However, is there a kernel task force working to prevent this from happening and, if so, what's the best way of giving them feedback when my system locks up so they can plug up the hole? I hate to think what a malicious program could do to a web server if Eclipse can do this to my computer. With thanks, Borden Rhodes -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=jY3bT=ridkxje+8neclkfnna...@mail.gmail.com
