Re: [Discuss] memory management

[Jerry Feldman]

[gaf@gaf ~]$ ps -ef | grep firefox
gaf   2734  2259  8 06:31 tty2 00:03:27 /usr/lib64/firefox/firefox
gaf   3143  2734  0 06:32 tty2 00:00:01 
/usr/lib64/firefox/plugin-container 
/opt/google/talkplugin/libnpgoogletalk.so -greomni 
/usr/lib64/firefox/omni.ja -appomni /usr/lib64/firefox/browser/omni.ja 
-appdir /usr/lib64/firefox/browser 2734 true plugin




On 06/23/2015 08:48 PM, John Abreau wrote:

I just checked three different machines firefox (Fedora 22, CentOS 6.5, and
MacOS Snow Leopard).

All three have an active firefox browser running, but none show a running
process named "plugin-container", nor any process with a name containing
the string "plug" There are also no processes whose parent PPID is the
firefox process.

All three instances are running a number of plugins.

Shouldn't plugin-container be running if firefox is running?


On Tue, Jun 23, 2015 at 12:17 PM, Steve Litt 
wrote:


On Tue, 23 Jun 2015 11:38:30 -0400
Matthew Gillen  wrote:


On 06/23/2015 10:18 AM, John Abreau wrote:

A bit of googling turned up a page about using cgroups to limit
firefox's memory usage.



http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html


ulimit, and prlimit could do the trick I suppose, but the hard-limits
there would be quite a bit of use-case-specific tuning.
CGroups are much closer to what I want, but not for the rogue
processes: I think making a cgroup for core processes and setting
their swappiness to zero actually gets me closer to what I'm looking
for.

What I really wanted was the rogue process to pay the cost of memory
access, instead of spreading that pain throughout the system.  But
CGroups gets me close I think:

According to this:


https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html

you can create a group, and set swappiness and oom-killer eligibility
for that group.  So ideally I would put certain critical things needed
for recovery (e.g. ssh daemon, agetty, maybe even the window manager;
any process that allows me to find and kill what I need to help the
system recover) in a group that would effectively exempt them from the
thrashing.

HOWEVER, there is still a problem. For instance, my current system
doesn't actually launch an 'agetty' (login) process on the virtual
terminals until you switch to them.  That means you need some
'reserved' memory, which my quick reading of cgroups doesn't seem to
allow you to do.  I'd be happy if there were just a small amount of
memory reserved, enough to:
  - launch agetty and login
  - launch root's login shell
  - run killall eclipse

Wait a minute. Once upon a time I had a daemon for almost this same use
case, but it was for rogue dbus-daemons instead of rogue eclipses or
rogue firefoxes.

It doesn't even need to be a daemon. You could run it every 5 seconds
with cron, and if twice in a row it shows evidence of a rogue eclipse,
it killalls eclipse. Same with firefox, etc. With firefox, I'd
recommend killall plugin-container first, because that's the usual
subject, and you can keep your windows so you know which to bookmark.

To do this, you need the following:

* A command to define runaway eclipse. Probably some sort of parsed ps
   command.

* A file to store the last value of Eclipse's "runaway value", so you
   can tell whether it's two in a row.

* A script to combine the preceding with a killall

* Connecting the preceding script to cron. Every 5 seconds ought to do
   it.

HTH,

SteveT

Steve Litt
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss






--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Steve Litt]

On Fri, 26 Jun 2015 11:02:01 -0500
Derek Martin  wrote:

> On Fri, Jun 26, 2015 at 02:45:13AM -0400, Peter Olson wrote:
> > > On June 25, 2015 at 4:39 PM Derek Martin 
> > > wrote: If you browse daily, and browsing is causing your problem,
> > > then it seems clear to me that you do not. 
> >
> > Blaming the victim is unproductive.
> 
> No, sticking your head in the sand and blaming someone else is
> unproductive.
> 
> Ever hear of contributory negligence?  The victim has some
> responsibility to do what is necessary to prevent from being the
> victim, in a great many cases.  

Please tell me you're kidding and just mocking judgemental people.

The OP (Original Poster) had a problem with his machine slowing to a
crawl when Firefox had been running awhile. He asked for advice. Plain,
simple advice. Several people (myself included) acknowledged that we
had seen the same thing, and some of us gave suggestions.

Anyway, to the OP I suggest this:

   Write a little cron-driven program to kill any plugin-containe or
   Firefox processes that have used more than 90% of CPU or more than
   50% RAM during two consecutive five second scans.

That way, regardless of whose "fault" it is, your machine will remain
useable. By outfitting your little cron driven program to maintain a
log of kills, you can find out a lot about the nature of this problem,
perhaps eventually leading to a solution.

HTH,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Peter Olson]

> On June 26, 2015 at 2:04 PM Derek Martin  wrote:
> 
> On Fri, Jun 26, 2015 at 01:45:51PM -0400, Peter Olson wrote:
> > Unreasonable demands such as leaving Netsol Webmail running overnight and
> > having
> > to kill Firefox in the morning :-)
> > 
> > (This may have been fixed in a recent upgrade of Webmail.)
> 
> Well that certainly seems to suggest that it was not the fault of
> Firefox...

Well, possibly.  My solution at the time was to use Chromium instead of Firefox.
 Chromium never gobbled up all the memory when left reading Web mail.

My intermediate solution is to autoforward all my Netsol email to a different
account I have on Gandi.  I only need Netsol webmail to send/reply from my main
account, as I am doing now.

The ultimate solution is to move my email hosting from Netsol to somewhere else
(probably Gandi. not sure yet).

Peter Olson
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Fri, Jun 26, 2015 at 01:45:51PM -0400, Peter Olson wrote:
> Unreasonable demands such as leaving Netsol Webmail running overnight and 
> having
> to kill Firefox in the morning :-)
> 
> (This may have been fixed in a recent upgrade of Webmail.)

Well that certainly seems to suggest that it was not the fault of
Firefox...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Peter Olson]

> On June 26, 2015 at 10:24 AM Richard Pieri  wrote:
> 
> On 6/26/2015 2:45 AM, Peter Olson wrote:
> > Blaming the victim is unproductive.
> 
> M-x snark-mode
> 
> What? We're not blaming the computer for thrashing and crashing when 
> unreasonable demands are being made of it. We're blaming the user for 
> abusing and overwhelming the poor, mistreated computer.
> 
> M-x snark-mode

Meta-snark-coke-bottle

Unreasonable demands such as leaving Netsol Webmail running overnight and having
to kill Firefox in the morning :-)

/meta

(This may have been fixed in a recent upgrade of Webmail.)

Peter
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Fri, Jun 26, 2015 at 02:45:13AM -0400, Peter Olson wrote:
> > On June 25, 2015 at 4:39 PM Derek Martin  wrote:
> > If you browse daily, and browsing is causing your problem, then it
> > seems clear to me that you do not. 
>
> Blaming the victim is unproductive.

No, sticking your head in the sand and blaming someone else is
unproductive.

Ever hear of contributory negligence?  The victim has some
responsibility to do what is necessary to prevent from being the
victim, in a great many cases.  This is one of them:  There's no
other recourse for you, so you simply must protect yourself.  If you
ask for help and then ignore the good advice you get, then you tend to
get what you deserve.

> What I know is that for years Firefox has consumed all of the CPU time and
> progressively all of the memory of the machine until I kill it.

And I would ask you the same questions that have already been asked
about maintaining your system, opening too many tabs, visiting
malicious/poorly written sites, having enough memory, etc..
 
> Folklore says that that Firefox has had memory leaks for years, maybe at this
> point dozens of years.

Folklore is just that--it's not to be believed (at least not without
verifiable proof).  Javascript is a programming language, which can
cause memory leaks just as well as any other language.  Did it ever
occur to you that the author of the site you're visiting might be at
fault?

  http://javascript.info/tutorial/memory-leaks

Leaks can also occur because of poorly implemented plugins or browser
extensions that you're using.  Firefox is not to blame for any of
that.  There's plenty of other software involved on which to put the
blame for your browser's memory leaks.  

Any or all of those might explain why you suffer from this and I
don't... I don't use a lot of that junk and I avoid sites that load a
ton of garbage and/or use adblock to block the garbage (which FWIW
made a HUGE difference with the performance of certain sites I do
use).  I also don't have less than 8GB of RAM in any of my systems,
and 16 is more the norm for me now.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/26/2015 2:45 AM, Peter Olson wrote:

Blaming the victim is unproductive.


M-x snark-mode

What? We're not blaming the computer for thrashing and crashing when 
unreasonable demands are being made of it. We're blaming the user for 
abusing and overwhelming the poor, mistreated computer.


M-x snark-mode

--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Peter Olson]

> On June 25, 2015 at 4:39 PM Derek Martin  wrote:
> 
> On Mon, Jun 22, 2015 at 03:13:13PM -0400, Matthew Gillen wrote:
> > I'll chime in on this one more time just to be clear about what my beef
> > with linux is here.  Several people have said, in effect, "Have more
> > RAM" or "Have enough RAM for what you need".  Which is obviously true,
> > but missing the point.
>
> ...
>
> If you browse daily, and browsing is causing your problem, then it
> seems clear to me that you do not. ;-)  But I use firefox daily, leave
> it up for WEEKS at a time, and don't have the problems you are
> describing, so I think it must be something about your particular
> usage patterns... Like maybe visiting sites that put too much garbage
> on a page, or some such.  I certainly have seen THAT--it's usually
> sites that have one seemingly endless page of content, with multiple
> (or even tens) of flash ads.  Don't visit those sites. ;-)  Or, maybe
> your system is woefully out of date.

Blaming the victim is unproductive.

What I know is that for years Firefox has consumed all of the CPU time and
progressively all of the memory of the machine until I kill it.

A few years ago, this was a disaster but now Firefox will offer to reopen the
pages I had open prior to the crash.  I do that, and there is no crisis of CPU
or memory.  For a while, perhaps not too soon.

Javascript seems to be the bad boy here: all those pages that are not front are
probably looping Javascript stuff draining the CPU.  That would account for the
CPU being pegged at 100%.  I am ignorant of the API for pages not at front, so I
cannot comment further.

Folklore says that that Firefox has had memory leaks for years, maybe at this
point dozens of years.

Who will fix this?

Do the developers test the case of multiple URLs open, perhaps dozens at a time,
over long periods of time?

Maybe that's why these problems don't get fixed.

Peter Olson
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Mike Small]

On Thu, Jun 25, 2015 at 02:32:24PM -0500, Derek Martin wrote:
> On Sun, Jun 21, 2015 at 03:18:03PM +0200, Bill Bogstad wrote:
> > On Sun, Jun 21, 2015 at 1:10 PM, Jerry Feldman  wrote:
> 
> > I'm curious though, how this other user account gains access to your
> > X server.   Allowing other user ids to write on your screen/capture
> > key & mouse events seem to me to be a potential issue.
> 
> Only if someone else can log in as that user.
> 
> It's been my experience that I didn't need to fix display access, but
> maybe it's because typically I'm switching to root.  But if you need
> to, it's not hard... just arcane.
...
> xauth add myhost/unix:0  MIT-MAGIC-COOKIE-1 
> 
> Should now work fine, without allowing access to anyone else on the
> box.  Just tested it in my Ubuntu VM, closed WORKSFORME. ;-)

I figured Bill was concerned with an exploit owning firefox and
being able to run arbitrary code as that user. Arbitrary code would
include Xlib calls so they're home free. You'd need to give your
unprivileged user untrusted access to the xserver to be safer. See
xauth(1), the generate command and the untrusted argument to it.
That brings the SECURITY extension into play, restricting their
access to the XServer and limiting which X extensions can be used.
Give it a try, but I'm not sure you'll be happy with the resulting
behaviour of firefox or your ability to use the clipboard or
selection. There's also something called XACE, but I couldn't
make heads or tails of it. Sounds like SELinux in terms of 
complexity.

On the memory topic, I tried dillo this morning again. VSZ around 4MB,
but maybe not up to most of what you'd want to throw at it. It may
be loading everything sequentially in a single thread too. Pretty 
slow bringing up pages compared to firefox (when not swapping).

-- 
sma...@sdf.org
SDF Public Access UNIX System - http://sdf.org
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Mon, Jun 22, 2015 at 03:13:13PM -0400, Matthew Gillen wrote:
> I'll chime in on this one more time just to be clear about what my beef
> with linux is here.  Several people have said, in effect, "Have more
> RAM" or "Have enough RAM for what you need".  Which is obviously true,
> but missing the point.

I don't think it is.  You don't seem to like the answer, but it IS the
answer.  4GB really is not that much these days.  The kernel itself
uses ~.5GB - 1GB or so...  A browser with a lot of tabs open can
easily have a VSIZE > 1GB.  That's half your RAM gone right there.

> For my day-to-day, I do have enough RAM.  

If you browse daily, and browsing is causing your problem, then it
seems clear to me that you do not. ;-)  But I use firefox daily, leave
it up for WEEKS at a time, and don't have the problems you are
describing, so I think it must be something about your particular
usage patterns... Like maybe visiting sites that put too much garbage
on a page, or some such.  I certainly have seen THAT--it's usually
sites that have one seemingly endless page of content, with multiple
(or even tens) of flash ads.  Don't visit those sites. ;-)  Or, maybe
your system is woefully out of date.

> What strikes me as odd and wrong is that the OS doesn't seem to protect
> itself from thrashing.  The system is perfectly happy to render itself
> inoperative in the service of some lone process sucking up memory.

What if the process in question is running the dialysis equipment that
is currently filtering your blood? 

The VM design assumes that the manager of the system has some clue...
If you need more memory you'll add it, you'll keep your software
updated, or if need be you'll use ulimit to limit misbehaving
processes (or, you know, fix them if you have the code).

Its goal is not only to allocate resources, but keep the system
stable.  If it starts killing processes randomly that's kind of the
opposite of stable.  So it tries really hard to not kill processes.
Maybe a pending I/O will complete and free up the necessary pages, or
maybe the user will kill the process...  It plays a "wait and see"
game that's hard for it to win because there are too many variables.
But it tries anyway because the consequences are potentially very
severe if it has to kill a process.

In contrast, when you run out of file descriptors, you've run out of
file descriptors.  There's no back-up plan, nothing else to be done
but fail.  But the consequences are far less severe--the application
can do a number of things in response:  Depending on the specifics, it
might ignore the problem, or retry later, or tell the user about the
problem and let them decide what to do.

You can't do much of anything if you can't get any memory to do it
in... including notify the user in many cases, as the code for doing
that is probably going to allocate space for a string, or some such.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Sun, Jun 21, 2015 at 05:57:06PM +0200, Bill Bogstad wrote:
> > xhost +SI:localuser:myffuser
> > sudo -u ffuser /usr/bin/firefox
> > xhost -SI:localuser:myffuser
> >
> > It's not an issue on a single user box; it's the same user (human) with a
> > different UID.
> >
> 
> This is where I disagree.   If it doesn't increase security over using the
> same UID, why bother.  

It does though... it enables you to access the user's display without
allowiong you to access their files (at least directly; it's possible
there's some exploit but I'm not aware of one).

> Second, if that user id has the privileges to pop up windows on the same X
> server as my "real" user id; I might get spoofed, have my screen or even
> possibly my keystrokes captured.   

The method I just posted will prevent that too.  But it's
extraordinarily unlikely that anything you're doing with your browser
is going to result in such an attack.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Sun, Jun 21, 2015 at 03:18:03PM +0200, Bill Bogstad wrote:
> On Sun, Jun 21, 2015 at 1:10 PM, Jerry Feldman  wrote:

> I'm curious though, how this other user account gains access to your
> X server.   Allowing other user ids to write on your screen/capture
> key & mouse events seem to me to be a potential issue.

Only if someone else can log in as that user.

It's been my experience that I didn't need to fix display access, but
maybe it's because typically I'm switching to root.  But if you need
to, it's not hard... just arcane.

$ xauth list
myhost/unix:0  MIT-MAGIC-COOKIE-1  

Then:

1. su - other

2. make sure XAUTHORITY is not left over from your original UID:

$ export XAUTHORITY=/home/other/.Xauthority # or whatever

3. make sure that file exists

$ touch /home/other/.Xauthority

4. add the cookie you got from your original UID

$ xauth add myhost/unix:0  MIT-MAGIC-COOKIE-1 

Should now work fine, without allowing access to anyone else on the
box.  Just tested it in my Ubuntu VM, closed WORKSFORME. ;-)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

I use Firefox on several Linux systems and one MacOS Snow Leopard system.
I've never seen this problem on the Linux systems, but both Firefox and
Google Chrome would routinely use up the memory on the MacOS system to the
point where the entire system became unresponsive, and all too often I had
to forcibly power-cycle the machine to recover.

Since setting up the launch script I posted a few days ago that uses ulimit
before invoking Firefox,the problem has been far less severe. It now takes
a lot longer before memory starts to run out, and when it reaches that
point, Firefox crashes instead of rendering the whole system unresponsive.

In terms of usability, I find these Firefox crashes a *HUGE* improvement.


On Thu, Jun 25, 2015 at 3:04 PM, Derek Martin 
wrote:

> On Fri, Jun 19, 2015 at 10:46:33AM -0400, Dan Ritter wrote:
> > > The memory leak wrt Firefox has been an issue for those users (not
> > > community...users) for years.
> > > There are so many reasons to use a distro over Ubuntu.
> > >
> > > Note I did not refer to Ubuntu as a "distro"...
> > > At any rate, there is a fix for the Ubuntu Memory issue. The best
> > > solution though- Don't use Ubuntu.
> > >
> >
> > Could you provide references for any of this, please?
>
> Seconded.  I use Ubuntu almost exclusively (for everything except
> gaming and casual web browsing in between gaming), have for a few
> years now, and I have no idea what you are talking about.
>
> --
> Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
> -=-=-=-=-
> This message is posted from an invalid address.  Replying to it will
> result in
> undeliverable mail due to spam prevention.  Sorry for the inconvenience.
>
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email: abre...@gmail.com / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Derek Martin]

On Fri, Jun 19, 2015 at 10:46:33AM -0400, Dan Ritter wrote:
> > The memory leak wrt Firefox has been an issue for those users (not
> > community...users) for years.
> > There are so many reasons to use a distro over Ubuntu.
> > 
> > Note I did not refer to Ubuntu as a "distro"...
> > At any rate, there is a fix for the Ubuntu Memory issue. The best
> > solution though- Don't use Ubuntu.
> > 
> 
> Could you provide references for any of this, please?

Seconded.  I use Ubuntu almost exclusively (for everything except
gaming and casual web browsing in between gaming), have for a few
years now, and I have no idea what you are talking about.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Tom Metro]

Matthew Gillen wrote:
> I think plugin-container doesn't get launched until it needs something
> like the flash plugin.

Right. That's been my observation. It doesn't get used for extensions.
Only plugins, and I think only ones using a legacy API at that. And is
loaded only when needed.

 -Tom

-- 
Tom Metro
The Perl Shop, Newton, MA, USA
"Predictable On-demand Perl Consulting."
http://www.theperlshop.com/
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Ricker]

> I think plugin-container doesn't get launched until it needs something
> like the flash plugin.
>

​which, if you're running NoScript, is under your control.
(If not, why not ? :-)​


-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

I think plugin-container doesn't get launched until it needs something
like the flash plugin.

On 6/23/2015 10:03 PM, John Abreau wrote:
> "ps x | grep plug" yields nothing but the "grep" process.
> 
> "ps x -o pid,ppid,cmd | grep 12345" (where 12345 is firefox's PID) yields
> nothing but the "grep" process.
> 
> I'm running Firefox 38.0.5, which is the latest release. I use a script I
> wrote long ago to fetch the latest tarball from ftp.mozilla.org and build
> an rpm from the tarball. Perhaps there's been a change in recent versions
> of firefox, or a difference between the vanilla tarballs and the rpm
> packages that the distro builds?
> 
> 
> On Tue, Jun 23, 2015 at 9:54 PM, Bill Ricker  wrote:
> 
>> On Tue, Jun 23, 2015 at 8:48 PM, John Abreau  wrote:
>>
>>> Shouldn't plugin-container be running if firefox is running?
>>
>>
>> ​'top' truncates mine to 'plugin-containe'​, but that does contain 'plug'.
>>
>>
>>
>> --
>> Bill Ricker
>> bill.n1...@gmail.com
>> https://www.linkedin.com/in/n1vux
>> ___
>> Discuss mailing list
>> Discuss@blu.org
>> http://lists.blu.org/mailman/listinfo/discuss
>>
> 
> 
> 

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

"ps x | grep plug" yields nothing but the "grep" process.

"ps x -o pid,ppid,cmd | grep 12345" (where 12345 is firefox's PID) yields
nothing but the "grep" process.

I'm running Firefox 38.0.5, which is the latest release. I use a script I
wrote long ago to fetch the latest tarball from ftp.mozilla.org and build
an rpm from the tarball. Perhaps there's been a change in recent versions
of firefox, or a difference between the vanilla tarballs and the rpm
packages that the distro builds?


On Tue, Jun 23, 2015 at 9:54 PM, Bill Ricker  wrote:

> On Tue, Jun 23, 2015 at 8:48 PM, John Abreau  wrote:
>
> > Shouldn't plugin-container be running if firefox is running?
>
>
> ​'top' truncates mine to 'plugin-containe'​, but that does contain 'plug'.
>
>
>
> --
> Bill Ricker
> bill.n1...@gmail.com
> https://www.linkedin.com/in/n1vux
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email j...@blu.org / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Ricker]

On Tue, Jun 23, 2015 at 8:48 PM, John Abreau  wrote:

> Shouldn't plugin-container be running if firefox is running?


​'top' truncates mine to 'plugin-containe'​, but that does contain 'plug'.



-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

I just checked three different machines firefox (Fedora 22, CentOS 6.5, and
MacOS Snow Leopard).

All three have an active firefox browser running, but none show a running
process named "plugin-container", nor any process with a name containing
the string "plug" There are also no processes whose parent PPID is the
firefox process.

All three instances are running a number of plugins.

Shouldn't plugin-container be running if firefox is running?


On Tue, Jun 23, 2015 at 12:17 PM, Steve Litt 
wrote:

> On Tue, 23 Jun 2015 11:38:30 -0400
> Matthew Gillen  wrote:
>
> > On 06/23/2015 10:18 AM, John Abreau wrote:
> > > A bit of googling turned up a page about using cgroups to limit
> > > firefox's memory usage.
> > >
> > >
> http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html
> >
> >
> > ulimit, and prlimit could do the trick I suppose, but the hard-limits
> > there would be quite a bit of use-case-specific tuning.
> > CGroups are much closer to what I want, but not for the rogue
> > processes: I think making a cgroup for core processes and setting
> > their swappiness to zero actually gets me closer to what I'm looking
> > for.
> >
> > What I really wanted was the rogue process to pay the cost of memory
> > access, instead of spreading that pain throughout the system.  But
> > CGroups gets me close I think:
> >
> > According to this:
> >
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html
> >
> > you can create a group, and set swappiness and oom-killer eligibility
> > for that group.  So ideally I would put certain critical things needed
> > for recovery (e.g. ssh daemon, agetty, maybe even the window manager;
> > any process that allows me to find and kill what I need to help the
> > system recover) in a group that would effectively exempt them from the
> > thrashing.
> >
> > HOWEVER, there is still a problem. For instance, my current system
> > doesn't actually launch an 'agetty' (login) process on the virtual
> > terminals until you switch to them.  That means you need some
> > 'reserved' memory, which my quick reading of cgroups doesn't seem to
> > allow you to do.  I'd be happy if there were just a small amount of
> > memory reserved, enough to:
> >  - launch agetty and login
> >  - launch root's login shell
> >  - run killall eclipse
>
> Wait a minute. Once upon a time I had a daemon for almost this same use
> case, but it was for rogue dbus-daemons instead of rogue eclipses or
> rogue firefoxes.
>
> It doesn't even need to be a daemon. You could run it every 5 seconds
> with cron, and if twice in a row it shows evidence of a rogue eclipse,
> it killalls eclipse. Same with firefox, etc. With firefox, I'd
> recommend killall plugin-container first, because that's the usual
> subject, and you can keep your windows so you know which to bookmark.
>
> To do this, you need the following:
>
> * A command to define runaway eclipse. Probably some sort of parsed ps
>   command.
>
> * A file to store the last value of Eclipse's "runaway value", so you
>   can tell whether it's two in a row.
>
> * A script to combine the preceding with a killall
>
> * Connecting the preceding script to cron. Every 5 seconds ought to do
>   it.
>
> HTH,
>
> SteveT
>
> Steve Litt
> June 2015 featured book: The Key to Everyday Excellence
> http://www.troubleshooters.com/key
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email j...@blu.org / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Steve Litt]

On Tue, 23 Jun 2015 11:38:30 -0400
Matthew Gillen  wrote:

> On 06/23/2015 10:18 AM, John Abreau wrote:
> > A bit of googling turned up a page about using cgroups to limit
> > firefox's memory usage.
> > 
> > http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html
> 
> 
> ulimit, and prlimit could do the trick I suppose, but the hard-limits
> there would be quite a bit of use-case-specific tuning.
> CGroups are much closer to what I want, but not for the rogue
> processes: I think making a cgroup for core processes and setting
> their swappiness to zero actually gets me closer to what I'm looking
> for.
> 
> What I really wanted was the rogue process to pay the cost of memory
> access, instead of spreading that pain throughout the system.  But
> CGroups gets me close I think:
> 
> According to this:
>  
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html
> 
> you can create a group, and set swappiness and oom-killer eligibility
> for that group.  So ideally I would put certain critical things needed
> for recovery (e.g. ssh daemon, agetty, maybe even the window manager;
> any process that allows me to find and kill what I need to help the
> system recover) in a group that would effectively exempt them from the
> thrashing.
> 
> HOWEVER, there is still a problem. For instance, my current system
> doesn't actually launch an 'agetty' (login) process on the virtual
> terminals until you switch to them.  That means you need some
> 'reserved' memory, which my quick reading of cgroups doesn't seem to
> allow you to do.  I'd be happy if there were just a small amount of
> memory reserved, enough to:
>  - launch agetty and login
>  - launch root's login shell
>  - run killall eclipse

Wait a minute. Once upon a time I had a daemon for almost this same use
case, but it was for rogue dbus-daemons instead of rogue eclipses or
rogue firefoxes.

It doesn't even need to be a daemon. You could run it every 5 seconds
with cron, and if twice in a row it shows evidence of a rogue eclipse,
it killalls eclipse. Same with firefox, etc. With firefox, I'd
recommend killall plugin-container first, because that's the usual
subject, and you can keep your windows so you know which to bookmark.

To do this, you need the following:

* A command to define runaway eclipse. Probably some sort of parsed ps
  command.

* A file to store the last value of Eclipse's "runaway value", so you
  can tell whether it's two in a row.

* A script to combine the preceding with a killall

* Connecting the preceding script to cron. Every 5 seconds ought to do
  it.

HTH,

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

On 06/23/2015 10:18 AM, John Abreau wrote:
> A bit of googling turned up a page about using cgroups to limit
> firefox's memory usage.
> 
> http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html


ulimit, and prlimit could do the trick I suppose, but the hard-limits
there would be quite a bit of use-case-specific tuning.
CGroups are much closer to what I want, but not for the rogue processes:
 I think making a cgroup for core processes and setting their swappiness
to zero actually gets me closer to what I'm looking for.

What I really wanted was the rogue process to pay the cost of memory
access, instead of spreading that pain throughout the system.  But
CGroups gets me close I think:

According to this:
 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html

you can create a group, and set swappiness and oom-killer eligibility
for that group.  So ideally I would put certain critical things needed
for recovery (e.g. ssh daemon, agetty, maybe even the window manager;
any process that allows me to find and kill what I need to help the
system recover) in a group that would effectively exempt them from the
thrashing.

HOWEVER, there is still a problem. For instance, my current system
doesn't actually launch an 'agetty' (login) process on the virtual
terminals until you switch to them.  That means you need some 'reserved'
memory, which my quick reading of cgroups doesn't seem to allow you to
do.  I'd be happy if there were just a small amount of memory reserved,
enough to:
 - launch agetty and login
 - launch root's login shell
 - run killall eclipse

That you can't easily do this is odd, because other system resources
have reserves (e.g. filesystems tend to have a small percentage reserved
for root; the system-wide limit for process handles has some set-aside
for root).  For some reason having some mlock()'d memory reserved is
harder...

Thanks,
Matt

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

Also tried the following bash script on MacOS Snow Leopard desktop, and the
system is running much better now, after I killed the running instance of
firefox and then ran the script from an xterm to relaunch firefox.


#! /bin/bash
ulimit -d 350 ; ulimit -v 350 ; ulimit -m 300
exec /Applications/Firefox.app/Contents/MacOS/firefox "$@"



On Tue, Jun 23, 2015 at 10:18 AM, John Abreau  wrote:

> A bit of googling turned up a page about using cgroups to limit firefox's
> memory usage.
>
>
> http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html
>
>
> On Fri, Jun 19, 2015 at 10:01 AM, Matthew Gillen 
> wrote:
>
> > I'm looking for some advice on tuning my linux box's memory management.
> >  I've got an older workstation that has merely 4GB of memory.  If I try
> > to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
> > about and effectively locks up because of out-of-memory issues.
> >
> > For example: the mouse will continue to move, but won't change it's icon
> > contextually.  If I hit cntl-alt-f2 and try to log in to a virtual
> > console, mgetty will eventually ask for the username, but after I hit
> > enter, it just hangs, not popping up the password prompt, and after 60
> > seconds the login times out.  Trying to ssh into the machine from
> > somewhere else ends up timing out.
> >
> > After going on like this for literally 10 minutes, OOM-killer sometimes
> > kills the right thing (one of the two processes hogging the most memory:
> > firefox or eclipse), and the machine becomes usable again sometime later.
> >
> > I have heftier workstations I can use, but this behavior is really
> > frustrating to me, because I'd like to think linux does good memory
> > management.  I've tried using huge swap (2x physical memory).  I've
> > tried with virtually no swap (on the theory that without swap, there
> > would be no thrashing and at least oom-killer would have to do its thing
> > without locking up the machine for 10 minutes first).  The problem there
> > was oom-killer making bad decisions about what to kill (e.g., the window
> > manager, and then whatever out-of-control process is sucking up memory
> > just sucks up whatever got freed, and nothing gets better).  At least
> > with some swap oom-killer seems to make better guesses about who to
> murder.
> >
> > Does anyone have any tips on how to prevent linux from thrashing like
> > that?  The behavior when low on memory seems atrociously bad.
> >
> > Thanks,
> > Matt
> > ___
> > Discuss mailing list
> > Discuss@blu.org
> > http://lists.blu.org/mailman/listinfo/discuss
> >
>
>
>
> --
> John Abreau / Executive Director, Boston Linux & Unix
> Email: abre...@gmail.com / WWW http://www.abreau.net / PGP-Key-ID
> 0x920063C6
> PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email j...@blu.org / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

A bit of googling turned up a page about using cgroups to limit firefox's
memory usage.

http://jlebar.com/2011/6/15/Limiting_the_amount_of_RAM_a_program_can_use.html


On Fri, Jun 19, 2015 at 10:01 AM, Matthew Gillen  wrote:

> I'm looking for some advice on tuning my linux box's memory management.
>  I've got an older workstation that has merely 4GB of memory.  If I try
> to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
> about and effectively locks up because of out-of-memory issues.
>
> For example: the mouse will continue to move, but won't change it's icon
> contextually.  If I hit cntl-alt-f2 and try to log in to a virtual
> console, mgetty will eventually ask for the username, but after I hit
> enter, it just hangs, not popping up the password prompt, and after 60
> seconds the login times out.  Trying to ssh into the machine from
> somewhere else ends up timing out.
>
> After going on like this for literally 10 minutes, OOM-killer sometimes
> kills the right thing (one of the two processes hogging the most memory:
> firefox or eclipse), and the machine becomes usable again sometime later.
>
> I have heftier workstations I can use, but this behavior is really
> frustrating to me, because I'd like to think linux does good memory
> management.  I've tried using huge swap (2x physical memory).  I've
> tried with virtually no swap (on the theory that without swap, there
> would be no thrashing and at least oom-killer would have to do its thing
> without locking up the machine for 10 minutes first).  The problem there
> was oom-killer making bad decisions about what to kill (e.g., the window
> manager, and then whatever out-of-control process is sucking up memory
> just sucks up whatever got freed, and nothing gets better).  At least
> with some swap oom-killer seems to make better guesses about who to murder.
>
> Does anyone have any tips on how to prevent linux from thrashing like
> that?  The behavior when low on memory seems atrociously bad.
>
> Thanks,
> Matt
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email: abre...@gmail.com / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/23/2015 2:04 AM, John Abreau wrote:

So the OP is asking for help figuring out how to manage these processes,
and your response is to tell him that it's up to him to manage them? Not a
particularly useful response.


No, he was complaining about how the kernel doesn't do it for him.

As a matter of fact, I *did* point him in the right direction for doing 
it himself.


Hint: read the hint that left in the last message.

--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Robert Krawitz]

On Mon, 22 Jun 2015 15:13:13 -0400, Matthew Gillen wrote:
> I'll chime in on this one more time just to be clear about what my beef
> with linux is here.  Several people have said, in effect, "Have more
> RAM" or "Have enough RAM for what you need".  Which is obviously true,
> but missing the point.
>
> For my day-to-day, I do have enough RAM.  What sometimes happens is that
> the cesspool of problems that is javascript engines, or eclipse, go
> completely off the rails, and start gobbling up memory.  Seeing the
> forest for the trees, what I'm getting at here is that this will always
> be an issue, and vastly over-provisioning RAM might mask the problem for
> a while, eventually your day-to-day is going to start including multiple
> VMs and you're back to square one.
>
> Operating systems have concerned themselves for a long time with not
> letting unprivileged processes destroy a system.  If a process tries to
> touch memory that doesn't belong to it, BAM! The OS says "bad process!"
> and hits it with a seg-fault.  The signal can kill the offending
> process, or the process can catch it and try to recover, but either way,
> the integrity of the rest of the system is preserved at the expense of
> not letting the naughty process do what it was trying to do.
>
> What strikes me as odd and wrong is that the OS doesn't seem to protect
> itself from thrashing.  The system is perfectly happy to render itself
> inoperative in the service of some lone process sucking up memory.

If you're concerned about that, ulimit is your friend.  Look at the
bash man page, not the man page for ulimit itself.

-- 
Robert Krawitz 

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[John Abreau]

So the OP is asking for help figuring out how to manage these processes,
and your response is to tell him that it's up to him to manage them? Not a
particularly useful response.

On Mon, Jun 22, 2015 at 7:34 PM, Richard Pieri 
wrote:

> On 6/22/2015 6:40 PM, Matthew Gillen wrote:
>
>> That doesn't (necessarily) address the issue I was raising about an out
>> of control app wreaking havoc on the rest of the system.  Just a
>> different flavor of havoc.
>>
>
> Yes, it does. The kernel isn't there to baby sit. It's there to allocate
> resources when processes request them. It is up to you, the user or the
> administrator, to manage those processes. It isn't the kernel's
> responsibility to do this for you.
>
> Hint: man prlimit
>
> --
> Rich P.
>
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>



-- 
John Abreau / Executive Director, Boston Linux & Unix
Email: abre...@gmail.com / WWW http://www.abreau.net / PGP-Key-ID 0x920063C6
PGP-Key-Fingerprint A5AD 6BE1 FEFE 8E4F 5C23  C2D0 E885 E17C 9200 63C6
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/22/2015 6:40 PM, Matthew Gillen wrote:

That doesn't (necessarily) address the issue I was raising about an out
of control app wreaking havoc on the rest of the system.  Just a
different flavor of havoc.


Yes, it does. The kernel isn't there to baby sit. It's there to allocate 
resources when processes request them. It is up to you, the user or the 
administrator, to manage those processes. It isn't the kernel's 
responsibility to do this for you.


Hint: man prlimit

--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

On 6/22/2015 3:38 PM, Richard Pieri wrote:
> On 6/22/2015 3:13 PM, Matthew Gillen wrote:
>> What strikes me as odd and wrong is that the OS doesn't seem to protect
>> itself from thrashing.  The system is perfectly happy to render itself
>> inoperative in the service of some lone process sucking up memory.
> 
> Don't like swap thrashing? Shut off swap. There can be no swap trashing
> if there is no swap.
> 
> Just don't whine when the OOM killer reaps the applications that you're
> using.

That doesn't (necessarily) address the issue I was raising about an out
of control app wreaking havoc on the rest of the system.  Just a
different flavor of havoc.

Matt


___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/22/2015 3:13 PM, Matthew Gillen wrote:

What strikes me as odd and wrong is that the OS doesn't seem to protect
itself from thrashing.  The system is perfectly happy to render itself
inoperative in the service of some lone process sucking up memory.


Don't like swap thrashing? Shut off swap. There can be no swap trashing 
if there is no swap.


Just don't whine when the OOM killer reaps the applications that you're 
using.


--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

I'll chime in on this one more time just to be clear about what my beef
with linux is here.  Several people have said, in effect, "Have more
RAM" or "Have enough RAM for what you need".  Which is obviously true,
but missing the point.

For my day-to-day, I do have enough RAM.  What sometimes happens is that
the cesspool of problems that is javascript engines, or eclipse, go
completely off the rails, and start gobbling up memory.  Seeing the
forest for the trees, what I'm getting at here is that this will always
be an issue, and vastly over-provisioning RAM might mask the problem for
a while, eventually your day-to-day is going to start including multiple
VMs and you're back to square one.

Operating systems have concerned themselves for a long time with not
letting unprivileged processes destroy a system.  If a process tries to
touch memory that doesn't belong to it, BAM! The OS says "bad process!"
and hits it with a seg-fault.  The signal can kill the offending
process, or the process can catch it and try to recover, but either way,
the integrity of the rest of the system is preserved at the expense of
not letting the naughty process do what it was trying to do.

What strikes me as odd and wrong is that the OS doesn't seem to protect
itself from thrashing.  The system is perfectly happy to render itself
inoperative in the service of some lone process sucking up memory.

I guess we should count ourselves lucky that more of the processes we
run every day don't go off the rails in terms of memory usage.  And it
might be that this behavior isn't easy to create in a test environment:
it's possible that what causes the problem is sucking up a lot of memory
and then trying to /use/ all of it constantly.  Hard to say what the JS
engine is doing...

I'll note that I've pushed linux systems to resource limits in other
ways on plenty of occasions.  For example: running out of file
descriptors.  While there are certainly ways that you can abuse the
system and make the /system/ run out of file descriptors, the
protections that were in place seemed reasonable: my user was out of
file-descriptors and therefore couldn't start any new processes, but
'root' could still do things.

Matt
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

On 06/21/2015 05:58 PM, Dan Ritter wrote:

On Sun, Jun 21, 2015 at 03:39:46PM -0400, Richard Pieri wrote:

When a process tries to allocate more pages than the total of clean
and unallocated pages in the page cache, well, that's an overload.
Overloaded system is overloaded. Game over. Install more RAM or
don't overload the system. If neither option is viable then put your
page file or partition on a fast SSD. That won't solve the problem
but it will make it seem less severe.

Remember that a lot of our expectations around VM and swapping
come from a time when a multiuser system might have 16 MB of
RAM. Hard disks transferred 25MB/s.

Suppose you had a 16MB system and allocated 16MB of disk for
swap. A request to evict a 4MB process and pull in another one
would take about 350ms -- not too bad.

Now let's say you have a 16GB system with 16GB of swap on disk,
and you want to evict a 4GB browser process and pull in another
one. Disk speeds went up to 100MB/s, so it only takes 80 seconds
or so! With a consumer SSD that gets lucky, you have a 500MB/s
transfer rate and can do it in 16 seconds, practically no time at
all. It's certainly less severe, but it isn't much fun.

Conclusion: swap is a last-ditch mechanism to save your system from
having to kill processes. The other two uses -- Linux trades out pages
that are likely to never be called again, or uses the swap space as
a sleep-time mechanism to store memory -- are still viable. It would
probably be best for a desktop machine that *is* going to be put to
sleep overnight to have a minimal amount of swap, and mount a larger,
pre-prepared swapspace for the purpose of sleeping. Unmount it
when awake.

Linux does not swap out pages that are not dirty. For instance code and 
initialized data that has not been written to is never swapped out to 
backing store. In the old days before virtual memory processes were 
swapped in and swapped out. But today, VM is much more efficient. 
Certainly if the system needs to free up memory, less recently used 
pages may be swapped out to make room.


--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Robert Krawitz]

On Sun, 21 Jun 2015 17:58:12 -0400, Dan Ritter wrote:
> On Sun, Jun 21, 2015 at 03:39:46PM -0400, Richard Pieri wrote:
>> When a process tries to allocate more pages than the total of clean
>> and unallocated pages in the page cache, well, that's an overload.
>> Overloaded system is overloaded. Game over. Install more RAM or
>> don't overload the system. If neither option is viable then put your
>> page file or partition on a fast SSD. That won't solve the problem
>> but it will make it seem less severe.
>
> Remember that a lot of our expectations around VM and swapping come
> from a time when a multiuser system might have 16 MB of RAM. Hard
> disks transferred 25MB/s.

That would have been early-mid 1990's -- disk throughput at that time
for IDE drives was maybe 2 MB/sec.  SCSI drives might have been a bit
faster, and with command queuing, could optimize head seeks.  But
throughput wasn't the problem; latency (and I/O rate) was the real
issue.

> Suppose you had a 16MB system and allocated 16MB of disk for
> swap. A request to evict a 4MB process and pull in another one
> would take about 350ms -- not too bad. 

And as you note below, swapping is only an emergency mechanism.
Swapping is a lot more efficient than paging, since it has some hope
of using close to the full bandwidth of the disk, but at 2 MB/sec, the
overhead is significant.  The problem with swapping is that it greatly
reduces responsiveness; the process is completely evicted from memory
and has to be brought back in, and something else removed by paging or
swapping, so you're going to have a lot of latency (possibly many
seconds).

> Now let's say you have a 16GB system with 16GB of swap on disk,
> and you want to evict a 4GB browser process and pull in another
> one. Disk speeds went up to 100MB/s, so it only takes 80 seconds
> or so! With a consumer SSD that gets lucky, you have a 500MB/s
> transfer rate and can do it in 16 seconds, practically no time at
> all. It's certainly less severe, but it isn't much fun.

I've certainly seen my laptop thrash (at least before I added an SSD
and put the swap partition on it), but I think it's paging rather than
swapping that usually did me in.  With swapping, processes will stall
entirely for extended periods of time, but with paging, they'll keep
running, pulling in pages as needed.  From an SSD, even a cheap
consumer-grade mSATA, that isn't too painful; 50K IO/sec of even 4K
pages isn't too bad unless access patterns are really awful (the
process will take under 100 usec to pull a page in).  If your process
can keep running without needing another page for 1 msec, the impact
won't be bad at all.  With a spinning drive that might top out around
200 IO/sec, it's another story; it will take more like 5 msec.  Then
you're really taking a hit.

System startup (particularly window system startup after boot) behaves
very much like a system in thrash; a lot of processes are kicking off,
competing against each other for the bandwidth and IOPS needed to pull
in enough program text to get to the point where they're executing out
of RAM rather than constantly pulling stuff off disk a page or few at
a time.

(I know about the recommendation not to put swap on an SSD because
you'll shorten its lifetime.  Personally, I think that's bollocks.
Consumer-grade SSD's have a lifetime around 500 TB of writes; unless
you're chronically severely memory-starved it will take ages for you
to get there.  SSD's are great for small random accesses; demand
paging of anonymous pages is done with small random accesses.  Pretty
good match.)

> Conclusion: swap is a last-ditch mechanism to save your system from
> having to kill processes. The other two uses -- Linux trades out pages
> that are likely to never be called again, or uses the swap space as
> a sleep-time mechanism to store memory -- are still viable. It would
> probably be best for a desktop machine that *is* going to be put to
> sleep overnight to have a minimal amount of swap, and mount a larger,
> pre-prepared swapspace for the purpose of sleeping. Unmount it
> when awake.

What it really boils down to is that you need enough memory for your
worst case working set, or you're going to get unhappy awfully fast.
-- 
Robert Krawitz 

***  MIT Engineers   A Proud Tradition   http://mitathletics.com  ***
Member of the League for Programming Freedom  --  http://ProgFree.org
Project lead for Gutenprint   --http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Dan Ritter]

On Sun, Jun 21, 2015 at 03:39:46PM -0400, Richard Pieri wrote:
> When a process tries to allocate more pages than the total of clean
> and unallocated pages in the page cache, well, that's an overload.
> Overloaded system is overloaded. Game over. Install more RAM or
> don't overload the system. If neither option is viable then put your
> page file or partition on a fast SSD. That won't solve the problem
> but it will make it seem less severe.

Remember that a lot of our expectations around VM and swapping 
come from a time when a multiuser system might have 16 MB of
RAM. Hard disks transferred 25MB/s.

Suppose you had a 16MB system and allocated 16MB of disk for
swap. A request to evict a 4MB process and pull in another one
would take about 350ms -- not too bad. 

Now let's say you have a 16GB system with 16GB of swap on disk,
and you want to evict a 4GB browser process and pull in another
one. Disk speeds went up to 100MB/s, so it only takes 80 seconds
or so! With a consumer SSD that gets lucky, you have a 500MB/s
transfer rate and can do it in 16 seconds, practically no time at
all. It's certainly less severe, but it isn't much fun.

Conclusion: swap is a last-ditch mechanism to save your system from
having to kill processes. The other two uses -- Linux trades out pages
that are likely to never be called again, or uses the swap space as
a sleep-time mechanism to store memory -- are still viable. It would
probably be best for a desktop machine that *is* going to be put to
sleep overnight to have a minimal amount of swap, and mount a larger,
pre-prepared swapspace for the purpose of sleeping. Unmount it
when awake.

-dsr-
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/21/2015 2:23 PM, Tom Metro wrote:

That doesn't entirely make sense. If setting swappiness to zero means
swap is never used, and setting it to a high value means it is
aggressively used, then in the latter case it means the kernel is moving
stuff from RAM to swap. If it is moved to swap, it ought to free up
physical RAM.


swappiness does not prevent the kernel from swapping. It controls how 
aggressive the kernel is in writing out least-used pages to disk or 
other non-volatile storage.


When set to 0 the kernel will not write pages to disk until all physical 
RAM is allocated. Then it will start writing out pages normally. The 
higher the setting, the more aggressive the kernel will be in writing 
out least-used pages.


Clean pages (allocated pages that have been written to disk) remain in 
the page cache in case they are needed. swappiness = 100 does not mean 
that every page gets read from disk every time it is accessed. Pages are 
dropped from the page cache when their processes release the allocations 
or processes try to allocate more RAM. The kernel drops least-used clean 
pages from the page cache to meet the allocation request. swappiness has 
no effect on the page cache.


When a process tries to allocate more pages than the total of clean and 
unallocated pages in the page cache, well, that's an overload. 
Overloaded system is overloaded. Game over. Install more RAM or don't 
overload the system. If neither option is viable then put your page file 
or partition on a fast SSD. That won't solve the problem but it will 
make it seem less severe.


--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

I've done that a while back. I very rarely have a lockup like that, but 
in most cases I can ssh in and accomplish the same task.


On 06/21/2015 03:28 PM, Steve Litt wrote:

On Sun, 21 Jun 2015 14:23:28 -0400
Tom Metro  wrote:



I've considered a low-tech solution, like having a background script
pop-up a notification when free RAM drops below some threshold to
prompt me to restart various long running and leaking processes.

That's not a half bad idea. I might do that too.

Another thing that might help is to have one of your gettys (Let's say
Ctrl+Alt+F6) running with a nice value of -20. You use it for nothing
except using top and killing things when everything bogs down.





--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

At work I have a version that does. I'll post that tomorrow.

On 06/21/2015 09:18 AM, Bill Bogstad wrote:



On Sun, Jun 21, 2015 at 1:10 PM, Jerry Feldman > wrote:


You can override its behavior my modifying the desktop file
(/usr/share/applications/firefox.desktop in Gnome 3)
The statement 'Exec=firefox %u' is the line to modify.

You could place your modified copy in ~/.local/share/applications

I have not tried this, but it should work.

Instead of su -, use 'sudo -u  firefox', and update
/etc/sudoers not to require a password for this.
For instance:
you  = NOPASSWD: /usr/bin/firefox


I use multiple Firefox user profiles instead.   Some of them allow 
cookies/javascript and others do not.
This probably doesn't help memory usage, but it does allow some 
(small?) security benefits.


I'm curious though, how this other user account gains access to your X 
server.   Allowing other
user ids to write on your screen/capture key & mouse events seem to me 
to be a potential issue.


Bill Bogstad


--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Steve Litt]

On Sun, 21 Jun 2015 14:23:28 -0400
Tom Metro  wrote:


> I've considered a low-tech solution, like having a background script
> pop-up a notification when free RAM drops below some threshold to
> prompt me to restart various long running and leaking processes.

That's not a half bad idea. I might do that too.

Another thing that might help is to have one of your gettys (Let's say
Ctrl+Alt+F6) running with a nice value of -20. You use it for nothing
except using top and killing things when everything bogs down.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Tom Metro]

Matthew Gillen wrote:
> If I try to run Firefox, and a few java apps (e.g., Eclipse), my
> machine thrashes about and effectively locks up because of
> out-of-memory issues.
> 
> After going on like this for literally 10 minutes, OOM-killer sometimes
> kills the right thing...

I've noticed this as well on my laptop with 6 GB RAM. Once you get over
a certain threshold of free RAM, things just drop off a cliff, and even
with manual killing of memory hogs, it takes a while for the needed
stuff to swap back into RAM. Meanwhile the system is locked up or near
locked up.

I've considered a low-tech solution, like having a background script
pop-up a notification when free RAM drops below some threshold to prompt
me to restart various long running and leaking processes.

(This wouldn't help on the occasion when I've hit a bug in a Thunderbird
add-on that results in runaway memory allocations.)


> The behavior when low on memory seems atrociously bad.

Yes. It's disappointing that after decades Linux systems still have
their performance fall off a cliff when free RAM runs low. I expect
performance to be significantly impacted, but it should be swapping
pages out more gradually as RAM starts running low, and the swapping
shouldn't lockup the entire machine. It seems to assume the memory
allocation for that last program that pushed over the limit is so
important that every CPU and I/O cycle should go to swapping to meet
that need. I'd rather just see that one process block. (Of course just
about every interaction allocates some RAM, so once you are out, if
there are no reserves you're stuck waiting to swap on all processes.)


Dan Ritter wrote:
> - play with config settings for the kernel; see
>   https://www.kernel.org/doc/Documentation/sysctl/vm.txt
>   for documentation. Swappiness, OOM behavior, reserved
>   memory...

Swappiness has some potential to help here. I've tweaked it on servers,
but haven't experimented with it on a desktop. It could be time
consuming to arrive at a value that has a benefit. Anyone have
recommendations for values, backed up by experiments that showed it helped?

There are lots of knobs to twiddle there. I'd be surprised if the
situation couldn't be somewhat improved by adjusting them right.


Matthew Gillen wrote:
> Swappiness doesn't really help here. That just controls how eager
> linux is to swap something out.  Once you're out of physical RAM, the 
> OS is going to start swapping if it can.

That doesn't entirely make sense. If setting swappiness to zero means
swap is never used, and setting it to a high value means it is
aggressively used, then in the latter case it means the kernel is moving
stuff from RAM to swap. If it is moved to swap, it ought to free up
physical RAM.

What I do believe is that even with a high swappiness you still might
(will) get DoSed when you run out of physical RAM, but when you hit that
point should take longer.

Although I don't know the granularity with which the swapping decisions
occur. My current FF is running 2 GB VM, 650 MB resident. That ratio is
about typical. Does increasing swappiness result in a lower resident to
VM ratio for FF, or does it just mean other idle processes are more apt
to be swapped out? If the later, and most of your RAM is taken up by a
few huge foreground processes, then yeah, increasing swappiness won't
change much.


> What I want for desktop environments is behavior like: if you run out 
> of memory, kill the thing that's hogging the most.

See Dan's link:
https://www.kernel.org/doc/Documentation/sysctl/vm.txt

  oom_kill_allocating_task

  If this is set to zero, the OOM killer will scan through the entire
  tasklist and select a task based on heuristics to kill.  This normally
  selects a rogue memory-hogging task that frees up a large amount of
  memory when killed.

  If this is set to non-zero, the OOM killer simply kills the task that
  triggered the out-of-memory condition. This avoids the expensive
  tasklist scan.

The default is zero, so it supposedly is trying to implement exactly
what you want. Clearly their heuristics aren't working for you. Maybe
you should disable it and use a custom OOM killer. (Or adjust the
scoring, as you suggested.)

The problem with improving OOM killer process choice is that by the time
it gets invoked, your system is already locking up. You really want a
solution that intervenes before the swapping starts.

I wonder if overcommit_memory set to 2 would help here. See:
https://www.kernel.org/doc/Documentation/vm/overcommit-accounting

That prevents the allocating process from asking for too much memory. If
they do, it returns an error (rather than triggering the OOM killer).
Not clear if this applies to an accumulation of small allocations, or
only single large allocations.

 -Tom

-- 
Tom Metro
The Perl Shop, Newton, MA, USA
"Predictable On-demand Perl Consulting."
http://www.theperlshop.com/
___
Discuss mailing list
Discuss@blu.org
http

Re: [Discuss] memory management

[Bill Bogstad]

On Sun, Jun 21, 2015 at 4:19 PM, Richard Pieri 
wrote:

> On 6/21/2015 9:18 AM, Bill Bogstad wrote:
>
>> I use multiple Firefox user profiles instead.   Some of them allow
>> cookies/javascript and others do not.
>> This probably doesn't help memory usage, but it does allow some (small?)
>> security benefits.
>>
>
> Or use a script blocker like NoScript or uBlock. These offer significant
> security benefits and significantly reduce memory footprint.
>

 I do that as well.   Some of my FireFox profiles have NoScript and others
do not.   I have have a "junk"
profile which has nothing installed, but allows everything, but discards
all history/cookies/etc. when I
exit it.


>
>  I'm curious though, how this other user account gains access to your X
>> server.   Allowing other
>> user ids to write on your screen/capture key & mouse events seem to me to
>> be a potential issue.
>>
>
> May need to use xhost to allow the second user access to the X server,
> something like this:
>
> xhost +SI:localuser:myffuser
> sudo -u ffuser /usr/bin/firefox
> xhost -SI:localuser:myffuser
>
> It's not an issue on a single user box; it's the same user (human) with a
> different UID.
>

This is where I disagree.   If it doesn't increase security over using the
same UID, why bother.  And I'm not sure it really increases security all
that much.For example, breaking out of a browser to run arbitrary code
on the same box as my "real" user id is still a potential security problem.
  Any OS level bugs that aren't network exploitable are now in play.  A bit
like having a guest account on the machine.   Not something
that most people do anymore.

Second, if that user id has the privileges to pop up windows on the same X
server as my "real" user id; I might get spoofed, have my screen or even
possibly my keystrokes captured.   It will depend on how my X server is
setup (and its security).   While it isn't a bad idea to run the browser as
a different user, I think it is more like a speed bump or a chain link
fence than a vault door.   Better might be a chrooted environment, linux
container (docker?), or even VM.

Now, I have to say that I'm not paranoid enough to bother with this.   I
guess it depends on why you
do it.  If it is for user tracking control, I think different user profiles
are sufficient.  If the intent is better
security, I'm not sure it is an improvement.

Bill Bogstad
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Ricker]

On Sun, Jun 21, 2015 at 12:20 AM, Matthew Gillen  wrote:

> I've found that if you use
>   su - 
> then you can run X programs as another user without trouble.
>

IIRC that requires a permissive setting on the X "server" to allow any to
connect to execute graprics, which may not be harmful in a home setting but
feels wrong.​

​gksu works as a sudo for graphical programs and i think it avoids the
above .


-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/21/2015 9:18 AM, Bill Bogstad wrote:

I use multiple Firefox user profiles instead.   Some of them allow
cookies/javascript and others do not.
This probably doesn't help memory usage, but it does allow some (small?)
security benefits.


Or use a script blocker like NoScript or uBlock. These offer significant 
security benefits and significantly reduce memory footprint.



I'm curious though, how this other user account gains access to your X
server.   Allowing other
user ids to write on your screen/capture key & mouse events seem to me to
be a potential issue.


May need to use xhost to allow the second user access to the X server, 
something like this:


xhost +SI:localuser:myffuser
sudo -u ffuser /usr/bin/firefox
xhost -SI:localuser:myffuser

It's not an issue on a single user box; it's the same user (human) with 
a different UID.


--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Bogstad]

On Sun, Jun 21, 2015 at 1:10 PM, Jerry Feldman  wrote:

> You can override its behavior my modifying the desktop file
> (/usr/share/applications/firefox.desktop in Gnome 3)
> The statement 'Exec=firefox %u' is the line to modify.
>
> You could place your modified copy in ~/.local/share/applications
>
> I have not tried this, but it should work.
>
> Instead of su -, use 'sudo -u  firefox', and update /etc/sudoers not
> to require a password for this.
> For instance:
> you  = NOPASSWD: /usr/bin/firefox


I use multiple Firefox user profiles instead.   Some of them allow
cookies/javascript and others do not.
This probably doesn't help memory usage, but it does allow some (small?)
security benefits.

I'm curious though, how this other user account gains access to your X
server.   Allowing other
user ids to write on your screen/capture key & mouse events seem to me to
be a potential issue.

Bill Bogstad
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

You can override its behavior my modifying the desktop file 
(/usr/share/applications/firefox.desktop in Gnome 3)

The statement 'Exec=firefox %u' is the line to modify.

You could place your modified copy in ~/.local/share/applications

I have not tried this, but it should work.

Instead of su -, use 'sudo -u  firefox', and update /etc/sudoers 
not to require a password for this.

For instance:
you  = NOPASSWD: /usr/bin/firefox



On 06/21/2015 12:20 AM, Matthew Gillen wrote:

On 6/20/2015 4:18 PM, Mike Small wrote:

Matthew Gillen  writes:

going to start swapping if it can. What I want for desktop environments
is behavior like: if you run out of memory, kill the thing that's
hogging the most.  My typical case is that if there is a process using a
ton of memory, it's probably doing something wrong (e.g. javascript, or
eclipse going into a death spiral because of the awful Android plugin),
and /that/ is what I want OOM-killer to murder.

I suppose the right answer is to wrap the problem programs in a script
so that every time I start them I can
   echo 999 > /proc/[firefox-pid]/oom_score_adj

What about creating a second, less privileged user for running firefox
and using ulimit to keep it down to size?  There are good reasons to not
run firefox as your main user anyway, at least not for general browsing.
I do this (minus the ulimit part), with the non-privileged firefox also
having restrictive plugins. For banking and a small number of other
sites I run firefox as my main user with no plugins. That way I don't
have to worry about librejs or requestpolicy messing up a financial
transaction. And if a site takes advantage of a firefox exploit it's
somewhat contained, assuming it's not my bank that hosts the exploit.

That's not a bad idea.  I've found that if you use
   su - 
then you can run X programs as another user without trouble.

Matt

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss




--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

On 6/20/2015 4:18 PM, Mike Small wrote:
> Matthew Gillen  writes:
>> going to start swapping if it can. What I want for desktop environments
>> is behavior like: if you run out of memory, kill the thing that's
>> hogging the most.  My typical case is that if there is a process using a
>> ton of memory, it's probably doing something wrong (e.g. javascript, or
>> eclipse going into a death spiral because of the awful Android plugin),
>> and /that/ is what I want OOM-killer to murder.
>>
>> I suppose the right answer is to wrap the problem programs in a script
>> so that every time I start them I can
>>   echo 999 > /proc/[firefox-pid]/oom_score_adj
> 
> What about creating a second, less privileged user for running firefox
> and using ulimit to keep it down to size?  There are good reasons to not
> run firefox as your main user anyway, at least not for general browsing.
> I do this (minus the ulimit part), with the non-privileged firefox also
> having restrictive plugins. For banking and a small number of other
> sites I run firefox as my main user with no plugins. That way I don't
> have to worry about librejs or requestpolicy messing up a financial
> transaction. And if a site takes advantage of a firefox exploit it's
> somewhat contained, assuming it's not my bank that hosts the exploit.

That's not a bad idea.  I've found that if you use
  su - 
then you can run X programs as another user without trouble.

Matt

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Mike Small]

Matthew Gillen  writes:
> going to start swapping if it can. What I want for desktop environments
> is behavior like: if you run out of memory, kill the thing that's
> hogging the most.  My typical case is that if there is a process using a
> ton of memory, it's probably doing something wrong (e.g. javascript, or
> eclipse going into a death spiral because of the awful Android plugin),
> and /that/ is what I want OOM-killer to murder.
>
> I suppose the right answer is to wrap the problem programs in a script
> so that every time I start them I can
>   echo 999 > /proc/[firefox-pid]/oom_score_adj

What about creating a second, less privileged user for running firefox
and using ulimit to keep it down to size?  There are good reasons to not
run firefox as your main user anyway, at least not for general browsing.
I do this (minus the ulimit part), with the non-privileged firefox also
having restrictive plugins. For banking and a small number of other
sites I run firefox as my main user with no plugins. That way I don't
have to worry about librejs or requestpolicy messing up a financial
transaction. And if a site takes advantage of a firefox exploit it's
somewhat contained, assuming it's not my bank that hosts the exploit.

-- 
Mike Small
sma...@panix.com
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Matthew Gillen]

Thanks for the suggestions.

Addressing some random questions:
 -this is a Fedora box.  Currently v21, but I've had these types of
issues for years.
 - top, when I can get it to run, shows virtually no CPU use.  The only
thing  that is getting to run is kswapd0 if I recall correctly.
 - I do have way to many tabs open in firefox.  I thought the recent
changes to not load pages in other tabs on startup meant that it was a
little more forgiving in terms of memory of having a 100+ tabs open that
you never actually load.  I can certainly change my use-pattern there.

Swappiness doesn't really help here. That just controls how eager linux
is to swap something out.  Once you're out of physical RAM, the OS is
going to start swapping if it can. What I want for desktop environments
is behavior like: if you run out of memory, kill the thing that's
hogging the most.  My typical case is that if there is a process using a
ton of memory, it's probably doing something wrong (e.g. javascript, or
eclipse going into a death spiral because of the awful Android plugin),
and /that/ is what I want OOM-killer to murder.

I suppose the right answer is to wrap the problem programs in a script
so that every time I start them I can
  echo 999 > /proc/[firefox-pid]/oom_score_adj

Anyone ever had to use cpuset(7) for anything?

Thanks,
Matt
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

I don't agree with shutting firefox down every time you are finished. 
Virtual memory should mitigate this if you focus on another task, but.
Many web sites are very active so they keep requiring system resources 
even if firefox is not actively being used. Maybe just closing some tabs 
of some of the more active web sites will help.


On 06/19/2015 11:30 AM, Jerry Natowitz wrote:
My advise on Firefox is to close it down completely whenever you have 
finished using it.  There seems to be a lot of memory leaking, I've 
seen Firefox grow into multi-gigabyte virtual size in a matter of hours.


I have not experimented with shutting down Firefox with multiple 
windows and tabs, and then restarted it, using the Restore Previous 
Session option on the History pull-down.  It would be interesting to 
see how much memory was saved, and for how long.


Jerry Natowitz
===>j.natowitz (at) rcn.com
if rcn.com bounces, try gmail.com

On 06/19/15 11:02, Steve Litt wrote:

On Fri, 19 Jun 2015 10:01:57 -0400
Matthew Gillen  wrote:


I'm looking for some advice on tuning my linux box's memory
management. I've got an older workstation that has merely 4GB of
memory.  If I try to run Firefox, and a few java apps (e.g.,
Eclipse), my machine thrashes about and effectively locks up because
of out-of-memory issues.

For example: the mouse will continue to move, but won't change it's
icon contextually.  If I hit cntl-alt-f2 and try to log in to a
virtual console, mgetty will eventually ask for the username, but
after I hit enter, it just hangs, not popping up the password prompt,
and after 60 seconds the login times out.  Trying to ssh into the
machine from somewhere else ends up timing out.

After going on like this for literally 10 minutes, OOM-killer
sometimes kills the right thing (one of the two processes hogging the
most memory: firefox or eclipse), and the machine becomes usable
again sometime later.

I have heftier workstations I can use, but this behavior is really
frustrating to me, because I'd like to think linux does good memory
management.  I've tried using huge swap (2x physical memory). I've
tried with virtually no swap (on the theory that without swap, there
would be no thrashing and at least oom-killer would have to do its
thing without locking up the machine for 10 minutes first). The
problem there was oom-killer making bad decisions about what to kill
(e.g., the window manager, and then whatever out-of-control process
is sucking up memory just sucks up whatever got freed, and nothing
gets better).  At least with some swap oom-killer seems to make
better guesses about who to murder.

Does anyone have any tips on how to prevent linux from thrashing like
that?  The behavior when low on memory seems atrociously bad.

Thanks,
Matt


Hi Matt,

I haven't seen any stats quoted in your email, from the  "top" program,
that indicate it's a RAM problem. Firefox and its pet
"plugin-container" use a heck of a lot of CPU. Until very recently I
was using a 4GB machine, and when things got crawly, the top program
indicated that both my cores were near 100%, but there was plenty more
RAM.

Today I have a 16GB RAM box, with dual core CPU (I wanted things to
stay cool), and things still get crawly. When they do, I run the top
command to see what's taking all the CPU, and kill it if necessary.
It's usually one or more instances of Firefox and plugin-container. I
typically killall plugin-container, and then start closing
no-longer-needed tabs on the various Firefox windows. I'll often drag
all the tags to *one* Firefox window, and kill the others.

I like Firefox, but it's no doubt a pig. My recommendation when using
Firefox is to close any tabs you're finished with. Often, good
housekeeping with Firefox is the key to avoiding the crawlies.

When the crawlies rear their ugly head, my first step on the path
is the top command, to see who is consuming what resource, and what
resource is becoming a choke point. Then I take care of the choke
point, and if it involves Firefox at all, I'm ruthless in closing tabs
I'm finished with.

SteveT

Steve Litt
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss


___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss




--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Feldman]

Agreed. My netbook has 1G of memory and actually ran virtualbox 
surprisingly well while I was giving a presentation using Libre Office 
Impress. It might be possible that you have an issue with your memory. 
Try running a memory diagnostic.


On 06/19/2015 10:32 AM, Eric Chadbourne wrote:



On Jun 19, 2015, at 10:01 AM, Matthew Gillen  wrote:

I'm looking for some advice on tuning my linux box's memory management.
I've got an older workstation that has merely 4GB of memory.  If I try
to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
about and effectively locks up because of out-of-memory issues.

4GB is a ton for regular tasks.  That’s all two of my boxes have and I can even 
run virtualbox with a linux guest without issue.  I don’t see how those two 
apps are going to max out your system.  I bet your issues are not firefox and 
eclipse.

What is really using your system resources?

- Eric
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss


--
Jerry Feldman 
Boston Linux and Unix
PGP key id:B7F14F2F
PGP Key fingerprint: D937 A424 4836 E052 2E1B  8DC6 24D7 000F B7F1 4F2F

___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Ricker]

On Fri, Jun 19, 2015 at 11:02 AM, Steve Litt  wrote:
> I haven't seen any stats quoted in your email, from the  "top" program,
> that indicate it's a RAM problem. Firefox and its pet
> "plugin-container" use a heck of a lot of CPU. Until very recently I
> was using a 4GB machine, and when things got crawly, the top program
> indicated that both my cores were near 100%, but there was plenty more
> RAM.

I just killed and restarted FF (on Ubuntu LTS) ... down from 3GB total
1.5G Resident to 1.7G total 0.9G Resident, same tabs. And it peaks at
100% CPU. I should be thankful it's not multiCPU thread capable ?

-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Drew Van Zandt]

Hardware design, several things.  Primarily EM simulation and big FPGA sim,
complex analog sim, or just big schematics.  When doing a schematic review,
it is helpful to have half a dozen or more large PDFs (datasheets) open
while viewing schematics, a few spreadsheets, some Word & Visio design
docs, and other miscellaneous goodies.  I can open/close things as needed,
but that makes it take 2-3 times as long (a week instead of a day and a
half).

32GB is OK for this.  64GB would be nicer.  I might ask for it next year.
It would be nice to still be able to work on other things effectively while
a signal integrity/crosstalk sim is crunching numbers.

*Drew Van ZandtArtisan's Asylum Board of DirectorsFirefly Arts Collective
Board of Directors*

On Fri, Jun 19, 2015 at 11:48 AM, Mike Small  wrote:

> Drew Van Zandt  writes:
>
> > 16GB seems like a small amount of memory to me; I generally use 24GB
> > or more.  Different strokes...
> >
> > --DTVZ
>
> Gaming or simulation?  What struggles at 16GB?
>
> --
> Mike Small
> sma...@panix.com
>
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Mike Small]

Drew Van Zandt  writes:

> 16GB seems like a small amount of memory to me; I generally use 24GB
> or more.  Different strokes...
>
> --DTVZ

Gaming or simulation?  What struggles at 16GB?

-- 
Mike Small
sma...@panix.com
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Mike Small]

Jerry Natowitz  writes:

> My advise on Firefox is to close it down completely whenever you have
> finished using it.  There seems to be a lot of memory leaking, I've
> seen Firefox grow into multi-gigabyte virtual size in a matter of
> hours.

I leave it running all the time and don't see this. I mean, it's bloated
as all hell for what it is, but I've had it up since last week and it's
stable at 1 GB virtual memory, peak working set 560 MB with many tabs
open.  This make me think it's less firefox having severe leaks than the
abomidable state of javascript on websites today. I don't let very much
javascript run.

-- 
Mike Small
sma...@panix.com
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Jerry Natowitz]

My advise on Firefox is to close it down completely whenever you have 
finished using it.  There seems to be a lot of memory leaking, I've seen 
Firefox grow into multi-gigabyte virtual size in a matter of hours.


I have not experimented with shutting down Firefox with multiple windows 
and tabs, and then restarted it, using the Restore Previous Session 
option on the History pull-down.  It would be interesting to see how 
much memory was saved, and for how long.


Jerry Natowitz
===>j.natowitz (at) rcn.com
if rcn.com bounces, try gmail.com

On 06/19/15 11:02, Steve Litt wrote:

On Fri, 19 Jun 2015 10:01:57 -0400
Matthew Gillen  wrote:


I'm looking for some advice on tuning my linux box's memory
management. I've got an older workstation that has merely 4GB of
memory.  If I try to run Firefox, and a few java apps (e.g.,
Eclipse), my machine thrashes about and effectively locks up because
of out-of-memory issues.

For example: the mouse will continue to move, but won't change it's
icon contextually.  If I hit cntl-alt-f2 and try to log in to a
virtual console, mgetty will eventually ask for the username, but
after I hit enter, it just hangs, not popping up the password prompt,
and after 60 seconds the login times out.  Trying to ssh into the
machine from somewhere else ends up timing out.

After going on like this for literally 10 minutes, OOM-killer
sometimes kills the right thing (one of the two processes hogging the
most memory: firefox or eclipse), and the machine becomes usable
again sometime later.

I have heftier workstations I can use, but this behavior is really
frustrating to me, because I'd like to think linux does good memory
management.  I've tried using huge swap (2x physical memory).  I've
tried with virtually no swap (on the theory that without swap, there
would be no thrashing and at least oom-killer would have to do its
thing without locking up the machine for 10 minutes first).  The
problem there was oom-killer making bad decisions about what to kill
(e.g., the window manager, and then whatever out-of-control process
is sucking up memory just sucks up whatever got freed, and nothing
gets better).  At least with some swap oom-killer seems to make
better guesses about who to murder.

Does anyone have any tips on how to prevent linux from thrashing like
that?  The behavior when low on memory seems atrociously bad.

Thanks,
Matt


Hi Matt,

I haven't seen any stats quoted in your email, from the  "top" program,
that indicate it's a RAM problem. Firefox and its pet
"plugin-container" use a heck of a lot of CPU. Until very recently I
was using a 4GB machine, and when things got crawly, the top program
indicated that both my cores were near 100%, but there was plenty more
RAM.

Today I have a 16GB RAM box, with dual core CPU (I wanted things to
stay cool), and things still get crawly. When they do, I run the top
command to see what's taking all the CPU, and kill it if necessary.
It's usually one or more instances of Firefox and plugin-container. I
typically killall plugin-container, and then start closing
no-longer-needed tabs on the various Firefox windows. I'll often drag
all the tags to *one* Firefox window, and kill the others.

I like Firefox, but it's no doubt a pig. My recommendation when using
Firefox is to close any tabs you're finished with. Often, good
housekeeping with Firefox is the key to avoiding the crawlies.

When the crawlies rear their ugly head, my first step on the path
is the top command, to see who is consuming what resource, and what
resource is becoming a choke point. Then I take care of the choke
point, and if it involves Firefox at all, I'm ruthless in closing tabs
I'm finished with.

SteveT

Steve Litt
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss


___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Drew Van Zandt]

16GB seems like a small amount of memory to me; I generally use 24GB or
more.  Different strokes...

--DTVZ

On Fri, Jun 19, 2015 at 11:08 AM, Mike Small  wrote:

> 4.0 GB seems like a large amount of memory to me. It's way more than
> I have a use for.
>



*Drew Van ZandtArtisan's Asylum Board of DirectorsFirefly Arts Collective
Board of Directors*
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Mike Small]

Dan Ritter  writes:
> - Firefox is a memory hog. (So is Chrome.)
>
> - All Java apps are memory hogs, but especially Eclipse

Which caused me puzzlement at the last BLU meeting.  How has it come to
be that some scientific software is written in Java?

> - Open less stuff in Firefox. Especially, run an ad blocker like
>   uBlock Origin, and don't load Flash at all.

Being selective about what javascript gets to run helps a lot too. The
most popular plugin for this is noscript I think, but see a recent
thread here for other options.

You could also stop using the web as much. Sometimes there are
alternatives: e.g. mailing list announcements instead of Meetup site,
mailing list or usenet group instead of web discussion forum, gopher or
telnet weather forecasts instead of web site with same info.  That's
more advice for < 512 MB, but there are other advantages to these
methods if you give them a chance.

4.0 GB seems like a large amount of memory to me. It's way more than 
I have a use for.

-- 
Mike Small
sma...@panix.com
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Steve Litt]

On Fri, 19 Jun 2015 10:01:57 -0400
Matthew Gillen  wrote:

> I'm looking for some advice on tuning my linux box's memory
> management. I've got an older workstation that has merely 4GB of
> memory.  If I try to run Firefox, and a few java apps (e.g.,
> Eclipse), my machine thrashes about and effectively locks up because
> of out-of-memory issues.
> 
> For example: the mouse will continue to move, but won't change it's
> icon contextually.  If I hit cntl-alt-f2 and try to log in to a
> virtual console, mgetty will eventually ask for the username, but
> after I hit enter, it just hangs, not popping up the password prompt,
> and after 60 seconds the login times out.  Trying to ssh into the
> machine from somewhere else ends up timing out.
> 
> After going on like this for literally 10 minutes, OOM-killer
> sometimes kills the right thing (one of the two processes hogging the
> most memory: firefox or eclipse), and the machine becomes usable
> again sometime later.
> 
> I have heftier workstations I can use, but this behavior is really
> frustrating to me, because I'd like to think linux does good memory
> management.  I've tried using huge swap (2x physical memory).  I've
> tried with virtually no swap (on the theory that without swap, there
> would be no thrashing and at least oom-killer would have to do its
> thing without locking up the machine for 10 minutes first).  The
> problem there was oom-killer making bad decisions about what to kill
> (e.g., the window manager, and then whatever out-of-control process
> is sucking up memory just sucks up whatever got freed, and nothing
> gets better).  At least with some swap oom-killer seems to make
> better guesses about who to murder.
> 
> Does anyone have any tips on how to prevent linux from thrashing like
> that?  The behavior when low on memory seems atrociously bad.
> 
> Thanks,
> Matt

Hi Matt,

I haven't seen any stats quoted in your email, from the  "top" program,
that indicate it's a RAM problem. Firefox and its pet
"plugin-container" use a heck of a lot of CPU. Until very recently I
was using a 4GB machine, and when things got crawly, the top program
indicated that both my cores were near 100%, but there was plenty more
RAM.

Today I have a 16GB RAM box, with dual core CPU (I wanted things to
stay cool), and things still get crawly. When they do, I run the top
command to see what's taking all the CPU, and kill it if necessary.
It's usually one or more instances of Firefox and plugin-container. I
typically killall plugin-container, and then start closing
no-longer-needed tabs on the various Firefox windows. I'll often drag
all the tags to *one* Firefox window, and kill the others.

I like Firefox, but it's no doubt a pig. My recommendation when using
Firefox is to close any tabs you're finished with. Often, good
housekeeping with Firefox is the key to avoiding the crawlies.

When the crawlies rear their ugly head, my first step on the path
is the top command, to see who is consuming what resource, and what
resource is becoming a choke point. Then I take care of the choke
point, and if it involves Firefox at all, I'm ruthless in closing tabs
I'm finished with.

SteveT

Steve Litt 
June 2015 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Dan Ritter]

On Fri, Jun 19, 2015 at 02:44:35PM +, Marcia K Wilbur wrote:
> I'm not sure what you are using for the OS but I'll guess - Ubuntu
> or some variant of Ubuntu..
> 
> If so, stop doing that. If not, let us know. Then, maybe there are
> other solutions.
> 
> The memory leak wrt Firefox has been an issue for those users (not
> community...users) for years.
> There are so many reasons to use a distro over Ubuntu.
> 
> Note I did not refer to Ubuntu as a "distro"...
> At any rate, there is a fix for the Ubuntu Memory issue. The best
> solution though- Don't use Ubuntu.
> 

Could you provide references for any of this, please?

-dsr-
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Marcia K Wilbur]

I'm not sure what you are using for the OS but I'll guess - Ubuntu or  
some variant of Ubuntu..


If so, stop doing that. If not, let us know. Then, maybe there are  
other solutions.


The memory leak wrt Firefox has been an issue for those users (not  
community...users) for years.

There are so many reasons to use a distro over Ubuntu.

Note I did not refer to Ubuntu as a "distro"...
At any rate, there is a fix for the Ubuntu Memory issue. The best  
solution though- Don't use Ubuntu.



Quoting Eric Chadbourne :


On Jun 19, 2015, at 10:01 AM, Matthew Gillen  wrote:

I'm looking for some advice on tuning my linux box's memory management.
I've got an older workstation that has merely 4GB of memory.  If I try
to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
about and effectively locks up because of out-of-memory issues.


4GB is a ton for regular tasks.  That’s all two of my boxes have and  
I can even run virtualbox with a linux guest without issue.  I don’t  
see how those two apps are going to max out your system.  I bet your  
issues are not firefox and eclipse.


What is really using your system resources?

- Eric
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss




___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Eric Chadbourne]

> On Jun 19, 2015, at 10:01 AM, Matthew Gillen  wrote:
> 
> I'm looking for some advice on tuning my linux box's memory management.
> I've got an older workstation that has merely 4GB of memory.  If I try
> to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
> about and effectively locks up because of out-of-memory issues.

4GB is a ton for regular tasks.  That’s all two of my boxes have and I can even 
run virtualbox with a linux guest without issue.  I don’t see how those two 
apps are going to max out your system.  I bet your issues are not firefox and 
eclipse.

What is really using your system resources?

- Eric
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Richard Pieri]

On 6/19/2015 10:01 AM, Matthew Gillen wrote:

Does anyone have any tips on how to prevent linux from thrashing like
that?  The behavior when low on memory seems atrociously bad.


Install more RAM or stop using programs that use more RAM than you have.

--
Rich P.
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Bill Ricker]

How Swap and Memory are used is controlled largely by the 'swappiness'
setting.  The default is correct for servers but not for workstations.

swap - How do I configure swappiness? - Ask Ubuntu
The Linux kernel provides a tweakable setting that controls how often
the swap file is used, called swappiness. A swappiness setting of zero
means that the disk will ...
askubuntu.com/questions/103915/how-do-i-configure-swapp...

SwapFaq - Community Help Wiki - Ubuntu
The hibernation implementation currently used in Ubuntu, swsusp, needs
a swap or suspend partition. ... The default setting in Ubuntu is
swappiness=60.
help.ubuntu.com/community/SwapFaq

On Fri, Jun 19, 2015 at 10:01 AM, Matthew Gillen  wrote:
> I'm looking for some advice on tuning my linux box's memory management.
>  I've got an older workstation that has merely 4GB of memory.  If I try
> to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
> about and effectively locks up because of out-of-memory issues.
>
> For example: the mouse will continue to move, but won't change it's icon
> contextually.  If I hit cntl-alt-f2 and try to log in to a virtual
> console, mgetty will eventually ask for the username, but after I hit
> enter, it just hangs, not popping up the password prompt, and after 60
> seconds the login times out.  Trying to ssh into the machine from
> somewhere else ends up timing out.
>
> After going on like this for literally 10 minutes, OOM-killer sometimes
> kills the right thing (one of the two processes hogging the most memory:
> firefox or eclipse), and the machine becomes usable again sometime later.
>
> I have heftier workstations I can use, but this behavior is really
> frustrating to me, because I'd like to think linux does good memory
> management.  I've tried using huge swap (2x physical memory).  I've
> tried with virtually no swap (on the theory that without swap, there
> would be no thrashing and at least oom-killer would have to do its thing
> without locking up the machine for 10 minutes first).  The problem there
> was oom-killer making bad decisions about what to kill (e.g., the window
> manager, and then whatever out-of-control process is sucking up memory
> just sucks up whatever got freed, and nothing gets better).  At least
> with some swap oom-killer seems to make better guesses about who to murder.
>
> Does anyone have any tips on how to prevent linux from thrashing like
> that?  The behavior when low on memory seems atrociously bad.
>
> Thanks,
> Matt
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss



-- 
Bill Ricker
bill.n1...@gmail.com
https://www.linkedin.com/in/n1vux
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss

Re: [Discuss] memory management

[Dan Ritter]

On Fri, Jun 19, 2015 at 10:01:57AM -0400, Matthew Gillen wrote:
> I'm looking for some advice on tuning my linux box's memory management.
>  I've got an older workstation that has merely 4GB of memory.  If I try
> to run Firefox, and a few java apps (e.g., Eclipse), my machine thrashes
> about and effectively locks up because of out-of-memory issues.
> 
> For example: the mouse will continue to move, but won't change it's icon
> contextually.  If I hit cntl-alt-f2 and try to log in to a virtual
> console, mgetty will eventually ask for the username, but after I hit
> enter, it just hangs, not popping up the password prompt, and after 60
> seconds the login times out.  Trying to ssh into the machine from
> somewhere else ends up timing out.
> 
> After going on like this for literally 10 minutes, OOM-killer sometimes
> kills the right thing (one of the two processes hogging the most memory:
> firefox or eclipse), and the machine becomes usable again sometime later.
> 
> I have heftier workstations I can use, but this behavior is really
> frustrating to me, because I'd like to think linux does good memory
> management.  I've tried using huge swap (2x physical memory).  I've
> tried with virtually no swap (on the theory that without swap, there
> would be no thrashing and at least oom-killer would have to do its thing
> without locking up the machine for 10 minutes first).  The problem there
> was oom-killer making bad decisions about what to kill (e.g., the window
> manager, and then whatever out-of-control process is sucking up memory
> just sucks up whatever got freed, and nothing gets better).  At least
> with some swap oom-killer seems to make better guesses about who to murder.
> 
> Does anyone have any tips on how to prevent linux from thrashing like
> that?  The behavior when low on memory seems atrociously bad.

You have three problems:

- Firefox is a memory hog. (So is Chrome.)

- All Java apps are memory hogs, but especially Eclipse

- You only have 4GB of RAM. Phones are coming with 4GB of RAM
  this year.

Swap will not save you. Swap has three purposes:

- stash a small amount of stuff that will likely never be called
  again but is important to have "in memory"

- provide a place for RAM to be stored when putting the machine
  to sleep

- last ditch "keep the machine alive" when it has too many
  requests for not enough RAM.

You can:

- Add more RAM.

- Open less stuff in Firefox. Especially, run an ad blocker like
  uBlock Origin, and don't load Flash at all.

- Start Java programs with a restricted amount of memory:
  -Xmx is the flag you want. java -Xmx512M, for example.

- play with config settings for the kernel; see
  https://www.kernel.org/doc/Documentation/sysctl/vm.txt
  for documentation. Swappiness, OOM behavior, reserved
  memory...

-dsr-
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss