[perl #131915] [REGRESSION] Proc using more memory since being a Proc::Async

[jn...@jnthn.net via RT]

On Wed, 16 Aug 2017 14:24:48 -0700, scoli...@gmail.com wrote:
> Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
> that there is a huge increase of memory usage after running a simple
> echo command.
> Result are in Kbytes.
> 
> <AlexDaniel> commit:
> 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
> mem(); run 'echo'; say mem()
> <committable6test> AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820»
> ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660»
> ¦9658dd9: «139024␤␤305320»

We won't get it back to where it was, because threads cost something. And we'd 
not really want to go back to where we were, because Proc had all kinds of ways 
of not working. It's easy to be cheaper if you can be brokener too.

The cost is now very significantly lower, however, thanks to:

* The new scheduler, which doesn't wastefully start a load of threads for a 
single Proc::Async usage. This was the major win.
* Smarter handling of memory for new threads in MoarVM; now they start with a 
128KB nursery and grow it as needed, as opposed to starting with a 4MB nursery.

So rather than it causing a 3.8 times growth in program memory size, I now 
measure it as less than 1.5 (and, better, it won't increase with follow-up 
calls to run the way it used to either).