Re: [Mojolicious] Reducing daemon memory footprint?

2018-09-26 Thread Charlie Brady


This is why I need to run a persistent daemon rather than use CGI mode.

root@jy-cx:~# time perl -e exit

real0m0.035s
user0m0.000s
sys 0m0.030s
root@jy-cx:~# time perl -MMojolicious::Lite -e exit

real0m11.011s
user0m8.690s
sys 0m0.400s
root@jy-cx:~# 

Looking at an strace there doesn't seem to be a single culprit for the 
slow startup - there's just a lot of files to find, open, read, parse.

root@jy-cx:~# grep open ml.trace  | wc -l
211
root@jy-cx:~# 

...
root@jy-cx:~# grep open ml.trace | grep 12:21:26 | wc -l
16
root@jy-cx:~# grep open ml.trace | grep 12:21:27 | wc -l
28
root@jy-cx:~# grep open ml.trace | grep 12:21:28 | wc -l
16
root@jy-cx:~# grep open ml.trace | grep 12:21:29 | wc -l
19
root@jy-cx:~# grep open ml.trace | grep 12:21:30 | wc -l
13
root@jy-cx:~# grep open ml.trace | grep 12:21:31 | wc -l
2
root@jy-cx:~# grep open ml.trace | grep 12:21:32 | wc -l
15
root@jy-cx:~# grep open ml.trace | grep 12:21:33 | wc -l
8
root@jy-cx:~# grep open ml.trace | grep 12:21:34 | wc -l
18
root@jy-cx:~# grep open ml.trace | grep 12:21:35 | wc -l
24
root@jy-cx:~# grep open ml.trace | grep 12:21:36 | wc -l
8
root@jy-cx:~# grep open ml.trace | grep 12:21:37 | wc -l
19
root@jy-cx:~# grep open ml.trace | grep 12:21:38 | wc -l
11
root@jy-cx:~# grep open ml.trace | grep 12:21:39 | wc -l
14
root@jy-cx:~# grep open ml.trace | grep 12:21:40 | wc -l
0
root@jy-cx:~# 
...

Initialising Time::HiRes::HiRes seems to be the single individual 
parse/init step:

12:21:30.734184 open("/usr/lib/perl/5.22.0/Pod/Simple/BlackBox.pm", 
O_RDONLY|O_L
ARGEFILE) = 16
12:21:31.607709 open("/usr/lib/perl/5.22.0/Time/HiRes.pm", 
O_RDONLY|O_LARGEFILE) = 12
12:21:31.643254 open("/usr/lib/perl/5.22.0//auto/Time/HiRes/HiRes.so", 
O_RDONLY|O_CLOEXEC) = 12
12:21:32.356811 open("/usr/lib/perl/vendor_perl/5.22.0/Mojo/Log.pm", 
O_RDONLY|O_LARGEFILE) = 7



On Thu, 20 Sep 2018, Charlie Brady wrote:

> 
> On Thu, 20 Sep 2018, Stefan Adams wrote:
> 
> > Wow!  That's fascinating!  Thanks for sharing!  I'm surprised by how 
> > much the memory footprint grows for a minimal perl program to one 
> > running M::L!
> 
> Bear in mind this is one M::L program which has loaded (via the Mount 
> plugin) a few dozen other M::L programs and then run for a few months 
> (mostly idle).
> 
> FWIW, this is on a PPC processor (perl 5.22.0, Mojolicious 7.33).
> 
> > On Thu, Sep 20, 2018 at 2:57 PM Charlie Brady <
> > charlieb-m...@budge.apana.org.au> wrote:
> > 
> > >
> > > Hi Stefan, thanks for trying to help.
> > >
> > > I know the perl executable will make some contribution to the memory
> > > footprint, but not much.
> > >
> > > I can measure that separately by running a minimum perl program and
> > > looking at its resource usage. If I run 'perl -e "<>"', I see this in
> > > /proc/$pid/status:
> > >
> > > ...
> > > VmPeak: 5476 kB
> > > VmSize: 5476 kB
> > > VmLck: 0 kB
> > > VmPin: 0 kB
> > > VmHWM:  3184 kB
> > > VmRSS:  3184 kB
> > > VmData:  468 kB
> > > VmStk:   132 kB
> > > VmExe: 4 kB
> > > VmLib:  4232 kB
> > > VmPTE:24 kB
> > > VmPMD: 0 kB
> > > VmSwap:0 kB
> > > Threads:1
> > > ...
> > >
> > > The same data from my M::L daemon is:
> > >
> > > ...
> > > VmPeak:   135548 kB
> > > VmSize:   133540 kB
> > > VmLck: 0 kB
> > > VmPin: 0 kB
> > > VmHWM: 67016 kB
> > > VmRSS: 65988 kB
> > > VmData:86940 kB
> > > VmStk:   176 kB
> > > VmExe: 4 kB
> > > VmLib: 31664 kB
> > > VmPTE:   132 kB
> > > VmPMD: 0 kB
> > > VmSwap:0 kB
> > > Threads:6
> > > ...
> > >
> > > CPAN has a few modules which might help gather some stats
> > > (Memory::Process, Memory::Stats, Memory::Usage). '-Dm' flag to perl might
> > > help too, if perl is compiled with DEBUG enabled.
> > >
> > > I notice that I can disable Mojolicious response cacheing; it'll be
> > > interesting to see what difference that makes.
> > >
> > > On Wed, 19 Sep 2018, Stefan Adams wrote:
> > >
> > > > I shouldn't involve myself in this topic because I haven't the 
> > > > knowledge,
> > > > but it sounds fascinating. Wouldn't the majority of the 50MB memory
> > > > footprint be from the Perl interpreter itself? Wouldn't the question be,
> > > > how to reduce the size of the Perl interpreter? Different compile
> > > options?
> > > > e.g. I remember once compiling a "minimal" perl that was 2MB. I imagine
> > > > that would probably be insufficient to run Mojolicious, though I really
> > > > haven't a clue, but it suggests to me that it's possible to reduce the
> > > size
> > > > of the perl interpreter and, therefore it seems, the size of the
> > > > Mojolicious daemon?
> > > >
> > > > On Wed, Sep 19, 2018, 8:19 PM Charlie Brady 

Re: [Mojolicious] Reducing daemon memory footprint?

2018-09-26 Thread sri


Am Mittwoch, 26. September 2018 18:28:10 UTC+2 schrieb Charlie Brady:
>
>
> This is why I need to run a persistent daemon rather than use CGI mode. 
>
> root@jy-cx:~# time perl -e exit 
>
> real0m0.035s 
> user0m0.000s 
> sys 0m0.030s 
> root@jy-cx:~# time perl -MMojolicious::Lite -e exit 
>
> real0m11.011s 
> user0m8.690s 
> sys 0m0.400s 
> root@jy-cx:~# 
>
> Looking at an strace there doesn't seem to be a single culprit for the 
> slow startup - there's just a lot of files to find, open, read, parse. 
>

If that's the case, why not just fatpack your mojo app?

--
sebastian 

-- 
You received this message because you are subscribed to the Google Groups 
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mojolicious+unsubscr...@googlegroups.com.
To post to this group, send email to mojolicious@googlegroups.com.
Visit this group at https://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.


Re: [Mojolicious] Reducing daemon memory footprint?

2018-09-26 Thread Charlie Brady


On Wed, 26 Sep 2018, sri wrote:

> If that's the case, why not just fatpack your mojo app?

Um, because I didn't know about fatpack? :-)

Thanks, looks a good suggestion. The system will continue to be slow, but 
not as slow...