Hi, <helmut> we're using HTML::Template <h01ger> "BENCHMARKS" seems interesting <helmut> http://xslate.org/benchmark.html also claims to be 40 times faster. <h01ger> Setting case_sensitive to 1, loop_context_vars to 0 and global_vars to 0 saves time. <helmut> we set loop_context_vars and global_Vars to 1 <helmut> likely, we currently need it <h01ger> it seems "memory cache" has the biggest speed gain, not sure if we are using it <helmut> h01ger: memory cache only makes sense in fcgi setting <helmut> using file_cache=1 is claimed to boost 50% <weasel> http://www.palfrader.org/volatile/2013-06-13-JxtE9ayM1tI/munin.debian.org <weasel> helmut: duplicating how? <h01ger> weasel, can i attach http://www.palfrader.org/volatile/2013-06-13-123456/munin.debian.org to the bug? <weasel> h01ger: ok
that url showed:
<VirtualHost *:80>
ServerName munin.debian.org
ServerAlias munin2.debian.org
ServerAdmin debian-ad...@debian.org
ErrorLog /var/log/apache2/munin.debian.org-error.log
CustomLog /var/log/apache2/munin.debian.org-access.log privacy
FcgidIOTimeout 120
#
# RewriteEngine on
# RewriteRule ^/(.*)$ https://munin.debian.org/$1 [R]
#</VirtualHost>
#
#<VirtualHost *:443>
# ServerName munin.debian.org
# ServerAlias munin2.debian.org
# ServerAdmin debian-ad...@debian.org
<IfModule mod_userdir.c>
UserDir disabled
</IfModule>
# Use common-ssl-wildcard.tpo
# Use common-ssl-HSTS
#
# ErrorLog /var/log/apache2/munin.debian.org-ssl-error.log
# CustomLog /var/log/apache2/munin.debian.org-ssl-access.log privacy
DocumentRoot /var/cache/munin/www
<DirectoryMatch /var/cache/munin/www>
Options FollowSymLinks
DirectoryIndex index.html
AllowOverride None
Order Allow,Deny
Allow From All
</DirectoryMatch>
#1#SuexecUserGroup munin munin
RewriteEngine On
#RewriteLog /var/log/apache2/rewrite.log
#RewriteLogLevel 5
RewriteRule ^/favicon.ico /var/cache/munin/www/static/favicon.ico [L]
RewriteRule ^/static/(.*) /var/cache/munin/www/static/$1 [L]
RewriteCond %{REQUEST_URI} !^/munin-cgi/munin-cgi-html
RewriteRule ^(/.*\.html)?$ /munin-cgi/munin-cgi-html/$1 [PT]
RewriteRule ^/munin-cgi/munin-cgi-graph/(.*) /$1
RewriteCond %{REQUEST_URI} !^/static
RewriteRule ^/(.*.png)$ /munin-cgi/munin-cgi-graph/$1 [L,PT]
Alias /munin-cgi/munin-cgi-html/static /var/cache/munin/www/static
ScriptAlias /munin-cgi/munin-cgi-graph /var/www/wrappers/munin-cgi-graph
<Location /munin-cgi/munin-cgi-graph>
Options +ExecCGI
SetHandler fcgid-script
Allow From All
</Location>
ScriptAlias /munin-cgi/munin-cgi-html /var/www/wrappers/munin-cgi-html
<Location /munin-cgi/munin-cgi-html>
Options +ExecCGI
SetHandler fcgid-script
Allow From All
</Location>
#<Location /munin-cgi/munin-cgi-html/static>
# Options -ExecCGI
# SetHandler None
#</Location>
<Location />
AuthName "Munin Access"
AuthType Basic
AuthUserFile /etc/apache2/munin-htpasswd
require valid-user
</Location>
</VirtualHost>
<helmut> weasel: dbdir is specified twice when not on menotti
<weasel> indeed. not sure it matters much. I'd just like to kill the
non-menotti munin host
<helmut> h01ger, weasel: could you verify that html cgi really is the issue, by
loading the page with image rendering completely disabled? (iceweasel: edit ->
pref -> content -> load images)
<h01ger> helmut, there are no images there
<h01ger> helmut, and as i can login on menotti feel free to tell me to look for
configs there
<helmut> h01ger: next question: I would like to know whether IO is an issue. I
guess that rendering the main page will load many files. load it and watch the
process with e.g. top/ps for cpu usage. how much does cpu time and wall time
differ?
<helmut> h01ger: does /var/lib/munin/htmlconf.storable exist, and is writeable
by the user running the cgi script?
<h01ger> i dont believe IO is an issue on that host. weasel?
<weasel> I concur.
<h01ger> -rw-r--r-- 1 munin munin 120413523 Jun 13 11:33
/var/lib/munin/htmlconf.storable
<weasel> that's probably a no.
<helmut> => no
<h01ger> so it seems its not writeable by the cgi user
<helmut> making it writeable to the user should get you a massive speedup.
<helmut> hmm. no.
<helmut> the cgi cares to not write to it.
<weasel> weasel@menotti:/var/lib/munin$ echo `date` `ls -l htmlconf.storable `
<weasel> Thu Jun 13 11:43:25 UTC 2013 -rw-rw-r-- 1 munin www-data 119990603 Jun
13 11:37 htmlconf.storable
<weasel> weasel@menotti:/var/lib/munin$ echo `date` `ls -l htmlconf.storable `
<weasel> Thu Jun 13 11:43:33 UTC 2013 -rw-r--r-- 1 munin munin 120329454 Jun 13
11:43 htmlconf.storable
<helmut> keeping it up to date by other means is necessary.
<weasel> and munin re-writes it anyway
<helmut> sorry, bad guess.
<helmut> so. next thing would be profiling. :-/
<TheSnide> sorry, I was busy @work. [ reading backlog ]
<TheSnide> h01ger: olasd suggested that 2.0.6-4 was reading the storable on
*each* http request, which it obviously wrong.
<TheSnide> h01ger: he also suggested that 2.0.14-1~bpo70+1 wasn't affected.
<TheSnide> on the "cron" html, our HTML template engine isn't particularly
speedy. but i guess that most of the time is currently taken by "reading the
storable" part. which should be cached in the fastcgi context.
<weasel> strace suggests munin-cgi-html is not doing any syscalls other than
time() when it's eating all the cpu
<h01ger> weasel, did you try 2.0.14?
<TheSnide> weasel: oh, so you don't seem to hit the issue olasd was speaking
about.
<weasel> open("/home/weasel/munin/db/datafile.storable", O_RDONLY) = 3
<weasel> stat("/var/lib/munin/datafile.storable", {st_mode=S_IFREG|0644,
st_size=18524040, ...}) = 0
<weasel> something in munin-cgi-html ignores dbdir
<weasel> h01ger: before I downgraded for the fixes
<weasel> http://www.palfrader.org/volatile/2013-06-13-123456/munin.tar.gz
<weasel> TheSnide: download that, untar, edit etc/munin.conf to fix paths
<h01ger> so 2.0.14 also was slow
<weasel> then run MUNIN_CONFIG=~/munin/etc/munin.conf
/usr/lib/munin/cgi/munin-cgi-html
<weasel> h01ger: don't remember
<TheSnide> weasel: thx
<weasel> MUNIN_CONFIG=~/munin/etc/munin.conf /usr/lib/munin/cgi/munin-cgi-html
47.73s user 0.66s sys 99% cpu 48.453 total 538 kB max
<weasel> I hope it has everything you need.
<TheSnide> (installing)
<TheSnide> weasel: thx, i do reproduce the issue here.
<weasel> even with HEAD/master/whateveryoucallit?
<TheSnide> weasel: using epel 2.0.14.
<TheSnide> weasel: yet, with 2.0.14, the cpu "tax" is only paid once.
<weasel> hm?
<TheSnide> weasel: as in later calls are quite quick.
<TheSnide> i lied a little in saying "reading the storable".
<TheSnide> as is "reading the storable" == ""reading the storable + compiling
to the datastruct that HTML::Template expects"
<TheSnide> weasel: now, i'm speeding up the compiling part.
<TheSnide> weasel: as in, even when doing it initially. 47s isn't acceptable to
me. And i'm looking @ what difference there is between your setup and mine,
which are of comparable size ( i have 142 nodes on the CGI HTML enabled one )
<TheSnide> weasel: so, to sum up : I think I have everything I need to work on
my side. thx.
<weasel> yay
yay indeed.
cheers,
Holger
signature.asc
Description: This is a digitally signed message part.
