The httpd-supplied write callback is the leak culprit under Perl
5.16.3. undef-ing it immediately after use keeps a repeated
"git fetch" loop from monotonically increasing memory and FD use
on the Perl shipped with RHEL/CentOS 7.x.
Other endpoints tested showed no increase in memory use under
constant load with "ab -HAccept-Encoding:gzip -k", including the
async psgi_qx code path used by $INBOX_URL/$OBJECT_ID/s/ via
SolverGit module.
---
Note: I initially tried this change, but thought it only slowed
down the leaking because I had not yet discovered the
workaround in commit cd71a869c7e9c811
("ds: workaround a memory leak in Perl 5.16.x").
Now that both leaks are worked around, memory usage is completely
flat when repeating a single request of any type with gzip-enabled.
lib/PublicInbox/Qspawn.pm | 4 ++++
script/public-inbox-httpd | 8 +-------
2 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/lib/PublicInbox/Qspawn.pm b/lib/PublicInbox/Qspawn.pm
index 5a30064..cb3dc51 100644
--- a/lib/PublicInbox/Qspawn.pm
+++ b/lib/PublicInbox/Qspawn.pm
@@ -281,6 +281,10 @@ sub psgi_return {
$buf, $filter);
$wcb->($r);
}
+
+ # Workaround a leak under Perl 5.16.3 when combined with
+ # Plack::Middleware::Deflater:
+ $wcb = undef;
};
$limiter ||= $def_limiter ||= PublicInbox::Qspawn::Limiter->new(32);
my $start_cb = sub { # may run later, much later...
diff --git a/script/public-inbox-httpd b/script/public-inbox-httpd
index 9b869f9..b2464f4 100755
--- a/script/public-inbox-httpd
+++ b/script/public-inbox-httpd
@@ -24,13 +24,7 @@ my $refresh = sub {
my $www = PublicInbox::WWW->new;
$www->preload;
$app = builder {
- # Perl 5.16.3 leaks in our "push" response code path
- # (e.g. Qspawn) due to something in
- # Plack::Util::response_cb, regardless of whether the
- # client is sending Accept-Encoding:gzip requests.
- # perl5180delta documents many leak fixes, so assume
- # 5.18+ is safe for now and bump the check as-need:
- $] >= 5.018000 and eval {
+ eval {
enable 'Deflater',
content_type => [ qw(
text/html
--
EW
--
unsubscribe: meta+unsubscr...@public-inbox.org
archive: https://public-inbox.org/meta/
