Your message dated Tue, 11 Jul 2017 19:06:34 +0000
with message-id <[email protected]>
and subject line Bug#867982: fixed in strip-nondeterminism 0.036-1
has caused the Debian Bug report #867982,
regarding libfile-stripnondeterminism-perl: [PATCH] Optimize load times of
File::StripNondeterminism
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
867982: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867982
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: libfile-stripnondeterminism-perl
Version: 0.035-2
Severity: normal
Tags: upstream patch
Hi,
Thanks to a very minimalistic performance test case for dh from Adam
Borowski, I realised that dh_strip-nondeterminism accounts for ~4.5%
of the total runtime for a (mostly) no-op dh package build[1]. This
cost applies even to packages for which dh_strip-nondeterminism does
not normalize any files.
Attached, I have included a few patches to optimize the start up time
a bit (Feel free to squash them into a single commit).
In my simplified testing, the start up time is reduced to ~0.075s
(down from ~0.170s) in the no-op case. The impact should be minimal
in the case where dh_strip-nondeterminism actually need to do
anything.
Thanks,
~Niels
[1] dpkg-buildpackage -us -uc -tc -b
>From 5a1ffdd72f55b17b11869b10a85529a0048fec91 Mon Sep 17 00:00:00 2001
From: Niels Thykier <[email protected]>
Date: Mon, 10 Jul 2017 20:19:54 +0000
Subject: [PATCH 1/3] File::SND: Lazy load most handlers
This reduces the start up time of dh_strip_nondeterminism to ~0.10s
from ~0.17s in a "no-op" case.
Signed-off-by: Niels Thykier <[email protected]>
---
lib/File/StripNondeterminism.pm | 73 ++++++++++++++++++++++++-----------------
1 file changed, 42 insertions(+), 31 deletions(-)
diff --git a/lib/File/StripNondeterminism.pm b/lib/File/StripNondeterminism.pm
index c29d4df..c153b0e 100644
--- a/lib/File/StripNondeterminism.pm
+++ b/lib/File/StripNondeterminism.pm
@@ -22,16 +22,9 @@ use strict;
use warnings;
use POSIX qw(tzset);
-use File::StripNondeterminism::handlers::ar;
-use File::StripNondeterminism::handlers::cpio;
-use File::StripNondeterminism::handlers::gettext;
-use File::StripNondeterminism::handlers::gzip;
-use File::StripNondeterminism::handlers::jar;
use File::StripNondeterminism::handlers::javadoc;
use File::StripNondeterminism::handlers::pearregistry;
-use File::StripNondeterminism::handlers::png;
use File::StripNondeterminism::handlers::javaproperties;
-use File::StripNondeterminism::handlers::zip;
our($VERSION, $canonical_time, $clamp_time);
@@ -59,29 +52,29 @@ sub get_normalizer_for_file($) {
# ar
if (m/\.a$/ && _get_file_type($_) =~ m/ar archive/) {
- return \&File::StripNondeterminism::handlers::ar::normalize;
+ return _handler('ar');
}
# cpio
if (m/\.cpio$/ && _get_file_type($_) =~ m/cpio archive/) {
- return \&File::StripNondeterminism::handlers::cpio::normalize;
+ return _handler('cpio');
}
# gettext
if (m/\.g?mo$/ && _get_file_type($_) =~ m/GNU message catalog/) {
- return
\&File::StripNondeterminism::handlers::gettext::normalize;
+ return _handler('gettext');
}
# gzip
if (m/\.(gz|dz)$/ && _get_file_type($_) =~ m/gzip compressed data/) {
- return \&File::StripNondeterminism::handlers::gzip::normalize;
+ return _handler('gzip');
}
# jar
if (m/\.(jar|war|hpi|apk)$/
&& _get_file_type($_) =~ m/(Java|Zip) archive data/) {
- return \&File::StripNondeterminism::handlers::jar::normalize;
+ return _handler('jar');
}
# javadoc
if (m/\.html$/
&&
File::StripNondeterminism::handlers::javadoc::is_javadoc_file($_)) {
- return
\&File::StripNondeterminism::handlers::javadoc::normalize;
+ return _handler('javadoc');
}
# pear registry
if (
@@ -89,11 +82,11 @@ sub get_normalizer_for_file($) {
&&
File::StripNondeterminism::handlers::pearregistry::is_registry_file(
$_)
) {
- return
\&File::StripNondeterminism::handlers::pearregistry::normalize;
+ return _handler('pearregistry');
}
# PNG
if (m/\.png$/ && _get_file_type($_) =~ m/PNG image data/) {
- return \&File::StripNondeterminism::handlers::png::normalize;
+ return _handler('png');
}
# pom.properties, version.properties
if (
@@ -101,32 +94,50 @@ sub get_normalizer_for_file($) {
&&
File::StripNondeterminism::handlers::javaproperties::is_java_properties_file(
$_)
) {
- return
-
\&File::StripNondeterminism::handlers::javaproperties::normalize;
+ return _handler('javaproperties');
}
# zip
if (m/\.(zip|pk3|epub|whl|xpi|htb|zhfst|par)$/
&& _get_file_type($_) =~ m/Zip archive data|EPUB document/) {
- return \&File::StripNondeterminism::handlers::zip::normalize;
+ return _handler('zip');
}
return undef;
}
-our %typemap = (
- ar => \&File::StripNondeterminism::handlers::ar::normalize,
- cpio => \&File::StripNondeterminism::handlers::cpio::normalize,
- gettext => \&File::StripNondeterminism::handlers::gettext::normalize,
- gzip => \&File::StripNondeterminism::handlers::gzip::normalize,
- jar => \&File::StripNondeterminism::handlers::jar::normalize,
- javadoc => \&File::StripNondeterminism::handlers::javadoc::normalize,
- pearregistry =>
\&File::StripNondeterminism::handlers::pearregistry::normalize,
- png => \&File::StripNondeterminism::handlers::png::normalize,
- javaproperties =>
\&File::StripNondeterminism::handlers::javaproperties::normalize,
- zip => \&File::StripNondeterminism::handlers::zip::normalize,
+
+our %HANDLER_CACHE;
+our %KNOWN_HANDLERS = (
+ ar => 1,
+ cpio => 1,
+ gettext => 1,
+ gzip => 1,
+ jar => 1,
+ javadoc => 1,
+ pearregistry => 1,
+ png => 1,
+ javaproperties => 1,
+ zip => 1,
);
+
+sub _handler {
+ my ($handler_name) = @_;
+ return $HANDLER_CACHE{$handler_name} if
exists($HANDLER_CACHE{$handler_name});
+ die("Unknown handler: ${handler_name}\n")
+ if not exists($KNOWN_HANDLERS{$handler_name});
+ my $pkg = "File::StripNondeterminism::handlers::${handler_name}";
+ my $mod = "File/StripNondeterminism/handlers/${handler_name}.pm";
+ my $sub_name = "${pkg}::normalize";
+ require $mod;
+ no strict 'refs';
+ if (not defined &{$sub_name}) {
+ die("Internal error: No handler for $handler_name!?\n");
+ }
+ my $handler = \&{$sub_name};
+ return $HANDLER_CACHE{$handler_name} = $handler;
+}
+
sub get_normalizer_by_name($) {
- $_ = shift;
- return $typemap{$_};
+ return _handler(shift);
}
1;
--
2.13.2
>From 93c46d5976126573c6621dcea064aa6f65dcea52 Mon Sep 17 00:00:00 2001
From: Niels Thykier <[email protected]>
Date: Mon, 10 Jul 2017 20:26:25 +0000
Subject: [PATCH 2/3] File::SND: Lazy load remaining handlers
With this and the previous patch, the start up time of
dh_strip_nondeterminism is now reduced to ~0.075s (from a start of
~0.17s) in the noop case.
Signed-off-by: Niels Thykier <[email protected]>
---
lib/File/StripNondeterminism.pm | 33 +++++++++++++++------------------
1 file changed, 15 insertions(+), 18 deletions(-)
diff --git a/lib/File/StripNondeterminism.pm b/lib/File/StripNondeterminism.pm
index c153b0e..b50d41e 100644
--- a/lib/File/StripNondeterminism.pm
+++ b/lib/File/StripNondeterminism.pm
@@ -22,9 +22,6 @@ use strict;
use warnings;
use POSIX qw(tzset);
-use File::StripNondeterminism::handlers::javadoc;
-use File::StripNondeterminism::handlers::pearregistry;
-use File::StripNondeterminism::handlers::javaproperties;
our($VERSION, $canonical_time, $clamp_time);
@@ -72,29 +69,29 @@ sub get_normalizer_for_file($) {
return _handler('jar');
}
# javadoc
- if (m/\.html$/
- &&
File::StripNondeterminism::handlers::javadoc::is_javadoc_file($_)) {
- return _handler('javadoc');
+ if (m/\.html$/) {
+ # Loading the handler forces the load of the javadoc package as
well
+ my $handler = _handler('javadoc');
+ return $handler
+ if
File::StripNondeterminism::handlers::javadoc::is_javadoc_file($_);
}
# pear registry
- if (
- m/\.reg$/
- &&
File::StripNondeterminism::handlers::pearregistry::is_registry_file(
- $_)
- ) {
- return _handler('pearregistry');
+ if (m/\.reg$/) {
+ # Loading the handler forces the load of the pearregistry
package as well
+ my $handler = _handler('pearregistry');
+ return $handler
+ if
File::StripNondeterminism::handlers::pearregistry::is_registry_file($_);
}
# PNG
if (m/\.png$/ && _get_file_type($_) =~ m/PNG image data/) {
return _handler('png');
}
# pom.properties, version.properties
- if (
- m/\.properties$/
- &&
File::StripNondeterminism::handlers::javaproperties::is_java_properties_file(
- $_)
- ) {
- return _handler('javaproperties');
+ if (m/\.properties$/) {
+ # Loading the handler forces the load of the javaproperties
package as well
+ my $handler = _handler('javaproperties');
+ return $handler
+ if
File::StripNondeterminism::handlers::javaproperties::is_java_properties_file($_);
}
# zip
if (m/\.(zip|pk3|epub|whl|xpi|htb|zhfst|par)$/
--
2.13.2
>From a533527c927f79877c83bcfb90c66f786990cb69 Mon Sep 17 00:00:00 2001
From: Niels Thykier <[email protected]>
Date: Mon, 10 Jul 2017 20:30:56 +0000
Subject: [PATCH 3/3] File::SND: Apply perltidy
Signed-off-by: Niels Thykier <[email protected]>
---
lib/File/StripNondeterminism.pm | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/lib/File/StripNondeterminism.pm b/lib/File/StripNondeterminism.pm
index b50d41e..8ee0b94 100644
--- a/lib/File/StripNondeterminism.pm
+++ b/lib/File/StripNondeterminism.pm
@@ -73,14 +73,16 @@ sub get_normalizer_for_file($) {
# Loading the handler forces the load of the javadoc package as
well
my $handler = _handler('javadoc');
return $handler
- if
File::StripNondeterminism::handlers::javadoc::is_javadoc_file($_);
+ if
File::StripNondeterminism::handlers::javadoc::is_javadoc_file($_);
}
# pear registry
if (m/\.reg$/) {
- # Loading the handler forces the load of the pearregistry
package as well
+ # Loading the handler forces the load of the pearregistry package as
well
my $handler = _handler('pearregistry');
return $handler
- if
File::StripNondeterminism::handlers::pearregistry::is_registry_file($_);
+ if
+
File::StripNondeterminism::handlers::pearregistry::is_registry_file(
+ $_);
}
# PNG
if (m/\.png$/ && _get_file_type($_) =~ m/PNG image data/) {
@@ -88,10 +90,12 @@ sub get_normalizer_for_file($) {
}
# pom.properties, version.properties
if (m/\.properties$/) {
- # Loading the handler forces the load of the javaproperties
package as well
+ # Loading the handler forces the load of the javaproperties package as
well
my $handler = _handler('javaproperties');
return $handler
- if
File::StripNondeterminism::handlers::javaproperties::is_java_properties_file($_);
+ if
+
File::StripNondeterminism::handlers::javaproperties::is_java_properties_file(
+ $_);
}
# zip
if (m/\.(zip|pk3|epub|whl|xpi|htb|zhfst|par)$/
@@ -101,7 +105,6 @@ sub get_normalizer_for_file($) {
return undef;
}
-
our %HANDLER_CACHE;
our %KNOWN_HANDLERS = (
ar => 1,
@@ -118,14 +121,16 @@ our %KNOWN_HANDLERS = (
sub _handler {
my ($handler_name) = @_;
- return $HANDLER_CACHE{$handler_name} if
exists($HANDLER_CACHE{$handler_name});
+ return $HANDLER_CACHE{$handler_name}
+ if exists($HANDLER_CACHE{$handler_name});
die("Unknown handler: ${handler_name}\n")
- if not exists($KNOWN_HANDLERS{$handler_name});
+ if not exists($KNOWN_HANDLERS{$handler_name});
my $pkg = "File::StripNondeterminism::handlers::${handler_name}";
my $mod = "File/StripNondeterminism/handlers/${handler_name}.pm";
my $sub_name = "${pkg}::normalize";
require $mod;
no strict 'refs';
+
if (not defined &{$sub_name}) {
die("Internal error: No handler for $handler_name!?\n");
}
--
2.13.2
--- End Message ---
--- Begin Message ---
Source: strip-nondeterminism
Source-Version: 0.036-1
We believe that the bug you reported is fixed in the latest version of
strip-nondeterminism, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Chris Lamb <[email protected]> (supplier of updated strip-nondeterminism package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Tue, 11 Jul 2017 19:41:39 +0100
Source: strip-nondeterminism
Binary: libfile-stripnondeterminism-perl strip-nondeterminism
dh-strip-nondeterminism
Architecture: source
Version: 0.036-1
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks
<[email protected]>
Changed-By: Chris Lamb <[email protected]>
Description:
dh-strip-nondeterminism - file non-deterministic information stripper —
Debhelper add-on
libfile-stripnondeterminism-perl - file non-deterministic information stripper
— Perl module
strip-nondeterminism - file non-deterministic information stripper —
stand-alone tool
Closes: 867982 868064
Changes:
strip-nondeterminism (0.036-1) unstable; urgency=medium
.
[ Niels Thykier ]
* File::StripNondeterminism:
+ Optimize load times of File::StripNondeterminism.
dh_strip-nondeterminism was accounting for ~4.5% of the total runtime
for a (mostly) no-op dh package build. This cost applied even to
packages for which dh_strip-nondeterminism did not normalise any files.
(Closes: #867982)
+ Apply perltidy.
* dh_strip_nondeterminism: Don't assume tmpdir() exists. This will become
more relevant for Debhelper compat level 11's `dh_auto_install` which will
only create destination directories it needs. (Closes: #868064)
Checksums-Sha1:
10e9c83a7f4102f3c1d973d5410a948464ed71b2 2526 strip-nondeterminism_0.036-1.dsc
0d046099e4c766a491a5b19f00ca2f1953e3cbb0 170779
strip-nondeterminism_0.036.orig.tar.gz
67e40d7236db74ce59aea67c37262763c52ba9c4 11760
strip-nondeterminism_0.036-1.debian.tar.xz
52d1b963acf9c8d4fc1420f4ec8e108eaf7d85da 6446
strip-nondeterminism_0.036-1_amd64.buildinfo
Checksums-Sha256:
ca16a5851b13719ecfe1918a3bd59601719b809d01adbf7f591f3f86e95d1cc7 2526
strip-nondeterminism_0.036-1.dsc
cbc28046eb825de4139467ceb09921f7f065b23d626743702852846ad3e4200d 170779
strip-nondeterminism_0.036.orig.tar.gz
0d8f4e940c4c9a4be8e7e201af82db5d277f304d60e09a2377e3f48b933b160f 11760
strip-nondeterminism_0.036-1.debian.tar.xz
44af0432c8cd20eb69cbe04a3b2fb4aac45c83bacbba7a080f898f33b1443459 6446
strip-nondeterminism_0.036-1_amd64.buildinfo
Files:
243878be6b338d014e5048942579a6bd 2526 devel optional
strip-nondeterminism_0.036-1.dsc
19e828ca48bb657dcee003295053976a 170779 devel optional
strip-nondeterminism_0.036.orig.tar.gz
539dc0736479c649abf59f438d0543d6 11760 devel optional
strip-nondeterminism_0.036-1.debian.tar.xz
1266e5f960cfb8dad508ba40702888a2 6446 devel optional
strip-nondeterminism_0.036-1_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAlllHC0ACgkQHpU+J9Qx
HljrcRAAso5OjS8u6x117P2EGEVerXb0YI0JcmjatMvIGc18jB23Gt7Nfzgcoj4q
aI+1QIWpa+4c92qeEhF42FdEDg8bJkkFdPK6RsdZ2MGHoberlQW/BID+2Ssqf86t
fSVu4kNbESN0VYH4dwWpajtcuiQSj0fjJQ0IJ7arXUtXPpWbiaFuuSZHbyJUNyWw
qg6dt5W5B3hk4Xx+SOpGLdeXPTc0HGJlzbZWMVEvXnmRdGxcdDpF9TsR9SY6bFQR
ylCV/mg0soXS/zpwdeXTfXZ089k0rvVIWgSXuhH1i6FycJbwyA4DuKsSsR1EY/8h
nucA5OXx5kZ5tC5wN8IM6cReMiSLsW6pOPWcCxWE195zWOW4aO0XjC8Pe+LfNHFh
c8+g9bk/V3SKOKFNW2NX5Il24cC5aiqyL66mhvq8HFqXrBUmDWfcwqcB9thu8y9O
MubhdozzvZhSHqDI4/YjASAkb5sxP36VeBKCytbFhzD4tB1nA4E7zVyGKKgdoktp
LpAbdw5gyg4/e8s9YbHpEk9rOJT+ZjvTdz0mVtlipAn1ZAQjUin631AO41HMpXi8
P56XZ5yplcsrZeNkvjNEjCfHHS2EgYQl1Fq800enbX/CXwUfw7GKJvHO7ii13QTS
YhjxyogTpWC8l8H7reAonVD5XMTkm5A54fLeGi2CNcVOljHOJ1o=
=JD/z
-----END PGP SIGNATURE-----
--- End Message ---
