Re: Dead locks in t/runtests

[Niels Thykier]

On 2011-08-18 21:24, Niels Thykier wrote:
> Hi
> 
> I have been working a bit on debugging the dead-lock issue in t/runtests
> and I have found something interesting.  Unfortunately I cannot
> pin-point the issue...
> 
> When I see a dead-lock it, I noticed that we tend to have two lintian
> proceses (ps: [lintian] ) and two /usr/bin/sort.  Do we know
> that IPC::Run/Lintian::Command is thread-safe?
> 
> ~Niels
> 
> 

As you may have noticed, I have decided to follow that hunch and removed
all (read: "both") uses of Lintian::Command/IPC::Run in t/runtests.
  So far I have not experienced any dead-locks.  It is not a proof that
it now works, but I am very pleased with apparent sudden absence of
dead-locks ,,, even when running with 16 threads (on a related note; the
runtime does not really improve after 8 threads). :)

~Niels


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4d7a25.40...@thykier.net

[SCM] Debian package checker branch, master, updated. 2.5.2-41-gf85156c

[Niels Thykier]

The following commit has been merged in the master branch:
commit f85156c2469de1f9637f444a4c4e8dd9396ee106
Author: Niels Thykier 
Date:   Thu Aug 18 22:23:18 2011 +0200

Do not use Lintian::Command (or IPC::Run) in t/runtests

Testing suggested that the dead-lock test dead-lock appears to be
related to the lintian sub-process and only spawn (using IPC::Run)
started that process.

diff --git a/t/runtests b/t/runtests
index 9ddd7de..133cf38 100755
--- a/t/runtests
+++ b/t/runtests
@@ -58,7 +58,6 @@ BEGIN {
 
 use lib "$ENV{'LINTIAN_ROOT'}/lib";
 
-use Lintian::Command qw(spawn);
 use Util;
 
 # --- Global configuration
@@ -441,24 +440,31 @@ sub _builder_tests {
 sub run_lintian {
 my ($testdata, $file, $out) = @_;
 msg_print 'testing... ';
-my $opts = { err => "$out", fail => 'never' };
-my $status;
-my @options = split(' ', $testdata->{options}//'');
+my $status = 0;
+# Quote (A test use -L <=, which blows up if we don't... plus it is safer 
that way)
+my @options = map { quotemeta $_ } split(' ', $testdata->{options}//'');
+my $cmd;
+my $ret;
 unshift(@options, '--allow-root', '--no-cfg');
 unshift(@options, '--profile', $testdata->{profile}) if 
$testdata->{profile};
-if ($testdata->{sort}) {
-   $status = spawn($opts, [ $LINTIAN, @options, $file ], '|', [ 'sort' ]);
+$cmd = "$LINTIAN " . join(' ', @options). " $file 2>&1";
+if (open my $in, '-|', $cmd) {
+   my @data = <$in>;
+   close $in;
+   $status = ($? >> 8) & 255;
+   @data = sort @data if $testdata->{sort};
+   open my $fd, '>', $out or fail "opening $out: $!";
+   print $fd $_ for @data;
+   close $fd or fail "closing $out: $!";
 } else {
-   $status = spawn($opts, [ $LINTIAN, @options, $file ]);
+   # Set it to 255 so it fails below
+   $status = 255;
 }
+
 unless ($status == 0 or $status == 1) {
msg_print "FAILED:\n";
fail("$LINTIAN @options $file exited with status $status\n");
 }
-open(my $fd, '>>', "$out")
-   or fail("cannot append to $out: $!");
-print $fd ${ $opts->{out} };
-close $fd or fail "closing $out: $!";
 return 1;
 }
 

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qu9e6-0005ne...@vasks.debian.org

[SCM] Debian package checker branch, master, updated. 2.5.2-41-gf85156c

[Niels Thykier]

The following commit has been merged in the master branch:
commit ccf06f9f483e5a2c1855ca93c6458af683f419db
Author: Niels Thykier 
Date:   Thu Aug 18 22:20:18 2011 +0200

flush messages via a sub rather than sending messages to self

The message handler now uses a sub to flush messages rather than
sending itself a "flush"-message.  This has the advantage of not
involving an enqueue + a dequeue on a threaded queue.

diff --git a/t/runtests b/t/runtests
index c546e5f..9ddd7de 100755
--- a/t/runtests
+++ b/t/runtests
@@ -836,7 +836,26 @@ sub msg_print {
 $MSG_Q->enqueue(\%msg);
 }
 
+sub _flush {
+my ($thrs, $id, $length) = @_;
+print (' 'x$length,"\r");
+while (my $m = shift @{$thrs->{$id}}) {
+   print $m;
+}
+print "\n";
+delete $thrs->{$id};
+}
+
 sub msg_queue_handler {
+# if _msg_qh fails
+eval {
+   _msg_qh();
+};
+die "Error (msg_queue_handler): $@\n" if $@;
+die "_msg_qh returned!?\n";
+}
+
+sub _msg_qh {
 my %thrs;
 my $length = 0;
 
@@ -845,18 +864,13 @@ sub msg_queue_handler {
# master thread calls msg_flush to flush all messages
if ($id == 0) {
for my $tid (keys %thrs) {
-   my %msg = (id => $tid);
-   $MSG_Q->insert(0, \%msg);
+   _flush(\%thrs, $tid, $length);
}
+   $length = 0;
} else {
if (!exists($msg->{'msg'}) && exists($thrs{$id})) {
-   print (' 'x$length,"\r");
+   _flush(\%thrs, $id, $length);
$length = 0;
-   while (my $m = shift @{$thrs{$id}}) {
-   print $m;
-   }
-   print "\n";
-   delete $thrs{$id};
} elsif (exists($msg->{'msg'})) {
$thrs{$id} = []
unless (exists($thrs{$id}));
@@ -870,7 +884,10 @@ sub msg_queue_handler {
}
 
# Insert a flush request, if needed
-   $MSG_Q->insert(0, { id => $id }) if $flush;
+   if ($flush) {
+   _flush(\%thrs, $id, $length);
+   $length = 0;
+   }
}
}
 

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qu9e0-0005io...@vasks.debian.org

Dead locks in t/runtests

[Niels Thykier]

Hi

I have been working a bit on debugging the dead-lock issue in t/runtests
and I have found something interesting.  Unfortunately I cannot
pin-point the issue...

When I see a dead-lock it, I noticed that we tend to have two lintian
proceses (ps: [lintian] ) and two /usr/bin/sort.  Do we know
that IPC::Run/Lintian::Command is thread-safe?

~Niels


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4d66f6.9040...@thykier.net

Re: Lintian.d.o upgraded to 2.5.2

[Russ Allbery]

Niels Thykier  writes:

> Upgrading was slightly complicated by two things;
> reporting/checkout-release did not quite work for me (mostly due to a
> merge-conflict).  I think a "git merge -s recursive -X theirs" could
> help the automation here.

Yeah, I don't think I usually actually use that script (I just do what it
does manually, since I'm used to it), so it's probably bit-rotted a bit.

> The second upgrade issue was caused by debhelper (or dh_testdir at
> least) not being present on lintian.d.o, so I had to workaround that
> (trivial, but still annoying).

Oh, we should request that be installed.

>   I do not know if it is custom to run the testsuite on lintian.d.o; I
> didn't - mostly because they blow up due to lacking
> debhelper^Wdependencies (Test::Pod and debhelper etc.).  If we do not
> want to run them, we should probably either make a special target in
> d/rules for lintian.d.o or/and use DEB_BUILD_OPTIONS=nocheck in
> reporting/checkout-release (and /srv/lintian.debian.org/README).

No, I never run the tests there, just build the documentation so that it
will show up on the web site.  Since it's running stable, I would actually
expect some of the tests to fail if they rely on functionality (to build
the test packages, not in Lintian itself, obviously) added since stable,
so I'm not sure running the tests is all that useful.  Although now that I
think about it some more, I'm not sure what functionality we'd rely on.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87mxf6bp30@windlord.stanford.edu

[SCM] Debian package checker branch, master, updated. 2.5.2-35-g5606b4f

[Niels Thykier]

The following commit has been merged in the master branch:
commit 5606b4ff5b6fe5ddc0f58b81d43c6dc57c6a9557
Author: Niels Thykier 
Date:   Thu Aug 18 10:57:41 2011 +0200

Mentioned that the umask problem with the tests are now fixed

This has been fixed for a while (tested with  and 0077).

diff --git a/debian/changelog b/debian/changelog
index 6149a4d..6ae9075 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -46,6 +46,10 @@ lintian (2.5.3) UNRELEASED; urgency=low
 + [JW] Use LC_ALL rather than LANG, since LC_ALL overrules
   LANG.
 
+  * t/*:
++ [JW,NT] Fixed test failures caused by unpacking the source
+  with a umask different from 0022.  (Closes: #637649)
+
  -- Niels Thykier   Wed, 10 Aug 2011 21:09:24 +0200
 
 lintian (2.5.2) unstable; urgency=low

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qtyrp-00078f...@vasks.debian.org

[SCM] Debian package checker branch, master, updated. 2.5.2-36-gd9b8a0d

[Niels Thykier]

The following commit has been merged in the master branch:
commit d9b8a0d9de50e709b1702dfcbb86bd6fcc21fe71
Author: Niels Thykier 
Date:   Thu Aug 18 13:46:13 2011 +0200

Added missing import statement in reporting/harness

diff --git a/debian/changelog b/debian/changelog
index 6ae9075..99cc41a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -46,6 +46,9 @@ lintian (2.5.3) UNRELEASED; urgency=low
 + [JW] Use LC_ALL rather than LANG, since LC_ALL overrules
   LANG.
 
+  * reporting/harness:
++ [NT] Added missing import statement.
+
   * t/*:
 + [JW,NT] Fixed test failures caused by unpacking the source
   with a umask different from 0022.  (Closes: #637649)
diff --git a/reporting/harness b/reporting/harness
index 81c8ff0..dcece30 100755
--- a/reporting/harness
+++ b/reporting/harness
@@ -64,6 +64,7 @@ use vars qw($LINTIAN_ROOT $LINTIAN_LAB $LINTIAN_ARCHIVEDIR 
$LINTIAN_DIST
 # import perl libraries
 unshift @INC, "$LINTIAN_ROOT/lib";
 require Read_pkglists;
+import Read_pkglists;
 require Util;
 
 # turn file buffering off

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qu16k-0002k4...@vasks.debian.org

[SCM] Debian package checker branch, master, updated. 2.5.2-37-gda49213

[Niels Thykier]

The following commit has been merged in the master branch:
commit da492134d7d154823eb1afa052f4798139f0a447
Author: Niels Thykier 
Date:   Thu Aug 18 13:52:57 2011 +0200

Removed a duplicate "allowed to" in doc/lintian.xml

diff --git a/debian/changelog b/debian/changelog
index 99cc41a..aaa308e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -28,6 +28,8 @@ lintian (2.5.3) UNRELEASED; urgency=low
 + [JW] Use LC_ALL rather than LANG, since LC_ALL overrules
   LANG.
 
+  * doc/lintian.xml:
++ [NT] Removed duplicate "allowed to" in same sentence.
   * doc/README.developers:
 + [NT] New file.  Thanks to Jeremiah C. Foster for the initial
   contribution.
diff --git a/doc/lintian.xml b/doc/lintian.xml
index 1716f74..c3c37aa 100644
--- a/doc/lintian.xml
+++ b/doc/lintian.xml
@@ -702,7 +702,7 @@ foo source: configure-generated-file-in-source config.cache
 Enable-Tags-From-Check and Disable-Tags-From-Check.
   
   
-It is allowed to allowed to list a tag in Enable-Tags or
+It is allowed to list a tag in Enable-Tags or
 Disable-Tags even if the check that provides this tag is
 listed in the Disable-Tags-From-Check or
 Enable-Tags-From-Check field. In case of conflict,

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qu1co-0002jy...@vasks.debian.org

[SCM] Debian package checker branch, master, updated. 2.5.2-39-g45b90f1

[Niels Thykier]

The following commit has been merged in the master branch:
commit 45b90f13d3a5017bd4ddac950a2c351fa288b204
Author: Niels Thykier 
Date:   Thu Aug 18 17:18:57 2011 +0200

Inform the user if overrides have been ignored

Lintian will now print an informal message if it sees an override
for a tag that was marked non-overridable.  This message can be
silenced with "--quiet" and extended with "--verbose".

diff --git a/debian/changelog b/debian/changelog
index aaa308e..0174e89 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -34,6 +34,12 @@ lintian (2.5.3) UNRELEASED; urgency=low
 + [NT] New file.  Thanks to Jeremiah C. Foster for the initial
   contribution.
 
+  * frontend/lintian:
++ [NT] Lintian now mentions if overrides may have been ignored.
+  This happens if a package contains an override for a tag
+  that has been marked "non-overridable" by the current vendor
+  profile.
+
   * lib/Util.pm:
 + [JW] Use LC_ALL rather than LANG, since LC_ALL overrules
   LANG.
@@ -43,6 +49,8 @@ lintian (2.5.3) UNRELEASED; urgency=low
 + [JW,NT] Replace use of fail with croak.  (Closes: #637741)
   * lib/Lintian/Profile.pm:
 + [NT] Reject profiles containing an unknown field.
+  * lib/Lintian/Tags.pm:
++ [NT] Track which tags had overrides that have been dropped.
 
   * private/*:
 + [JW] Use LC_ALL rather than LANG, since LC_ALL overrules
diff --git a/doc/lintian.xml b/doc/lintian.xml
index c3c37aa..1c6ced4 100644
--- a/doc/lintian.xml
+++ b/doc/lintian.xml
@@ -740,8 +740,10 @@ foo source: configure-generated-file-in-source config.cache
   

  Either "Yes" or "No", which decides whether these
-  tags can be overriden.  Overrides for these tags
-  will silently be ignored.
+  tags can be overriden.  Lintian will print an
+  informal message if it sees an override for a tag
+  marked as non-overridable (except if --quiet is
+  passed).

   
 
diff --git a/frontend/lintian b/frontend/lintian
index 9f867c1..595539c 100755
--- a/frontend/lintian
+++ b/frontend/lintian
@@ -1266,6 +1266,19 @@ if ($action eq 'check' and not $opt{'no-override'} and 
not $opt{'show-overrides'
 }
 }
 
+my $ign_over = $TAGS->ignored_overrides;
+if (keys %$ign_over) {
+msg('Some overrides were ignored, since the tags were marked 
"non-overridable".');
+if ($opt{'verbose'}) {
+   v_msg('The following tags were "non-overridable" and had at least one 
override');
+   foreach my $tag (sort keys %$ign_over) {
+   v_msg("  - $tag");
+   }
+} else {
+   msg('Use --verbose for more information.');
+}
+}
+
 # }}}
 
 
diff --git a/lib/Lintian/Tags.pm b/lib/Lintian/Tags.pm
index 78e47a6..57e3184 100644
--- a/lib/Lintian/Tags.pm
+++ b/lib/Lintian/Tags.pm
@@ -154,6 +154,7 @@ sub new {
 display_source   => {},
 files=> {},
 non_overridable_tags => {},
+ignored_overrides=> {},
 only_issue   => {},
 respect_display  => 1,
 show_experimental=> 0,
@@ -537,7 +538,7 @@ file cannot be opened.
 
 sub file_overrides {
 my ($self, $overrides) = @_;
-my $ignored = $self->{non_overridable_tags};
+my $noover = $self->{non_overridable_tags};
 unless (defined $self->{current}) {
 die 'no current file when adding overrides';
 }
@@ -582,7 +583,10 @@ sub file_overrides {
 }
 next unless $found;
 }
-next if $ignored->{$tag};
+if ( $noover->{$tag} ) {
+$self->{ignored_overrides}{$tag}++;
+next;
+}
 $extra = '' unless defined $extra;
 $info->{overrides}{$tag}{$extra} = 0;
 } else {
@@ -746,13 +750,26 @@ Marks all tags (given as arguments) for non-overridable.
 
 sub non_overridable_tags {
 my ($self, @tags) = @_;
-my $ignored = $self->{non_overridable_tags};
+my $noover = $self->{non_overridable_tags};
 foreach my $tag (@tags){
-$ignored->{$tag} = 1;
+$noover->{$tag} = 1;
 }
 return 1;
 }
 
+=item ignored_overrides()
+
+Returns a hash of tags, for which overrides have been ignored.  The
+keys are tag names and the value is the number of overrides that has
+been ignored.
+
+=cut
+
+sub ignored_overrides {
+my ($self) = @_;
+return $self->{ignored_overrides};
+}
+
 =item respect_display_level([BOOL])
 
 Whether or not the display level should be considered for

-- 
Debian package checker


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/e1qu4rt-0001eb...@vasks.debian.org

[SCM] Debian package checker branch, master, updated. 2.5.2-39-g45b90f1

[Niels Thykier]

The following commit has been merged in the master branch:
commit a6144a09f2ece3ffb36bac669de674c061181e10
Author: Niels Thykier 
Date:   Thu Aug 18 16:28:21 2011 +0200

Renamed ignore-overrides to non-overridable-tags

Renamed variables and methods to better reflect their purpose.

diff --git a/frontend/lintian b/frontend/lintian
index 0625b61..9f867c1 100755
--- a/frontend/lintian
+++ b/frontend/lintian
@@ -796,11 +796,11 @@ if ($opt{'LINTIAN_PROFILE'}) {
 my $profile = Lintian::Profile->new($opt{'LINTIAN_PROFILE'},
[@prof_inc, 
"$opt{'LINTIAN_ROOT'}/profiles"]);
 my @ptags = $profile->tags;
-my @ign_overrides = $profile->ignored_overrides;
+my @non_overridable = $profile->non_overridable_tags;
 my $severities = $profile->severity_changes;
 v_msg('Using profile ' . $profile->name . '.');
 $TAGS->only(@ptags) if @ptags;
-$TAGS->ignore_overrides(@ign_overrides) if @ign_overrides;
+$TAGS->non_overridable_tags(@non_overridable) if @non_overridable;
 while ( my ($tagname, $severity) = each(%$severities) ){
my $tag = Lintian::Tag::Info->new($tagname);
$tag->set_severity($severity);
diff --git a/lib/Lintian/Profile.pm b/lib/Lintian/Profile.pm
index 76b5fcc..5e4739d 100644
--- a/lib/Lintian/Profile.pm
+++ b/lib/Lintian/Profile.pm
@@ -129,12 +129,12 @@ sub new {
 if $name =~ m,^/,o or $name =~ m/\./o;
 _load_checks() unless %TAG_MAP;
 my $self = {
-'parent-map'=> {},
-'parents'   => [],
-'profile-path'  => $ppath,
-'enabled-tags'  => {},
-'ignored-overrides' => {},
-'severity-changes'  => {},
+'parent-map'   => {},
+'parents'  => [],
+'profile-path' => $ppath,
+'enabled-tags' => {},
+'non-overridable-tags' => {},
+'severity-changes' => {},
 };
 $self = bless $self, $type;
 $profile = $self->find_profile($name);
@@ -189,7 +189,7 @@ sub severity_changes {
 return $self->{'severity-changes'};
 }
 
-=item $prof->ignored_overrides
+=item $prof->non_overridable_tags
 
 List of tags that has been marked as non-overridable.
 
@@ -197,9 +197,9 @@ Note: This list nor its contents should be modified.
 
 =cut
 
-sub ignored_overrides {
+sub non_overridable_tags {
 my ($self) = @_;
-return keys %{ $self->{'ignored-overrides'} };
+return keys %{ $self->{'non-overridable-tags'} };
 }
 
 =item Lintian::Profile->find_profile($pname, @dirs), 
$prof->find_profile($pname[, @dirs])
@@ -292,7 +292,7 @@ sub _read_profile_section {
 my @tags = $self->_split_comma_sep_field($section->{'tags'});
 my $overridable = $self->_parse_boolean($section->{'overridable'}, -1, 
$pname, $sno);
 my $severity = $section->{'severity'}//'';
-my $ignore_map = $self->{'ignored-overrides'};
+my $noover = $self->{'non-overridable-tags'};
 my $sev_map = $self->{'severity-changes'};
 $self->_check_for_invalid_fields($section, \%SEC_FIELDS, $pname, "section 
$sno");
 croak "Profile \"$pname\" is missing Tags field (or it is empty) in 
section $sno.\n" unless @tags;
@@ -303,9 +303,9 @@ sub _read_profile_section {
 $sev_map->{$tag} = $severity if $severity;
 if ( $overridable != -1 ) {
 if ($overridable) {
-delete $ignore_map->{$tag};
+delete $noover->{$tag};
 } else {
-$ignore_map->{$tag} = 1;
+$noover->{$tag} = 1;
 }
 }
 }
diff --git a/lib/Lintian/Tags.pm b/lib/Lintian/Tags.pm
index 9dcf59f..78e47a6 100644
--- a/lib/Lintian/Tags.pm
+++ b/lib/Lintian/Tags.pm
@@ -151,16 +151,16 @@ sub new {
 important => { 'wild-guess' => 1, possible => 1, certain => 1 },
 serious   => { 'wild-guess' => 1, possible => 1, certain => 1 },
 },
-display_source=> {},
-files => {},
-ignored_overrides => {},
-only_issue=> {},
-respect_display   => 1,
-show_experimental => 0,
-show_overrides=> 0,
-show_pedantic => 0,
-statistics=> {},
-suppress  => {},
+display_source   => {},
+files=> {},
+non_overridable_tags => {},
+only_issue   => {},
+respect_display  => 1,
+show_experimental=> 0,
+show_overrides   => 0,
+show_pedantic=> 0,
+statistics   => {},
+suppress => {},
 };
 bless($self, $class);
 $GLOBAL = $self unless $GLOBAL;
@@ -537,7 +537,7 @@ file cannot be opened.
 
 sub file_overrides {
 my ($self, $overrides) = @_;
-my $ignored = $self->{ignored_overrides};
+my $ignored = $self->{non_overridable_tags};
 unless (defined $self->{current}) {
 die 'no current file when adding overrides'

Bug#638278: Bits from the Lintian maintainers

[Thorsten Glaser]

Niels Thykier dixit:

>I did (BCC it), the bug is #638278.  I took the liberty of marking you
>as the submitter.

OK, no problem.

>Emails in my inbox has a way of disappearing even if I tag them TODO
>etc, so when I can I file bugs if I cannot handle them immediately.

Yeah, still before doing all the BTS stuff I like to ask
off-“list” first for some things. I met many Debian pak-
kage maintainers who prefer the BTS for even simple pat-
ches, though, but I am not one of them myself…

>> Yes, probably. But you could just map 0 to root for both
>> uid and gid (everything else is not standardised anyway).

>That could solve a lot it, if done in the method reading the tar index.

That was the idea. And it really only works (reliably) with
the uid and gid 0 – although if Debian standardises certain
other uids or gids, that could be supported too. (On the
other hand, almost all files, if not all, in a DEB would be
root-owned anyway.)

bye,
//mirabilos
-- 
[...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh but
what about xfs, and if only i had waited until reiser4 was ready... in the be-
ginning, there was ffs, and in the middle, there was ffs, and at the end, there
was still ffs, and the sys admins knew it was good. :)  -- Ted Unangst über *fs



--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/pine.bsm.4.64l.1108181438580.12...@herc.mirbsd.org

Bug#633887: marked as done (lintian: Perl warnings from reporting/harness -f)

[Debian Bug Tracking System]

Your message dated Thu, 18 Aug 2011 15:02:32 +0200
with message-id <4e4d0d68.2060...@thykier.net>
and subject line Not present in 2.5.2
has caused the Debian Bug report #633887,
regarding lintian: Perl warnings from reporting/harness -f
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
633887: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633887
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: lintian
Version: 2.5.1
Severity: minor

This may already be fixed with the work that was done on master; if so,
feel free to close.

Running reporting/harness -f on lintian.debian.org with 2.5.1 produced the
following Perl warnings:

Use of uninitialized value $ENV{"LINTIAN_ROOT"} in concatenation (.) or string 
at /org/lintian.debian.org/root/lib/Read_pkglists.pm line 25.
Use of uninitialized value in numeric eq (==) at 
/org/lintian.debian.org/root/unpack/list-srcpkg line 140,  line 746.
Use of uninitialized value in numeric eq (==) at 
/org/lintian.debian.org/root/unpack/list-srcpkg line 140,  line 7558.

This doesn't seem to have affected the Lintian run.

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 2.6.39-2-686-pae (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages lintian depends on:
ii  binutils  2.21.52.20110606-2 The GNU assembler, linker and bina
ii  bzip2 1.0.5-6high-quality block-sorting file co
ii  diffstat  1.54-1 produces graph of changes introduc
ii  file  5.04-5+b1  Determines file type using "magic"
ii  gettext   0.18.1.1-3 GNU Internationalization utilities
ii  intltool-debian   0.35.0+20060710.1  Help i18n of RFC822 compliant conf
ii  libapt-pkg-perl   0.1.24+b2  Perl interface to libapt-pkg
ii  libclass-accessor-per 0.34-1 Perl module that automatically gen
ii  libdpkg-perl  1.16.0.3   Dpkg perl modules
ii  libemail-valid-perl   0.184-1Perl module for checking the valid
ii  libipc-run-perl   0.89-1 Perl module for running processes
ii  libparse-debianchange 1.2.0-1parse Debian changelogs and output
ii  libtimedate-perl  1.2000-1   collection of modules to manipulat
ii  liburi-perl   1.58-1 module to manipulate and access UR
ii  locales   2.13-7 Embedded GNU C Library: National L
ii  man-db2.6.0.2-1  on-line manual pager
ii  perl [libdigest-sha-p 5.12.4-1   Larry Wall's Practical Extraction 
ii  unzip 6.0-5  De-archiver for .zip files

lintian recommends no packages.

Versions of packages lintian suggests:
pn  binutils-multiarch (no description available)
ii  dpkg-dev  1.16.0.3   Debian package development tools
ii  libhtml-parser-perl   3.68-1+b1  collection of modules that parse H
ii  libtext-template-perl 1.45-2 Text::Template perl module
ii  man-db2.6.0.2-1  on-line manual pager
ii  xz-utils  5.0.0-2XZ-format compression utilities

-- no debconf information


--- End Message ---
--- Begin Message ---
Source: lintian
Source-Version: 2.5.2

Hi,

None of these were present in 2.5.2, so I am closing it as fixed.  Yay! :)

~Niels


--- End Message ---

Lintian.d.o upgraded to 2.5.2

[Niels Thykier]

Hi,

As the subject suggests, I updated lintian.d.o to using lintian 2.5.2.
I saw no initialized warnings, so I have closed #633887.  However there
was a missing an import in reporting/harness (oops! my bad), which I
fixed in the master branch (and have cherry-picked on lintian.d.o
without the changelog).

Upgrading was slightly complicated by two things;
reporting/checkout-release did not quite work for me (mostly due to a
merge-conflict).  I think a "git merge -s recursive -X theirs" could
help the automation here.
  I am not too sure the perl expression works, since it ate the variable
name in the code (rather than replacing the value).  I had to run it
manually, so it might have been a copy-waste fail.

Also it cd's into root, but uses root/frontend/lintian looks fishy at
best.  But since I never got past the merge-conflict I ran that part
manually, so I merely assuming there is an issue. <:-)


The second upgrade issue was caused by debhelper (or dh_testdir at
least) not being present on lintian.d.o, so I had to workaround that
(trivial, but still annoying).
  I do not know if it is custom to run the testsuite on lintian.d.o; I
didn't - mostly because they blow up due to lacking
debhelper^Wdependencies (Test::Pod and debhelper etc.).  If we do not
want to run them, we should probably either make a special target in
d/rules for lintian.d.o or/and use DEB_BUILD_OPTIONS=nocheck in
reporting/checkout-release (and /srv/lintian.debian.org/README).

~Niels


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4d101c.6010...@thykier.net

Bug#638278: Bits from the Lintian maintainers

[Niels Thykier]

On 2011-08-18 13:50, Thorsten Glaser wrote:
> Niels Thykier dixit:
> 
>> Package: lintian
> 
> Hm, don’t find it, did you Cc submit@b.d.o?
> (I usually file against Source for changes like this one…)
> 

Hi,

I did (BCC it), the bug is #638278.  I took the liberty of marking you
as the submitter.

>> Severity: normal
> 
> Would set it to minor or wishlist, even though it’s
> an improvement and “probably” correct.
> 

Personally I do not care if it is minor or normal.  I tend to view
wishlist as requests for new tags/features and I think this one is about
possibly correct a false-positive.

>> Not a 100% sure here, but I have submitted this as a bug against Lintian
> 
> Mh, I contacted you first about the opinion because I didn’t
> want to impose this as bug to you if you disagreed.
> 

Emails in my inbox has a way of disappearing even if I tag them TODO
etc, so when I can I file bugs if I cannot handle them immediately.
  Plus I am hoping a co-maintainer can step in and elaborate on this.
The sun might be up outside, but it isn't on the inside. :P

>> now.  As I recall, we tend to use user and group names rather than
>> numerical values in the code, so this smells like work. :P
> 
> Yes, probably. But you could just map 0 to root for both
> uid and gid (everything else is not standardised anyway).
> If you want I can have a look at the code (no guarantees,
> I don’t really speak Perl).
> 
> bye,
> //mirabilos

That could solve a lot it, if done in the method reading the tar index.
 Brilliant!

~Niels




--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4d0401.8060...@thykier.net

Processed: limit source to lintian, tagging 637649

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> #lintian (2.5.3) UNRELEASED; urgency=low
> #
> #  * t/*:
> #+ [JW,NT] Fixed test failures caused by unpacking the source
> #  with a umask different from 0022.  (Closes: #637649)
> #
> limit source lintian
Limiting to bugs with field 'source' containing at least one of 'lintian'
Limit currently set to 'source':'lintian'

> tags 637649 + pending
Bug #637649 [lintian] lintian: copyright files in test debs with wrong 
permissions
Added tag(s) pending.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
637649: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=637649
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/handler.s.c.131365798632610.transcr...@bugs.debian.org

Processed: retitle 638278 to lintian: errors if only numerical owner/group info is available

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> # that was probably a bad bug name
> retitle 638278 lintian: errors if only numerical owner/group info is available
Bug #638278 [lintian] Bits from the Lintian maintainers
Changed Bug title to 'lintian: errors if only numerical owner/group info is 
available' from 'Bits from the Lintian maintainers'
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
638278: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638278
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/handler.s.c.131365648923692.transcr...@bugs.debian.org

Bug#638278: Bits from the Lintian maintainers

[Niels Thykier]

Package: lintian
Severity: normal

On 2011-08-18 10:15, Thorsten Glaser wrote:
> Hi,
> 
> no idea whether you knew it, but lintian warns with Error
> severity if files in {control,data}.tar.gz have only the
> numerical user and group ID filled in, i.e. are owned by
> "0/0" instead of "root/root".
> 
> The difference is in the ustar header:
> 
> typedef struct {
>   char name[TNMSZ];   /* name of entry */
>   char mode[8];   /* mode */
>   char uid[8];/* uid */
>   char gid[8];/* gid */
>   char size[12];  /* size */
>   char mtime[12]; /* modification time */
>   char chksum[CHK_LEN];   /* checksum */
>   char typeflag;  /* type of file. */
>   char linkname[TNMSZ];   /* linked to name */
>   char magic[TMAGLEN];/* magic cookie */
>   char version[TVERSLEN]; /* version */
>   char uname[32]; /* ascii owner name */
>   char gname[32]; /* ascii group name */
>   char devmajor[8];   /* major device number */
>   char devminor[8];   /* minor device number */
>   char prefix[TPFSZ]; /* linked to name */
> } HD_USTAR;
> 
> Normally, uid and gid are "000", and uname and gname
> are optionally (normally yes, but several tar implementa-
> tions have gained the ability to not do it) filled in with
> the local values for that, such as "root" but the group
> with the ID 0 results as "wheel" when packaged on BSD sy-
> stems, which would violate Policy in the moment that a
> non-gid0 group called wheel exists on the Debian system
> the DEB is extracted on (since they have priority over
> the numerical values if they exist).
> That’s why I was playing with only-numeric uid/gid values…
> 
> Here’s a hexdump:
> 
> tglase@tglase:~ $ mirtar -M 0x000B -cf - emptyfile | hd
>   65 6d 70 74 79 66 69 6c  65 00 00 00 00 00 00 00  |emptyfile...|
> 0010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 0060  00 00 00 00 30 31 30 30  36 34 34 00 30 30 30 30  |0100644.|
> 0070  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30  |000.000.|
> 0080  30 30 30 30 30 30 30 00  31 31 36 32 33 31 34 34  |000.11623144|
> 0090  33 37 37 00 30 30 31 31  35 33 34 00 30 00 00 00  |377.0011534.0...|
> 00a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 0100  00 75 73 74 61 72 00 30  30 72 6f 6f 74 00 00 00  |.ustar.00root...|
> 0110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> 0120  00 00 00 00 00 00 00 00  00 72 6f 6f 74 00 00 00  |.root...|
> 0130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 2800
> tglase@tglase:~ $ mirtar -M 0x008B -cf - emptyfile | hd
>   65 6d 70 74 79 66 69 6c  65 00 00 00 00 00 00 00  |emptyfile...|
> 0010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 0060  00 00 00 00 30 31 30 30  36 34 34 00 30 30 30 30  |0100644.|
> 0070  30 30 30 00 30 30 30 30  30 30 30 00 30 30 30 30  |000.000.|
> 0080  30 30 30 30 30 30 30 00  31 31 36 32 33 31 34 34  |000.11623144|
> 0090  33 37 37 00 30 30 30 37  37 32 34 00 30 00 00 00  |377.0007724.0...|
> 00a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 0100  00 75 73 74 61 72 00 30  30 00 00 00 00 00 00 00  |.ustar.00...|
> 0110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 2800
> 
> 
> I’ll write more about DEB files in the wlog, which is syndicated
> on Planet Debian, later (which will include the reasons for me
> to do this; I wrote an "ar" backend for paxtar in the process).
> If you want to reproduce this, “paxmirabilis” currently lies at
> https://www.freewrt.org/~tg/debs/dists/sid/wtf/Pkgs/mircpio/
> (it doesn’t conflict with anything on the system).
> 
> Do you agree when I think ustar archives with numeric zero values
> as uid/gid should not violate Policy?
> 
> Thanks,
> //mirabilos

Hi,

Not a 100% sure here, but I have submitted this as a bug against Lintian
now.  As I recall, we tend to use user and group names rather than
numerical values in the code, so this smells like work. :P

~Niels




--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4e4ccd43.2070...@thykier.net

Processed: submitter 638278

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> # p.s. the real submitter is:
> submitter 638278 t.gla...@tarent.de
Bug #638278 [lintian] lintian: errors if only numerical owner/group info is 
available
Changed Bug submitter to 't.gla...@tarent.de' from 'Niels Thykier 
'
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
638278: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638278
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems


-- 
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/handler.s.c.131365653324024.transcr...@bugs.debian.org