Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Mon, Jan 19, 2009 at 10:02:31PM +, Neil Williams wrote:
On Mon, 19 Jan 2009 12:20:40 -0800
Russ Allbery r...@debian.org wrote:
If I check the generated templates in the binary deb, how do I check that
the string was marked for translation? We don't want to trigger this tag
on strings that aren't intended to be translated.
You can't. This only works properly at the source package level.
TBH I was expecting that all questions would be translated - at least
the help text (Description), even if not the possible answers.
Otherwise, doesn't it tend to indicate that debconf is being used as a
registry?
To be honest, I think you seem to have got the wrong end of the
debconf-is-not-a-registry stick somewhat.
The point of saying that debconf is not a registry is to discourage
maintainers from storing information long-term in the debconf database
that instead belongs in the filesystem. There is no prohibition on
people doing fancy stuff on a short-term basis, e.g. to communicate
between the config and postinst scripts, and nor should there be.
(I'm speaking with my debconf co-maintainer hat on here.)
A quick check finds these files on my system that contain unexpected
content like:
Template: debconf-apt-progress/info
Type: text
Description: ${DESCRIPTION}
Template: debconf-apt-progress/media-change
Type: text
Description: Media change
${MESSAGE}
How are those variables utilised? Where and how could these be
translated?
With the possible exception of the Media change string there, which I
think is probably a bug, debconf-apt-progress is passing through
already-translated strings supplied by apt.
Checking this at the binary package level loses all the paragraph-level
annotations applied to po-debconf input templates files (e.g.
flag:translate!:2), though. It's just the wrong place for the check.
Template: console-setup/modelcode
Type: string
Description: for internal use
Template: console-setup/layoutcode
Type: string
Description: for internal use
Template: console-setup/variantcode
Type: string
Description: for internal use
Template: console-setup/optionscode
Type: string
Description: for internal use
Template: console-setup/fontsize
Type: string
Description: for internal use
Template: console-setup/codesetcode
Type: string
Description: for internal use
Those all look wrong to me - debconf for internal use only?
See above - this is perfectly legitimate. In the case of console-setup,
this is a way for the config script to communicate information to the
postinst, specifically the results of mapping translated choices back to
the identifiers that need to be written to the configuration file. It's
perhaps a little awkward and there are some other ways to achieve a
similar effect, but the most elegant way to do this can vary depending
on the circumstances; it's certainly not intrinsically wrong.
I haven't analysed all the cases you provided, but in general Lintian is
not smart enough to second-guess a maintainer's choice of labelling a
template as internal-use with any remote degree of correctness, and it
should therefore not attempt to do so.
Template: tasksel/desktop
Type: multiselect
Choices: gnome, kde, xfce
Default: gnome
Description: The desktop environment to install when the desktop task
is selected This can be preseeded to change the default.
(That tasksel one looks like a true positive - I can't see why that is
not translated.)
Because it's never asked; it's preseeding-only. The installer
intentionally doesn't offer this as a choice by default (and let us not
reopen that most endless of discussions here), but it's available for
preseeding for people customising the installer.
Again, Lintian is not smart enough to second-guess this and shouldn't
try.
Cheers,
--
Colin Watson [cjwat...@debian.org]
--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Mon, Jan 19, 2009 at 04:15:21PM -0800, Russ Allbery wrote: Neil Williams codeh...@debian.org writes: OK, I think there is a way of identifying private templates. It could be as simple as agreeing (after Lenny) that a particular Description is uniformly used for all private templates. That would help translators too. Well, it shouldn't help translators because those templates should not be marked as translatable anyway. It would just be an additional requirement that we're adding for people writing debconf templates so that they don't get Lintian warnings. If you implement this check, please make it inactive for udebs. We already have a comprehensive and semi-automatic translation infrastructure, and concerns about questions asked on upgrade don't apply to the installer. In general the installer *does* use debconf as a registry, and this is sanctioned. The assumptions in this bug do not generally hold for it. Most already seem to use for internal use. After all, the description itself is completely arbitrary as far as internal templates are concerned. Some examples of untranslated templates that don't use that string are: cdebootstrap (a udeb special case) cpufrequtils etcinsvk kickseed (maybe an actual bug) kickseed is like that because it's asking user-visible questions before the language question has been asked; this is a consequence of the design of Kickstart and not something it's reasonably possible to do anything about. Adding for internal use would be inappropriate. lowmem lowmem also knows what it's doing, with this comment above the relevant templates: # Note: not translatable, runs before language is chosen I've added matching comments to kickseed. Many of them use variations on it. Those are just the ones that I can find because they don't use po-debconf at all and hence Lintian currently warns (but won't after I fix #492626). Looking in more depth, while Lintian currently uses the internal use part to suppress checks for grammar and the like, there is currently no check in Lintian for a template that isn't marked as translatable but should be. I think we therefore have no idea how many false positives we're likely to uncover. We could introduce this as experimental first and see what lintian.d.o digs up, though. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- Colin Watson [cjwat...@debian.org] -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
(keeping CC as I don't know who's subscribed to what) Quoting Colin Watson (cjwat...@debian.org): Template: tasksel/desktop Type: multiselect Choices: gnome, kde, xfce Default: gnome Description: The desktop environment to install when the desktop task is selected This can be preseeded to change the default. (That tasksel one looks like a true positive - I can't see why that is not translated.) Because it's never asked; it's preseeding-only. The installer intentionally doesn't offer this as a choice by default (and let us not reopen that most endless of discussions here), but it's available for preseeding for people customising the installer. Again, Lintian is not smart enough to second-guess this and shouldn't try. I basically agree with what Colin was saying up to there.but not to that last affirmation. The lintian check about translatability avoided a lot of non translatable debconf templates (at least this is a guess...but when it was introduced, I saw a bump in new strings for debconf templates, showing that many maintainers realized they were omitting something). So, I'd vote for this check to be kepteven though there are many cases where it will be legitimate to ignore that lintian warning. signature.asc Description: Digital signature
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Sun, 18 Jan 2009 19:37:29 -0800 Russ Allbery r...@debian.org wrote: Well that check only works on the source package and only uses msgfmt - it could probably be improved with a check on the binaries and the actual templates file(s). Could you explain a bit more about what that would catch that the current check doesn't catch, other than the (rather unusual) case of only running lintian on the .deb file without running it on the source package? It allows the check to isolate one particular case: A new question has been added to the templates file and *NONE* of the translations include that specific question. The check repeats for all other questions, uniquely matching the translation status of that one message against each language. It fails only if NONE of the languages include at least one specific question. The result is that this check would then warrant a higher certainty and a higher severity because it will not pick up the cases where the translators for some languages have replied and some have not. It will only pick up cases where *NONE* of the translators have replied. It gives a high certainty for the problem described by Christian: Of course, a first package with no translation at all is really something we don't like as this makes obvious that no call for translations was made and very very likely that no review happened (because such reviews always include a recommendation to do a call for translations). It also extends that to a package that includes a new debconf question that is completely untranslated, with the same degree of certainty. The test would not affect program translations, only debconf ones. All po-debconf checks are currently done against the source package, and if we can diagnose the problem on the source package, I'd rather do it there since fixing the problem really requires action on the whole source package rather than separate actions for each binary package. The source package does not contain the post-processed templates file that debconf actually reads. It is the file that is created after the PO files are merged into the DEBIAN/templates file during the final stages of the package build. In the original report, I only tested against the .deb. The no-complete-debconf-translation check is not high enough severity to show up without -I when checking the source package. Yes, we can change the severity, although I'd like to run that past debian-i18n first. Christian - this is a slightly different problem to what you first thought. It isn't that some translators have answered and some have not, it is that a new question has been added and nobody at all has replied. If a sane deadline is set, isn't it unlikely that not one of the language teams managed to get a translation to the maintainer in time? It is far more likely that the maintainer didn't ask the translation teams before uploading the new question. The tag name might have to change: new-question-without-translations If the binary check is added, the certainty can be raised and also the severity. With that change, the description could be made more strict: I don't see why a binary check would change the certainty. Maybe I'm missing something? The source package check can only process the msgfmt output which is overly brief. msgfmt does not say whether all translations are missing the *same* string, it just says that all translations are missing *a* string. Processing the templates file from the binary uniquely identifies situations where none of the translations include that question. As soon as any question is changed, all the old translations become fuzzy and podebconf (via gettext) refuses to include those translations against the modified question. It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. Then maybe an override can quote the message to debian-18n in the comments, just as other overrides are meant to quote the bug number? If other questions in the templates file *are* translated, it seems highly unlikely to me that none of the previous translators and none of the other language teams responded to the call for updates - as long as a sane deadline was set. debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) I disagree with that analysis of the discussion on mentors - I think a brand new package *should* go to translators before the first upload and gave my reasons in the thread. New packages using debconf should have their templates reviewed. New packages
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Quoting Neil Williams (codeh...@debian.org): Yes, we can change the severity, although I'd like to run that past debian-i18n first. Christian - this is a slightly different problem to what you first thought. It isn't that some translators have answered and some have not, it is that a new question has been added and nobody at all has replied. If a sane deadline is set, isn't it unlikely that not one of the language teams managed to get a translation to the maintainer in time? It is far more likely that the maintainer didn't ask the translation teams before uploading the new question. I agree here. If you manage to get through the problem of having to examine binary packages' templates, then I agree that having a template that's marked translatable and *no* translation at all makes it very likely that no call for translations was issued when the templated was added. By fairly likely, I mean quite certain, indeed The tag name might have to change: new-question-without-translations that would certainly help avoiding the case where maintainers entirely drop existing translations A properly worded long information will also help. It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. Not really, here. Given that some language teams try to commit self to stay 100%, nearly any new debconf template will be catched. debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) I disagree with that analysis of the discussion on mentors - I think a brand new package *should* go to translators before the first upload and gave my reasons in the thread. New packages using debconf should have their templates reviewed. Entirely agreed. I'd like to see severity important but normal would be OK for starters. Remember, the rule of thumb here is that severity should match the severity that you'd pick for the bug that you filed about the problem, were you to file a bug. Important is a rather large leap over the current severities used for translations. Debconf is a little different. It is a peculiar problem that if a new debconf question is not translated, the user does not get the chance to reconsider their answer when the translation finally arrives. Normal severity would be fine if important is deemed a step too far. I think that normal is a good compromise. signature.asc Description: Digital signature
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Mon, 19 Jan 2009 18:27:40 +0100
Christian Perrier bubu...@debian.org wrote:
Quoting Neil Williams (codeh...@debian.org):
Yes, we can change the severity, although I'd like to run that past
debian-i18n first.
Christian - this is a slightly different problem to what you first
thought. It isn't that some translators have answered and some have
not, it is that a new question has been added and nobody at all has
replied. If a sane deadline is set, isn't it unlikely that not one of
the language teams managed to get a translation to the maintainer in
time? It is far more likely that the maintainer didn't ask the
translation teams before uploading the new question.
I agree here. If you manage to get through the problem of having to
examine binary packages' templates, then I agree that having a
template that's marked translatable and *no* translation at all makes
it very likely that no call for translations was issued when the
templated was added.
Something like this should work:
# i18n -- lintian check script -*- perl -*-
package Lintian::codehelp;
use strict;
use Tags;
use Util;
sub run {
my $pkg = shift;
my $type = shift;
if (-f control/templates)
{
my $incomplete;
my $stanza;
open (I18N, '', control/templates) or
fail (cannot open lintian templates: $!);
while (I18N)
{
if (/^Description: /)
{
$stanza++;
undef $incomplete;
next;
}
undef ($stanza) if (/^Description-/);
if ((defined ($stanza) and (/^Template:/)))
{
$incomplete++;
last;
}
}
close (I18N);
tag (ch-package-contains-docs, templates) if (defined $incomplete);
}
}
1;
control/templates is safe as we're checking the binary here - we don't
want to be checking debian/foo.templates. It's not perfect, it probably
also need to check that this isn't a source package with:
return if ($type eq source);
and output the $pkg variable instead of the rather bland templates.
It simply says that once a Description has been found, there must be a
Description- (this too can be improved) before the next empty line and
following Template stanza. That might need improvement to catch the
last question in the file, especially if there is no terminal newline.
I've not used a sensible tagname either.
Under what circumstances are questions untranslated? There should
always be some help text that needs translation.
By fairly likely, I mean quite certain, indeed
The tag name might have to change:
new-question-without-translations
that would certainly help avoiding the case where maintainers entirely
drop existing translations
A properly worded long information will also help.
I'm sure we can come up with that. Something based on:
debconf only ever asks the same question once - to be effective, that
question should be translated the very first time that question is
offered to the user. Translating it after the user has answered the
question in English is pointless - at least as far as that user is
concerned.
.
This package contains a template file containing at least one question
that has not been translated at all. This means that translators
were not properly warned about new strings.
.
Translators may be notified of changes using podebconf-report-po, for
example:
.
podebconf-report-po --call --withtranslators --deadline=+10 days \
--languageteam
Ref: devref 6.5.2.2
I can spend some more time on this, if you'd like a fully tested patch Russ.
--
Neil Williams
=
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
pgpYwNSe5z91E.pgp
Description: PGP signature
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Neil Williams codeh...@debian.org writes: The source package check can only process the msgfmt output which is overly brief. msgfmt does not say whether all translations are missing the *same* string, it just says that all translations are missing *a* string. Oh! Yes, thank you. That was the point that I was missing. So the existing check does have false positives that the check you outline would not for the problem that you're trying to detect. If I check the generated templates in the binary deb, how do I check that the string was marked for translation? We don't want to trigger this tag on strings that aren't intended to be translated. It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. Then maybe an override can quote the message to debian-18n in the comments, just as other overrides are meant to quote the bug number? If other questions in the templates file *are* translated, it seems highly unlikely to me that none of the previous translators and none of the other language teams responded to the call for updates - as long as a sane deadline was set. That's a good point. If we check that other things are translated but this question was not, that's a fairly conclusive sign that the translators are willing to work on the package and just haven't had an opportunity to do so. debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) I disagree with that analysis of the discussion on mentors - I think a brand new package *should* go to translators before the first upload and gave my reasons in the thread. New packages using debconf should have their templates reviewed. Okay. I'll defer to the debian-i18n consensus on this, since it's y'all's resources that are at stake here. Remember, the rule of thumb here is that severity should match the severity that you'd pick for the bug that you filed about the problem, were you to file a bug. Important is a rather large leap over the current severities used for translations. Debconf is a little different. It is a peculiar problem that if a new debconf question is not translated, the user does not get the chance to reconsider their answer when the translation finally arrives. Normal severity would be fine if important is deemed a step too far. I still think it is worth enforcing no untranslated debconf questions in debian-mentors as a point of good practice - even if the lintian tag severity is not changed in order to avoid annoying existing DD's. Mentors should be about raising the quality of NEW packages and package updates (especially QA uploads etc.) and all packages coming through mentors should be up to the latest measures of Policy, Standards and general behaviour. Sure. I believe that when mentoring you should always run lintian with -I and ask mentees to fix info-level tags as well (unless they're just wrong). Info-level tags are intended to be best practices that people who don't have a solid reason for doing otherwise should follow, which describes fairly well the typical mentoring situation. Maybe lintian could be more aggressive for checks performed during sponsoring than when being used by more experienced DD's. ;-) My hope is that the existing distinction between info and warning, and the upcoming pedantic support, will provide that extra level of aggressiveness for those who want it. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Mon, 19 Jan 2009 12:20:40 -0800
Russ Allbery r...@debian.org wrote:
If I check the generated templates in the binary deb, how do I check that
the string was marked for translation? We don't want to trigger this tag
on strings that aren't intended to be translated.
TBH I was expecting that all questions would be translated - at least
the help text (Description), even if not the possible answers.
Otherwise, doesn't it tend to indicate that debconf is being used as a
registry?
The empirical answer, at the moment, would be as described earlier:
Template:
...
Description:
Template:
should fail - with the proviso that the last question in the file also
needs to be allowed to fail, so check for EOF as well.
Template:
...
Description:
Description-
...
Template:
would pass.
A quick check finds these files on my system that contain unexpected
content like:
Template: debconf-apt-progress/info
Type: text
Description: ${DESCRIPTION}
Template: debconf-apt-progress/media-change
Type: text
Description: Media change
${MESSAGE}
How are those variables utilised? Where and how could these be
translated?
debconf.templates failed
console-data.templates failed
x11-common.templates failed
gdm.templates failed
console-setup.templates failed
tasksel.templates failed
dbconfig-common.templates failed
linux-image-2.6.24-1-amd64.templates failed
linux-image-2.6.25-2-amd64.templates failed
linux-image-2.6.26-1-amd64.templates failed
That's using a script based on the perl script from the other message.
Out of 65 templates on this system, it's a start.
Choices is a little awkward but I cannot see a scenario where Choices
is *translated* but the Description *is not* so basing the check on
Description alone (to avoid a more common case where Choices is not
translatable) is appropriate, AFAICT.
x11 has:
Template: x11-common/xwrapper/actual_allowed_users
Type: string
Description: internal use only
This template is never shown to the user and does not require
translation.
Others include:
Template: console-data/bootmap-md5sum
Type: string
Default: none
Description: for internal use
Template: gdm/daemon_name
Type: string
Default: /usr/bin/gdm
Description: for internal use only
Template: console-setup/modelcode
Type: string
Description: for internal use
Template: console-setup/layoutcode
Type: string
Description: for internal use
Template: console-setup/variantcode
Type: string
Description: for internal use
Template: console-setup/optionscode
Type: string
Description: for internal use
Template: console-setup/fontsize
Type: string
Description: for internal use
Template: console-setup/codesetcode
Type: string
Description: for internal use
Those all look wrong to me - debconf for internal use only?
Template: tasksel/desktop
Type: multiselect
Choices: gnome, kde, xfce
Default: gnome
Description: The desktop environment to install when the desktop task
is selected This can be preseeded to change the default.
(That tasksel one looks like a true positive - I can't see why that is
not translated.)
Template: dbconfig-common/missing-db-package-error
Type: select
Choices: abort, retry, ignore
Default: abort
Description: Database package required.
To properly configure the database for ${pkg}, it is necessary
that you also have ${dbpackage} installed. Unfortunately, this can
not be done automatically.
.
If in doubt, you should choose abort, and install ${dbpackage} before
continuing with the configuration of this package. If you choose
retry, you will be allowed to choose different answers (in case you
chose the wrong database type by mistake). If you choose ignore,
then installation will continue as normal.
.
(Note to translators: don't bother translating this message yet, as the
text/wording is not stabilized)
Hmm. That's going to be tricky to identify but probably OK for an
override.
linux-image-2.6.24-1-amd64.templates - none of the strings appear to be
translated and to my eye, all should have been - true positive?
Ditto for linux-image-2.6.25-2-amd64.templates and
linux-image-2.6.26-1-amd64.templates
I'm attaching the script so that others can check their templates
files. (Consider it under the GPL-3+, Copyright me 2009. Note that
it does not currently check for EOF correctly.)
So, overall, only one package needs an override from this cursory
glance at one machine, two packages (four template files) look like
correct checks that should have been caught before (probably) and
another 4 that are using debconf for internal use which seems to be
against the spirit of debconf, to me anyway.
Oh and before anyone asks, I'm not saying that the true positive checks
or the internal use only templates need to be fixed for Lenny. ;-)
--
Neil Williams
=
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/
#!/usr/bin/perl
opendir (LINT, /var/lib/dpkg/info/);
@files=grep(/templates$/, readdir(LINT));
closedir (LINT);
foreach $file
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Neil Williams codeh...@debian.org writes:
Russ Allbery r...@debian.org wrote:
If I check the generated templates in the binary deb, how do I check
that the string was marked for translation? We don't want to trigger
this tag on strings that aren't intended to be translated.
TBH I was expecting that all questions would be translated - at least
the help text (Description), even if not the possible answers.
Otherwise, doesn't it tend to indicate that debconf is being used as a
registry?
Private templates are extremely common. We can't realistically do
anything that would warn about private templates. It will just annoy a
lot of people.
Whether that's using debconf as a registry or not is debatable, but I just
raised this point on debian-devel and several people commented that they
prefer private templates for things that can be adjusted by preseeding but
aren't worth prompting the user about. There are also reasons to use
private templates as a way of storing information about how the package
has handled debconf settings. For example, krb5-config uses a private
template of type boolean:
Template: krb5-config/read_conf
Type: boolean
Default: true
Description: For internal use only
We want to try and capture user changes when they edit a config file
manually. To do this, we look in the config script to read the file.
However, in the case of preconfigure, the config script is run twice
before the postinst is run. Thus, we may read the wrong value before the
edited value is written out in postinst. If this is false, we skip
reading config files until postinst runs.
I think this sort of thing is quite common.
for this sort of check, I want to trust the package maintainer when they
decide what is translatable and what isn't. We already warn them if we
think they're not marking something as translatable that should be in the
source package checks.
A quick check finds these files on my system that contain unexpected
content like:
Template: debconf-apt-progress/info
Type: text
Description: ${DESCRIPTION}
Template: debconf-apt-progress/media-change
Type: text
Description: Media change
${MESSAGE}
How are those variables utilised? Where and how could these be
translated?
debconf is currently exempted from this check because it does complex
things internally that Lintian can't analyze.
--
Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/
--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Mon, 19 Jan 2009 14:54:26 -0800 Russ Allbery r...@debian.org wrote: Neil Williams codeh...@debian.org writes: Russ Allbery r...@debian.org wrote: If I check the generated templates in the binary deb, how do I check that the string was marked for translation? We don't want to trigger this tag on strings that aren't intended to be translated. TBH I was expecting that all questions would be translated - at least the help text (Description), even if not the possible answers. Otherwise, doesn't it tend to indicate that debconf is being used as a registry? Private templates are extremely common. We can't realistically do anything that would warn about private templates. It will just annoy a lot of people. OK, I think there is a way of identifying private templates. It could be as simple as agreeing (after Lenny) that a particular Description is uniformly used for all private templates. That would help translators too. Most already seem to use for internal use. After all, the description itself is completely arbitrary as far as internal templates are concerned. Indeed, simply adding one line to the test script could be enough: next if ($line =~ /^Description: .*[for ]?internal use/); Which gives: (excluding debconf.templates as you described) tasksel.templates failed dbconfig-common.templates failed linux-image-2.6.24-1-amd64.templates failed linux-image-2.6.25-2-amd64.templates failed linux-image-2.6.26-1-amd64.templates failed dbconfig probably needs an override and the others look like correct checks. Until there is consensus on the syntax for private templates, the lintian test could stay at lower severity. Description: For internal use only That would match too - so it would be ignored for the test. I think this sort of thing is quite common. As long as a particular identifier can be used for all (and the lintian test can provide information on that identifier in the guide text), that shouldn't be a problem. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgprvn2hu4gOp.pgp Description: PGP signature
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Neil Williams codeh...@debian.org writes: OK, I think there is a way of identifying private templates. It could be as simple as agreeing (after Lenny) that a particular Description is uniformly used for all private templates. That would help translators too. Well, it shouldn't help translators because those templates should not be marked as translatable anyway. It would just be an additional requirement that we're adding for people writing debconf templates so that they don't get Lintian warnings. Most already seem to use for internal use. After all, the description itself is completely arbitrary as far as internal templates are concerned. Some examples of untranslated templates that don't use that string are: cdebootstrap (a udeb special case) cpufrequtils etcinsvk kickseed (maybe an actual bug) lowmem Many of them use variations on it. Those are just the ones that I can find because they don't use po-debconf at all and hence Lintian currently warns (but won't after I fix #492626). Looking in more depth, while Lintian currently uses the internal use part to suppress checks for grammar and the like, there is currently no check in Lintian for a template that isn't marked as translatable but should be. I think we therefore have no idea how many false positives we're likely to uncover. We could introduce this as experimental first and see what lintian.d.o digs up, though. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Package: lintian
Version: 2.1.6
Severity: wishlist
Tags: l10n
http://people.debian.org/~codehelp/#debconf
http://lists.debian.org/debian-mentors/2009/01/msg00178.html
I've been thinking about translation and debconf, partially due to my
work on TDebs etc. I think there are good reasons to consider any
untranslated debconf templates as automatically buggy.
debconf only ever asks the same question once - to be effective, that
question should be translated the very first time that question is
offered to the user. Translating it after the user has answered the
question in English is pointless - at least as far as that user is
concerned.
So, I wonder if lintian could extend checks/po-debconf to raise an error
not only if any package contains a completely untranslated set of templates
but also check if a single question is untranslated even if the rest
of the templates have one or more translations.
It is important that if a question is translated, even just once, that
the error is not issued - only questions that have no translations at
all should generate the error.
Maybe for Squeeze it could even be a release goal that no package using
debconf is uploaded without the translations being updated. A lintian
test would be a useful stage in such a process.
'msgfmt -c --statistics' will output the status of the debian/po files
or it may just be simpler to parse the templates file(s) in the binary
package(s) and check for any 'Description' that does not have a
'Description-[a-z]{2}\..*:' in the corresponding Template: stanza.
e.g.:
Template: emsource/svnusername
Type: string
Description: Subversion login to use on buildd.emdebian.org:
...
Description-cs.UTF-8: Uživatelské jméno pro subversion na
buildd.emdebian.org:
...
Description-eu.UTF-8: buildd.emdebian.org-en erabiliko den subversion
erabiltzaile izena:
...
Description-fi.UTF-8: Koneella buildd.emdebian.org käytettävä
Subversion-tunnus:
...
Template: emsetup/aptagent
...
Deliberately breaking that stanza produces:
Template: emsource/svnusername
Type: string
Description: Subversion login to use on buildd.emdebian.org:
...
.
Deliberately compromising the question to break translations
Template: emsetup/aptagent
...
No lintian warning is raised after this act of wanton vandalism. ;-)
$ lintian -Ii ../build-area/libemdebian-tools-perl_1.4.15_all.deb
$
lintian (2.1.6)
The lack of a Description-$foo at the start of a line after a
Description: but before the next empty line would indicate the error
with a high degree of certainty. (UTF-8 is by no means a reliable
token, sadly).
A lintian error would also provide a quite useful hook for various
l10n scripts that may exist or will soon be written by myself and/or
Christian Perrier, just by parsing lintian.debian.org. It would provide
very useful data on existing translation support and assist me in the
development of a range of TDeb support code.
An extension like this would complement the existing support for
newer-debconf-templates and no-complete-debconf-translation.
-- System Information:
Debian Release: 5.0
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.26-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages lintian depends on:
ii binutils2.18.1~cvs20080103-7 The GNU assembler, linker and bina
ii diffstat1.46-1 produces graph of changes introduc
ii dpkg-dev1.14.24 Debian package development tools
ii file4.26-2 Determines file type using magic
ii gettext 0.17-6 GNU Internationalization utilities
ii intltool-debian 0.35.0+20060710.1Help i18n of RFC822 compliant conf
ii libipc-run-perl 0.82-1 Perl module for running processes
ii libparse-debianchan 1.1.1-2 parse Debian changelogs and output
ii libtimedate-perl1.1600-9 Time and date functions for Perl
ii liburi-perl 1.37+dfsg-1 Manipulates and accesses URI strin
ii man-db 2.5.2-3 on-line manual pager
ii perl [libdigest-sha 5.10.0-19Larry Wall's Practical Extraction
lintian recommends no packages.
Versions of packages lintian suggests:
ii binutils-multiarch 2.18.1~cvs20080103-7 Binary utilities that support mult
ii libtext-template-pe 1.44-1.2 Text::Template perl module
ii man-db 2.5.2-3 on-line manual pager
-- no debconf information
--
To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
Neil Williams codeh...@debian.org writes: debconf only ever asks the same question once - to be effective, that question should be translated the very first time that question is offered to the user. Translating it after the user has answered the question in English is pointless - at least as far as that user is concerned. So, I wonder if lintian could extend checks/po-debconf to raise an error not only if any package contains a completely untranslated set of templates but also check if a single question is untranslated even if the rest of the templates have one or more translations. I must be missing something -- why doesn't no-complete-debconf-translation catch that? That's its intention as I understand it; in fact, it's even stricter than what you're proposing. Is it just buggy? -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
On Sun, 18 Jan 2009 10:22:09 -0800 Russ Allbery r...@debian.org wrote: Neil Williams codeh...@debian.org writes: debconf only ever asks the same question once - to be effective, that question should be translated the very first time that question is offered to the user. Translating it after the user has answered the question in English is pointless - at least as far as that user is concerned. So, I wonder if lintian could extend checks/po-debconf to raise an error not only if any package contains a completely untranslated set of templates but also check if a single question is untranslated even if the rest of the templates have one or more translations. I must be missing something -- why doesn't no-complete-debconf-translation catch that? That's its intention as I understand it; in fact, it's even stricter than what you're proposing. Is it just buggy? Well that check only works on the source package and only uses msgfmt - it could probably be improved with a check on the binaries and the actual templates file(s). In the original report, I only tested against the .deb. The no-complete-debconf-translation check is not high enough severity to show up without -I when checking the source package. If the binary check is added, the certainty can be raised and also the severity. With that change, the description could be made more strict: Info: Even though this package provides debconf translation support, there are either no translations or none of the translations are complete. This will mean that users are expected to answer the new question(s) exclusively in English. You should always call for translations for any new strings before uploading new debconf questions because debconf only asks any question once. I'd like to see severity important but normal would be OK for starters. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpVCCQa4KB97.pgp Description: PGP signature
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
I'll go ahead and copy debian-i18n on this reply as there are several questions below. The discussion is about detecting failure to notify translators to get new translations before upload. There is an existing Lintian tag, no-complete-debconf-translation, for this, but it's currently only severity: wishlist. (Which, I should note, matches the severity of the bugs filed about translations at present, although I personally treat them as higher priority than wishlist for my packages.) Neil Williams codeh...@debian.org writes: Well that check only works on the source package and only uses msgfmt - it could probably be improved with a check on the binaries and the actual templates file(s). Could you explain a bit more about what that would catch that the current check doesn't catch, other than the (rather unusual) case of only running lintian on the .deb file without running it on the source package? All po-debconf checks are currently done against the source package, and if we can diagnose the problem on the source package, I'd rather do it there since fixing the problem really requires action on the whole source package rather than separate actions for each binary package. In the original report, I only tested against the .deb. The no-complete-debconf-translation check is not high enough severity to show up without -I when checking the source package. Yes, we can change the severity, although I'd like to run that past debian-i18n first. If the binary check is added, the certainty can be raised and also the severity. With that change, the description could be made more strict: I don't see why a binary check would change the certainty. Maybe I'm missing something? It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) I'd like to see severity important but normal would be OK for starters. Remember, the rule of thumb here is that severity should match the severity that you'd pick for the bug that you filed about the problem, were you to file a bug. Important is a rather large leap over the current severities used for translations. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
[ Sigh. *This* time I'll copy debian-i18n. ] I'll go ahead and copy debian-i18n on this reply as there are several questions below. The discussion is about detecting failure to notify translators to get new translations before upload. There is an existing Lintian tag, no-complete-debconf-translation, for this, but it's currently only severity: wishlist. (Which, I should note, matches the severity of the bugs filed about translations at present, although I personally treat them as higher priority than wishlist for my packages.) Neil Williams codeh...@debian.org writes: Well that check only works on the source package and only uses msgfmt - it could probably be improved with a check on the binaries and the actual templates file(s). Could you explain a bit more about what that would catch that the current check doesn't catch, other than the (rather unusual) case of only running lintian on the .deb file without running it on the source package? All po-debconf checks are currently done against the source package, and if we can diagnose the problem on the source package, I'd rather do it there since fixing the problem really requires action on the whole source package rather than separate actions for each binary package. In the original report, I only tested against the .deb. The no-complete-debconf-translation check is not high enough severity to show up without -I when checking the source package. Yes, we can change the severity, although I'd like to run that past debian-i18n first. If the binary check is added, the certainty can be raised and also the severity. With that change, the description could be made more strict: I don't see why a binary check would change the certainty. Maybe I'm missing something? It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) I'd like to see severity important but normal would be OK for starters. Remember, the rule of thumb here is that severity should match the severity that you'd pick for the bug that you filed about the problem, were you to file a bug. Important is a rather large leap over the current severities used for translations. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-lint-maint-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#512210: lintian: [checks/po-debconf] Extend template check for updated strings
(I haven't looked #512210 yet) Quoting Russ Allbery (r...@debian.org): In the original report, I only tested against the .deb. The no-complete-debconf-translation check is not high enough severity to show up without -I when checking the source package. Yes, we can change the severity, although I'd like to run that past debian-i18n first. I'm not entirely enthusiast to increase the severity. Having incomplete PO files is not an error per se. And the maintainer might very well have sent a call for translaiton updates but not received any input from some translators. That happens very often. So, I would not like some maintainers to consider that the translation file is bogus and not include it, while it is just safe to keep it as is. If the binary check is added, the certainty can be raised and also the severity. With that change, the description could be made more strict: I don't see why a binary check would change the certainty. Maybe I'm missing something? It's certainty: possible right now because there may be cases where translators were warned but didn't have a chance to do any translations (for an obscure package, for instance). I think that will always be the case. *Yes*. Really, I see no real good way to actually check if translators were warned (as this is done by mail). So, the current warning is the only thing that should be done, imho... debian-mentors discussion also raises the valid point that a brand new package possibly shouldn't go to translators before the first upload. I'd like to get a debian-i18n opinion on that as well. Should we skip the Lintian tag for no complete translation if this is the first packaging? (We can detect this by noting that we only have one changelog entry.) On first packaging, what we would see is first a review of English messages in debian-l10n-english (yet another thing you can't really detect) before the call for translations is issued to -i18n. Of course, a first package with no translation at all is really something we don't like as this makes obvious that no call for translations was made and very very likely that no review happened (because such reviews always include a recommendation to do a call for translations). I'd like to see severity important but normal would be OK for starters. Remember, the rule of thumb here is that severity should match the severity that you'd pick for the bug that you filed about the problem, were you to file a bug. Important is a rather large leap over the current severities used for translations. Agreed. Even though we consider l10n to be important, I think that overflating severities for it would very likely make maintainers angry for no real benefit. signature.asc Description: Digital signature
