Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Mark Purcell]

On Friday 09 April 2010 08:30:14 Albert Astals Cid wrote:
> hm? eh? what? why would we remove a perfectly valid translation to a
> language spoken by 11 million people?

Albert,

I don't think the issue is the removal of the translations, rather the fact 
that the locale-code hne isn't defined in ISO 639-1 & ISO 639-2.

My reading of Chhattisgarhi_language[1] shows that the ISO 639-3 code is 
"hne", whilst the ISO 639-2 code for this family is "inc".

I am raising this as an inconsistency. If we are accepting ISO 639-3 locales 
then that is fine too and we will fix lintian.

Mark

W: kipi-plugins: unknown-locale-code hne
N: 
N:The package appears to ship locales for a language but uses an unknown
N:locale code as a subdirectory of /usr/share/locale. This usually results
N:in users of the intended target language not finding the locale. The
N:language codes used in the locale directories are those from the ISO
N:639-1 and ISO 639-2 standards, not those usually used as TLDs (which are
N:from the ISO 3166 standard).
N:
N:It is possible that the language code was mistyped or incorrectly
N:guessed from the language's or country's name.
N:
N:Refer to http://www.loc.gov/standards/iso639-2/php/code_list.php for
N:details.
N:
N:Severity: normal, Certainty: certain
N:
http://lintian.debian.org/tags/unknown-locale-code.html

[1] http://en.wikipedia.org/wiki/Chhattisgarhi_language


signature.asc
Description: This is a digitally signed message part.

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Russ Allbery]

Mark Purcell  writes:

> I don't think the issue is the removal of the translations, rather the
> fact that the locale-code hne isn't defined in ISO 639-1 & ISO 639-2.

> My reading of Chhattisgarhi_language[1] shows that the ISO 639-3 code is
> "hne", whilst the ISO 639-2 code for this family is "inc".

> I am raising this as an inconsistency. If we are accepting ISO 639-3
> locales then that is fine too and we will fix lintian.

I suspect that we should be accepting ISO 639-3 codes as well as -1 and -2
codes.  inc looks like a catch-all that wouldn't be very useful for the
end user in language selection.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-qt-kde-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ochtv56d@windlord.stanford.edu

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Raphael Geissert]

Mark Purcell wrote:

> On Friday 09 April 2010 08:30:14 Albert Astals Cid wrote:
>> hm? eh? what? why would we remove a perfectly valid translation to a
>> language spoken by 11 million people?
> 
> Albert,
> 
> I don't think the issue is the removal of the translations, rather the
> fact that the locale-code hne isn't defined in ISO 639-1 & ISO 639-2.
> 
> My reading of Chhattisgarhi_language[1] shows that the ISO 639-3 code is
> "hne", whilst the ISO 639-2 code for this family is "inc".
> 
> I am raising this as an inconsistency. If we are accepting ISO 639-3
> locales then that is fine too and we will fix lintian.
> 

Right, but the first faulty package is isoquery. The isoquery program is not 
able to display the ISO 639-3 codes:
> TODO for Isoquery
> =
> 
> - allow use of ISO 639-3

The reason I assume is that there's no (given the nature of the ISO 639-3 
standard) translation table between ISO 639-3 and ISO 639-2 codes.

So I guess the most we can do is just inject the 639-3 codes and hope for 
the best (i.e. hope people is going to use the best and most appropriate ISO 
639-1..3 code.)

Sorry for not noticing ISO 639-3 codes were being used back when I 
implemented the checks. I verified some of the cases that would trigger the 
warning but they were all true positives.

And like Russ said, switching to the "inc" code would be incorrect.

Cheers,
-- 
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net


-- 
To UNSUBSCRIBE, email to debian-qt-kde-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4bbe9167.9f15f10a.4bd2.2...@mx.google.com

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Christian PERRIER]

Jumping in that thread (you would have guessed I would, right? :-))

> Mark Purcell wrote:
> 
> > On Friday 09 April 2010 08:30:14 Albert Astals Cid wrote:
> >> hm? eh? what? why would we remove a perfectly valid translation to a
> >> language spoken by 11 million people?
> > 
> > Albert,
> > 
> > I don't think the issue is the removal of the translations, rather the
> > fact that the locale-code hne isn't defined in ISO 639-1 & ISO 639-2.
> > 
> > My reading of Chhattisgarhi_language[1] shows that the ISO 639-3 code is
> > "hne", whilst the ISO 639-2 code for this family is "inc".
> > 
> > I am raising this as an inconsistency. If we are accepting ISO 639-3
> > locales then that is fine too and we will fix lintian.

We are:

bubu...@mykerinos:~/src/debian/iso-codes/git/iso_639_3> ls -l 
/usr/share/i18n/locales/hne_IN
-rw-r--r-- 1 root root 5393  7 févr. 18:26 /usr/share/i18n/locales/hne_IN

We also have several other locales with ISO-639-3 codes:

/usr/share/i18n/locales/ast_ES: Asturian (Spain)
/usr/share/i18n/locales/byn_ER: Bilin (Eritrea)
/usr/share/i18n/locales/crh_UA: Crimean Turkish (Ukraine)
/usr/share/i18n/locales/csb_PL: Kashubian (Poland)
/usr/share/i18n/locales/fil_PH: Filipino (Philippines)
/usr/share/i18n/locales/fur_IT: Friulian( Italy)
/usr/share/i18n/locales/gez_ER: Geez (Eritrea) 
/usr/share/i18n/locales/gez...@abegede
/usr/share/i18n/locales/gez_ET: Geez (Ethiopia) 
/usr/share/i18n/locales/gez...@abegede
/usr/share/i18n/locales/hne_IN: Chhattisgarhi (India)
/usr/share/i18n/locales/hsb_DE: Upper Sorbian
/usr/share/i18n/locales/mai_IN: Maithili (India)
/usr/share/i18n/locales/nan...@latin: Min Nan Chinese (Taiwan)
/usr/share/i18n/locales/nds_DE: Lower Saxon (Germany)
/usr/share/i18n/locales/nds_NL: Lower Saxon (Netherlands)
/usr/share/i18n/locales/nso_ZA: Northern Sotho (South Africa)
/usr/share/i18n/locales/pap_AN: Papiamento (Netherlands Antilles)
/usr/share/i18n/locales/shs_CA: Shuswap (Canada)
/usr/share/i18n/locales/sid_ET: Sidamo (Ethiopia)
/usr/share/i18n/locales/tig_ER: Tigre (Eritrea)
/usr/share/i18n/locales/wal_ET: Wolayttia (Ethiopia)

There are even some locales that correspond to *no* ISO-639-3 code
/usr/share/i18n/locales/ber_DZ: Berber (Algeria)
/usr/share/i18n/locales/ber_MA: Berber (Morocco)

(here I think this is a bug. The code to use should be the code for
Tamazight...but indeed the Berber/Tamazight language uses two scripts,
the fact that Berber and Tamazight are different or not is debated
between Algeria and Morocco and all this is a highly sensitive
political issue  and that may explain this inconsistency...which
is, still, done the wrong way)


All of them are here (and come from glibc upstream) because the said
languages have no ISO-639-2 code.

> > 
> 
> Right, but the first faulty package is isoquery. The isoquery program is not 
> able to display the ISO 639-3 codes:
> > TODO for Isoquery
> > =
> > 
> > - allow use of ISO 639-3
> 
> The reason I assume is that there's no (given the nature of the ISO 639-3 
> standard) translation table between ISO 639-3 and ISO 639-2 codes.
> 
> So I guess the most we can do is just inject the 639-3 codes and hope for 
> the best (i.e. hope people is going to use the best and most appropriate ISO 
> 639-1..3 code.)

That's correct about isoquery's capabilities that should be enhanced.

Maybe, if possible, check that a locale exists for the said language.

We can quite safely assume that, if a locale exists with a given
ISO-639-3 code, then it has been carefully though by glibc maintainers
(either upstream or Debian/Ubuntu maintainers).

Indeed, if a translation file exists for a locale that doesn't exist,
there are chances that it isn't used.particularly because users
will have more problems in setting this locale in their environment.

So, maybe lintian could check that, if foo_BAR.po file exists in a
package, there is at least one foo_* file in /usr/share/i18n/locales

And, subliminal message to Chhattisgarhi translators: I would very
much welcome a work on localization for Debian Installer that would
help in adding Yet Another Indic Language to the already quite long
list of supported Indic languages in Debian and Ubuntu installers..:-)





signature.asc
Description: Digital signature

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Raphael Geissert]

On 9 April 2010 00:16, Christian PERRIER  wrote:
> Jumping in that thread (you would have guessed I would, right? :-))
>

:-)

> There are even some locales that correspond to *no* ISO-639-3 code
> /usr/share/i18n/locales/ber_DZ: Berber (Algeria)
> /usr/share/i18n/locales/ber_MA: Berber (Morocco)

(for lintian's current purposes, the country code is ignored; FWIW)

> Maybe, if possible, check that a locale exists for the said language.
>
> We can quite safely assume that, if a locale exists with a given
> ISO-639-3 code, then it has been carefully though by glibc maintainers
> (either upstream or Debian/Ubuntu maintainers).

I considered using /usr/share/i18n/SUPPORTED as the source of
information but it is far from being complete.

> So, maybe lintian could check that, if foo_BAR.po file exists in a
> package, there is at least one foo_* file in /usr/share/i18n/locales

Am I missing something or wouldn't adding the 639-3 codes to the list
of known and valid codes be enough?

Cheers,
-- 
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net


-- 
To UNSUBSCRIBE, email to debian-qt-kde-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/y2he9fb436d1004082325m81f94ec3qf0a5d78f65bc6...@mail.gmail.com

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Christian PERRIER]

> > So, maybe lintian could check that, if foo_BAR.po file exists in a
> > package, there is at least one foo_* file in /usr/share/i18n/locales
> 
> Am I missing something or wouldn't adding the 639-3 codes to the list
> of known and valid codes be enough?


Actually, if a locale doesn't exist in the glibc, translations for
that locale won't be used, so checking against
/usr/share/i18n/SUPPORTED seems to be the right thing to do.


I'm not sure, though, that it is really a good idea to check PO files
against supported locales, for "upstream" translations (ie non-debconf
ones) with lintian. The only case where it's relevant to lintian,
imho, is the case where the package is native (so "upstream" is
Debian).




signature.asc
Description: Digital signature

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Raphael Geissert]

[should probably drop some CCs as this is becoming partially
off-topic; adding CC to eglibc]

On 9 April 2010 12:59, Christian PERRIER  wrote:
> Actually, if a locale doesn't exist in the glibc, translations for
> that locale won't be used, so checking against
> /usr/share/i18n/SUPPORTED seems to be the right thing to do.

If that's true, then (after playing with Contents-i386.gz), by
comparing /usr/share/locale/*/ with the locales in SUPPORTED:
Locale 'ab' is not supported
Locale 'ace' is not supported
Locale 'ak' is not supported
Locale 'ang' is not supported
Locale 'ara' is not supported
Locale 'ar_PS' is not supported
Locale 'ay' is not supported
Locale 'az_IR' is not supported
Locale 'bal' is not supported
Locale 'b...@latin' is not supported
Locale 'bem' is not supported
Locale 'be-tarask' is not supported
Locale 'bi' is not supported
Locale 'bist' is not supported
Locale 'C' is not supported
Locale 'cat' is not supported
Locale 'c...@valencia' is not supported
Locale 'ckb' is not supported
Locale 'cn' is not supported
Locale 'cn_ZH' is not supported
Locale 'co' is not supported
Locale 'cpf' is not supported
Locale 'cpp' is not supported
Locale 'cs_CZ.utf8' is not supported
Locale 'cs-windows-1250' is not supported
Locale 'cz' is not supported
Locale 'de_DE.utf8' is not supported
Locale 'dk' is not supported
Locale 'dv' is not supported
Locale 'e...@boldquot' is not supported
Locale 'en_IGID' is not supported
Locale 'e...@quot' is not supported
Locale 'e...@shaw' is not supported
Locale 'en_UK' is not supported
Locale 'en...@piglatin' is not supported
Locale 'en_US.utf8' is not supported
Locale 'es_ES.utf8' is not supported
Locale 'fa_AF' is not supported
Locale 'ff' is not supported
Locale 'fi_FI.utf8' is not supported
Locale 'fr_FR.utf8' is not supported
Locale 'fr_FX' is not supported
Locale 'frp' is not supported
Locale 'gn' is not supported
Locale 'gos' is not supported
Locale 'gr' is not supported
Locale 'haw' is not supported
Locale 'io' is not supported
Locale 'it_IT.utf8' is not supported
Locale 'jbo' is not supported
Locale 'jp' is not supported
Locale 'jv' is not supported
Locale 'kab' is not supported
Locale 'kok' is not supported
Locale 'kos' is not supported
Locale 'la' is not supported
Locale 'lb' is not supported
Locale 'mal' is not supported
Locale 'malkovich' is not supported
Locale 'md' is not supported
Locale 'mus' is not supported
Locale 'mvo' is not supported
Locale 'my' is not supported
Locale 'my_MM' is not supported
Locale 'nap' is not supported
Locale 'nl_NL.utf8' is not supported
Locale 'no' is not supported
Locale 'no_NO' is not supported
Locale 'no_NY' is not supported
Locale 'oj' is not supported
Locale 'pis' is not supported
Locale 'pms' is not supported
Locale 'ps' is not supported
Locale 'pseudo' is not supported
Locale 'pt-br' is not supported
Locale 'qu' is not supported
Locale 'racv' is not supported
Locale 'rm' is not supported
Locale 'ru_RU.cp1251' is not supported
Locale 'ru_RU.koi8r' is not supported
Locale 'ru_RU.utf8' is not supported
Locale 'sco' is not supported
Locale 'son' is not supported
Locale 'sp' is not supported
Locale 's...@ije' is not supported
Locale 's...@ijekavian' is not supported
Locale 's...@ijekavianlatin' is not supported
Locale 's...@latin' is not supported
Locale 's...@latn' is not supported
Locale 'sw' is not supported
Locale 'tet' is not supported
Locale 'tlh' is not supported
Locale 'tpi' is not supported
Locale 'tvl' is not supported
Locale 'tw' is not supported
Locale 'twi' is not supported
Locale 'tzo' is not supported
Locale 'ua' is not supported
Locale 'urd' is not supported
Locale 'u...@cyrillic' is not supported
Locale 'wal' is not supported
Locale 'x-test' is not supported
Locale 'xx' is not supported
Locale 'zam' is not supported
Locale 'zh_cn' is not supported
Locale 'zh_TW.Big5' is not supported
Total: 106
Total excluding anything after [...@_.-]: 67

However, if I compare that list with the codes of ISO 639-1..3
(excluding anything after [...@_.-] -- similar to lintian's check):

Locale 'bist' is not supported
Locale 'C' is not supported
Locale 'cn' is not supported
Locale 'cz' is not supported
Locale 'dk' is not supported
Locale 'gr' is not supported
Locale 'jp' is not supported
Locale 'malkovich' is not supported
Locale 'md' is not supported
Locale 'pseudo' is not supported
Locale 'racv' is not supported
Locale 'sh' is not supported
Locale 'sp' is not supported
Locale 'ua' is not supported
Locale 'x' is not supported
Locale 'xx' is not supported
Total: 16

So, we have 67 - 16 = 51 locales that are not supported by glibc but
that we ship translations for. Great.

Something needs to be done.

> I'm not sure, though, that it is really a good idea to check PO files
> against supported locales, for "upstream" translations (ie non-debconf
> ones) with lintian. The only case where it's relevant to lintian,
> imho, is the case where the package is native (so "upstream" is
> Debian).

In fact, what lintian checks is the name of s

Re: [Bug 233674] kipi-plugins: unknown-locale-code hne i8n

[Bjørn Steensrud]

 Fredag 9. april 2010 23.21.45 skrev Raphael Geissert :
> [should probably drop some CCs as this is becoming partially
> off-topic; adding CC to eglibc]

Other NO people may be better qualified to reply than I am, but here goes:

> Locale 'no' is not supported
> Locale 'no_NO' is not supported 
> Locale 'no_NY' is not supported


Good!We have been trying to get rid of those locales for quite some time. 
The two we use are nb_NO and nn_NO, for Bomål and Nynorsk, respectively. 
I know of only one other active locale for _NO:  se_NO,  Northern Sámi in 
Norway.

Bjørn, who translates KDE to nb and occasionally has time to do a debconf 
translation.


--
To UNSUBSCRIBE, email to debian-qt-kde-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/201004100907.26005.bjor...@skogkatt.homelinux.org