Re: Language file `en` missing
On 12/1/19 10:10 PM, Wolfgang Scherer wrote: == Accept-Language bug == If there is no language file `en` installed, kallithea does not handle Accept-Language correctly, when `en` is the prioritized language, e.g.: Apparently related: https://kallithea-scm.org/repos/kallithea/changeset/04682d9fb33a https://kallithea-scm.org/repos/kallithea/changeset/89e50be94fcb https://kallithea-scm.org/repos/kallithea/changeset/f2f7a8c1281e Is your point that the problem would be fixed by providing an empty kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding .po)? And for clarity, the default .ini should perhaps set the fallback language as "i18n.lang = en" and update the description accordingly? /Mads ___ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
Re: Language file `en` missing
Am 01.12.19 um 22:58 schrieb Mads Kiilerich: > On 12/1/19 10:10 PM, Wolfgang Scherer wrote: >> == >> Accept-Language bug >> == >> >> If there is no language file `en` installed, kallithea does not handle >> Accept-Language correctly, when `en` is the prioritized language, e.g.: > > Apparently related: > > https://kallithea-scm.org/repos/kallithea/changeset/04682d9fb33a The English translation is not always redundant! For a language priority of `nothere` -> `anything but en` -> `fr` -> `default`, the English translation is in fact redundant for the transition `fr` -> `default`.. For a language priority of `nothere` -> `en` -> `fr` -> `default`, the English translation is **not* redundant for `nothere` -> `en`, because without an English translation, the French translation is incorrectly used. > > https://kallithea-scm.org/repos/kallithea/changeset/89e50be94fcb The reasons to give up a poorly maintained manual translation file are sound. A better option is to automatically create such a file. > > https://kallithea-scm.org/repos/kallithea/changeset/f2f7a8c1281e > With an automtically generated `en` translation file, it does not make a difference whether i8n.lang is set to `en` or left emtpy, the result is the same. > > Is your point that the problem would be fixed by providing an empty > kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding .po)? Unfortunately, the translation file cannot be empty. Anything missing in the translation file is looked up in the next translation file of the chain. However, it is quite simple to generate a full translation file from the automatically created .po template. I have just written such a generator at: http://sw-amt.ws/kallithea/public/kallithea-deploy/files/c92c49843628ac881d772cabf0c4ea445777f4fc/msgdup.py (relevant parts are :class:`MsgSection` and :func:`run`, the rest is boiler-plate). It handles the info attributes and plurals properly. The entire creation sequence is documented here: http://sw-amt.ws/kallithea-deploy/html/overview.html#install-a-language-file-for-en . I did not check, whether setup.py already provides such a facility. > > And for clarity, the default .ini should perhaps set the fallback language as > "i18n.lang = en" and update the description accordingly? I would not do that, since an older kallithea without an `en` translation file would choke on it with an error message. If I should prepare anything further, let me know. Wolfgang ___ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
Re: Language file `en` missing
On 12/2/19 3:08 AM, Wolfgang Scherer wrote: Is your point that the problem would be fixed by providing an empty kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding .po)? Unfortunately, the translation file cannot be empty. Anything missing in the translation file is looked up in the next translation file of the chain. So ... there is no problem in the simple case with one preferred translation, and fallback to "untranslated". But any i18n code that support multiple prioritized translations with fall-through between translations must know that if the default "untranslated" language is in the prioritized list of languages, then that language has 100% coverage even without any actual translations, and it should never look further in the list. I don't think Kallithea is doing anything in that area. It rely on TurboGears (and webob and what not). If it doesn't work correctly, then it must be because we don't use these modules correctly, or because a bug in these libraries (which we then may have to work around until fixed). But before trying to work around in Kallithea: Exactly where is the problem? /Mads ___ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
Re: Language file `en` missing
Am 04.12.19 um 02:02 schrieb Mads Kiilerich:
> On 12/2/19 3:08 AM, Wolfgang Scherer wrote:
>>> Is your point that the problem would be fixed by providing an empty
>>> kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding
>>> .po)?
>> Unfortunately, the translation file cannot be empty. Anything missing in the
>> translation file is looked up in the next translation file of the chain.
>
>
> So ... there is no problem in the simple case with one preferred translation,
> and fallback to "untranslated".
Yes.
>
> But any i18n code that support multiple prioritized translations with
> fall-through between translations must know that if the default
> "untranslated" language is in the prioritized list of languages, then that
> language has 100% coverage even without any actual translations, and it
> should never look further in the list.
I heartily agree. And it actually is implemented just like that in package
gettext.
> I don't think Kallithea is doing anything in that area. It rely on TurboGears
> (and webob and what not). If it doesn't work correctly, then it must be
> because we don't use these modules correctly, or because a bug in these
> libraries (which we then may have to work around until fixed).
Kallithea uses TurboGears (tg), which in turn uses the GNU gettext package.
> But before trying to work around in Kallithea: Exactly where is the problem?
The attached :func:`check_i18n.run.` identifies the relevant modules and
functions used for translation.
1. The first question is "what is the default language?". Somehow, there is the
assumption, that it would be "en". This is True for e.g., PHP: "en_US_POSIX"
https://www.php.net/manual/en/class.locale.php, Android: "en"
https://developer.android.com/guide/topics/resources/multilingual-support. The
Android example also shows, that the resolution rules are basically arbitrary.
But :mod:`gettext` uses "C" as the default language and **not** "en" (and is
correct in doing so, since "C" is not necessarily the same as "en").
:func:`gettext.find` also stops when "C" is found in the list of languages,
just as expected.
So :mod:`gettext` does not have a bug, just the standardized POSIX default
locale C instead of the rather adhoc and arbitrary "en".
2. TurboGear has no concept of a default locale. The "C" locale is implicitely
ignored, since the list of languages is fed to :func:`gettext.find` one
language at a time. I.e., if "C" appears somewhere between supported locales,
the list is not terminated.
This can be considered a bug, but it is not necessarily so.
So, even if TurboGear was to be modified to recognize the default locale "C" as
a termination point, that would not solve the problem of "en" not being
recognized as default locale, if followed by a supported language.
Summarily, nobody can actually be blamed, however the problem still exists. The
strategies to solve it are in order of (my subjective) usefulness:
1. Modify :func:`tg.i18n._gettranslator` to support a default locale ("C"
**not** "en") which terminates the fallback translator chain. The default
should be configurable, e.g. "i18n.default = en" (see attached
:file:`check_i18n.py`). The necessary changes are minimal if not trivial. I
have no idea, how welcoming the library maintainers are, but I will submit a
pull request to modify the current behavior.
2. An "en" language file is supplied. This is the fastest fix. (This was
probably the reason for the removed "en" language file in the first place).
3. The Accept-Language header is terminated at "en" by some middleware before
the request is passed on to :mod:`tg.i18n`. This is ugly, but does not depend
on a library modification.
>
> /Mads
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2019, Wolfgang Scherer,
#
# This file is part of Kallithea Deployment.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see
Re: Language file `en` missing
Am 04.12.19 um 09:45 schrieb Wolfgang Scherer: > Am 04.12.19 um 02:02 schrieb Mads Kiilerich: >> On 12/2/19 3:08 AM, Wolfgang Scherer wrote: Is your point that the problem would be fixed by providing an empty kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding .po)? >>> Unfortunately, the translation file cannot be empty. Anything missing in >>> the translation file is looked up in the next translation file of the chain. >> >> So ... there is no problem in the simple case with one preferred >> translation, and fallback to "untranslated". > Yes. > > [...] >> I don't think Kallithea is doing anything in that area. It rely on >> TurboGears (and webob and what not). If it doesn't work correctly, then it >> must be because we don't use these modules correctly, or because a bug in >> these libraries (which we then may have to work around until fixed). > Kallithea uses TurboGears (tg), which in turn uses the GNU gettext package. >> But before trying to work around in Kallithea: Exactly where is the problem? The problem is in TurboGears2. I have submitted a pull request with an option i18n.notrans to fix it (https://github.com/TurboGears/tg2/pull/115). I'll let you know, if it is accepted. Wolfgang ___ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
