Am 04.12.19 um 02:02 schrieb Mads Kiilerich: > On 12/2/19 3:08 AM, Wolfgang Scherer wrote: >>> Is your point that the problem would be fixed by providing an empty >>> kallithea/i18n/en/LC_MESSAGES/kallithea.mo (and building a corresponding >>> .po)? >> Unfortunately, the translation file cannot be empty. Anything missing in the >> translation file is looked up in the next translation file of the chain. > > > So ... there is no problem in the simple case with one preferred translation, > and fallback to "untranslated". Yes. > > But any i18n code that support multiple prioritized translations with > fall-through between translations must know that if the default > "untranslated" language is in the prioritized list of languages, then that > language has 100% coverage even without any actual translations, and it > should never look further in the list.
I heartily agree. And it actually is implemented just like that in package gettext. > I don't think Kallithea is doing anything in that area. It rely on TurboGears > (and webob and what not). If it doesn't work correctly, then it must be > because we don't use these modules correctly, or because a bug in these > libraries (which we then may have to work around until fixed). Kallithea uses TurboGears (tg), which in turn uses the GNU gettext package. > But before trying to work around in Kallithea: Exactly where is the problem? The attached :func:`check_i18n.run.` identifies the relevant modules and functions used for translation. 1. The first question is "what is the default language?". Somehow, there is the assumption, that it would be "en". This is True for e.g., PHP: "en_US_POSIX" https://www.php.net/manual/en/class.locale.php, Android: "en" https://developer.android.com/guide/topics/resources/multilingual-support. The Android example also shows, that the resolution rules are basically arbitrary. But :mod:`gettext` uses "C" as the default language and **not** "en" (and is correct in doing so, since "C" is not necessarily the same as "en"). :func:`gettext.find` also stops when "C" is found in the list of languages, just as expected. So :mod:`gettext` does not have a bug, just the standardized POSIX default locale C instead of the rather adhoc and arbitrary "en". 2. TurboGear has no concept of a default locale. The "C" locale is implicitely ignored, since the list of languages is fed to :func:`gettext.find` one language at a time. I.e., if "C" appears somewhere between supported locales, the list is not terminated. This can be considered a bug, but it is not necessarily so. So, even if TurboGear was to be modified to recognize the default locale "C" as a termination point, that would not solve the problem of "en" not being recognized as default locale, if followed by a supported language. Summarily, nobody can actually be blamed, however the problem still exists. The strategies to solve it are in order of (my subjective) usefulness: 1. Modify :func:`tg.i18n._gettranslator` to support a default locale ("C" **not** "en") which terminates the fallback translator chain. The default should be configurable, e.g. "i18n.default = en" (see attached :file:`check_i18n.py`). The necessary changes are minimal if not trivial. I have no idea, how welcoming the library maintainers are, but I will submit a pull request to modify the current behavior. 2. An "en" language file is supplied. This is the fastest fix. (This was probably the reason for the removed "en" language file in the first place). 3. The Accept-Language header is terminated at "en" by some middleware before the request is passed on to :mod:`tg.i18n`. This is ugly, but does not depend on a library modification. > > /Mads
#!/usr/bin/env python # -*- coding: utf-8 -*- # Copyright (C) 2019, Wolfgang Scherer, <Wolfgang.Scherer at gmx.de> # # This file is part of Kallithea Deployment. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. # (progn (forward-line 1) (snip-insert "py_doc.main" t t "python") (insert "\n")) r""" check_i18n.py - ::fillme:: ====== ==================== usage: check_i18n.py [OPTIONS] ::fillme:: or import check_i18n ====== ==================== :rem:`|||:sec:|||`\ Options =========================== .. compound:: ===================== ================================================== -q, --quiet suppress warnings -v, --verbose verbose test output -d, --debug[=NUM] show debug information -h, --help display this help message -t, --test run doc tests ===================== ================================================== .. adhoc Options ===================== ================================================== --template list show available templates --eide[=COMM] Emacs IDE template list (implies `--template` list) --template[=NAME] extract named template to standard output; default NAME is ``-`` --extract[=DIR] extract adhoc files to directory DIR (default: :file:`.`) --explode[=DIR] explode script with adhoc in directory DIR (default :file:`__adhoc__`) --setup[=install] explode script into temporary directory and call :command:`python setup.py install` --implode implode script with adhoc ===================== ================================================== :rem:`|||:sec:|||`\ Description =============================== :rem:`|||:sec:|||`\ Module ========================== :rem:`|||:sec:|||`\ Automatic Exports ===================================== >>> for ex in __all__: printf(sformat('from {0} import {1}', __name__, ex)) :rem:`|||:sec:|||`\ Explicit Exports ==================================== >>> if '__all_internal__' in globals(): ... for ex in __all_internal__: ... printf(sformat('from {0} import {1}', __name__, ex)) .. _END_OF_HELP_check_i18n: :rem:`|||:sec:|||`\ Details =========================== """ # (progn (forward-line 1) (snip-insert "py.b.future.with" t t "python") (insert "\n")) # for python 2.5 from __future__ import with_statement # (progn (forward-line 1) (snip-insert "py.main.pyramid.activate" t t "py") (insert "")) # -------------------------------------------------- # |||:sec:||| COMPATIBILITY # -------------------------------------------------- import sys # (progn (forward-line 1) (snip-insert "py.b.printf" t t "py") (insert "\n")) # adapted from http://www.daniweb.com/software-development/python/code/217214 try: printf = eval("print") # python 3.0 case except SyntaxError: printf_dict = dict() try: exec("from __future__ import print_function\nprintf=print", printf_dict) printf = printf_dict["printf"] # 2.6 case except SyntaxError: def printf(*args, **kwd): # 2.4, 2.5, define our own Print function fout = kwd.get("file", sys.stdout) w = fout.write if args: w(str(args[0])) sep = kwd.get("sep", " ") for a in args[1:]: w(sep) w(str(a)) w(kwd.get("end", "\n")) del printf_dict # (progn (forward-line 1) (snip-insert "py.b.sformat" t t "py") (insert "\n")) try: ('{0}').format(0) def sformat (fmtspec, *args, **kwargs): return fmtspec.format(*args, **kwargs) except AttributeError: try: import stringformat def sformat (fmtspec, *args, **kwargs): return stringformat.FormattableString(fmtspec).format( *args, **kwargs) except ImportError: printf('error: stringformat missing. Try `easy_install stringformat`.', file=sys.stderr) # (progn (forward-line 1) (snip-insert "py.b.isstring" t t "python" " --key isstring_onlyx") (insert "\n")) try: from ws_seq_type import isstring, issequence, sequence_type, UCHAR_FMT except ImportError: # (progn (forward-line 1) (snip-insert "py.f.isstring" t t "py") (insert "\n")) exec(''' def isstring(obj): return isinstance(obj, basestring) '''.strip()) try: isstring("") UCHAR_FMT = 'u"{0}u{1:04x}"' except NameError: def isstring(obj): return isinstance(obj, str) or isinstance(obj, bytes) UCHAR_FMT = '"{0}u{1:04x}"' # (progn (forward-line 1) (snip-insert "py.f.issequence" t t "py") (insert "\n")) def issequence(arg, or_dict=False, or_seq=True): # ||:fnc:|| if not isstring(arg): if hasattr(arg, 'items'): return or_dict if hasattr(arg, '__getitem__'): return True if hasattr(arg, '__iter__'): return or_seq return False # (progn (forward-line 1) (snip-insert-mode "py.f.sequence_type" t) (insert "\n")) _st_strg = (True, False, False, False) _st_list = (False, True, False, False) _st_dict = (False, False, True, False) _st_seq = (False, False, False, True) _st_none = (False, False, False, False) def sequence_type(value): # ||:fnc:|| if isstring(value): return _st_strg if hasattr(value, 'items'): return _st_dict if hasattr(value, '__getitem__'): return _st_list if hasattr(value, '__iter__'): return _st_seq return _st_none # (progn (forward-line 1) (snip-insert-mode "py.f.uchar" t) (insert "\n")) def uchar(num): '''Make UNICODE character.''' return eval(sformat(UCHAR_FMT,'\\', num)) # (progn (forward-line 1) (snip-insert "py.b.dict.items" t t "py") (insert "\n")) try: getattr(dict(), 'iteritems') except AttributeError: ditems = lambda d: getattr(d, 'items')() dkeys = lambda d: getattr(d, 'keys')() dvalues = lambda d: getattr(d, 'values')() else: ditems = lambda d: getattr(d, 'iteritems')() dkeys = lambda d: getattr(d, 'iterkeys')() dvalues = lambda d: getattr(d, 'itervalues')() # (progn (forward-line 1) (snip-insert "py.b.xrange" t t "py") (insert "\n")) # `xrange` returns a generator, which `range` already does for python3 # note: use l.. and g.. to get list/generator versions try: xrange(0) lrange = lambda *args, **kwargs: range(*args, **kwargs) except NameError: xrange = range lrange = lambda *args, **kwargs: list(range(*args, **kwargs)) grange = xrange # `xfilter` returns a list, `filter` may return a generator. This is # different from the range/xrange semantics! if isinstance(filter(str, []), list): xfilter = filter gfilter = lambda _f, _s, *args, **kwargs: (_e for _e in _s if _f(_e)) else: xfilter = lambda *args, **kwargs: list(filter(*args, **kwargs)) gfilter = filter lfilter = xfilter # `xmap` returns a list, `map` may return a generator. This is # different from the range/xrange semantics! if isinstance(map(str, []), list): xmap = map gmap = lambda _f, _s, *args, **kwargs: (_f(_e) for _e in _s) else: xmap = lambda *args, **kwargs: list(map(*args, **kwargs)) gmap = map lmap = xmap # `long` is gone in python3 try: isinstance(int, long) except NameError: long = int # (progn (forward-line 1) (snip-insert "py_f.lfind" t t "python") (insert "\n")) def lfind(l, elt): try: return l.index(elt) except ValueError: return -1 import os import re # -------------------------------------------------- # |||:sec:||| CONFIGURATION # -------------------------------------------------- __all__ = [] __all_internal__ = [] # (progn (forward-line 1) (snip-insert "py.b.dbg.def" t t "python") (insert "")) dbg_fwid = globals().get('dbg_fwid', 15) # (progn (forward-line 1) (snip-insert "py.b.canonize.module" t t "python") (insert "")) # (progn (forward-line 1) (snip-insert "py.b.strings" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.strclean" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.b.logging" t t "python") (insert "")) # (progn (forward-line 1) (snip-insert "py.b.ordereddict" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.b.dbg.setup" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.main.project.libdir" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.main.sql.alchemy" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.main.sql.ws" t t "py") (insert "\n")) # @:adhoc_run_time:@ #import adhoc # @:adhoc:@ # (progn (forward-line 1) (snip-insert "py.b.posix" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.b.os.system.sh" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.b.prog.path" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.line-loop" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.table_standalone" t t "python") (insert "")) # (progn (forward-line 1) (snip-insert "py.wsrfid.pylons.imports" t t "python") (insert "")) # (progn (forward-line 1) (snip-insert "py.main.wsgi.get.app" t t "python") (insert "")) # (progn (forward-line 1) (snip-insert "py_wsrfid.config_delayed" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.wsrfid.config_translate_shortcuts" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.dba_imports" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.dba_datainit" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_jsmo.imports" t t "python") (insert "\n")) # |:here:| # -------------------------------------------------- # |||:sec:||| DATA # -------------------------------------------------- # -------------------------------------------------- # |||:sec:||| CLASSES # -------------------------------------------------- # (progn (forward-line -1) (insert "\n") (snip-insert "py.s.class" t t "py") (backward-symbol-tag 1 "fillme" "::")) # -------------------------------------------------- # |||:sec:||| FUNCTIONS # -------------------------------------------------- # (progn (forward-line -1) (insert "\n") (snip-insert "py.s.func" t t "py") (backward-symbol-tag 1 "fillme" "::")) # (progn (forward-line 1) (snip-insert "py_wsrfid.customization" t t "python" " --key cust_delayed_skipx") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.dba_setup_sql" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.dba_id_maps" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.dba_commands_init" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.wsrfid.module.route" t t "py") (insert "")) # -------------------------------------------------- # |||:sec:||| UTILITIES # -------------------------------------------------- # (progn (forward-line 1) (snip-insert "py.wsrfid.dispatch.request" t t "py") (insert "")) # (progn (forward-line 1) (snip-insert "py.f.findfile" t t "py") (insert "")) # (progn (forward-line 1) (snip-insert "py_f.next_numfile_seq" t t "python" " --key xwith_docstrings --key xwith_all_append") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_f.add_prefix_indent" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.c.placeholder.template" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.c.key.hash.ordered.dict" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.c.progress" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.hl" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.single.quote" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.remove.match" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.printenv" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.uname.s" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_f.decoded_email_headers" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_f.print_utf8" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.printe" t t "py") (insert "")) def printe_(*args, **kwargs): kwargs['file'] = kwargs.get('file', sys.stderr) printf(*args, **kwargs) if 'printe' not in globals(): # or globals().get('_is_main_', (__name__ == '__main__')): printe = printe_ printd = printe_ printw = printe_ printx = printe_ # (progn (forward-line 1) (snip-insert "py.f.dbg.squeeze" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.dbg.indent" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.quick.dump" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_b.all.reverse" t t "python") (insert "\n")) if '__all_internal__' in globals(): import sys if 'sphinx.directives' in sys.modules: __all__[:0] = __all_internal__ __all_internal__ = list(reversed(__all_internal__)) __all__ = list(reversed(__all__)) def run(parameters): # ||:fnc:|| """Application runner, when called as __main__.""" # (progn (forward-line 1) (snip-insert "py.bf.sql.ws" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.bf.file.arg.loop" t t "py") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.wsrfid.wsuvv.run" t t "py" " --key py_wsrfid.wsuvv_run --key skip_for_new") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_shell.run" t t "python") (insert "\n")) # from pyjsmo.result import Result, IResult, ERR_NONE # result = IResult() # |:here:| import gettext as _gettext import tg import tg.i18n from tg.i18n import _parse_locale import kallithea tg.i18n.printf = printf tg.i18n.printe = printf tg.i18n.sformat = sformat tg.i18n.dbg_fwid = dbg_fwid # |:here:| APP_DOMAIN = 'kallithea' LOCALE_DIR = os.path.abspath('kallithea/i18n') TG_CONF = tg.config.current_conf() TG_CONF['localedir'] = LOCALE_DIR TG_CONF['package'] = kallithea # |:here:| ACCEPTED_LANGUAGE_SETS = ( ('fr', 'en_US', 'en', 'de_CH', 'de',), ('fr', 'c', 'de_ch', 'de',), ) def find_lang_gettext(lang, tgl=None, tg_config=None, **kwargs): r""" # -------------------------------------------------- # |||:LOG:||| find_lang_gettext, i18n.default: disable # -------------------------------------------------- # :DBG: accepted : ]('fr', 'en_US', 'en', 'de_CH', 'de')[ # :DBG: supported : ]['fr', 'de'][ # -------------------------------------------------- # :DBG: accepted : ]('fr', 'c', 'de_ch', 'de')[ # :DBG: supported : ]['fr'][ - `en` is not found - `c` terminates the fallback chain """ mofiles = _gettext.find(APP_DOMAIN, localedir=LOCALE_DIR, languages=lang, all=True) supported_languages = [_m.split('/')[-3] for _m in mofiles] return supported_languages, mofiles def find_lang_tg(lang, tgl=None, tg_config=None, **kwargs): r""" # -------------------------------------------------- # |||:LOG:||| find_lang_tg, i18n.default: disable # -------------------------------------------------- # :DBG: accepted : ]('fr', 'en_US', 'en', 'de_CH', 'de')[ # :DBG: supported : ]['fr', 'de_CH', 'de'][ # -------------------------------------------------- # :DBG: accepted : ]('fr', 'c', 'de_ch', 'de')[ # :DBG: supported : ]['fr', 'de_ch', 'de'][ - `en` is not found - `c` is not found """ translator = tg.i18n._get_translator((list(lang)), tgl, tg_config, **kwargs) return translator.tg_supported_lang, [] def find_lang_tg_dflt(lang, tgl=None, tg_config=None, **kwargs): r""" # -------------------------------------------------- # |||:LOG:||| find_lang_tg_dflt, i18n.default: C # -------------------------------------------------- # :DBG: accepted : ]('fr', 'en_US', 'en', 'de_CH', 'de')[ # :DBG: supported : ]['fr', 'de_CH', 'de'][ # -------------------------------------------------- # :DBG: accepted : ]('fr', 'c', 'de_ch', 'de')[ # :DBG: supported : ]['fr'][ - `en` is not found - `c` terminates the fallback chain # -------------------------------------------------- # |||:LOG:||| find_lang_tg_dflt, i18n.default: en # -------------------------------------------------- # :DBG: accepted : ]('fr', 'en_US', 'en', 'de_CH', 'de')[ # :DBG: supported : ]['fr'][ # -------------------------------------------------- # :DBG: accepted : ]('fr', 'c', 'de_ch', 'de')[ # :DBG: supported : ]['fr'][ - `en` terminates the fallback chain - `c` terminates the fallback chain """ # copied from tg.i18n._get_translator() if tg_config: conf = tg_config else: if tgl: conf = tgl.config else: # pragma: no cover #backward compatibility with explicit calls without #specifying local context or config. conf = tg.config.current_conf() # |:added:| default = conf.get('i18n.default') or 'C' # |:added:| mofiles = [] supported_languages = [] for l in lang: # |:added:| if _parse_locale(l)[0] in (default.lower(), 'c'): break # |:added:| mo = _gettext.find(APP_DOMAIN, localedir=LOCALE_DIR, languages=[l], all=False) if mo is not None: mofiles.append(mo) supported_languages.append(l) return supported_languages, mofiles for find_lang, i18n_default in ( (find_lang_gettext, 'disable'), # use non-existent locale to ensure old behavior (find_lang_tg, 'disable'), # use non-existent locale to ensure old behavior (find_lang_tg_dflt, ''), # use empty locale to ensure new default behavior (find_lang_tg_dflt, 'en'), # use empty locale to ensure new default behavior ): TG_CONF['i18n.default'] = i18n_default printe("# --------------------------------------------------") printe(sformat("# |||"":LOG:||| {0}, i18n.default: {i18n_default}", find_lang.__name__, i18n_default=i18n_default or 'C')) for accepted in ACCEPTED_LANGUAGE_SETS: supported, mofiles = find_lang(accepted) printe("# --------------------------------------------------") printe(sformat("# "":DBG: {1:<{0}s}: ]{2!s}[", dbg_fwid, "accepted", (accepted))) printe(sformat("# "":DBG: {1:<{0}s}: ]{2!s}[", dbg_fwid, "supported", (supported))) # |:here:| if False: printe("# --------------------------------------------------") printe(sformat("# |||"":LOG:||| {0}, i18n.default: {i18n_default}", '_expand_lang', i18n_default=i18n_default or 'C')) for accepted in ACCEPTED_LANGUAGE_SETS: printe("# --------------------------------------------------") printe(sformat("# "":DBG: {1:<{0}s}: ]{2!s}[", dbg_fwid, "accepted", (accepted))) for lang in accepted: expanded = _gettext._expand_lang(lang) printe(sformat("# "":DBG: {1:<{0}s}: {2!s:<5s} => ]{3!s}[", dbg_fwid, "expanded", lang, (expanded))) # |:here:| return # result.report() # return max(result.error, ERR_NONE) # -------------------------------------------------- # |||:sec:||| MAIN # -------------------------------------------------- _quiet = False _verbose = False _debug = False # (progn (forward-line 1) (snip-insert "py_f.argparse_callbacks" t t "python") (insert "\n")) # (progn (forward-line 1) (snip-insert "py_main.py" t t "python" " --key xpy_main_minimal") (insert "\n")) # (progn (forward-line 1) (snip-insert "py.f.setdefaultencoding" t t "py") (insert "\n")) #file_encoding_is_clean = True def setdefaultencoding(encoding=None, quiet=False, context=None): if context is None: context = globals() if context.get('file_encoding_is_clean'): return if encoding is None: encoding='utf-8' try: isinstance('', basestring) if not hasattr(sys, '_setdefaultencoding'): if not quiet: printf('''\ Add this to /etc/python2.x/sitecustomize.py, or put it in local sitecustomize.py and adjust PYTHONPATH=".:${PYTHONPATH}":: try: import sys setattr(sys, '_setdefaultencoding', getattr(sys, 'setdefaultencoding')) except AttributeError: pass Running with reload(sys) hack ... ''', file=sys.stderr) reload(sys) setattr(sys, '_setdefaultencoding', getattr(sys, 'setdefaultencoding')) sys._setdefaultencoding(encoding) except NameError: # python3 already has utf-8 default encoding ;-) pass def main(argv=None, context=None): # ||:fnc:|| if argv is None: argv = sys.argv if context is None: context = globals() context.get( 'setup_logger', globals().get( 'setup_logger', lambda *args: None))(context, True) try: import argparse except ImportError: printe('error: argparse missing. Try `easy_install argparse`.') sys.exit(1) parser = argparse.ArgumentParser(add_help=False, conflict_handler='resolve') # parser.add_argument('--sum', dest='accumulate', action='store_const', # const=sum, default=max, # help='sum the integers (default: find the max)') # |:opt:| add options context.get('_argparse_begin', lambda _p, *_args: _p)(parser, context) parser.add_argument( '-q', '--quiet', action='store_const', const=-2, dest='debug', default=0, help='suppress warnings') parser.add_argument( '-v', '--verbose', action='store_const', const=-1, dest='debug', default=0, help='verbose test output') parser.add_argument( '-d', '--debug', nargs='?', action='store', type=int, metavar='NUM', default = 0, const = 1, help='show debug information') parser.add_argument( '-h', '--help', action='store_true', help="display this help message") parser.add_argument( '-t', '--test', action='store_true', help='run doc tests') class AdHocAction(argparse.Action): options = ('implode', 'setup', 'explode', 'extract', 'template', 'eide') def __call__(self, parser, namespace, values, option_string=None): for _opt in self.options: setattr(namespace, 'adhoc_' + _opt, False) setattr(namespace, 'adhoc_' + option_string[2:], True) setattr(namespace, 'adhoc_arg', values) parser.add_argument( '--implode', nargs=0, action=AdHocAction, dest='adhoc_implode', default=False, help='implode script with adhoc') parser.add_argument( '--setup', nargs='?', action=AdHocAction, type=str, metavar='install', dest='adhoc_setup', default=False, const='install', help='explode script into temporary directory and call' ' `python setup.py install`') parser.add_argument( '--explode', nargs='?', action=AdHocAction, type=str, metavar='DIR', dest='adhoc_explode', default=False, const='__adhoc__', help='explode script with adhoc in directory DIR' ' (default: `__adhoc__`)') parser.add_argument( '--extract', nargs='?', action=AdHocAction, type=str, metavar='DIR', dest='adhoc_extract', default=False, const = '.', help='extract files to directory DIR (default: `.`)') parser.add_argument( '--template', nargs='?', action=AdHocAction, type=str, metavar='NAME', dest='adhoc_template', default=False, const = '-', help='extract named template to standard output. default NAME is ``-``') parser.add_argument( '--eide', nargs='?', action=AdHocAction, type=str, metavar='COMM', dest='adhoc_eide', default=False, const = '', help='Emacs IDE template list (implies --template list).') parser.add_argument( '--ap-help', action='store_true', help="internal help message") context.get('_argparse_end', lambda _p, *_args: _p)(parser, context) if not context.get('_argparse_have_sub_commands'): # all options and arguments are known # all non-option arguments are consumed by `_parameters.args` parser.add_argument( 'args', nargs='*', metavar='arg', #'args', nargs='+', metavar='arg', #type=argparse.FileType('r'), default=sys.stdin, help='a series of arguments') _parameters = parser.parse_args(argv[1:]) else: # (progn (forward-line 1) (snip-insert "py_f.args_split_range" t t "python") (insert "\n")) def args_split_range(args): next_range = [] for arg in args: next_range.append(arg) if not arg.startswith('-'): break if next_range and not next_range[0].startswith('-'): next_range = [] return next_range, args[len(next_range):] # for sub-commands with their own options: pre-parse to first # non-option argument _parameters = None args = argv[1:] while True: next_range, args = args_split_range(args) if not next_range and not _parameters is None: break _parameters, unknown_args = parser.parse_known_args(next_range, _parameters) if unknown_args: unknown_args.extend(args) args = unknown_args next_range = [] break _parameters.args = args # generate argparse help if _parameters.ap_help: parser.print_help() return 0 # standard help if _parameters.help: help_ = re.sub('\n+[.][.] _END_OF_HELP.*(?s)', '', context['__doc__']) sys.stdout.write(help_ + '\n') return 0 context['_debug'] = _parameters.debug if context['_debug'] > 0: context['_verbose'] = True context['_quiet'] = False elif context['_debug'] < 0: context['_verbose'] = (context['_debug'] == -1) context['_quiet'] = not(context['_verbose']) context['_debug'] = 0 _parameters.debug = context['_debug'] _parameters.verbose = context['_verbose'] _parameters.quiet = context['_quiet'] if context['_debug']: cmd_line = argv sys.stderr.write(sformat( "{0}{3:^{1}} {4:<{2}s}: ]{5!s}[\n", context.get('dbg_comm', '# '), context.get('dbg_twid', 11), context.get('dbg_fwid', 15), ':DBG:', 'cmd_line', cmd_line)) # at least use `quiet` to suppress the setdefaultencoding warning context.get('setdefaultencoding', globals().get('setdefaultencoding', lambda *_a, **_k: None))( quiet=context['_quiet'] or _parameters.test) # |:opt:| handle options # adhoc: implode/setup/explode/extract adhoc_get_opt = lambda opt: getattr( _parameters, 'adhoc_' + opt, None) adhoc_op = sum(((adhoc_get_opt(_opt) and 1) or 0 for _opt in AdHocAction.options)) if adhoc_op: for _opt in AdHocAction.options: setattr(_parameters, _opt, adhoc_get_opt(_opt)) adhoc_export = ( _parameters.setup or _parameters.explode or _parameters.extract) file_ = context['__file__'] source = None have_adhoc = 'AdHoc' in context have_rt_adhoc = 'RtAdHoc' in context # shall adhoc be imported if _parameters.implode or not have_rt_adhoc: # shall this file be compiled adhoc_compile = not (have_rt_adhoc) os_path = os.defpath for pv in ('PATH', 'path'): try: os_path = os.environ[pv] break except KeyError: pass os_path = os_path.split(os.pathsep) for path_dir in os_path: if not path_dir: continue if path_dir not in sys.path: sys.path.append(path_dir) if not have_adhoc: try: import adhoc context['AdHoc'] = adhoc.AdHoc except ImportError: adhoc_compile = False try: from rt_adhoc import RtAdHoc as Adhoc context['AdHoc'] = AdHoc except ImportError: pass else: adhoc_compile = False context['AdHoc'] = context['RtAdHoc'] AdHoc = context['AdHoc'] AdHoc.quiet = context['_quiet'] AdHoc.verbose = context['_verbose'] AdHoc.debug = context['_debug'] AdHoc.include_path.append(os.path.dirname(file_)) AdHoc.extra_templates = [ ] AdHoc.template_process_hooks = { } if _parameters.eide: AdHoc.tt_ide = True AdHoc.tt_comment = _parameters.adhoc_arg or '' AdHoc.tt_prefix = '. (shell-command "' AdHoc.tt_suffix = '")' _parameters.template = True _parameters.adhoc_arg = 'list' if adhoc_compile: ah = AdHoc() source = ah.compileFile(file_) else: file_, source = AdHoc.std_source_param(file_) # implode if _parameters.implode: # @:adhoc_enable:@ # if not context['_quiet']: # map(sys.stderr.write, # ["warning: ", os.path.basename(file_), # " already imploded!\n"]) # @:adhoc_enable:@ AdHoc.write_source('-', source) # explode elif _parameters.setup or _parameters.explode: _here = os.path.abspath('.') _clean_dir = False AdHoc.export_dir = _parameters.adhoc_arg if _parameters.setup: import tempfile _clean_dir = True AdHoc.export_dir = tempfile.mkdtemp('_setup', '__adhoc__') try: AdHoc.export(file_, source) if _parameters.setup: sq = lambda string: ''.join(("'", re.sub("'", """'\\''""", string), "'")) os.chdir(AdHoc.export_dir) os.system(sformat('{0} setup.py {1}', sq(sys.executable), sq(_parameters.adhoc_arg))) finally: if _clean_dir: try: os.chdir(_here) except: pass import shutil shutil.rmtree(AdHoc.export_dir) # extract elif _parameters.extract: AdHoc.extract_dir = _parameters.adhoc_arg AdHoc.extract(file_, source) # template elif _parameters.template: template_name = _parameters.adhoc_arg if not template_name: template_name = '-' if template_name == 'list': sys.stdout.write( '\n'.join(AdHoc.template_table(file_, source)) + '\n') else: template = AdHoc.get_named_template( template_name, file_, source) AdHoc.write_source('-', template) # restore for subsequent calls to main if not have_adhoc: del(AdHoc) return 0 # run doc tests if _parameters.test: import warnings warnings.simplefilter('default') import doctest # for :file:`__init__.py`, :func:`_canonize_module_` does not register the module in `sys.modules`. _canon_name = _module_name = context['__name__'] context.get('_canonize_module_', globals().get( '_canonize_module_', lambda *args: context.__setitem__('__name__', _canon_name)))(context['__name__'], context['__name__'] == '__main__') if context['__name__'] not in sys.modules: sys.modules[context['__name__']] = sys.modules[_module_name] try: logger = logging.getLogger() logger.setLevel(logging.DEBUG) except NameError: pass context.get('_doctest_hook_', lambda *args, **kwargs: None)(context) result = doctest.testmod(sys.modules[context['__name__']], verbose = context['_verbose']) return result.failed # run program final = False ecode = 0 try: try: ecode = context['run'](_parameters) except IOError: (t, e, tb) = sys.exc_info() del(tb) # ignore SIGPIPE import errno if e.errno != errno.EPIPE: raise except SystemExit: raise except: # |:info:| this is used, since module cgitb does not work so well ... (t, e, tb) = sys.exc_info() if not final or context['_debug']: import traceback printf(''.join(traceback.format_tb(tb)), file=sys.stderr, end='') printf(sformat('{0}: {1}', t.__name__, e), file=sys.stderr) del(tb) ecode = 1 return ecode if globals().get('_is_main_', (__name__ == '__main__')): #sys.argv.insert(1, '--debug') # |:debug:| result = main(sys.argv, globals()) sys.exit(result) # |:here:| # (progn (forward-line 1) (snip-insert "py.t.ide" t t "py") (insert "\n")) # # :ide-menu: Emacs IDE Main Menu - Buffer @BUFFER@ # . M-x `eIDE-menu' (eIDE-menu "z") # :ide: CSCOPE ON # . (cscope-minor-mode) # :ide: CSCOPE OFF # . (cscope-minor-mode (quote ( nil ))) # :ide: TAGS: forced update # . (compile (concat "cd /home/ws/project/ws-rfid && make -k FORCED=1 tags")) # :ide: TAGS: update # . (compile (concat "cd /home/ws/project/ws-rfid && make -k tags")) # :ide: +-#+ # . Utilities () # :ide: TOC: Generate TOC with py-toc.py # . (progn (save-buffer) (compile (concat "py-toc.py ./" (file-name-nondirectory (buffer-file-name)) " "))) # :ide: CMD: Fold region with line continuation # . (shell-command-on-region (region-beginning) (region-end) "fold --spaces -width 79 | sed 's, $,,;1!s,^, ,;$!s,$,\\\\,'" nil nil nil t) # :ide: CMD: Fold region and replace with line continuation # . (shell-command-on-region (region-beginning) (region-end) "fold --spaces --width 79 | sed 's, $,,;1!s,^, ,;$!s,$,\\\\,'" t nil nil t) # :ide: +-#+ # . Fold () # :ide: CMD: Remove 8 spaces and add `>>> ' to region # . (shell-command-on-region (region-beginning) (region-end) "sed 's,^ ,,;/^[ ]*##/d;/^[ ]*#/{;s,^ *# *,,p;d;};/^[ ]*$/!s,^,>>> ,'" nil nil nil t) # :ide: CMD: Remove 4 spaces and add `>>> ' to region # . (shell-command-on-region (region-beginning) (region-end) "sed 's,^ ,,;/^[ ]*##/d;/^[ ]*#/{;s,^ *# *,,p;d;};/^[ ]*$/!s,^,>>> ,'" nil nil nil t) # :ide: +-#+ # . Doctest () # :ide: LINT: Check 80 column width ignoring IDE Menus # . (let ((args " | /srv/ftp/pub/check-80-col.sh -")) (compile (concat "sed 's,^\\(\\|. \\|.. \\|... \\)\\(:ide\\|[.] \\).*,,' " (buffer-file-name) " " args " | sed 's,^-," (buffer-file-name) ",'"))) # :ide: LINT: Check 80 column width # . (let ((args "")) (compile (concat "/srv/ftp/pub/check-80-col.sh " (buffer-file-name) " " args))) # :ide: +-#+ # . Lint Tools () # :ide: DELIM: @: SYM :@ @:fillme:@ adhoc tag # . (symbol-tag-normalize-delimiter (cons (cons nil "@:") (cons ":@" nil)) t) # :ide: +-#+ # . Delimiters () # :ide: COMPILE: Run with --ap-help # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --ap-help"))) # :ide: COMPILE: Run with --help # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --help"))) # :ide: COMPILE: Run with --test # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --test"))) # :ide: COMPILE: Run with --test --verbose # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --test --verbose"))) # :ide: COMPILE: Run with --debug # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --debug"))) # :ide: +-#+ # . Compile with standard arguments () # :ide: OCCUR-OUTLINE: Python Source Code # . (x-symbol-tag-occur-outline "sec" '("|||:" ":|||") (cons (cons "^\\([ \t\r]*\\(def\\|class\\)[ ]+\\|[A-Za-z_]?\\)" nil) (cons nil "\\([ \t\r]*(\\|[ \t]*=\\)"))) # :ide: MENU-OUTLINE: Python Source Code # . (x-eIDE-menu-outline "sec" '("|||:" ":|||") (cons (cons "^\\([ \t\r]*\\(def\\|class\\)[ ]+\\|[A-Za-z_]?\\)" nil) (cons nil "\\([ \t\r]*(\\|[ \t]*=\\)"))) # :ide: +-#+ # . Outline () # :ide: INFO: SQLAlchemy - SQL Expression Language - Reference # . (let ((ref-buffer "*sqa-expr-ref*")) (if (not (get-buffer ref-buffer)) (shell-command (concat "w3m -dump -cols " (number-to-string (1- (window-width))) " 'http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/expressions.html'") ref-buffer) (display-buffer ref-buffer t))) # :ide: INFO: SQLAlchemy - SQL Expression Language - Tutorial # . (let ((ref-buffer "*sqa-expr-tutor*")) (if (not (get-buffer ref-buffer)) (shell-command (concat "w3m -dump -cols " (number-to-string (1- (window-width))) " 'http://www.sqlalchemy.org/docs/05/sqlexpression.html'") ref-buffer) (display-buffer ref-buffer t))) # :ide: INFO: SQLAlchemy - Query # . (let ((ref-buffer "*sqa-query*")) (if (not (get-buffer ref-buffer)) (shell-command (concat "w3m -dump -cols " (number-to-string (1- (window-width))) " 'http://www.sqlalchemy.org/docs/orm/query.html'") ref-buffer) (display-buffer ref-buffer t))) # :ide: +-#+ # . SQLAlchemy Reference () # :ide: INFO: Python - argparse # . (let ((ref-buffer "*python-argparse*")) (if (not (get-buffer ref-buffer)) (shell-command (concat "w3m -dump -cols " (number-to-string (1- (window-width))) " 'http://docs.python.org/library/argparse.html'") ref-buffer) (display-buffer ref-buffer t))) # :ide: INFO: Python Documentation # . (let ((ref-buffer "*w3m*")) (if (get-buffer ref-buffer) (display-buffer ref-buffer t)) (other-window 1) (w3m-goto-url "http://docs.python.org/index.html"; nil nil)) # :ide: INFO: Python Reference # . (let* ((ref-buffer "*python-ref*") (local "/home/ws/project/ws-util/python/reference/PQR2.7.html") (url (or (and (file-exists-p local) local) "'http://rgruet.free.fr/PQR27/PQR2.7.html'"))) (unless (get-buffer ref-buffer) (get-buffer-create ref-buffer) (with-current-buffer ref-buffer (shell-command (concat "snc txt.py.reference 2>/dev/null") ref-buffer) (goto-char (point-min)) (if (eobp) (shell-command (concat "w3m -dump -cols " (number-to-string (1- (window-width))) " " url) ref-buffer)))) (display-buffer ref-buffer t)) # :ide: +-#+ # . Python Reference () # :ide: COMPILE: Run with --eide # . (progn (save-buffer) (shell-command (concat "python ./" (file-name-nondirectory (buffer-file-name)) " --eide") (concat "*templates: " (file-name-nondirectory (buffer-file-name)) "*"))) # :ide: COMPILE: Run with python3 --test # . (progn (save-buffer) (compile (concat "python3 ./" (file-name-nondirectory (buffer-file-name)) " --test"))) # :ide: COMPILE: Run with python3 w/o args # . (progn (save-buffer) (compile (concat "python3 ./" (file-name-nondirectory (buffer-file-name)) " "))) # :ide: COMPILE: Run w/o args # . (progn (save-buffer) (compile (concat "python ./" (file-name-nondirectory (buffer-file-name)) " "))) # :ide: COMPILE: Run with --test # . (progn (save-buffer) (compile (concat "cd ../kallithea; \"${HOME}\"/project/kallithea/kallithea-venv/bin/python " (buffer-file-name) " --test"))) # :ide: COMPILE: Run w/o args # . (progn (save-buffer) (compile (concat "cd ../kallithea; \"${HOME}\"/project/kallithea/kallithea-venv/bin/python " (buffer-file-name) " "))) # :ide: +-#+ # . Compile () # # Local Variables: # mode: python # comment-start: "#" # comment-start-skip: "#+" # comment-column: 0 # truncate-lines: t # End:
_______________________________________________ kallithea-general mailing list kallithea-general@sfconservancy.org https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
