[issue44345] The first line (comment) of the parser.c is incorrect.
Change by Akira Nonaka : -- pull_requests: +25199 pull_request: https://github.com/python/cpython/pull/26615 ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Akira Nonaka added the comment: I have just signed the contributor agreement. -- ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Change by Akira Nonaka : -- pull_requests: +25184 pull_request: https://github.com/python/cpython/pull/26602 ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Change by Akira Nonaka : -- keywords: +patch pull_requests: +25175 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26591 ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The first line (comment) of the parser.c is incorrect.
Change by Akira Nonaka : -- title: The First-line (comment) of the parser.c is incorrect. -> The first line (comment) of the parser.c is incorrect. ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44345] The First-line (comment) of the parser.c is incorrect.
New submission from Akira Nonaka : The First-line (comment) of the parser.c is incorrect. "// @generated by pegen.py from ./Grammar/python.gram" pegen.py no longer exists. It is now "pegen" package. -- components: Demos and Tools messages: 395306 nosy: anonaka priority: normal severity: normal status: open title: The First-line (comment) of the parser.c is incorrect. versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue44345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29352] provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences
Changes by Akira Li <4kir4...@gmail.com>: -- pull_requests: +576 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29352> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29352] provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences
Akira Li added the comment: I prefer the wording in the current patch. Though I don't have strong feelings one way or the other as long as the behavior is specified explicitly. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29352> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Akira Li added the comment: > Akira, could you open a pull request on GitHub? Done. PR 699 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Changes by Akira Li <4kir4...@gmail.com>: -- pull_requests: +572 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29352] provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences
New submission from Akira Li: I've failed to find where the behavior for negative indices in s[i:j] expression (i, j < -len(s)) for standard sequences (str, list, etc) is formally defined. The observed behavior implemented in PySlice_GetIndicesEx(): If "len(s) + i" or "len(s) + j" is negative, use 0. [1] I don't see it in the docs. if (*start < 0) *start += length; if (*start < 0) *start = (*step < 0) ? -1 : 0; ... if (*stop < 0) *stop += length; if (*stop < 0) *stop = (*step < 0) ? -1 : 0; The tutorial mentions [2]: > out of range slice indexes are handled gracefully when used for > slicing" slice.indices() documentation says [3]: > Missing or out-of-bounds indices are handled in a manner consistent > with regular slices. Neither define it explicitly. The behavior for the upper boundary is defined explicitly [4]: > If *i* or *j* is greater than ``len(s)``, use ``len(s)`` I've added the documentation patch that defines the behavior for the lower boundary too. [1] Objects/sliceobject.c [2] Doc/tutorial/introduction.rst [3] Doc/reference/datamodel.rst [4] Doc/library/stdtypes.rst -- assignee: docs@python components: Documentation files: docs-negative-slice-indices.patch keywords: patch messages: 286098 nosy: akira, docs@python priority: normal severity: normal status: open title: provide the authorative source for s[i:j] negative slice indices (<-len(s)) behavior for standard sequences versions: Python 3.5, Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46393/docs-negative-slice-indices.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29352> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Akira Li added the comment: I've updated the patch to use 4-space indent (pep-7). I've added space around "=" (pep-7); unlike the usual "dict(designator=value)" -- no space around "=" for keyword argument (pep-8). -- Added file: http://bugs.python.org/file46391/range_bool-c99-designated-initializers-indent.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Akira Li added the comment: Following the python-dev discussion [1] I've added a variant of the patch that uses c99 designated initializers [2] [1] https://mail.python.org/pipermail/python-dev/2017-January/147175.html [2] https://gcc.gnu.org/onlinedocs/gcc/Designated-Inits.html -- Added file: http://bugs.python.org/file46378/range_bool-c99-designated-initializers.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28180] sys.getfilesystemencoding() should default to utf-8
Changes by Akira Li <4kir4...@gmail.com>: -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28180> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Akira Li added the comment: I've removed the documentation changes from the patch. -- Added file: http://bugs.python.org/file45773/range_bool-no_docs.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28876] bool of large range raises OverflowError
Akira Li added the comment: Here's a patch with range_bool() implementation, tests and the docs update. I'm not sure how it should be documented. I've specified it as versionchanged:: 3.6 -- keywords: +patch nosy: +akira Added file: http://bugs.python.org/file45765/range_bool.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28876> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27079] Bugs in curses.ascii predicates
Akira Li added the comment: I'm not sure anything should be done (e.g., it is "undefined behavior" to pass a negative value such as CHAR_MIN (if *char* type is signed) to a character classification function in C. Though EOF value (-1 traditionally) should be handled). If you want to explore it further; I've enumerated open questions in 2014 (inconsistent TypeError, ord(c) > 0x100, negative ints handling, etc) http://bugs.python.org/issue9770#msg221008 -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27079> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27079] Bugs in curses.ascii predicates
Akira Li added the comment: There is an overlapping issue from 2010: "curses.ascii.isblank() function is broken. It confuses backspace (BS 0x08) with tab (0x09)" http://bugs.python.org/issue9770 Your patch fixes it too (it should be closed). Note: the patch does not pass tests from Lib/test/test_curses_ascii.py attached to issue9770 (even if the code: `if char_class_name in ('cntrl', 'punct') test = unittest.expectedFailure(test)` is removed) e.g., iscntrl(-1) should be False but it returns True: $ ./python >>> import curses.ascii >>> curses.ascii.iscntrl(-1) #XXX expected False True If we ignore negative ints then isblank, ispunct, iscntrl provided in the curses_ascii.patch are ok. -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27079> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27273] subprocess.run(cmd, input='text') should pass universal_newlines=True to Popen
New submission from Akira Li: At the moment, subprocess.run(cmd, input='text') raises TypeError. It would be nice if universal_newlines=isinstance(input, str) if *input* is set. I've attached a corresponding patch with the necessary changes to the docs, tests and the subprocess.run() code. -- components: Library (Lib) files: text_input.diff keywords: patch messages: 267936 nosy: akira priority: normal severity: normal status: open title: subprocess.run(cmd, input='text') should pass universal_newlines=True to Popen type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file43314/text_input.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27273> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27050] Demote run() below the high level APIs in subprocess docs
Akira Li added the comment: > setting "universal_newlines=True" switches to UTF-8 encoded text pipes It uses locale.getpreferredencoding(False) encoding -- something like cp1252,cp1251,etc on Windows, and UTF-8 on *nix with proper locale settings. It is ASCII (C/POSIX locale default) if the locale is not set in cron, ssh, init.d scripts, etc. If you need a different character encoding, you could use (instead of universal_newlines=True): pipe = io.TextIOWrapper(process.stdout, encoding=character_encoding) A better spelling for universal_newlines=True would be text_mode=True. A summary table (like in itertools' module) would be nice. check_output() name is unfortunate but it does the right thing and it is not hard to use for a beginner -- once somebody discovers it e.g., via "Running shell command from Python and capturing the output" Stack Overflow question http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output Compare: output = check_output([sys.executable, '-c', 'print("abc")']) and output = run([sys.executable, '-c', 'print("abc)'], stdout=PIPE).stdout The latter command doesn't raise exception if the child process fails. A beginner has to know about check=True to do the right thing: output = run([sys.executable, '-c', 'print("abc")'], stdout=PIPE, check=True).stdout It is easier to refer to check_output() if someone asks "How do I get command's output in Python?" I wish call() did what check_call() does and the current call() behavior would be achieved by the opposite parameter e.g. no_raise_on_status=False being the default: rc = call(command, no_raise_on_status=True) If we can't change the interface then check_call() is the answer to "Calling an external command in Python" question http://stackoverflow.com/questions/89228/calling-an-external-command-in-python - check_call(command) -- run command, raise if it fails - output = check_output(command) -- get command's output, raise if it fails. To pass *data* to the command via its standard input, pass input=data. To get/pass text (Unicode) instead of bytes, pass universal_newlines=True - check_call("a -- *.jpg | b 2>&1 >output | c", shell=True) -- run a shell command as is It is a pity that a list argument such as ["ls", "-l"] is allowed with shell=True These cover the most common operations with a subprocess. Henceforth, run() is more convenient if we don't need to interact with the child process while it is running. For example, if we introduce the word PIPE (a magic object in the kernel that connects processes) then to capture both standard and error streams of the command: cp = run(command, stdout=PIPE, stderr=PIPE) output, errors = cp.stdout, cp.stderr run() allows to get the output and to get the exit status easily: cp.returncode. Explicit cp.stdout_text, cp.stdout_bytes regardless the text mode would be nice. To interact with a child process while it is running, Popen() have to be used directly. There could be buffering and other issues (tty vs. pipe), see "Q: Why not just use a pipe (popen())?" http://pexpect.readthedocs.io/en/latest/FAQ.html#whynotpipe Working with both stdout/stderr or a non-blocking read require threads or asyncio, fcntl, etc. A couple of words should be said about killing a command started with shell=True. (to kill a process tree: set start_new_session=True parameter and call os.killpg()). timeout option doesn't work in this case (it uses Popen.kill()). check_output() unlike check_call() may wait for grandchildren if they inherit the pipe. Mention Job object on Windows e.g., http://stackoverflow.com/questions/23434842/python-how-to-kill-child-processes-when-parent-dies -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22274] subprocess.Popen(stderr=STDOUT) fails to redirect subprocess stderr to stdout
Akira Li added the comment: Updated the patch to address vadmium's review comments. -- versions: -Python 3.4 Added file: http://bugs.python.org/file42777/subprocess-stderr_redirect_with_no_stdout_redirect-2.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22274> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23220] Documents input/output effects of how IDLE runs user code
Akira Li added the comment: IDLE can implement functionality similar to what colorama [1] module does on Windows: translate ANSI escape character sequences into corresponding GUI method calls. For example, \b might be implemented using a .delete() call, \r using .mark_set(), etc. [1] https://pypi.python.org/pypi/colorama -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23220> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23570] Change "with subprocess.Popen():" (context manager) to ignore broken pipe error
Akira Li added the comment: Should this issue be reopened in light of http://bugs.python.org/issue26372 (Popen.communicate not ignoring BrokenPipeError)? If .close() shouldn't raise BrokenPipeError in .communicate() (and it shouldn't) then it seems logical that .close() shouldn't raise BrokenPipeError in .__exit__() too (and in other subprocess.Popen methods that do not return until the child process is dead such as subprocess.run()) -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23570> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25536] use sys.platform instead of os.name in asyncio docs consistently
New submission from Akira Li: asyncio code uses "sys.platform == 'win32'" to detect OS. asyncio docs use both os.name and sys.platform. As far as I can tell there is no *practical* difference between "os.name == 'nt" and "sys.platform == 'win32'" for choosing asyncio.ProactorEventLoop() I've attached a patch that replaces all os.name occurrences in asyncio docs with sys.platform. -- components: asyncio files: doc-asyncio-os.name->sys.platform.patch keywords: patch messages: 253931 nosy: akira, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: use sys.platform instead of os.name in asyncio docs consistently versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file40930/doc-asyncio-os.name->sys.platform.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25536> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25288] readline.py file in current directory caused unexpected code execution.
Akira Li added the comment: python3 -I could be used as a workaround. -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25288> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25286] views are not sequences
Akira Li added the comment: Thank you for `view`, hint. I did look for :term:`view` that was obviously not enough. The new patch contains the renamed entry in the correct place. All `view`, ` occurrences dictionary view are updated now. -- Added file: http://bugs.python.org/file40654/dict-views-glossary-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25286> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25286] views are not sequences
New submission from Akira Li: The entry for *dict view* in the glossary may be clarified, to avoid confusion with collection.abc.Sequence i.e., from: They are lazy sequences that will see changes in the underlying dictionary. to something like: They provide a dynamic view on the dictionary’s entries, which means that when the dictionary changes, the view reflects these changes. See https://mail.python.org/pipermail/python-ideas/2015-October/036682.html I've attached the corresponding doc patch. -- assignee: docs@python components: Documentation files: dict-views-glossary.patch keywords: patch messages: 251995 nosy: akira, docs@python priority: normal severity: normal status: open title: views are not sequences type: behavior versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file40638/dict-views-glossary.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25286> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: > Would issue22798.diff patch address your issue? No. The issue is that C mktime() may update C tzname on some platforms but time.mktime() does not update time.tzname on these platforms while the time module docs suggest that it might be expected e.g.: Most of the functions defined in this module call platform C library functions with the same name. It may sometimes be helpful to consult the platform documentation, because the semantics of these functions varies among platforms. --- Unrelated: time.strftime('%Z') and time.strftime('%Z', time.localtime(time.time())) can differ on some platforms and python versions http://stackoverflow.com/questions/32353015/python-time-strftime-z-is-always-zero-instead-of-timezone-offset -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24773] Implement PEP 495 (Local Time Disambiguation)
Changes by Akira Li <4kir4...@gmail.com>: -- nosy: +akira ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24773> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: Marc-Andre Lemburg <rep...@bugs.python.org> writes: ... > tzname is set when the module is being loaded and not updated > afterwards (unless you call tzset()). I can't really see why you > would expect a module global in Python to follow the semantics > of a C global, unless this is explicitly documented. Python docs recommend reading platform docs. Platform docs say that mktime() may change tzname. Python docs do not contradict. Why wouldn't I assume that mktime() may change tzname? > Note: The fact that tzset() does update the module globals is > not documented. It is documented e.g., I see: "These will be propagated into time.tzname" in tzset() docs. Though tzset() has nothing to do with the issue (TZ envvar hasn't been changed). >> Unrelated: time.strftime('%Z') and >> time.strftime('%Z', time.localtime(time.time())) can differ on some >> platforms and python versions >> >> http://stackoverflow.com/questions/32353015/python-time-strftime-z-is-always-zero-instead-of-timezone-offset > > The StackOverflow discussion targets %z (lower case z), > not %Z (with capital Z). Look at the answers. The examples clearly shows both %Z and %z. > I think this is just a documentation bug. > > Overall, I think that relying on those module globals is > not a good idea. They are not threadsafe and their values > can only be trusted right after module import. It's much > safer to access the resp. values by using struct_time values > or strftime(). > Agree. Moreover, you can't trust them even "right after module import" sometimes. Though, some predictability in the behavior such as "time.tzname is whatever C tzname is -- consult the platform docs" would be nice. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: The C code produces correct values according to the tz database. If TZ=Europe/Moscow then tzname={"MSK", "MSD"} at 2010-07-01 and tzname={"MSK", "MSK"} at 2015-07-01. Notice the difference! The code calls C mktime() with corresponding time tuples and checks that *tzname* is equal to the expected values. That's all. If C code is incomprehensible; here's its Python analog: >>> import os >>> os.environ['TZ'] = 'Europe/Moscow' >>> import time >>> time.tzset() >>> time.mktime((2010,7,1,0,0,0,-1,-1,-1)) 1277928000.0 >>> time.tzname #XXX expected ('MSK', 'MSD') ('MSK', 'MSK') >>> time.mktime((2015,7,1,0,0,0,-1,-1,-1)) 1435698000.0 >>> time.tzname ('MSK', 'MSK') C tzname changes on my machine after the corresponding C mktime() calls but Python time.tzname does not change after the time.mktime() calls. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: > C mktime itself should not change timezone globals, but it may indirectly if > it calls tzset() and TZ changed between calls. You should have run the attached test_mktime_changes_tzname.c which demonstrates that (at least on some systems) C mktime *does* change tzname global even if TZ envvar is constant in the test. Nowhere in my bug report I had mentioned different TZ values. I did mentioned *past* *future* dates -- the same timezone may have different utc offset, timezone abbreviations at different times. These timezone globals can change even if TZ is constant. -- status: pending -> open ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue22798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24881] _pyio checks that `os.name == 'win32'` instead of 'nt'
Akira Li added the comment: To make _pyio correspond to the C version I've added sys.platform in {'win32', 'cygwin'} condition. See the attached pyio_setmode.diff It is not clear why the absence of _setmode(fd, os.O_BINARY) is not detected by tests. (a) a corresponding test should be added (b) OR _setmode() call should be removed from both Python and C io versions -- keywords: +patch nosy: +akira Added file: http://bugs.python.org/file40222/pyio_setmode.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24881 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24842] Mention SimpleNamespace in namedtuple docs
Akira Li added the comment: People do have problems that SimpleNamespace can solve: - Why Python does not support record type i.e. mutable namedtuple [1] - Does Python have anonymous classes? [2] - How to create inline objects with properties in Python? [3] - python create object and add attributes to it [4] [1] http://stackoverflow.com/questions/5227839/why-python-does-not-support-record-type-i-e-mutable-namedtuple [2] http://stackoverflow.com/questions/1123000/does-python-have-anonymous-classes [3] http://stackoverflow.com/questions/1528932/how-to-create-inline-objects-with-properties-in-python [4] http://stackoverflow.com/questions/2827623/python-create-object-and-add-attributes-to-it -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24842 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19007] precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime
Changes by Akira Li 4kir4...@gmail.com: -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19007 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1191964] add non-blocking read and write methods to subprocess.Popen
Akira Li added the comment: I'm going to be honest; seeing None being returned from a pipe read feels *really* broken to me. When I get None returned from an IO read operation, my first instinct is there can't be anything else coming, why else would it return None? It is how it is done in similar cases (returning `None` to indicate would block). Do you know a better method that would allow to distinguish between EOF (no future data, ever) and would block (future data possible) without calling process.poll()? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1191964 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10482] subprocess and deadlock avoidance
Changes by Akira Li 4kir4...@gmail.com: -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10482 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23574] datetime: support leap seconds
Akira Li added the comment: POSIX timestamp doesn't count (literally) past/future leap seconds. It allows to find out that the timestamp 2**31-1 corresponds to 2038-01-19T03:14:07Z (UTC) regardless of how many leap seconds will occur before 2038: from datetime import datetime, timedelta str(datetime(1970,1,1) + timedelta(seconds=2**31-1)) '2038-01-19 03:14:07' If you use right timezone then mktime() may count leap seconds: $ TZ=right/UTC ./python import time time.mktime((2012, 6, 30, 23, 59, 59, -1, -1, -1)) 1341100823.0 time.mktime((2012, 6, 30, 23, 59, 60, -1, -1, -1)) 1341100824.0 time.mktime((2012, 7, 1, 0, 0, 0, -1, -1, -1)) 1341100825.0 It is a different time scale. There are no leap seconds in TAI: str(datetime(1970,1,1, 0,0, 10) + timedelta(seconds=1341100825)) '2012-07-01 00:00:35' i.e., 2012-07-01 00:00:35 TAI that corresponds to 2012-07-01 00:00:00 UTC. Each positive leap second increases the difference TAI-UTC (on 2015-07-01UTC it will be 36 [1]). TAI-UTC in the future (more than 6 months) is unknown but it is less than ~200 seconds until 2100 [2]. It might be convenient to think about datetime as a broken-down timestamp and therefore (datetime(2012,6,30,23,59,60) - epoch) == (datetime(2012,7, 1, 0, 0, 0) - epoch) The code [3] that silently truncates 60 to 59 when datetime constructor is called implicitly should retire. Use case: parse timestamps that might include a leap second [4] [1] https://hpiers.obspm.fr/iers/bul/bulc/bulletinc.dat [2] http://www.ucolick.org/~sla/leapsecs/year2100.html [3] https://bugs.python.org/msg155689 [4] http://stackoverflow.com/q/21027639 -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23574 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21619] Cleaning up a subprocess with a broken pipe
Akira Li added the comment: On Windows behavior http://stackoverflow.com/questions/23688492/oserror-errno-22-invalid-argument-in-subprocess -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21619 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23494] adding timedelta to datetime object is not timezone aware
Akira Li added the comment: pytz explicitly documents this case (crossing DST boundary). There is tz.normalize() method. the tzinfo object is responsible for handling daylight savings time. This looks like a bug in pytz. Are any of tzinfo methods even called during `before + timedelta(days=2)`? Also a DST transition is not the only reason the utc offset or tzname may change. -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23494 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Akira Li added the comment: As I've mentioned in http://bugs.python.org/issue22524#msg231703 os.walk size 7925376343, scandir.walk size 5534939617 -- NOT EQUAL! os.walk and scandir.walk do a different work here. I don't see that it is acknowledged so I assume the benchmark is not fixed. If the fixed (same work) benchmark is used then there is no performance difference on Linux. See the attached file http://bugs.python.org/file37292/get_tree_size_listdir.diff Note: the performance improvements on Windows along should be enough to accept os.scandir() even if Linux version won't be improved (if the behavior is documented). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22799] wrong time.timezone
Akira Li added the comment: Isn't this a duplicate of #13466? In what way is it a duplicate? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22799 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22799] wrong time.timezone
Akira Li added the comment: I agree that time.timezone, time.altzone is not enough in the general case. Because UTC offset may be different at different dates for reasons unrelated to DST transitions therefore any solution that doesn't take into account a given date/time into account will fail. *Nothing* works in the general case. Nothing. But it doesn't mean that the current behaviour of time.timezone can't be improved for this particular use-case: lt = time.localtime() # in a short-lived script assertEqual(lt.tm_gmtoff, -[time.timezone, time.altzone][lt.tm_isdst]) The test checks values for the current time (time.localtime()). It should work in *most* cases if time.timezone/altzone have correct values at import time. Perhaps synchronizing time.timezone with C timezone variable as I've mentioned before http://bugs.python.org/issue22798 may fix this issue too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22799 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23251] mention in time.sleep() docs that it does not block other Python threads
Akira Li added the comment: I've removed mentioning of GIL and uploaded a new patch. -- Added file: http://bugs.python.org/file37850/docs-time.sleep-other-threads-are-not-blocked-2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23251 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23315] tempfile.mkdtemp fails with non-ascii paths on Python 2
New submission from Akira Li: Python 2.7.9 (default, Jan 25 2015, 13:41:30) [GCC 4.9.2] on linux2 Type help, copyright, credits or license for more information. import os, sys, tempfile d = u'\u20ac'.encode(sys.getfilesystemencoding()) # non-ascii if not os.path.isdir(d): os.makedirs(d) ... os.environ['TEMP'] = d tempfile.mkdtemp(prefix=u'') Traceback (most recent call last): File stdin, line 1, in module File .../python2.7/tempfile.py, line 331, in mkdtemp file = _os.path.join(dir, prefix + name + suffix) File .../python2.7/posixpath.py, line 80, in join path += '/' + b UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 13: ordinal not in range(128) Related: https://bugs.python.org/issue1681974 -- components: Unicode messages: 234662 nosy: akira, ezio.melotti, haypo priority: normal severity: normal status: open title: tempfile.mkdtemp fails with non-ascii paths on Python 2 type: behavior versions: Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23315 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23320] devguide should mention rules about paragraph reflow in the documentation
New submission from Akira Li: It is suggested in https://bugs.python.org/issue23251 that only a core Python developer may reflow paragraphs while submitting patches for the Python documentation. It should be codified in devguide: I haven't found the word *reflow* in it. -- components: Devguide messages: 234692 nosy: akira, ezio.melotti priority: normal severity: normal status: open title: devguide should mention rules about paragraph reflow in the documentation type: behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23320 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23251] mention in time.sleep() docs that it does not block other Python threads
Akira Li added the comment: Only if the behaviour was unintuitive (i.e. if it *didn't* release the GIL) would it make sense to document it. There is no intuitive interface, not even the nipple. It's all learned. [1] Yes, on consideration I agree with Antoine. That last sentence should be deleted. Otherwise we'd need to mention that the gil was released every place that the gil was released, which would be very redundant. Whether or not other places mention it is unrelated to the current issue. Though the documentation should mention it more. Many programmers are convinced that Python threads can't be executed in parallel. The general rule is that anything that blocks in python releases the GIL, therefore as Antoine says the only time we need to document GIL behavior is when that *doesn't* happen. The reader of Python documentation is intelegent but not all-knowing. Blocking is the first and only job for time.sleep() function. GIL blocks Python code. You can't understand what time.sleep does without knowing what happens to GIL. Ask yourself who and why reads the time.sleep() documentation (novice and/or exprerienced Python user). Put yourself into the mind of the reader. [1] http://www.greenend.org.uk/rjk/misc/nipple.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23251 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23251] mention in time.sleep() docs that it does not block other Python threads
Akira Li added the comment: I think it's superfluous to mention the GIL here, since it has no impact on the function. If GIL is not released then all Python code in other threads is effectively blocked. It is worth mentioning explicitly that it is guaranteed to be released during the sleep. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23251 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23251] mention in time.sleep() docs that it does not block other Python threads
New submission from Akira Li: There is the corresponding StackOverflow question with 60K view time.sleep — sleeps thread or process? [1] The documentation patch is attached. [1] http://stackoverflow.com/questions/92928/time-sleep-sleeps-thread-or-process -- assignee: docs@python components: Documentation files: docs-time.sleep-other-threads-are-not-blocked.diff keywords: patch messages: 234135 nosy: akira, docs@python priority: normal severity: normal status: open title: mention in time.sleep() docs that it does not block other Python threads type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file37730/docs-time.sleep-other-threads-are-not-blocked.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23251 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23251] mention in time.sleep() docs that it does not block other Python threads
Akira Li added the comment: I do not understand. Have you tried to look at the patch in Rietveld? The new content is highlighted in a darker green. It is clearly visible. I've tested on Chromium, Firefox, Safari. If I won't reflow then the first line will be longer than the recommended 80 in devguide: The maximum line length is 80 characters for normal text, but tables, deeply indented code samples and long links may extend beyond that. I've set *fill-column* to 80 in emacs. Do you suggest other settings? Anyway, it doesn't affect how the final text is shown in a web browser. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23251 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22932] email.utils.formatdate uses unreliable time.timezone constant
Akira Li added the comment: @mitya57: Please, combine the code changes, tests, docs into a single rietveld-compatible patch (hg diff); read devguide and http://bugs.python.org/issue13963 Make sure review link appears on the right near the patch. Example: http://bugs.python.org/issue22798 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22932 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22932] email.utils.formatdate uses unreliable time.timezone constant
Changes by Akira Li 4kir4...@gmail.com: -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22932 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22673] document the special features (eg: fdclose=False) of the standard streams
Akira Li added the comment: Two minor details: 1. It is possible that `fileno(stdout) != 1` even in C [1]. I don't know what happens if the code from the answer is run on Windows. In principle, it may break eryksun's workaround. I don't know how likely it is in practice. 2. you can redirect at the file descriptor level in Python [2] using os.dup2(). I don't know whether the code in the answer works on Windows. [1] http://stackoverflow.com/questions/25516375/is-it-possible-that-filenostdout-1-on-a-posix-system [2] http://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python/22434262#22434262 -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22673 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23017] string.printable.isprintable() returns False
Akira Li added the comment: C standard defines locale-specific *printing characters* that are [ -~] in C locale for implementations that use 7-bit US ASCII character set i.e., SP (space, 0x20) is a printing character in C (isprint() returns nonzero). There is isgraph() function that returns zero for the space but otherwise is equivalent to isprint(). POSIX definition is aligned with the ISO C standard. I don't know what RFC 5822 has to do with this issue but the rfc contradicts itself e.g., in one place it has: printable US-ASCII characters except SP that imlies that SP *is* printable but in other places it considers isprint==isgraph. The authors probably meant characters for which isgraph() is nonzero when they use printable US-ASCII (that is incorrect according to C standard). Tests from issue9770 show the relation between C character classes and string constants [1]: set(string.printable) == set(C['graph']) + set(C['space']) where C['space'] is '\t\n\v\f\r ' (the standard C whitespace). It is a documented behavior [2]: This is a combination of digits, ascii_letters, punctuation, and whitespace where *whitespace* is C['space']. In Python 2, *printable* is locale-dependent and it coincides with the corresponding Python 3 definition in C locale with ASCII charset. Unlike other string constants, *printable* differs from C['print'] on both Python 2 and 3 because it includes whitespace characters other than space. str.isprintable [3] obeys C['print'] (in ASCII range) and considers SP to be printable. --- It might be too late to change string.printable to correspond to C isprint() (for ASCII characters). I've uploaded a documentation patch that mentions that string.printable and str.isprintable differ. [1] http://bugs.python.org/review/9770/diff/12212/Lib/test/test_curses_ascii.py [2] https://hg.python.org/cpython/file/3.4/Doc/library/string.rst#l62 [3] https://docs.python.org/3.4/library/stdtypes.html#str.isprintable -- nosy: +akira Added file: http://bugs.python.org/file37441/docs-string.printable.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23017 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22356] mention explicitly that stdlib assumes gmtime(0) epoch is 1970
Akira Li added the comment: Alexander Belopolsky added the comment: I've provide the direct quote from *C* standard ... I understand that C standard uses the word encoding, but it does so for a reason that is completely unrelated to the choice of epoch. Encoding is how the bytes in memory should be interpreted as number of seconds or some other notion of time. For, example two's complement little-endian 32-bit signed int is an example of valid time_t encoding, another example would be IEEE 754 big-endian 64-bit double. Note that these choices are valid for both C and POSIX standards. I agree one *part* of encoding is how time_t is *represented* in memory but it is not the only part e.g.: The mktime function converts the broken-down time, expressed as local time, in the structure pointed to by timeptr into a calendar time value with the same encoding as that of the values returned by the time function. notice: the same encoding as ... returned by the time function. time() function can return values with different epoch (implementation defined). mktime() is specified to use the *same* encoding i.e., the same epoch, etc. i.e., [in simple words] we have calendar time (Gregorian date, time) and we can convert it to a number (e.g., Python integer), we can call that number seconds and we can represent that number as some (unspecified) bit-pattern in C. I consider the whole process of converting time to a bit-pattern in memory as encoding i.e., 32/64, un/signed int/754 double is just *part* of it e.g., 1. specify that 1970-01-01T00:00:00Z is zero (0) 2. specify 0 has time_t type 3. specify how time_t type is represented in memory. I may be wrong that C standard includes the first item in time encoding. If you google for your phrase time in POSIX encoding, this issue is the only hit. This strongly suggests that your choice of words is not the most natural. I've googled the phrase (no surrounding quotes) and the links talk about time encoded as POSIX time [1] and some *literally* contain the phrase *POSIX encoding* [2] because *Python* documentation for calendar.timegm contains it [3]: [timegm] returns the corresponding Unix timestamp value, assuming an epoch of 1970, and the POSIX encoding. In fact, time.gmtime() and timegm() are each others’ inverse. In an effort to avoid personal influence, I've repeated the expreriment using Tor browser and other search engines -- the result is the same. timegm() documentation might be the reason why I've used the phrase. I agree POSIX encoding might be unclear. The patch could be replaced by any phrase that expresses that some functions in stdlib assume that time.time() returns (+/- fractional part) seconds since the Epoch as defined by POSIX [4]. [1] http://en.wikipedia.org/wiki/Unix_time#Encoding_time_as_a_number [2] http://ruslanspivak.com/2011/07/20/how-to-convert-python-utc-datetime-object-to-unix-timestamp/ [3] https://docs.python.org/3/library/calendar.html#calendar.timegm [4] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_15 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22356 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22356] mention explicitly that stdlib assumes gmtime(0) epoch is 1970
Akira Li added the comment: Alexander Belopolsky added the comment: 1. It is not the job of the time module documentation to warn about many functions in the stdlib. What are these functions, BTW? The e-mail linked in the first message of this issue msg226539 enumerates some of the functions: https://mail.python.org/pipermail/python-ideas/2014-September/029228.html 2. What is calendar time in POSIX encoding? This sounds like what time.asctime() returns. It is the language used by C standard for time() function: The time function determines the current calendar time. The encoding of the value is unspecified. I think an improvement would be to spell Epoch with a capital E and define it as The time zero hours, zero minutes, zero seconds, on January 1, 1970 Coordinated Universal Time (UTC). See http://pubs.opengroup.org/onlinepubs/9699919799. The word *epoch* (lowercase) is used by C standard. It is not enough to say that time module uses POSIX epoch (Epoch) e.g., a machine may use right zoneinfo (the same epoch year 1970) but the timestamp for the same UTC time are different by number of leap seconds (10+25 since 2012). POSIX encoding implies that the formula works: utc_time = datetime(1970, 1, 1) + timedelta(seconds=posix_timestamp) if time.time() doesn't return posix_timestamp than many functions in the stdlib will break. It is possible to inspect all stdlib functions that use time module and determine for some of them whether they will break if gmtime(0) is not 1970 or right zoneinfo is used or any non-POSIX time encoding is used. But it is hard to maintain such a list because any future code change may affect the behavior. I prefer a vague statement (many functions) over a possible lie (the documentation shouldn't make promises that the implementation can't keep). POSIX language is (intentionally) vague and avoids SI seconds vs. UT1 (mean solar) seconds distinction. I don't consider systems where seconds doesn't mean SI seconds used by UTC time scale. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22356 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22356] mention explicitly that stdlib assumes gmtime(0) epoch is 1970
Akira Li added the comment: Alexander Belopolsky added the comment: In the context of Python library documentation, the word encoding strongly suggests that you are dealing with string/bytes. The situation may be different in C. If you want to refer to something that is defined by the POSIX standard you should use the words that can actually be found in that standard. When I search for encoding at http://pubs.opengroup.org/onlinepubs/9699919799/, I get crypt - string encoding function (CRYPT) encrypt - encoding function (CRYPT) setkey - set encoding key (CRYPT) and nothing related to time. I've provide the direct quote from *C* standard in my previous message msg231957: 2. What is calendar time in POSIX encoding? This sounds like what time.asctime() returns. It is the language used by C standard for time() function: The time function determines the current calendar time. The encoding of the value is unspecified. ^ - from the C standard notice the word *encoding* in the quote. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22356 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22799] wrong time.timezone
Akira Li added the comment: This issue could be fixed using sync-time-timezone-attr-with-c.diff patch from http://bugs.python.org/issue22798 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22799 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Akira Li added the comment: STINNER Victor added the comment: scandir is slower on my machine: Please share more information about your config: OS, disk type (hard drive, SSD, something else), filesystem, etc. Ubuntu 14.04, SSD, ext4 filesystem. Results for different python versions are the same (scandir on pypy is even slower due to ctype usage). What other information could be useful? Ben Hoyt added the comment: Akira, note the Using slower ctypes version of scandir -- this is the older, ctypes implementation of scandir. On Linux, depending on file system etc, that can often be slower. You need to at least be using the fast C version in _scandir.c, which is then half C, half Python. But ideally you'd use the all-C version that I've provided as a CPython 3.5 patch. Yes. I've noticed it, that is why I asked :) *What commands should I run to benchmark the attached patch (scandir-2.patch)?* I've read benchmark.py's source; It happens that it already supports benchmarking of os.scandir. In my python checkout: cpython$ hg import --no-commit https://bugs.python.org/file36963/scandir-2.patch cpython$ make cd ../scandir/ scandir$ ../cpython/python benchmark.py -s /usr/ Using Python 3.5's builtin os.scandir() Comparing against builtin version of os.walk() Priming the system's cache... Benchmarking walks on /usr/, repeat 1/3... Benchmarking walks on /usr/, repeat 2/3... Benchmarking walks on /usr/, repeat 3/3... os.walk size 7925376343, scandir.walk size 5534939617 -- NOT EQUAL! os.walk took 13.552s, scandir.walk took 6.032s -- 2.2x as fast It seems os.walk and scandir.walk do a different work here. I've written get_tree_size_listdir_fd() that mimics get_tree_size() (see get_tree_size_listdir.diff patch) to get the same size: scandir$ ../cpython/python benchmark.py -s /usr Using Python 3.5's builtin os.scandir() Comparing against builtin version of os.walk() Comparing against get_tree_size_listdir_fd Priming the system's cache... Benchmarking walks on /usr, repeat 1/3... Benchmarking walks on /usr, repeat 2/3... Benchmarking walks on /usr, repeat 3/3... os.walk size 5534939617, scandir.walk size 5534939617 -- equal os.walk took 5.697s, scandir.walk took 5.621s -- 1.0x as fast scandir is not noticeably faster but scandir-based code is much nicer here. -- Added file: http://bugs.python.org/file37284/get_tree_size_listdir.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Akira Li added the comment: To see what happens at syscall level, I've run various implementations of get_tree_size() functions (see get_tree_size_listdir.diff) with strace: get_tree_size_listdir_fd -- os.listdir(fd) + os.lstat get_tree_size -- os.scandir(path) + entry.lstat get_tree_size_listdir_stat -- os.listdir(path) + os.lstat get_tree_size_listdir -- os.listdir(path) + os.path.isdir + os.lstat Summary: - os.listdir() and os.scandir()-based variants use the same number of getdents() syscalls - and the number of openat, lstat (newfstatat) syscalls is also comparable (except for the variant that uses os.path.isdir) Log: scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_fd as f; import os; print(f(os.open(/usr/, os.O_RDONLY)))' head -7 py.strace 5535240217 11.29user 8.14system 0:17.78elapsed 109%CPU (0avgtext+0avgdata 13460maxresident)k 0inputs+8outputs (0major+6781minor)pagefaults 0swaps % time seconds usecs/call callserrors syscall -- --- --- - - 46.510.075252 0264839 newfstatat 16.880.027315 1 50460 getdents 11.530.018660 0 75621 fcntl 9.740.015758 0 50531 close 6.870.06 0 25214 openat scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size as f; print(f(/usr/))' head -7 py.strace 5535240217 22.56user 8.47system 0:29.77elapsed 104%CPU (0avgtext+0avgdata 13280maxresident)k 0inputs+8outputs (0major+6306minor)pagefaults 0swaps % time seconds usecs/call callserrors syscall -- --- --- - - 62.000.032822 0239644 lstat 19.970.010570 0 50460 getdents 8.520.004510 0 25215 openat 6.090.003224 0 25326 close 0.550.000292 395 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir_stat as f; print(f(/usr/))' head -7 py.strace 5535240217 13.70user 6.30system 0:18.84elapsed 106%CPU (0avgtext+0avgdata 13456maxresident)k 0inputs+8outputs (0major+6769minor)pagefaults 0swaps % time seconds usecs/call callserrors syscall -- --- --- - - 64.790.050217 0264849 lstat 19.370.015011 0 50460 getdents 8.220.006367 0 25215 openat 5.760.004465 0 25326 close 0.320.000247 2 114 mmap scandir$ /usr/bin/time strace -fco py.strace ../cpython/python -c 'from benchmark import get_tree_size_listdir as f; print(f(/usr/))' head -7 py.strace 5535240217 19.53user 10.61system 0:28.16elapsed 107%CPU (0avgtext+0avgdata 13452maxresident)k 0inputs+8outputs (0major+6733minor)pagefaults 0swaps % time seconds usecs/call callserrors syscall -- --- --- - - 42.040.063050 0265195 37259 stat 37.400.056086 0265381 lstat 11.460.017187 0 50460 getdents 4.820.007232 0 25215 openat 3.430.005139 0 25326 close -- Added file: http://bugs.python.org/file37292/get_tree_size_listdir.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Changes by Akira Li 4kir4...@gmail.com: Removed file: http://bugs.python.org/file37284/get_tree_size_listdir.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: One of the ways to fix this issue is to synchronize time.tzname attribute with the corresponding C tzname variable. I've uploaded sync-time-timezone-attr-with-c.diff patch that synchronizes tzname, timezone, altzone, daylight attributes. The patch also includes tests. No docs changes are necessary. The code follows C, POSIX standards and the time module documentation. Any differences are unintetional. The patch also fixes wrong time.timezone issue http://bugs.python.org/issue22799 The patch doesn't use tm_gmtoff, tm_zone C values from jan, jul to set timezone, tzname time module attributes therefore it may break software that relies on that behaviour. I would be interested to hear about such instances. I've removed the cygwin branch in the code (I haven't found cygwin buildbot). It could be added back as is if necessary. -- Added file: http://bugs.python.org/file37273/sync-time-timezone-attr-with-c.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Changes by Akira Li 4kir4...@gmail.com: Removed file: http://bugs.python.org/file37132/test_mktime_changes_tzname.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Akira Li added the comment: scandir is slower on my machine: $ git clone https://github.com/benhoyt/scandir $ cd scandir/ $ ../cpython/python benchmark.py /usr/ Using slower ctypes version of scandir Comparing against builtin version of os.walk() Priming the system's cache... Benchmarking walks on /usr/, repeat 1/3... Benchmarking walks on /usr/, repeat 2/3... Benchmarking walks on /usr/, repeat 3/3... os.walk took 7.761s, scandir.walk took 10.420s -- 0.7x as fast What commands should I run to benchmark the attached patch (scandir-2.patch)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22752] incorrect time.timezone value
Akira Li added the comment: C standard delegates to implementations: The local time zone and Daylight Saving Time are implementation-defined. gcc (one of the implementations) says [1]: [timezone] contains the difference between UTC and the latest local standard time, in seconds west of UTC. Notice the word latest. To be fair, gcc (the actual implementation) uses a weird logic to assign timezone, daylight values in Europe/Moscow timezone in 2011-2015. Python own tests assume that time.timezone reflects the *current* (most recent) time http://bugs.python.org/issue22799 C tzname, timezone, daylight are not constants. They may change e.g., see http://bugs.python.org/issue22798 (the C code demonstrates that tzname changes even for the same local timezone). POSIX specifies a profile (additional restrictions but no conflict) of C standard for the time functions e.g., tzset(), mktime() [2], ctime(), localtime() may change tzname, timezone, daylight (but *not* gmtime() or asctime()). To summarize: - timezone is implementation-defined in C - gcc (one of implementations) says that timezone corresponds to the latest time (the actual behavior is complicated) - POSIX says that time functions shall set timezone info i.e., timezone is not a constant (it is obvious due to tzset() existence) - Python tests assume that timezone corresponds to the current time (not january or july) I'm not sure what time.timezone behaviour should be: - leave it as is (unsynchronized with C values and it may be different from the current correct value) -- it makes time module unusable in some timezones (60% all time, 11% since 2001), some of the time - use latest correct (gmtoff where available) value during importing the time module -- it would fix this issue22752 - synchronize with C values -- it improves some things e.g., it may fix http://bugs.python.org/issue22426 but it might make the behavior more platform dependent (tests needed) and it make things worse in some timezones e.g., where dst() != 1 hour (13% all time, 0.9% since 2000) [1] http://www.gnu.org/software/libc/manual/html_node/Time-Zone-Functions.html [2] http://pubs.opengroup.org/onlinepubs/9699919799/functions/mktime.html -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22752 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21872] LZMA library sometimes fails to decompress a file
Akira Li added the comment: If lzma._BUFFER_SIZE is less than 2048 then all example files are decompressed successfully (at least lzma module produces the same results as xz utility) -- Added file: http://bugs.python.org/file37241/decompress-example-files.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21872 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21872] LZMA library sometimes fails to decompress a file
Changes by Akira Li 4kir4...@gmail.com: Removed file: http://bugs.python.org/file37240/decompress-example-files.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21872 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21872] LZMA library sometimes fails to decompress a file
Akira Li added the comment: @Esa changing the buffer size helps with some bad files but lzma module still fails on some files. I've uploaded decompress-example-files.py script that demonstrates it. -- nosy: +akira Added file: http://bugs.python.org/file37239/decompress-example-files.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21872 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21872] LZMA library sometimes fails to decompress a file
Changes by Akira Li 4kir4...@gmail.com: Added file: http://bugs.python.org/file37240/decompress-example-files.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21872 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21872] LZMA library sometimes fails to decompress a file
Changes by Akira Li 4kir4...@gmail.com: Removed file: http://bugs.python.org/file37239/decompress-example-files.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21872 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22536] subprocess should include filename in FileNotFoundError exception
Akira Li added the comment: It would be inconsitent to provide filename only if exec is called e.g.: import subprocess subprocess.call(not used, cwd=nonexistent) FileNotFoundError: [Errno 2] No such file or directory: 'nonexistent' The error message shows the filename (cwd) despite exec not being called therefore it would be logical to provide non-None `filename` attribute here too. If we ignore the backward compatibility issue I've mentioned in msg231297 then the current code: if errno_num == errno.ENOENT: if child_exec_never_called: # The error must be from chdir(cwd). err_msg += ': ' + repr(cwd) else: err_msg += ': ' + repr(orig_executable) raise child_exception_type(errno_num, err_msg) could be replaced with: if errno_num == errno.ENOENT: if child_exec_never_called: # The error must be from chdir(cwd). filename = cwd else: filename = orig_executable raise child_exception_type(errno_num, err_msg, filename) raise child_exception_type(errno_num, err_msg) [1] https://hg.python.org/cpython/file/23ab1197df0b/Lib/subprocess.py#l1443 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22536 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22536] subprocess should include filename in FileNotFoundError exception
Akira Li added the comment: I can confirm that without the patch the filename attribute is None despite being mentioned in strerror. Travis, you should use `orig_executable` instead of `args[0]` to cover: subprocess.call(exit 0, shell=True, executable='/nonexistent bash') case. And use `cwd` if `child_exec_never_called`, to be consistent with the error message (see if/else statements above the raise statement). It seems appropriate to set filename even if errno is not ENOENT but to be conservative, you could provide filename iff `err_msg` is also changed i.e., iff errno is ENOENT. -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22536 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22536] subprocess should include filename in FileNotFoundError exception
Akira Li added the comment: If the_oserror.filename is not None then str(the_oserror) appends the filename twice: [Errno 2] No such file or directory: 'nonexistent': 'nonexistent' You could remove `err_msg += ':' ...` statements to avoid the repeatition. It may break the code that uses strerror attribute. But exception error messages are explicitly excluded from backward compatibility considirations therefore it might be ok to break it here. I can't find the reference so it should probably be resolved as a new issue (independent from providing the filename attribute value). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22536 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22791] datetime.utcfromtimestamp() shoud have option for create tz aware datetime
Akira Li added the comment: I agree the documentation should nudge towards aware datetime objects. I've attached a documentation patch as an example. -- keywords: +patch Added file: http://bugs.python.org/file37162/issue22791-utcfromtimestamp-aware.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22791 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22791] datetime.utcfromtimestamp() shoud have option for create tz aware datetime
Akira Li added the comment: from datetime import datetime, timezone datetime.fromtimestamp(0, timezone.utc) datetime.datetime(1970, 1, 1, 0, 0, tzinfo=datetime.timezone.utc) already works and it is documented [1] [1] https://docs.python.org/3/library/datetime.html#datetime.datetime.fromtimestamp Or it can be written as: epoch = datetime(1970, 1, 1, tzinfo=timezone.utc) aware_utc = epoch + timedelta(seconds=posix_timestamp) -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22791 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16353] add function to os module for getting path to default shell
Akira Li added the comment: Matt Frank added the comment: In msg230720 Akira Li (akira) wrote: os.defpath is supposed to be ':'+CS_PATH, e.g., look at glibc (C library used on Linux) sysdeps/posix/spawni.c I don't know whether it is possible to change CS_PATH without recompiling every statically linked executable on a system, sysdeps/unix/confstr.h: #define CS_PATH /bin:/usr/bin Though this issue is about the path to the standard shell sh/cmd.exe. It is not about os.defpath. I understand this issue is about the path to the standard shell. My argument is that os.defpath is the wrong tool for finding that path, and os.confstr('CS_PATH') is the correct tool, as you originally suggested in msg224514. I'll repeat os.defpath definition that I've cited in the same message msg224514: The default search path used by exec*p* and spawn*p* if the environment doesn’t have a 'PATH' key. Also available via os.path. 1. os.defpath should work (contain the path to the standard shell) everywhere where os.confstr(CS_PATH) works. 2. And it might be easier to customize e.g., on systems where there is no os.confstr or where changing CS_PATH involves recompiling crucial system processes. If these two statements are not true then os.defpath could be dropped. The patch [1] has tests. Have you tried to run them? The tests *pass* at least on one system ;) [...] os.get_shell_executable() does not use cwd. Neither the documentation nor the code in the patch suggest that. I ran the tests (and the entire Python test suite) before I wrote my first message. But you didn't test what happens when there is a bogus 'sh' somewhere along the path. You also have a bug in your path searching loop that interprets an empty element on the path as /. Here's the current failure mode: Go to root on your Posix system. With su or sudo create an executable file named sh. (for example cd /; sudo touch sh; sudo chmod +x sh). Then go back to your python interpreter (with the patch applied) and run os.get_shell_executable(). The returned result will be '/sh'. But if you fix the loop to iterate over the path correctly (including 'cwd') then you will find that putting an executable named 'sh' in the current working directory will make os.get_shell_executable() return the sh in the current working directory (because os.defpath says it should). Comment in the patch explicitly says that '' is interpreted as '/'. The intent is to exclude the current working directory, thinking that /sh executable can create only root. And root can do anything to the machine anyway. I agree. It is a misfeature. I've uploaded a new patch that excludes '' completely. The previous code tried to be too cute. The loop _should_ be treating empty path elements as current working directory. (In the glibc sysdeps/posix/spawni.c file you pointed to look around line 281, by the comment that says /* Two adjacent colons, or a colon at the beginning or the end of 'PATH' means to search the current directory.*/) yes. Empty path element stands for the current working directory e.g., $ PATH= python $ PATH=: python $ PATH=:/ python $ PATH=/: python run ./python on my system. The current working directory is intentionally excluded (in the original and in the current patches) otherwise os.get_exec_path(env={}) would be used. Note also, that glibc's spawni() uses two different default paths. One is for if the user didn't specify their PATH variable (that's the one where they use :/bin:/usr/bin) and a completely different (built in path is used in the spawn*p* version) if it turns out that the file is a script that needs to run under the system sh. (On line 290 they call maybe_script_execute(), which calls script_execute(), which uses _PATH_BSHELL.) glibc hardcodes the path (like subprocess module does currently): #define _PATH_BSHELL/bin/sh os.get_shell_executable() is an enhancement that allows (Unix) systems to configure the path via os.defpath. But os.defpath doesn't work on Android either (because it is hardcoded to ':/bin:/usr/bin'). As I've mentioned in msg224514 before posting my original patch: - Andriod uses /system/bin/sh - os.defpath could be tweaked on Android i.e., yes. Default os.defpath doesn't work there and it should be configured on Android to include the path to the standard shell. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16353 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16353] add function to os module for getting path to default shell
Changes by Akira Li 4kir4...@gmail.com: Added file: http://bugs.python.org/file37140/os.get_shell_executable-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16353 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
New submission from Akira Li: time.tzname is initialized from C tzname variable or tm_zone around Jan, Jul of the current year. If time.mktime() is called with a time tuple from the past/future then after the call time.tzname might be out-of-sync with the corresponding C tzname and tm_zone values. Because C mktime may change tzname, tm_zone values on some systems and time.mktime calls C mktime. I've attached test_mktime_changes_tzname.c file that demonstrates that mktime() may change tzname. -- components: Library (Lib) files: test_mktime_changes_tzname.c messages: 230674 nosy: akira priority: normal severity: normal status: open title: time.mktime doesn't update time.tzname type: behavior versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file37132/test_mktime_changes_tzname.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Akira Li added the comment: I've attached test-timezone-info-is-updated.diff file -- a patch for Lib/test/test_time.py that demonstrates that time functions fail to update the timezone info. The test uses Europe/Moscow timezone but most timezones around the world had/will have different tzname/utc offset (unrelated to DST changes) in the past/future. -- keywords: +patch Added file: http://bugs.python.org/file37133/test-timezone-info-is-updated.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22798] time.mktime doesn't update time.tzname
Changes by Akira Li 4kir4...@gmail.com: Added file: http://bugs.python.org/file37134/test_mktime_changes_tzname.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22798 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22799] wrong time.timezone
New submission from Akira Li: $ TZ=:Europe/Moscow ./python -mtest -v test_time == FAIL: test_localtime_timezone (test.test_time.TestPytime) -- Traceback (most recent call last): File .../Lib/test/test_time.py, line 721, in test_localtime_timezone self.assertEqual(lt.tm_gmtoff, -[time.timezone, time.altzone][lt.tm_isdst]) AssertionError: 10800 != 14400 -- Ran 45 tests in 1.832s FAILED (failures=1, skipped=3) test test_time failed 1 test failed: test_time UTC offset had changed on 2014-10-26 in Europe/Moscow timezone from MSK+0400 to MSK+0300. Python time.timezone returns -14400 (old utc offset) despite C timezone variable having the correct value (Python uses tm_gmtoff from Jan here). Similar case where timezone, altzone may be wrong http://bugs.python.org/msg31138 The issue again http://bugs.python.org/issue22798 is that time timezone attribute is out-of-sync with the corresponding C variable i.e., C library provides correct values but time module doesn't use them. -- components: Library (Lib) messages: 230684 nosy: akira priority: normal severity: normal status: open title: wrong time.timezone type: behavior versions: Python 3.4, Python 3.5, Python 3.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22799 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16353] add function to os module for getting path to default shell
Akira Li added the comment: Matt Frank added the comment: Unfortunately os.defpath seems to be hardcoded. And hardcoded to the wrong value on every system I have looked at, including Linux. os.defpath is supposed to be ':'+CS_PATH, e.g., look at glibc (C library used on Linux) sysdeps/posix/spawni.c I don't know whether it is possible to change CS_PATH without recompiling every statically linked executable on a system, sysdeps/unix/confstr.h: #define CS_PATH /bin:/usr/bin Though this issue is about the path to the standard shell sh/cmd.exe. It is not about os.defpath. The patch [1] has tests. Have you tried to run them? The tests *pass* at least on one system ;) [1] http://bugs.python.org/review/16353/#ps12569 To run the tests, download the patch into your python checkout: $ curl -O \ http://bugs.python.org/file36196/os.get_shell_executable.patch and apply it: $ patch -p1 os.get_shell_executable.patch to run only test_os.py: $ ./python -mtest test_os Lib/posixpath.py sets defpath=':/bin:/usr/bin' which is _not_ what getconf CS_PATH` returns on my Linux (the extra ':' at the beginning means include the cwd which is almost certainly not what anyone wants.) os.get_shell_executable() does not use cwd. Neither the documentation nor the code in the patch suggest that. The hardcoded value '/bin:/usr/bin' would also be wrong for Solaris and AIX installations that are trying to be POSIX (I think they use /usr/xpg4/bin/sh). os.defpath is the default search path used by exec*p* and spawn*p* i.e., if os.defpath is incorrect for these system then os.exec*p* and os.spawn*p* functions could also be broken. I expect that Windows, OS X, Linux work as is. If the tests fail on Solaris, AIX then os.defpath should be tweaked on these systems if it hasn't been already. Note: os.defpath is a very conservative value: it should be set at python's installation time at the very latest. Henceforth it should remain the same. And Lib/ntpath.py sets defpath='.;C:\\bin', which doesn't resemble a path that even works (as pointed out in now closed http://bugs.python.org/issue5717). (The correct value may be %SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\'.) Please, look at the patch. Neither the documentation nor the code suggest that os.defpath is used on Windows. So I don't know where to go next. I'm happy to cook up a different patch, but I have no idea what it should be. Here are some possibilities: 1. Kick the can down the road: file a new bug against os.defpath and (somehow) fix Lib/*path.py so they do something more reasonable in more cases. One of which might be to have posixpath.py try to call os.confstr() (and then, perhaps, special-code something in Modules/posixmodule.c in the case that HAVE_CONFSTR is not defined.) 2. Modify @akira's patch to call os.confstr('CS_PATH') instead of os.defpath, and then deal with the fact that very few systems actually implement confstr() (perhaps by special-coding something in Modules/posixmodule.c as described above.) Note I'm the author of http://bugs.python.org/issue16353#msg224514 message that references the POSIX recommendation about `getconf PATH` and *explicitly* mentions os.confstr('CS_PATH'). I don't remember exactly the motivation to use os.defpath instead of os.confstr('CS_PATH'). A guess: the result is the same (a lone `:` is ignored in the patch) but os.defpath is easier to modify for uncommon systems where os.confstr might not be available. I expect that the tests in the patch will pass on all stable buildbots [2] successfully without any modifications (with os.defpath as is). [2] http://buildbot.python.org/all/waterfall?category=3.x.stable Other systems should probably configure os.defpath appropriately (it should include the path to the standard shell). 3. Modify this patch to fall back to `PATH` if `sh` can't be found on os.defpath (or os.confstr('CS_PATH') fails). The standard shell should not depend on PATH envvar. The tests show that os.get_shell_executable() works even with an empty environment. The motivation is the same as why `getconf PATH` exists in the first place. The documentation for os.get_shell_executable() (from the patch): Return a path to the standard system shell executable -- ``sh`` program on POSIX (default: ``'/bin/sh'``) or ``%ComSpec%`` command processor on Windows (default: ``%SystemRoot%\system32\cmd.exe``). Availability: Unix, Windows The intent is that os.get_shell_executable() returns the same *shell path* as used by system(3) on POSIX i.e., (ignoring signals, etc) os.system(command) could be implemented: subprocess.call(command, shell=True, executable=os.get_shell_executable()) (presumably subprocess can use get_shell_executable() internally instead of /bin/sh but it is another issue) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org
[issue22749] remove obsolete remark in time.clock() docs
New submission from Akira Li: time.clock() documentation [1] says: this is the function to use for benchmarking Python or timing algorithms. and Deprecated since version 3.3: The behaviour of this function depends on the platform: use perf_counter() or process_time() instead, depending on your requirements, to have a well defined behaviour. [1]: https://hg.python.org/cpython/file/a22ef88143b9/Doc/library/time.rst#l127 The first remark is incorrect since 3.3. I've attached a documentation patch that removes it. -- assignee: docs@python components: Documentation files: docs-time-clock-remove-stale-remark.diff keywords: patch messages: 230124 nosy: akira, docs@python priority: normal severity: normal status: open title: remove obsolete remark in time.clock() docs type: behavior versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file37048/docs-time-clock-remove-stale-remark.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22749 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22715] PEP 257: drop the recommendation for a blank line between the class line and the docstring
New submission from Akira Kitada: The PEP 257 says: Insert a blank line before and after all docstrings (one-line or multi-line) that document a class Looking at stdlib and popular open source software, I couldn't find packages that follow this style, so I think this is not really a convention. -- files: pep-0257.patch keywords: patch messages: 229921 nosy: akitada priority: normal severity: normal status: open title: PEP 257: drop the recommendation for a blank line between the class line and the docstring Added file: http://bugs.python.org/file37003/pep-0257.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22715 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22715] PEP 257: drop the recommendation for a blank line between the class line and the docstring
Akira Kitada added the comment: Guido is with me :) On Wed, Oct 22, 2014 at 2:16 AM, Guido van Rossum guido at python.org wrote: I think the argument is about the blank line between the class line and the docstring. I agree with Akira that very few packages follow this style, and I think we would do well to remove this recommendation. There's no point in fighting the prevailing convention. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22715 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22715] PEP 257: drop the recommendation for a blank line between the class line and the docstring
Akira Kitada added the comment: Good point. I looked at some Python files but, as you pointed out, I couldn't find ones what follow the recommendation. In some files there is a blank line after a docstring but it seemed it's not because it is written as a number of sections. The patch updated. -- resolution: - not a bug status: open - closed Added file: http://bugs.python.org/file37005/pep-0257.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22715 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22709] restore accepting detached stdin in fileinput binary mode
New submission from Akira Li: The patch for Issue #21075: fileinput.FileInput now reads bytes from standard stream if binary mode is specified broke code that used sys.stdin = sys.stdin.detach() with FileInput(mode='rb') in Python 3.3 I've attached the patch that makes FileInput to accept detached sys.stdin (without 'buffer' attribute) in binary mode. -- components: Library (Lib) files: fileinput-detached-stdin.diff keywords: patch messages: 229859 nosy: akira priority: normal severity: normal status: open title: restore accepting detached stdin in fileinput binary mode type: behavior versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file36997/fileinput-detached-stdin.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22709 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22709] restore accepting detached stdin in fileinput binary mode
Akira Li added the comment: It is incorrect that sys.stdin is *always* a text stream. It often is, but not always. There are cases when it is not e.g., $ tar zcf - stuff | gpg -e | ssh user@server 'cat - stuff.tar.gz.gpg' tar's stdout is *not* a text stream. gpg's stdin/stdout are *not* text streams. ssh's stdin is *not* a text stream. etc. If any of the steps are implemented in Python then it is useful to consider sys.stdin as a binary stream. Any script written before Python 3.4.1 (#21075) that used FileInput binary mode *had to* use sys.stdin = sys.stdin.detach() A bugfix release should not break working code. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22709 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22709] restore accepting detached stdin in fileinput binary mode
Akira Li added the comment: This is not related to Python. Terms character, string, text, file can have different meaning in different domains. In Python we use Python terminology. There is no such thing as sys.stdin in Posix-compatible shell, because Posix-compatible shell has no the sys module and doesn't use a dot to access attributes. I use Python terminology (text - Unicode string, binary data - bytes). Though text vs. binary data distinction is language independent ( it doesn't matter how Unicode type is called in a particular language). Python can be used to implement `tar`, `gpg`, `ssh`, `7z`, etc. I don't see what POSIX has anything to do with that fact. It is very simple actually: text - encode character encoding - bytes bytes - decode character encoding - text In most cases text should be human readable. It doesn't make sense to encode/decode input/output of gpg-like utilities using a character encoding. *Therefore* the notion of sys.stdin being a bytes stream (io.BufferedReader) can be useful in this case. The lines produced by FileInput are often (after optional processing) written to sys.stdout. If binary mode is used then FileInput(mode='rb') yields bytes therefore it is also useful to consider sys.stdout a binary stream (io.BufferedWriter) in this case. It introduces a nice symmetry: text FileInput mode - text streams binary FileInput mode - binary streams By design, FileInput treats stdin as any other file. It even supports a special name for it: '-'. A file may be in binary mode; stdin should be able too. sys.stdout is used outside of FileInput therefore no changes in FileInput itself are necessary but sys.stdin is used inside FileInput that is why the change is needed. Correct solution in this case would be to use the workaround sys.stdin = sys.stdin.detach() conditionally, only in Python versions which have a bug. Do you mean every Python 3 version before Python 3.4.1? Correct solution is to avoid blaming users (your fault - you change your programs) for our mistakes and fix the bug in Python itself. The patch is attached. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22709 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11820] idle3 shell os.system swallows shell command output
Akira Li added the comment: It looks like the issue can be reduced to whether or not to show this output: import os os.write(1, b'should we see this in idle?\n') should we see this in idle? 28 assuming sys.__stdout__.fileno() == fileno(stdout) == 1. If should we see this in idle? is shown in idle then the output that writes to the inherited C stdout such as os.system('ls'), subprocess.call('ls') will also be shown automatically. And in reverse If should we see this in idle? should not be shown then other C stdout output such as produced by os.system('ls') should not be redirected too. The output written to the same file should be shown/not shown consistently. os.write(2, b'the same goes for stderr\n') -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11820 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22627] Calling timestamp() on a datetime object modifies the timestamp of a different datetime object.
Akira Li added the comment: Christopher, About your script http://paste.ubuntu.com/8562027/ dateutil may break if the local timezone had different UTC offset in the past. You could use tzlocal module to get pytz timezone that can handle such timezones. To get the correct time for 1414274400 timezone in Europe/Moscow timezone, you need the latest tz database e.g., `pytz` works but fromtimestamp, dateutil that use the local tz database fail (2:00 instead of 1:00): import time import os os.environ['TZ'] = 'Europe/Moscow' time.tzset() from datetime import datetime, timezone from tzlocal import get_localzone datetime.fromtimestamp(1414274400, get_localzone()) datetime.datetime(2014, 10, 26, 1, 0, tzinfo=DstTzInfo 'Europe/Moscow' MSK+3:00:00 STD) datetime.utcfromtimestamp(1414274400).replace(tzinfo=timezone.utc).astimezone(get_localzone()) datetime.datetime(2014, 10, 26, 1, 0, tzinfo=DstTzInfo 'Europe/Moscow' MSK+3:00:00 STD) datetime.fromtimestamp(1414274400) # wrong datetime.datetime(2014, 10, 26, 2, 0) datetime.fromtimestamp(1414274400, timezone.utc).astimezone() # wrong datetime.datetime(2014, 10, 26, 2, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 14400), 'MSK')) datetime.utcfromtimestamp(1414274400).replace(tzinfo=timezone.utc).astimezone() # wrong datetime.datetime(2014, 10, 26, 2, 0, tzinfo=datetime.timezone(datetime.timedelta(0, 14400), 'MSK')) from dateutil.tz import gettz, tzutc datetime.fromtimestamp(1414274400, gettz()) # wrong datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow')) datetime.fromtimestamp(1414274400, tzutc()).astimezone(gettz()) # wrong datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow')) datetime.utcfromtimestamp(1414274400).replace(tzinfo=tzutc()).astimezone(gettz()) # wrong datetime.datetime(2014, 10, 26, 2, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Moscow')) To avoid surprises, always use UTC time to perform date arithmetics: EPOCH = datetime(1970, 1,1, tzinfo=pytz.utc) utc_dt = EPOCH + timedelta(seconds=timestamp) should work for dates after 2038 too. To convert it to the local timezone: from tzlocal import get_localzone local_dt = utc_dt.astimezone(get_localzone()) ts = (local_dt - EPOCH) // timedelta(seconds=1) assert ts == timestamp # if timestamp is an integer Python stdlib assumes POSIX encoding for time.time() value (issue22356) therefore the formulae work on all platforms where Python works. -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22627 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22524] PEP 471 implementation: os.scandir() directory scanning function
Changes by Akira Li 4kir4...@gmail.com: -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22524 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22472] OSErrors should use str and not repr on paths
Akira Li added the comment: I meant, in general, repr() is better for an error message because it should be unambigous unlike str() but in your particular case you could use filename attribute to format the way you like it for your UI. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22472] OSErrors should use str and not repr on paths
Akira Li added the comment: OSError has *filename* attribute. Could it be passed to the UI instead? -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22472 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22477] GCD in Fractions
Changes by Akira Li 4kir4...@gmail.com: -- nosy: -akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22477 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22477] GCD in Fractions
Akira Li added the comment: Whether or not gcd(a, b) == gcd(|a|, |b|) depends on the definition if we believe to Stepanov of C++ STL fame who mentions in his lecture [1] [1] http://www.stepanovpapers.com/gcd.pdf that the current implementation that uses two operation __bool__ and __mod__: def gcd(a, b): while b: a, b = b, a%b return a can be applied to Euclidean ring elements (not just positive or negative integers). Despite Knuth’s objection to gcd(1, -1) = -1, adding `if a 0: a = -a` at the end changes the requirements for the type. Here's the definition from the lecture [1]: Greatest common divisor is a common divisor that is divisible by any other common divisor. I have no opinion on whether or not fractions.gcd() should be changed. I thought that I should mention that different definitions exist. -- nosy: +akira ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22477 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22442] subprocess.check_call hangs on large PIPEd data.
Akira Li added the comment: What do you think? I would prefer to deprecate PIPE argument for subprocess.call(): issue DeprecationWarning in 3.5 and raise ValueError in 3.6+ I've uploaded a patch that issues the warning. -- keywords: +patch type: - enhancement versions: +Python 3.5 -Python 2.7 Added file: http://bugs.python.org/file36675/subprocess.call-deprecate-PIPE.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22442 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21332] subprocess bufsize=1 docs are misleading
Changes by Akira Li 4kir4...@gmail.com: Added file: http://bugs.python.org/file36679/subprocess-line-buffering-issue21332-ps5.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21332 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22442] subprocess.check_call hangs on large PIPEd data.
Akira Li added the comment: @juj: DeprecationWarning is generated if PIPE is passed to call() as any positional or keyword argument in particular stdin, stdout, stderr. It also applies to check_call() that uses call() internally. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22442 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22442] subprocess.check_call hangs on large PIPEd data.
Akira Li added the comment: Victor, the message in my patch is copied almost verbatim from the current subprocess' documentation [1] [1] https://hg.python.org/cpython/file/850a62354402/Doc/library/subprocess.rst#l57 People use `call(cmd, stdout=PIPE)` as a *broken* way to suppress output i.e., when they actually want `call(cmd, stdout=DEVNULL)` The issue with `call(cmd, stdout=PIPE)` that it *appears* to work if cmd doesn't produce much output i.e., it might work in tests but may hang in production. It is unrelated to check_output(), getstatusouptut() or getoutput(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22442 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com