[issue43340] json.load() can raise UnicodeDecodeError, but this is not documented
New submission from Matthew Woodcraft : The documentation for json.load() and json.loads() says: « If the data being deserialized is not a valid JSON document, a JSONDecodeError will be raised. » But this is not currently entirely true: if the data is provided in bytes form and is not properly encoded in one of the three accepted encodings, UnicodeDecodeError is raised instead. (I have no opinion on whether the documentation or the behaviour should be changed.) -- components: Library (Lib) messages: 387780 nosy: mattheww priority: normal severity: normal status: open title: json.load() can raise UnicodeDecodeError, but this is not documented type: behavior versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43340> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33302] The search for pyvenv.cfg doesn't match PEP 405
New submission from Matthew Woodcraft <matt...@woodcraft.me.uk>: PEP 405 says that the pyvenv.cfg file is found as follows: « a pyvenv.cfg file is found either adjacent to the Python executable or one directory above it (if the executable is a symlink, it is not dereferenced), » But in cpython if the executable is a symlink, it _is_ dereferenced before searching for pyvenv.cfg . I've checked this behaviour with Python 3.5, 3.6, and today's 3.7 tip. Looking in 3.7's getpath.c, calculate_path_impl() calls, in order: calculate_program_full_path() calculate_argv0_path() calculate_read_pyenv() It looks like the symlink resolution happens near the end of calculate_argv0_path(). I think this means that the 'home=' line in pyvenv.cfg files generated by the 'venv' module is irrelevant (and seems likely to confuse anyone who might try to construct a Python environment by imitating a built venv). -- components: Interpreter Core messages: 315424 nosy: mattheww priority: normal severity: normal status: open title: The search for pyvenv.cfg doesn't match PEP 405 type: behavior versions: Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33302> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30755] locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8
Matthew Woodcraft added the comment: (For PEP 537 please read PEP 538, sorry) -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue30755> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30755] locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8
Matthew Woodcraft added the comment:
I've investigated a bit more.
First, I've tried with Python 3.7.0a1 . As you'd expect, PEP 537 means
this behaviour now also occurs when no locale environment variables at
all are set.
Second, I've looked through locale.py a bit. I believe what it calls the
"aliasing engine" is applied for:
- getlocale()
- getdefaultlocale()
- setlocale() when passed a tuple, but not when passed a string
This leads to some rather odd results.
With 3.7.0a1 and no locale environment variables:
>>> import locale
>>> locale.getlocale()
('en_US', 'UTF-8')
# getlocale() is lying: the effective locale is really C.UTF-8
>>> sorted("abcABC", key=locale.strxfrm)
['A', 'B', 'C', 'a', 'b', 'c']
Third, I've checked on a system which does have en_US.UTF-8 installed,
and (as you'd expect) instead of crashing it gives wrong results:
>>> import locale
>>> locale.setlocale(locale.LC_ALL, ('C', 'UTF-8'))
'en_US.UTF-8'
>>> locale.getlocale()
('en_US', 'UTF-8')
# now getlocale() is telling the truth, and the user isn't getting the
# collation they requested
>>> sorted("abcABC", key=locale.strxfrm)
['a', 'A', 'b', 'B', 'c', 'C']
--
versions: +Python 3.7
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue30755>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18378] locale.getdefaultlocale() fails on Mac OS X with default language set to English
Matthew Woodcraft added the comment: That alias (C.UTF-8 to en_US.UTF-8) is surely a bug in itself nowadays. I've filed #30755 . -- nosy: +mattheww ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue18378> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30755] locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8
New submission from Matthew Woodcraft:
I have a system where the default locale is C.UTF-8, and en_US.UTF-8 is
not installed.
But locale.normalize() unhelpfully converts "C.UTF-8" to "en_US.UTF-8".
So the following crashes for me:
python3.6 -c "import locale;locale.setlocale(locale.LC_ALL, ('C', 'UTF-8'))"
Similarly getdefaultlocale() returns ('en_US', 'UTF-8'), so this crashes too:
export LANG=C.UTF-8
unset LC_CTYPE
unset LC_ALL
unset LANGUAGE
python3.6 -c "import locale;locale.setlocale(locale.LC_ALL,
locale.getdefaultlocale())"
This behaviour is caused by a locale_alias entry in Lib/locale.py .
https://bugs.python.org/issue20076 documents its addition but doesn't
provide a rationale.
I can see that it might be helpful to provide such a conversion if
C.UTF-8 doesn't exist and en_US.UTF-8 does, but the current code is
breaking modern correctly-configured systems for the benefit of old
misconfigured ones (C.UTF-8 shouldn't really be in the environment if it
isn't available on the system, after all).
--
messages: 296828
nosy: mattheww
priority: normal
severity: normal
status: open
title: locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8
versions: Python 3.4, Python 3.5, Python 3.6
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30755>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13475] Add '-p'/'--path0' command line option to override sys.path[0] initialisation
Matthew Woodcraft added the comment: For the record: the '-I' option (#16499) in Python 3.4 disables sys.path[0] initialisation (among other things). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13475 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16217] Tracebacks are unnecessarily verbose when using 'python -m'
New submission from Matthew Woodcraft: If I run my code using 'python -m' and there is an unhandled exception, the tracebacks include lines from runpy.py (and now sometimes from importlib._bootstrap) which don't provide useful information, and tend to overwhelm the valuable part of the traceback. I suppose this is in some sense a regression from Python 2.4. echo fail fail.py python3.3 -m fail python -m fail Traceback (most recent call last): File /usr/lib/python3.3/runpy.py, line 160, in _run_module_as_main __main__, fname, loader, pkg_name) File /usr/lib/python3.3/runpy.py, line 73, in _run_code exec(code, run_globals) File ./fail.py, line 2, in module fail NameError: name 'fail' is not defined Here I think it would be better to omit the first two traceback entries. Syntax errors are worse: echo syntax error fail.py python3.3 -m fail Traceback (most recent call last): File /usr/lib/python3.3/runpy.py, line 140, in _run_module_as_main mod_name, loader, code, fname = _get_module_details(mod_name) File /usr/lib/python3.3/runpy.py, line 114, in _get_module_details code = loader.get_code(mod_name) File frozen importlib._bootstrap, line 981, in get_code File frozen importlib._bootstrap, line 313, in _call_with_frames_removed File ./fail.py, line 1 syntax error Here there are four traceback entries which could be omitted. -- messages: 172806 nosy: mattheww priority: normal severity: normal status: open title: Tracebacks are unnecessarily verbose when using 'python -m' type: enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16217 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13995] sqlite3 Cursor.rowcount documentation for old sqlite bug
New submission from Matthew Woodcraft matt...@woodcraft.me.uk: The documentation for the sqlite3 module contains the following statement, under 'Cursor.rowcount': For DELETE statements, SQLite reports rowcount as 0 if you make a DELETE FROM table without any condition. This doesn't happen for me (with sqlite 3.7.9): rowcount returns the correct value in this case. According to http://www.sqlite.org/lang_delete.html#truncateopt , this was a bug that was fixed in SQLite 3.6.5 (in 2008). So I think the Python documentation should either omit this paragraph, or else explain that it only applies to older versions of SQLite. Also, the first example under 'Using shortcut methods' has code to work around this bug, which should perhaps be removed. -- assignee: docs@python components: Documentation messages: 153136 nosy: docs@python, mattheww priority: normal severity: normal status: open title: sqlite3 Cursor.rowcount documentation for old sqlite bug ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13995 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13475] Add '-p'/'--path0' command line option to override sys.path[0] initialisation
Matthew Woodcraft matt...@woodcraft.me.uk added the comment: The proposed --nopath0 option is something I've wished I had in the past. If this is added, it would be good if it could be given a single-letter form too, because it's an option that would be useful in #! lines (they don't reliably support using more than one command-line argument, and single-letter switches can be combined while long-form ones can't). -- nosy: +mattheww ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13475 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4106] multiprocessing occasionally spits out exception during shutdown
Changes by Matthew Woodcraft matt...@woodcraft.me.uk: -- nosy: +mattheww ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4106 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9977] TestCase.assertItemsEqual's description of differences
Matthew Woodcraft matt...@woodcraft.me.uk added the comment: Terry J. Reedy wrote: If I understand correctly, you are requesting that .assertItemsEqual only use the 2nd (multiset comparison) method, so that if one want the first method, one should directly call .assertSequenceEqual(sorted(a), sorted(b)). Yes, that would make me happy. So would a new assertXXXEqual method that always did the multiset comparison. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9977 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2972] arguments and default path not set in site.py and sitecustomize.py
Matthew Woodcraft matt...@woodcraft.me.uk added the comment: open(/proc/self/cmdline).read() should work on linux (note that the arguments are separated by NULs). -- nosy: +mattheww ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2972 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9976] Make TestCase._formatMessage public
New submission from Matthew Woodcraft matt...@woodcraft.me.uk: It would be pleasant if TestCase._formatMessage, or something similar, could be made part of the documented API; I think pretty much anyone writing a custom TypeEqualityFunc is going to end up reimplementing it. (This is the code that makes the TestCase.longMessage attribute take effect.) -- messages: 117543 nosy: mattheww, michael.foord priority: normal severity: normal status: open title: Make TestCase._formatMessage public type: feature request ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9976 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9977] TestCase.assertItemsEqual's description of differences
New submission from Matthew Woodcraft matt...@woodcraft.me.uk:
TestCase.assertItemsEqual uses two different techniques to describe the
differences in the inputs that it compares.
If the inputs are sortable, it sorts them and then uses
assertSequenceEqual to describe the difference between them considered
as ordered sequences.
Otherwise, it uses unittest.util.unorderable_list_difference, which
is essentially a multiset comparison.
In practice, I think the output from unorderable_list_difference is
usually more readable, so I wonder if something of that kind should be
made the default.
Example:
a = [('b', (2, 3)), ('w', (3, 4))]
b = [('x', (2, 3)), ('w', (3, 4))]
case.assertItemsEqual(a, b)
unorderable_list_difference gives
Expected, but missing:
[('b', (2, 3))]
Unexpected, but present:
[('x', (2, 3))]
while the current assertItemsEqual gives
Sequences differ: [('b', (2, 3)), ('w', (3, 4))] != [('w', (3, 4)), ('x', (2,
3))]
First differing element 0:
('b', (2, 3))
('w', (3, 4))
- [('b', (2, 3)), ('w', (3, 4))]
+ [('w', (3, 4)), ('x', (2, 3))]
In general, I think that the 'first differing element' paragraph that
assertSequenceEqual produces is as likely to be misleading as it is to
be helpful (when we're really comparing unordered sequences).
--
messages: 117545
nosy: mattheww, michael.foord
priority: normal
severity: normal
status: open
title: TestCase.assertItemsEqual's description of differences
type: feature request
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9977
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9802] Document 'stability' of builtin min() and max()
Matthew Woodcraft matt...@woodcraft.me.uk added the comment: (1) Shouldn't 'reverse=True' be omitted in the second doc addition? Yes, of course, sorry. (2) I'd also suggest adding a brief comment about what this means for distinct, but equal, objects; otherwise it's not really obvious what the point of the doc addition is. (3) As a matter of clarity, perhaps replace this is with max(iterable, key=key) is, and similarly for min. I've attached a new patch incorporating these suggestions. -- Added file: http://bugs.python.org/file18858/functions.rst.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9802 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9802] Document 'stability' of builtin min() and max()
New submission from Matthew Woodcraft matt...@woodcraft.me.uk: In CPython, the builtin max() and min() have the property that if there are items with equal keys, the first item is returned. From a quick look at their source, I think this is true for Jython and IronPython too. I propose making this a documented guarantee. On Python-dev, Raymond Hettinger said: That seems like a reasonable request. This behavior has been around for a very long time is unlikely to change. Elsewhere, we've made efforts to document sort stability (i.e. sorted(), heapq.nlargest(), heapq.nsmallest, merge(), etc). (http://mail.python.org/pipermail/python-dev/2010-September/103543.html) I'm attaching a patch with a concrete suggestion for a change to functions.rst, modelled on the documentation of heapq.nlargest(). -- assignee: d...@python components: Documentation files: maxmin.patch keywords: patch messages: 115892 nosy: d...@python, mattheww priority: normal severity: normal status: open title: Document 'stability' of builtin min() and max() type: feature request Added file: http://bugs.python.org/file18802/maxmin.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9802 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5173] What's new claims StandardError was removed in 2.6
New submission from Matthew Woodcraft m...@pearson.co.uk: In the current What's New In Python 3.0 documentation, under Changes To Exceptions, it is claimed that the removal of StandardError is in 2.6 already. But according to the 2.6.1 documentation, StandardError is still there in its usual place. -- assignee: georg.brandl components: Documentation messages: 81288 nosy: georg.brandl, mjwpp severity: normal status: open title: What's new claims StandardError was removed in 2.6 versions: Python 3.0 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5173 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
