[issue47237] Inheritance from Protocol with property in dataclass makes them non-instantiatable
New submission from Daniel Draper : Hi, According to https://peps.python.org/pep-0544/#explicitly-declaring-implementation it should be possible to explicitly inherit from Protocols. This however breaks the dataclass constructor when using the @property decorator in the protocol, see this example: ```python from typing import Protocol from dataclasses import dataclass class SomeProtocol(Protocol): @property def some_value(self) -> str: ... @dataclass class SomeDataclasss(SomeProtocol): some_value: str if __name__ == '__main__': a = SomeDataclasss(some_value="value") # this crashes with AttributeError: can't set attribute 'some_value' ``` The pattern of @property in the protocol is one taken from the mypy docs (see https://mypy.readthedocs.io/en/stable/protocols.html#recursive-protocols for example). When removing the explicit inheritiance mypy also correctly typechecks the dataclass implementation when doing something like, only the explicit inheritance seems to fail in python ```python a: SomeProtocol = SomeDataclass() ``` -- components: Library (Lib) messages: 416846 nosy: Germandrummer92 priority: normal severity: normal status: open title: Inheritance from Protocol with property in dataclass makes them non-instantiatable type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue47237> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40172] ZipInfo corrupts file names in some old zip archives
Daniel Hillier added the comment: Related to issue https://bugs.python.org/issue28080 which has a patch that covers a bit of this issue -- ___ Python tracker <https://bugs.python.org/issue40172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46958] json dump/dumps prints each array element on a new line (bad for readability)
Change by Daniel : -- keywords: +patch pull_requests: +29873 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31762 ___ Python tracker <https://bugs.python.org/issue46958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46958] json dump/dumps prints each array element on a new line (bad for readability)
Change by Daniel : -- components: Library (Lib) nosy: Entirity priority: normal severity: normal status: open title: json dump/dumps prints each array element on a new line (bad for readability) type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46958> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46440] ArgumentParser.parse_args exits on missing required argument with exit_on_error=False
New submission from Daniel Schulte : When calling ArgumentParser.parse_args(list_of_things_to_parse) sys.exit gets called even though the parser was constructed with exit_on_error=False. It doesn't matter if the parser has any subparsers added or not. The docs say > Normally, when you pass an invalid argument list to the parse_args() > method of an ArgumentParser, it will exit with error info. > If the user would like to catch errors manually, the feature can be > enabled by setting exit_on_error to False: > [example code follows] The docs make me believe what I'm trying to do should work. I've attached a minimal working example to show what happens (with and without subparsers). -- components: Library (Lib) files: bug.py messages: 410984 nosy: trilader priority: normal severity: normal status: open title: ArgumentParser.parse_args exits on missing required argument with exit_on_error=False type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50567/bug.py ___ Python tracker <https://bugs.python.org/issue46440> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32912] Raise non-silent warning for invalid escape sequences
Daniel Carpenter added the comment:
I'm not sure if this is an issue or by design, but this DeprecationWarning
behaves differently to other DeprecationWarnings.
A normal DeprecationWarning triggered by code in __main__ is printed by default:
$ python -c 'import warnings; warnings.warn("test", DeprecationWarning)'
:1: DeprecationWarning: test
But this one is silent:
$ python -c '"\,"'
[no output]
To see this DeprecationWarning at all, I need to type:
$ python -Wdefault -c '"\,"'
:1: DeprecationWarning: invalid escape sequence '\,'
But that enables this DeprecationWarning for all modules, not just __main__ .
I've tested this with Python 3.9 on debian bullseye and the 3.10 docker image.
--
nosy: +dansebcar
___
Python tracker
<https://bugs.python.org/issue32912>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22833] The decode_header() function decodes raw part to bytes or str, depending on encoded part
Change by Daniel Lenski : -- keywords: +patch pull_requests: +28748 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30548 ___ Python tracker <https://bugs.python.org/issue22833> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46236] PyFunction_GetAnnotations returning Tuple vs Dict
New submission from Daniel McCarney : Hi there, This is my first Python bug, hope I haven't missed anything important. I noticed that the PyFunction_GetAnnotations function from the C API is returning a PyTuple when I link Python 3.10 where historically it returned a PyDict. The documentation for PyFunction_GetAnnotations for 3.10 (here[0]) says its return can be: > a mutable dictionary or NULL I'm not sure if: * This is user error and I should adapt to this case in my dependent code when using Python 3.10+. * This is a documentation error, and it should say "a mutable dictionary, set, or NULL" * This is a regression in the C API that should be fixed. I've made a small reproduction program (here[1]) that can be used to quickly demonstrate the difference in return type between the two Python versions for the same C program and .py inputs. I also noticed that issue42202 (here[2]) landed an optimization in 3.10.x that changed the internal representation of annotations to a set - could that possibly be related? I'm not 100% sure! Thanks! [0]: https://docs.python.org/3/c-api/function.html#c.PyFunction_GetAnnotations [1]: https://github.com/cpu/pyfunction_getannotations_test [2]: https://bugs.python.org/issue42202 -- components: C API messages: 409555 nosy: cpu priority: normal severity: normal status: open title: PyFunction_GetAnnotations returning Tuple vs Dict versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46236> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22833] The decode_header() function decodes raw part to bytes or str, depending on encoded part
Daniel Lenski added the comment: Due to this bug, any user of this function in Python 3.0+ *already* has to be able to handle all of the following outputs in order to use it reliably: decode_header(...) -> [(str, None)] or decode_header(...) -> [(bytes, str)] or decode_header(...) -> [(bytes, (str|None)), (bytes, (str|None)), ...] == Fix str/bytes inconsistency == We could eliminate the inconsistency, and make the function only ever return bytes instead of str, with the following changes to https://github.com/python/cpython/blob/3.10/Lib/email/header.py. ``` diff --git a/Lib/email/header.py.orig b/Lib/email/header.py index 4ab0032..41e91f2 100644 --- a/Lib/email/header.py +++ b/Lib/email/header.py @@ -61,7 +61,7 @@ _max_append = email.quoprimime._max_append def decode_header(header): """Decode a message header value without converting charset. -Returns a list of (string, charset) pairs containing each of the decoded +Returns a list of (bytes, charset) pairs containing each of the decoded parts of the header. Charset is None for non-encoded parts of the header, otherwise a lower-case string containing the name of the character set specified in the encoded string. @@ -78,7 +78,7 @@ def decode_header(header): for string, charset in header._chunks] # If no encoding, just return the header with no charset. if not ecre.search(header): -return [(header, None)] +return [header.encode(), None)] # First step is to parse all the encoded parts into triplets of the form # (encoded_string, encoding, charset). For unencoded strings, the last # two parts will be None. ``` With these changes, decode_header() would return one of the following: decode_header(...) -> [(bytes, None)] or decode_header(...) -> [(bytes, str)] or decode_header(...) -> [(bytes, (str|None)), (bytes, (str|None)), ...] == Ensure that charset is always str, never None == A couple more small changes: ``` @@ -92,7 +92,7 @@ def decode_header(header): unencoded = unencoded.lstrip() first = False if unencoded: -words.append((unencoded, None, None)) +words.append((unencoded, None, 'ascii')) if parts: charset = parts.pop(0).lower() encoding = parts.pop(0).lower() @@ -133,7 +133,8 @@ def decode_header(header): # Now convert all words to bytes and collapse consecutive runs of # similarly encoded words. collapsed = [] -last_word = last_charset = None +last_word = None +last_charset = 'ascii' for word, charset in decoded_words: if isinstance(word, str): word = bytes(word, 'raw-unicode-escape') ``` With these changes, decode_header() would return only: decode_header(...) -> List[(bytes, str)] -- ___ Python tracker <https://bugs.python.org/issue22833> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22833] The decode_header() function decodes raw part to bytes or str, depending on encoded part
Daniel Lenski added the comment:
I recently ran into this bug as well.
For those looking for a reliable workaround, here's an implementation of a
'decode_header_to_string' function which should Just Work™ in all possible
cases:
#!/usr/bin/python3
import email.header
# Workaround for https://bugs.python.org/issue22833
def decode_header_to_string(header):
'''Decodes an email message header (possibly RFC2047-encoded)
into a string, while working around
https://bugs.python.org/issue22833'''
return ''.join(
alleged_string if isinstance(alleged_string, str) else
alleged_string.decode(
alleged_charset or 'ascii')
for alleged_string, alleged_charset in
email.header.decode_header(header))
for header in ('=?utf-8?B?ZsOzbw==',
'=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?=',
'bar=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?=',
'plain string',):
print("Header value: %r" % header)
print("email.header.decode_header(...) -> %r" %
email.header.decode_header(header))
print("decode_header_to_string(...)-> %r" %
decode_header_to_string(header))
print("---")
Outputs:
Header value: '=?utf-8?B?ZsOzbw=='
email.header.decode_header(...) -> [('=?utf-8?B?ZsOzbw==', None)]
decode_header_to_string(...)-> '=?utf-8?B?ZsOzbw=='
---
Header value: '=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?='
email.header.decode_header(...) -> [(b'hello', 'ascii'), (b'f\xc3\xb3o',
'utf-8')]
decode_header_to_string(...)-> 'hellofóo'
---
Header value: 'bar=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?='
email.header.decode_header(...) -> [(b'bar', None), (b'hello', 'ascii'),
(b'f\xc3\xb3o', 'utf-8')]
decode_header_to_string(...)-> 'barhellofóo'
---
Header value: 'plain string'
email.header.decode_header(...) -> [('plain string', None)]
decode_header_to_string(...)-> 'plain string'
---
Header value: 'foo=?blah?Q??='
email.header.decode_header(...) -> [(b'foo', None), (b'', 'blah')]
decode_header_to_string(...)-> 'foo'
---
--
nosy: +dlenski
___
Python tracker
<https://bugs.python.org/issue22833>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25044] bring BTPROTO_SCO inline with other Bluetooth protocols
Change by Daniel Diniz : -- versions: +Python 3.11 -Python 3.4, Python 3.5, Python 3.6 ___ Python tracker <https://bugs.python.org/issue25044> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7687] Bluetooth support untested
Change by Daniel Diniz : -- versions: +Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue7687> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14844] netrc does not handle accentuated characters
Daniel Diniz added the comment: Now that a PR has landed in #28806 to improve shlex, we need to check whether this issue can/needs to move forward. -- nosy: +ajaksu2, asvetlov type: -> behavior versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.2 ___ Python tracker <https://bugs.python.org/issue14844> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1438480] shutil.move raises OSError when copystat fails
Change by Daniel Diniz : -- keywords: -easy versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.3 ___ Python tracker <https://bugs.python.org/issue1438480> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue978604] wait_variable hangs at exit
Daniel Diniz added the comment: Confirmed on Python 3.11.0a3+, assuming older versions also affected. This issue has different versions of tests to confirm it, but I think not in a test suite-friendly format yet. There's also a patch and detailed analysis by gpolo indicating that it might need editing _tkinter.c to handle fallout from his proposed Python-side patch. -- versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.2, Python 3.3 ___ Python tracker <https://bugs.python.org/issue978604> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46169] Same-moment datetimes with different ZoneInfo timezones are not considered ==
Change by Daniel Diniz : -- nosy: +belopolsky ___ Python tracker <https://bugs.python.org/issue46169> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46169] Same-moment datetimes with different ZoneInfo timezones are not considered ==
Daniel Diniz added the comment: Confirmed for 3.11 in Windows. The C datetime code can be trivially fixed so your equality test returns True, but there are two Python tests that depend on current behavior so it might not be so easy. They were added with current code in issue 24773, to implement PEP 495 -- Local Time Disambiguation. Also, using a timezone implementation from the test suite makes the equality work, so maybe the bug depends on zoneinfo. If you change datetimemodule.c line 5761 from "diff = 1;" to "diff = 0;", two tests fail and your code works. See attached file for the tests that fail and using the timezone implementation mentioned above. -- nosy: +ajaksu2 Added file: https://bugs.python.org/file50517/dt_equality.py ___ Python tracker <https://bugs.python.org/issue46169> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue504219] locale.resetlocale is broken
Change by Daniel Diniz : -- nosy: +ajaksu2 versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.3, Python 3.4 ___ Python tracker <https://bugs.python.org/issue504219> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46168] Incorrect format specified for the "style" key in the configuration file format formatter example
Daniel Diniz added the comment: This example was added in issue 43047. It only seems to affect 3.10+ docs. Ian, is this something you'd like to tackle? -- keywords: +easy nosy: +ajaksu2, iwienand stage: -> needs patch versions: +Python 3.10, Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue46168> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] Improvement mktime error reporting
Daniel McDonald added the comment: Sounds good, thank you, Terry -- type: enhancement -> behavior ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment: A definitive assessment was obtained by Aurelien with Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001774 In brief, the OverflowError is expected and correct. Two factors contribute to the difference in observed behavior. First, the production of the -1 from mktime which Python uses to raise OverflowError depends on whether timezone is or is not set (e.g. "Etc/UTC"). This was confirmed on the Github Actions 20.04 ubuntu instance independent of Python directly with the mktime() function from C. Second, mktime returning -1 to indicate overflow is dependent on glibc >= 2.29, and versions prior "do not report any error if the date is not representable." This second factor is supported by testing the lack of the -1 (and subsequent OverflowError) on Github Actions ubuntu 18.04, which uses an older version of glibc. The full thread with Debian is here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001774 With regards to CPython, a possible change to consider would be to test if tm_isdst < 0 prior to the mktime call, and if that evaluates true to provide an error indicating use requires a timezone being set. I think that change would make Python's behavior consistent with glibc >= 2.29, and smooth over a potential platform dependency, while also providing an informative error message. However, I have limited knowledge of how timezones are implemented and am unsure if more than "Etc/UTC" would result in tm_isdst being < 0. If that change makes sense, I'd be happy to submit a pull request to that effect. However, I recognize there may be considerations not accounted for here particularly as this is a central module, so I'll hold off issuing a pull request until hearing back from on this matter. -- ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46138] Strange behavior with lists
New submission from Daniel Frey :
Hi
I found a strange behavior with lists. When I want to save the list {3,1,8}, it
saves the 8 at the first entry. Only when I add the integers 7 and 9, the 8 is
placed correctly. (See the attached picture)
Can anyone help me understand what is going on here?
Thanks in advance!
--
files: python_strange_behavior.png
messages: 408960
nosy: daniel.frey
priority: normal
severity: normal
status: open
title: Strange behavior with lists
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file50505/python_strange_behavior.png
___
Python tracker
<https://bugs.python.org/issue46138>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23224] bz2/lzma: Compressor/Decompressor objects are only initialized in __init__
Change by Daniel Diniz : -- versions: +Python 3.10, Python 3.11 -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue23224> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46127] Missing HTML span element in exceptions.html
Daniel Diniz added the comment: That happens because EncodingWarning isn't highlighted as an Exception by Pygments[0]. The doc page gets its exceptions diagram by: ".. literalinclude:: ../../Lib/test/exception_hierarchy.txt" and all other entries are recognized by Pygments, so we get the "ne" class for them. A PR for Pygments adding EncodingWarning could be interesting, but I'm not sure the doc build process would pick it up and fix the 3.10 docs issue. For 3.11, the diagram drawing was changed so no highlighting happens anymore, which fixes the issue for us in the long run. [0] https://github.com/pygments/pygments/blob/8630e033313647d9579e314f3e8e2882f4558933/pygments/lexers/python.py#L225 -- nosy: +ajaksu2 ___ Python tracker <https://bugs.python.org/issue46127> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41033] readline.c: endless loop on SIGWINCH when loaded twice
Change by daniel hahler : -- versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue41033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41033] readline.c: endless loop on SIGWINCH when loaded twice
daniel hahler added the comment: Yes, the example also does not segfault for me either (also with Python 3.8.12 I have now as `python3.8`), so this was likely only happening in the more complex setup, and/or maybe with some specific version of readline back then. -- title: readline.c: SEGFAULT on SIGWINCH when loaded twice -> readline.c: endless loop on SIGWINCH when loaded twice ___ Python tracker <https://bugs.python.org/issue41033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue678264] test_resource fails when file size is limited
Change by Daniel Diniz : -- versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.4, Python 3.5 ___ Python tracker <https://bugs.python.org/issue678264> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9917] resource max value represented as signed when should be unsigned
Change by Daniel Diniz : -- versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.4, Python 3.5 ___ Python tracker <https://bugs.python.org/issue9917> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1062277] Pickle breakage with reduction of recursive structures
Change by Daniel Diniz : -- versions: +Python 3.11 -Python 3.5 ___ Python tracker <https://bugs.python.org/issue1062277> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45959] Teach pprint about dict views
Daniel Diniz added the comment: I have tried to add this. The PR adds: - PrettyPrinter._pprint_dict_view to handle dict_keys and dict_values (sorted with _safe_key). = PrettyPrinter._pprint_dict_items_view to handle dict_items (sorted using _safe_tuple). - Tests. Would a NEWS entry or other form of documentation be necessary? -- ___ Python tracker <https://bugs.python.org/issue45959> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45959] Teach pprint about dict views
Change by Daniel Diniz : -- keywords: +patch nosy: +ajaksu2 nosy_count: 4.0 -> 5.0 pull_requests: +28355 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30135 ___ Python tracker <https://bugs.python.org/issue45959> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment: For reference, the bug reports with Debian and Ubuntu are at the following URLs: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001774 https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1954963 -- ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment: Thank you, Christian, I apologize for missing your reply. That is great advice, and I will do so. -- ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment:
The use of tm_isdst=1 appears to trigger the overflow, and occurs when varying
other aspects of the timetuple.
Python's mktime wrapper can throw OverflowError in two places. The thrown error
is the second location, following the call to glibc's mktime occurring in this
specific if block:
https://github.com/python/cpython/blob/f62420c3d3f5d87f2b57e54b2a98682bc835f7b6/Modules/timemodule.c#L1038-L1046
Modification of Modules/timemodule.c as to printf tm struct members confirms
that when tm_isdst=1, the result from mktime() is -1, and the tm_wday member is
set to -1, the combination of which triggers the conditional.
mktime within the Github Action ubuntu-latest (20.04) environment is sourced
from glibc 2.31 as confirmed by ldd and nm on the compiled Python binary.
As a proof of concept, C program was written that calls mktime independent of
CPython (can be compiled with "gcc bug.c -o bug"). The code below succeeds on
both OSX 11.6.1 and Centos 7.9:
#include
#include
void do_test() {
struct tm tm_works = { .tm_year=117,
.tm_mon=4,
.tm_mday=26,
.tm_hour=15,
.tm_min=30,
.tm_sec=16,
.tm_wday=4,
.tm_yday=145,
.tm_isdst=-1 };
struct tm tm_fails = { .tm_year=117,
.tm_mon=4,
.tm_mday=26,
.tm_hour=15,
.tm_min=30,
.tm_sec=16,
.tm_wday=4,
.tm_yday=145,
.tm_isdst=1 };
time_t works = mktime(_works);
time_t fails = mktime(_fails);
if(works == -1) {
printf("Unexpected failure\n");
} else {
if(works == fails) {
printf("Test passed\n");
} else {
printf("Test failed: works=%d; fails=%d\n", (int)works, (int)fails);
}
}
}
int main(int argc, char **argv) {
do_test();
}
When compiled and run within in the Github Actions ubuntu-latest (20.04)
environment, the erroneous behavior occurs:
https://github.com/wasade/cpython/runs/4541212472?check_suite_focus=true#step:17:57
The use of tm_isdst=1 is valid according to multiple sources, a few examples
are linked below. The sources are consistent, and indicate a positive value
means Daylight Savings Time is in effect.
https://www.gnu.org/software/libc/manual/html_node/Broken_002ddown-Time.html
https://www.cplusplus.com/reference/ctime/tm/
At this time, I believe this remains unexpected behavior, however the source
appears upstream of CPython as mktime is part of glibc. Unless suggested
otherwise, I'll open an issue with on the glibc tracker.
--
___
Python tracker
<https://bugs.python.org/issue44413>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment: Thank you, Terry. I'm currently exploring modifications to the test Andrei made within a fork of CPython using Github Actions (ubuntu-latest). These modifications include debug prints in the CPython mktime call, and some parameter exploration. I expect to have a summary of the observations this afternoon. -- ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44413] OverflowError: mktime argument out of range after 2019
Daniel McDonald added the comment: I'd like to politely request this issue be reopened. We recently observed a similar, if not the same, OverflowError during continuous integration using Github Actions on ubuntu-latest (20.04). We can produce the error using pure Python without pytz. We have been unable to replicate the error using Github Actions on macos-latest, Centos 7.9, Ubuntu 18.04 or OSX 11.6.1. Specifically, the following code, which I believe is pure python, will trigger the overflow. We've observed this using Python 3.7 and 3.9. import time print(time.mktime((2017,5,26,15,30,16,4,146,1))) Exact error output from CI can be found at the link below: https://github.com/biocore/microsetta-private-api/runs/4536611219?check_suite_focus=true#step:4:117 On a passing system, we receive "1495837816.0" as output. On a failing system, we observe "OverflowError: mktime argument out of range". More detail can be found on our issue with pytz and stub42 who helped guide a more definitive example of the bug showing it could be pure Python. That issue can be found here: https://github.com/stub42/pytz/issues/66 Last, I apologize in advance if any detail is missing here or if this is not formatted as well as it should be. This is my first bug report with Python and am unfamiliar with the norms. Please let me know if further information may be helpful. -- nosy: +wasade versions: +Python 3.7, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue766910] fix one or two bugs in trace.py
Change by Daniel Diniz : -- nosy: +ajaksu2 versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.7 ___ Python tracker <https://bugs.python.org/issue766910> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1182143] making builtin exceptions more informative
Change by Daniel Diniz : -- nosy: +iritkatriel ___ Python tracker <https://bugs.python.org/issue1182143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4733] Add a "decode to declared encoding" version of urlopen to urllib
Daniel Diniz added the comment: As Victor notes, this is a controversial issue. And I'll add that the need for this feature seems not to have been brought up up in over a decade. So I'm closing this. -- resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue4733> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46036] Single-phase initialized modules gets initialized multiple times in 3.10.0
New submission from Daniel :
The documentation
(https://docs.python.org/3/c-api/init.html#c.Py_NewInterpreter) states:
For modules using single-phase initialization, e.g. PyModule_Create(), the
first time a particular extension is imported, it is initialized normally, and
a (shallow) copy of its module’s dictionary is squirreled away. When the same
extension is imported by another (sub-)interpreter, a new module is initialized
and filled with the contents of this copy; the extension’s init function is not
called. Objects in the module’s dictionary thus end up shared across
(sub-)interpreters, which might cause unwanted behavior (see Bugs and caveats
below).
This does however seem to have changed (sometime between 3.6.9 and 3.10.0).
Consider the following code:
#include
/*
* Create a module "my_spam" that uses single-phase initialization
*/
static PyModuleDef EmbModule = {
PyModuleDef_HEAD_INIT, "my_spam", NULL, -1,
NULL,
NULL, NULL, NULL, NULL
};
/*
* According to the docs this function is only called once when dealing with
* subinterpreters, the next time a shallow copy of the initial state is
* returned. This does however not seem to be the case in Python 3.10.0..
*/
static PyObject* PyInit_emb(void) {
PyObject *module = PyModule_Create();
PyModule_AddObject(module, "test", PyDict_New());
printf("Init my_spam module %p\n", module);
return module;
}
/*
* Main program
*/
int main(int argc, char *argv[]) {
PyImport_AppendInittab("my_spam", _emb);
Py_Initialize();
// Save the main state
PyThreadState *mainstate = PyThreadState_Get();
// Create two new interpreters
PyThreadState *inter1 = Py_NewInterpreter();
PyThreadState *inter2 = Py_NewInterpreter();
// Import the my_spam module into the first subinterpreter
// and change the global variable of it
PyThreadState_Swap(inter1);
PyRun_SimpleString("import sys; print(sys.version_info)");
PyRun_SimpleString("import my_spam; print('my_spam.test: ', my_spam.test)");
PyRun_SimpleString("my_spam.test[1]=1; print('my_spam.test: ',
my_spam.test)");
// Import the my_spam module into the second subinterpreter
// and change the global variable of it
PyThreadState_Swap(inter2);
PyRun_SimpleString("import sys; print(sys.version_info)");
PyRun_SimpleString("import my_spam; print('my_spam.test: ', my_spam.test)");
PyRun_SimpleString("my_spam.test[2]=2; print('my_spam.test: ',
my_spam.test)");
// Close the subinterpreters
Py_EndInterpreter(inter2);
PyThreadState_Swap(inter1);
Py_EndInterpreter(inter1);
// Swap back to the main state and finalize python
PyThreadState_Swap(mainstate);
if (Py_FinalizeEx() < 0) {
exit(120);
}
return 0;
}
Compiled with python 3.6.9 this does act according to the documentation:
$ gcc test_subinterpreters.c
-I/home/daniel/.pyenv/versions/3.6.9/include/python3.6m
-L/home/daniel/.pyenv/versions/3.6.9/lib -lpython3.6m &&
LD_LIBRARY_PATH=/home/daniel/.pyenv/versions/3.6.9/lib ./a.out
sys.version_info(major=3, minor=6, micro=9, releaselevel='final', serial=0)
Init my_spam module 0x7ff7a63d1ef8
my_spam.test: {}
my_spam.test: {1: 1}
sys.version_info(major=3, minor=6, micro=9, releaselevel='final', serial=0)
my_spam.test: {1: 1}
my_spam.test: {1: 1, 2: 2}
But compiled with 3.10.0 the module is reinitialized and thus objects in the
module are not shared between the subinterpreters:
$ gcc test_subinterpreters.c
-I/home/daniel/.pyenv/versions/3.10.0/include/python3.10
-L/home/daniel/.pyenv/versions/3.10.0/lib -lpython3.10 &&
LD_LIBRARY_PATH=/home/daniel/.pyenv/versions/3.10.0/lib ./a.out
sys.version_info(major=3, minor=10, micro=0, releaselevel='final', serial=0)
Init my_spam module 0x7f338a9a9530
my_spam.test: {}
my_spam.test: {1: 1}
sys.version_info(major=3, minor=10, micro=0, releaselevel='final', serial=0)
Init my_spam module 0x7f338a9a96c0
my_spam.test: {}
my_spam.test: {2: 2}
To me the new behavior seems nicer, but at the very least the documentation
should be updated. It also seems like if this could break integrations, albeit
it is an unlikely "feature" to rely on.
--
components: C API
messages: 408209
nosy: daniel-falk
priority: normal
severity: normal
status: open
title: Single-phase initialized modules gets initialized multiple times in
3.10.0
type: behavior
versions: Python 3.10
___
Python tracker
<https://bugs.python.org/issue46036>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45981] Get raw file name in bytes from ZipFile
Daniel Hillier added the comment: Handling different character sets is not completely supported yet. There are a couple of open issues relating to this: https://bugs.python.org/issue40407 (reading file names), https://bugs.python.org/issue41928 (support for reading and writing filenames using the unicode filename extra field) and https://bugs.python.org/issue40172 (issues with reading and then writing a filename from and back into a zip where the initial filename isn't encoded in cp437). Most modern zip programs that deal with characters outside ascii or cp437 either set the utf-8 flag or write both an ascii or cp437 compatible filename (to the original filename field in the zip header) and the actual filename with all non-ascii characters in the unicode filename extra field. I think adding support for the unicode field to Python would probably cover the majority files generated by modern zip programs. For complete support, including older zip programs that don't support the utf-8 flag or unicode filename extra field, we may need to provide another parameter in Python's ZipFile's read and write functions to be able to override the charset used for the filename stored directly in the zip file header. I've added my thoughts on how to approach this in https://bugs.python.org/issue40172 but haven't had time to implement these myself. -- nosy: +dhillier ___ Python tracker <https://bugs.python.org/issue45981> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45973] Not possible to clear PyImport_Inittab after PyImport_AppendInittab
New submission from Daniel :
Hello,
When embedding Python into a C application and not using Py_RunMain it is not
possible to remove a module added to PyImport_Inittab even if the interpreter
is finalized and a new is created.
One reason for not using Py_RunMain is to use python as a subprocessor to my
primary C application by initializing an interpreter, run some python code,
then go back to execute some C code and again execute some python code in the
same interpreter. After a while I might want to finalize the interpreter and
create a new one without the module added to inittab, still being in the same
process.
See example:
#include
/*
* Create a python interpreter, run a command and delete the interpreter
*/
void run_in_new_interpreter(char *cmd) {
Py_Initialize();
PyRun_SimpleString(cmd);
if (Py_FinalizeEx() < 0) {
exit(120);
}
}
/*
* Create a module "my_spam" but do not append it to inittab
*/
static PyModuleDef EmbModule = {
PyModuleDef_HEAD_INIT, "my_spam", NULL, -1,
NULL,
NULL, NULL, NULL, NULL
};
static PyObject* PyInit_emb(void) {
return PyModule_Create();
}
/*
* Main program
*/
char *LIST_MODULES_CMD="try:\n import my_spam; print('SPAM!');\nexcept:\n
print('no mod my_spam')";
int main(int argc, char *argv[]) {
// Run with no "my_spam" module
run_in_new_interpreter(LIST_MODULES_CMD);
// Run with "my_spam" module
PyImport_AppendInittab("my_spam", _emb);
run_in_new_interpreter(LIST_MODULES_CMD);
// How to run without my_spam? The module is still in the PyImport_Inittab
// despite the Py_FinalizeEx() call. This list is not reset until
// _PyImport_Fini2() is called, but that is never exposed in anyi public
header,
// only in Include/internal/pycore_pylifecycle.h
run_in_new_interpreter(LIST_MODULES_CMD);
return 0;
}
The output of the program is:
>>> gcc test_embed.c `pkg-config --cflags --libs python-3.6` && ./a.out
no mod my_spam
SPAM!
SPAM!
--
components: C API
messages: 407590
nosy: daniel-falk
priority: normal
severity: normal
status: open
title: Not possible to clear PyImport_Inittab after PyImport_AppendInittab
type: behavior
___
Python tracker
<https://bugs.python.org/issue45973>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45737] assertLogs to optionally not disable existing handlers
Change by Daniel <3dan...@hotmail.com>: -- type: -> enhancement ___ Python tracker <https://bugs.python.org/issue45737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45737] assertLogs to optionally not disable existing handlers
Change by Daniel <3dan...@hotmail.com>: -- components: +Library (Lib) versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue45737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45737] assertLogs to optionally not disable existing handlers
New submission from Daniel <3dan...@hotmail.com>: At the moment, assertLogs removes the handlers attached to the logger. In general this is good, because it reduces message spamming in the test logs. However, if the code being tested is relying on a handler to do something, then the test fails because the handler is being removed. This leads to the situation that the same exact test must be run twice: - first time within the context manager, to assert that specific messages were logged (using `with self.assertLogs()`) - second time, without the assertLogs to ensure the code that uses a handler does the right thing The proposal is to have `self.assertLogs()` accept a key word argument such as `keep_handlers=False`, which can be set to True, whenever the handlers should be preserved. It would probably be also useful to add a note in the documentation that makes users aware that the existing handlers will be removed. -- messages: 405858 nosy: dandiez priority: normal severity: normal status: open title: assertLogs to optionally not disable existing handlers versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue45737> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30238] 2to3 doesn't detect or fix Exception indexing
Daniel Lenski added the comment: Why was this closed? As I wrote in a previous comment, this situation can be automatically identified and addressed via static analysis in the vastly-predominant case, where the Exception object is assigned via an `exception ... as exc:` clause. -- ___ Python tracker <https://bugs.python.org/issue30238> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45249] Update doctect SyntaxErrors for location range
Change by daniel hahler : -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue45249> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45249] Update doctect SyntaxErrors for location range
daniel hahler added the comment:
I've noticed a regression/change with the code change for this issue.
When not catching the exception from `compile("invalid(", "", "single")`
it has a caret below the opening parenthesis:
```
Traceback (most recent call last):
File "…/t-syntaxerror-chained.py", line 2, in
compile("invalid(", "", "single")
File "", line 1
invalid(
^
SyntaxError: '(' was never closed
```
When using `traceback.print_exc` however this is missing:
```
Traceback (most recent call last):
File "…/t-syntaxerror-chained.py", line 2, in
compile("invalid(", "", "single")
File "", line 1
invalid(
SyntaxError: '(' was never closed
```
The file used for testing:
```
try:
compile("invalid(", "", "single")
except Exception:
# raise
__import__("traceback").print_exc()
```
(this change was noticed between 3.10.0rc2 and the final release with pdbpp's
test suite)
I've not investigated further (yet), and also feel free to ask for creating a
new issue, but I've figured it would be good to notify you here first (where
the code was changed).
--
nosy: +blueyed
___
Python tracker
<https://bugs.python.org/issue45249>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45347] datetime subject to rounding?
Change by Daniel Fortunov : -- nosy: +dfortunov ___ Python tracker <https://bugs.python.org/issue45347> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message
Daniel Hillier added the comment: I agree it is bad form but I would accidentally do it when I couldn't remember the proper API and took a stab in the dark without looking up the docs. I unfortunately used it in an example in the docs for pyzipper and started getting a few bug reports even after fixing my docs. -- ___ Python tracker <https://bugs.python.org/issue39359> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39359] zipfile: add missing "pwd: expected bytes, got str" exception message
Daniel Hillier added the comment: Thanks Łukasz! -- ___ Python tracker <https://bugs.python.org/issue39359> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45164] int.to_bytes()
daniel capelle added the comment: An ASCII representation is shown instead of the hexadecimal value, if there is any. I was not aware this is an intended behaviour. -- ___ Python tracker <https://bugs.python.org/issue45164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45164] int.to_bytes()
daniel capelle added the comment:
for example check:
(6500).to_bytes(2, 'big')
result is:
b'\x19d'
but was expected to be:
b'\x19\x64'
since
ord('d') is 100 = 0x64 there seems to be hex and char representation mixed up.
--
___
Python tracker
<https://bugs.python.org/issue45164>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45164] int.to_bytes()
New submission from daniel capelle :
for example check:
(6500).to_bytes(2, 'big')
result is:
b'\x19d'
but was expected to be:
b'\x1964'
since
ord('d') is 100 = 0x64 there seems to be hex and char representation mixed up.
--
messages: 401571
nosy: hypnoticum
priority: normal
severity: normal
status: open
title: int.to_bytes()
type: behavior
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue45164>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36550] Avoid creating AttributeError exceptions in the debugger
daniel hahler added the comment:
Given code like the following the try/except handling of Pdb (via `Cmd.onecmd`,
see https://github.com/python/cpython/pull/4666) will mess with
`sys.exc_info()`, which could be avoided:
```
try:
raise ValueError()
except Exception as exc:
e = exc
__import__('pdb').set_trace()
```
```
% ./python t_issue36550.py
--Return--
> …/t_issue36550.py(5)()->None
-> __import__('pdb').set_trace()
(Pdb) import sys; sys.exc_info()
(, AttributeError("'Pdb' object has no attribute
'do_import'"), )
```
The initial / better motivation was described in the original issue: with
pdb++/pdbpp I want to display tracebacks/errors with errors that might occur
via Pdb's prompt, where this then showed up as interfering with it.
(Sorry for not responding on https://github.com/python/cpython/pull/4666
earlier, but I think it is only part of this issue, and therefore it should not
get closed, and also creating a new one instead does not sound useful to me, so
please consider to re-open it instead.)
--
versions: +Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue36550>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44963] anext_awaitable is not a collections.abc.Generator
New submission from Daniel Pope : The anext_awaitable object returned by anext(..., default) does not support .send()/.throw(). It only supports __next__(). So we can pass messages from the suspending coroutine to the event loop but not from the event loop to the suspending coroutine. trio and curio rely on both directions working. (I don't know about asyncio.) For example, this trio code fails: import trio async def produce(): for v in range(3): await trio.sleep(1) yield v async def consume(): p = produce() while True: print(await anext(p, 'finished')) trio.run(consume) raising AttributeError: 'anext_awaitable' object has no attribute 'send'. I realise that any awaitable that wants to await another awaitable must return not an iterator from __await__() but something that implements the full PEP-342 generator protocol. Should PEP-492 section on __await__()[1] say something about that? [1] https://www.python.org/dev/peps/pep-0492/#await-expression -- components: Library (Lib) messages: 399982 nosy: lordmauve priority: normal severity: normal status: open title: anext_awaitable is not a collections.abc.Generator type: behavior versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue44963> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: Hey Raymond, > "Changing the dict implementation to a linked list would make your case perform better but would make most users worse off." Is there any standard benchmark? I would like to implement my idea, as I think that it is very unlikely that "most users would be made worse off" by a measurable amount if I implement what I have in mind. If there aren't standard benchmarks, I can always make artificial ones (insert a lot of entries, remove them, access, iterate, etc). Thank you, Daniel -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: Thank you for your reply. I didn't know that this was the expected behavior (found it at https://wiki.python.org/moin/TimeComplexity): "For these operations, the worst case n is the maximum size the container ever achieved, rather than just the current size. For example, if N objects are added to a dictionary, then N-1 are deleted, the dictionary will still be sized for N objects (at least) until another insertion is made." Thank you for pointing out! I still disagree with this design since it's not how a person using a hash map expects it to behave. That said, this link clearly shows that it's *not* a bug, and that we just have different opinions on what is the best trade-off between space, code complexity, and speed, especially on something as ubiquitous as a dictionary. Just one final attempt to gain support for my idea before I close the issue. When I proposed a linked list I didn't mean an "intro to programming" linked list, with pointers and a new malloc for every node. I meant simply adding two fields to PyDictKeyEntry with the indices of the next and previous valid entries. There would be a tiny memory overhead, and we would get a data structure that behaves how one would reasonably expect it to (compare with hash table found in the standard libraries of other languages. It's usually unexpected for an operation in a data structure to take time proportional to its historical value). I will close this issue in two days if there are no further replies. Thank you for the discussion! -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: Thank you again, Dennis. I would disagree with your conclusion for mainly 3 reasons: 1. The linked list proposal would increase the memory usage by 2 Py_ssize_t per entry, plus another 2 for the whole dictionary. I don't think this is an overwhelming amount of extra memory for Python, but of course it's open for discussion. Insertions and removals would become marginally slower (I'll try to measure it), membership wouldn't suffer, and iteration would become O(1) per element iterated on. 2. I think that this case is covered by "Dynamic Mappings" in the document you've linked to. 3. This is not about the ordering in ducts being first or second nature. Please read the example in bad_dict_example.py to see a bad case where hearing over a dict of size 1 can take milliseconds. I want to reiterate (pun not intended) that this is not a use case for me, but it surprised me that dictionaries display this behavior. It's not hard to imagine a real use case where it just happens that someone deletes elements from a dictionary in insertion order. Or that someone deletes all keys but the last inserted (in any order), resulting in a dictionary that takes way too long to iterate on. Thank you for the discussion. :) -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: I think the idea of augmenting the struts to maintain a linked list of elements together with periodically shrinking on removal would solve both time and space issues, right? Space will be always linear, and operations would still be constant amortized time (of course the worst case of removing will be linear because of the shrinking, but I guess this behavior is expected). I wrote a worse example in bad_dict_example.py submitted to this issue. This example would be fixed simply by the shrinking, but as is it's a pretty unexpected bug, as a simple sum(d.values()) can take milliseconds on a dictionary with only one entry (both key and value are int). -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: Please find attached a more complete example of the issue I am reporting. tl;dr: I can make `sum(d.values())` run in O(maximum_size_in_d's_history) instead of O(len(d)), even when len(d) == 1. The linked list approach would work in terms of making it faster, but we would still be using too much space. -- Added file: https://bugs.python.org/file50138/bad_dict_example.py ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: Thank you for your message! I'm not particularly looking for an implementation, I was just surprised by this behavior. It can get even worse. Consider this example: ``` d = large_dict() # remove all but one element of d, runs in quadratic time as explained above while len(d) > 1: del d[next(iter(d))] # now iterating over d takes O(d), even when d has only one item: # the following prints one key, but takes O(N) for key in d: print(key) ``` I think this example is better, since a person would expect iterating over a singleton dictionary would be really fast, but it can be made as slow as one wants. A "print_keys()" function would reasonably be expected to take O(size of dictionary), but it doesn't. Am I right to think that this is a performance bug? On Fri, Jul 2, 2021, 8:10 PM Dennis Sweeney wrote: > > Dennis Sweeney added the comment: > > For what it's worth, using collections.OrderedDict gives constant-time > behavior you're looking for. > > -- > nosy: +Dennis Sweeney > > ___ > Python tracker > <https://bugs.python.org/issue44555> > ___ > -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
Daniel Fleischman added the comment: The linked list solution is not as easy to implement as I expected, because of insertions. :( -- ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44555] Dictionary operations are LINEAR for any dictionary (for a particular code).
New submission from Daniel Fleischman : The following code takes quadratic time on the size of the dictionary passed, regardless of the dictionary (explanation below): ``` def slow_dictionary(d): while len(d) > 0: # Remove first element key = next(iter(d)) del d[key] ``` The problem is that when an element is deleted a NULL/NULL placeholder is set in its place (https://github.com/python/cpython/blob/818628c2da99ba0376313971816d472c65c9a9fc/Objects/dictobject.c#L1534) and when we try to find the first element with `next(iter(d))` the code needs to skip over all the NULL elements until it finds the first non-NULL element (https://github.com/python/cpython/blob/818628c2da99ba0376313971816d472c65c9a9fc/Objects/dictobject.c#L1713). I'm not sure of what is the best way to fix it, but note that simply adding a field to the struct with the position of the first non-NULL element is not enough, since a code that always deletes the SECOND element of the dictionary would still have linear operations. An easy (but memory-wasteful) fix would be to augment the struct PyDictKeyEntry with the indices of the next/previous non empty elements, and augment _dictkeysobject with the index of the first and last non empty elements (in other words, maintain an underlying linked list of the non empty entries). With this we can always iterate in O(1) per entry. (I tested it only on version 3.9.2, but I would be surprised if it doesn't happen in other versions as well). -- components: Interpreter Core messages: 396880 nosy: danielfleischman priority: normal severity: normal status: open title: Dictionary operations are LINEAR for any dictionary (for a particular code). type: performance versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue44555> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30238] 2to3 doesn't detect or fix Exception indexing
Daniel Lenski added the comment: > There is no way to know through static analysis that the subscript is on an > object of type exception. I think this should be closed as won't fix. In almost all cases, the variable in question will receive its value via `except ... as e:`. try: ... except (Exception1, Exception2, ...) as e: e[0] Seems to me that it should be possible for 2to3 to handle this large subset of applicable cases via static analysis. -- status: pending -> open ___ Python tracker <https://bugs.python.org/issue30238> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44410] Exception in AsyncMock side_effect cases incorrect refcount
New submission from Daniel Andersson : Dear maintainers, I discovered an unexpected behavior when the `side_effect` of an `AsyncMock` includes an exception. The test case below fails but I expected it to pass: ``` import sys import unittest from unittest.mock import AsyncMock class A: async def foobar(self): while True: try: return await self.mock() except Exception: continue class TestA(unittest.IsolatedAsyncioTestCase): async def test_refcount(self): a = A() a.mock = AsyncMock(side_effect=[Exception(), None]) refc = sys.getrefcount(a) await a.foobar() self.assertEqual(refc, sys.getrefcount(a)) if __name__ == "__main__": unittest.main() ``` If `side_effect=[Exception(), None]` is changed to `side_effect=[None, None]` the test case pass. I discovered this in a bigger codebase while debugging why a weakref.finalize did not trigger as expected. -- components: asyncio messages: 395752 nosy: asvetlov, penlect, yselivanov priority: normal severity: normal status: open title: Exception in AsyncMock side_effect cases incorrect refcount type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44410> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44238] Unable to install Python 3.9.5 - Windows Server
Ricardo Daniel Fuentes added the comment: Hi Steve, I am attaching the latest log files I see on temp. This is the first version of python I have installed on this server, we do install and uninstall SW on this machine all the time. As we run the operations and need to update sw there often. Thanks, Regards, Ricardo On Tue, Jun 1, 2021 at 10:19 PM Steve Dower wrote: > > Steve Dower added the comment: > > Unfortunately, I've got no idea whether it's failing to uninstall the MSI > or if it's specific to removing the PATH environment variable. If you've > still got the rest (no more than 2-3 I'd expect) of the uninstall log files > in your %TEMP% directory that may show something. > > Have you successfully installed/uninstalled other versions of Python > before on this machine/OS? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue44238> > ___ > -- Added file: https://bugs.python.org/file50082/archive (1).zip ___ Python tracker <https://bugs.python.org/issue44238> ___<> ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40172] ZipInfo corrupts file names in some old zip archives
Daniel Hillier added the comment: Looking into this more and it appears that while Appendix D of https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT says "If general purpose bit 11 is unset, the file name and comment SHOULD conform to the original ZIP character encoding" where the original encoding is IBM 437 (cp437), this is not always followed. This isn't too surprising as cp437 doesn't have every character for every language! In particular, some archive programs on windows will use the user's locale code page. https://superuser.com/questions/1321371/proper-encoding-for-file-names-in-zip-archives-created-in-windows-and-unpacked-i A UTF filename can be stored in the extra field 0x7075 in addition to a filename encoded in an arbitrary code page stored in the header's filename section. There is a open issue to add handling these fields (for reading) to zipfile: https://bugs.python.org/issue41928 and that issue may be related to this one https://bugs.python.org/issue40407 For this issue, with regards to encoding, I prefer the current situation where general purpose bit 11 for UTF is preferentially used because it doesn't change the behaviour compared to previous Python versions and it reduces file size as the filename isn't repeated in the extra field. For compatibility with other archive programs that don't support the general purpose bit 11, I suggest we add an additional mechanism to allow the code page for the path name (and comment) to be set and use the 0x7075 extra field to store the UTF name in those cases where the filename can't be encoded in ascii (and 0x6075 to store the utf comment where it can't be encoded in ascii) -- ___ Python tracker <https://bugs.python.org/issue40172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44238] Unable to install Python 3.9.5 - Windows Server
Change by Ricardo Daniel Fuentes : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue44238> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44238] Unable to install Python 3.9.5 - Windows Server
New submission from Ricardo Daniel Fuentes : Hi, I am Admin of a Windows Server 2012 R2, I have installed successfully Python 3.9.5 (64-bit) however I cannot uninstall it, I get the error: "0x80070659 - The installation is forbidden by system policy. Contact your system administrator" I don't have any software system policies defined on this server so this could be a Python bug. attached the log. Regards, Ricardo -- components: Installation files: Python 3.9.5 (64-bit)_20210526084254.txt messages: 394421 nosy: RFuentes priority: normal severity: normal status: open title: Unable to install Python 3.9.5 - Windows Server versions: Python 3.9 Added file: https://bugs.python.org/file50064/Python 3.9.5 (64-bit)_20210526084254.txt ___ Python tracker <https://bugs.python.org/issue44238> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40172] ZipInfo corrupts file names in some old zip archives
Daniel Hillier added the comment: zipfile decodes filenames using cp437 or unicode and encodes using ascii or unicode. It seems like zipfile has a preference for writing filenames in unicode rather than cp437. Is zipfile's preference for writing filenames in unicode rather than cp437 intentional? Is the bug you're seeing related to using zipfile to open and rewrite old zips and not being able to open the rewritten files in an old program that doesn't support the unicode flag? We could address this two ways: - Change ZipInfo._encodeFilenameFlags() to always encode to cp437 if possible - Add a flag to write filenames in cp437 or unicode, otherwise the current situation of ascii or unicode I guess the choice will depend on if preferring unicode rather than cp437 is intentional and if writing filenames in cp437 will break anything (it shouldn't break anything according to Appendix D of https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) Here's a test for your current patch (I'd probably put it alongside OtherTests.test_read_after_write_unicode_filenames as this test was adapted from that one) class OtherTests(unittest.TestCase): ... def test_read_after_write_cp437_filenames(self): fname = 'test_cp437_é' with zipfile.ZipFile(TESTFN2, 'w') as zipfp: zipfp.writestr(fname, b'sample') with zipfile.ZipFile(TESTFN2) as zipfp: zinfo = zipfp.infolist()[0] # Ensure general purpose bit 11 (Language encoding flag # (EFS)) is unset to indicate the filename is not unicode self.assertFalse(zinfo.flag_bits & 0x800) self.assertEqual(zipfp.read(fname), b'sample') -- nosy: +dhillier ___ Python tracker <https://bugs.python.org/issue40172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14965] super() and property inheritance behavior
Daniel Urban added the comment: @habnabit I believe I've already signed some contributor form some years ago. If there is a new one, I can sign that one too. -- ___ Python tracker <https://bugs.python.org/issue14965> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44129] zipfile: Add descriptive global variables for general purpose bit flags
Change by Daniel Hillier : -- keywords: +patch pull_requests: +24763 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26118 ___ Python tracker <https://bugs.python.org/issue44129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44129] zipfile: Add descriptive global variables for general purpose bit flags
New submission from Daniel Hillier : In the zipfile module, masking of bit flags is done against hex numbers eg. if flags & 0x800... To increase readability I suggest we replace these with global variables named for the purpose of the flag. From the example above: if flags & 0x800 becomes: if flags & _MASK_UTF_FILENAME -- components: Library (Lib) messages: 393622 nosy: dhillier, serhiy.storchaka priority: normal severity: normal status: open title: zipfile: Add descriptive global variables for general purpose bit flags versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue44129> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44128] zipfile: Deduplicate ZipExtFile code for init and resetting when seeking
Change by Daniel Hillier : -- keywords: +patch pull_requests: +24761 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26116 ___ Python tracker <https://bugs.python.org/issue44128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44128] zipfile: Deduplicate ZipExtFile code for init and resetting when seeking
New submission from Daniel Hillier : Integrating a refactor suggested in https://bugs.python.org/issue38334 The logic for preparing a ZipExtFile for reading (setting CRC state, read positions, etc) is currently in two locations: first initialisation and when seeking back to the start of a file. This change moves the logic into the method `ZipExtFile.init_read()` -- components: Library (Lib) messages: 393619 nosy: dhillier, serhiy.storchaka priority: normal severity: normal status: open title: zipfile: Deduplicate ZipExtFile code for init and resetting when seeking type: enhancement versions: Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue44128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.
Daniel Watkins added the comment: (Accidentally dropped Ned from nosy list; apologies!) -- nosy: +ned.deily ___ Python tracker <https://bugs.python.org/issue43882> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43882] [security] urllib.parse should sanitize urls containing ASCII newline and tabs.
Daniel Watkins added the comment: Hey folks, Thanks for all the work on this: I really appreciate the efforts to keep Python as secure as possible! This change _is_ causing us problems in the cloud-init codebase, which thankfully have been caught by our testing in Ubuntu's development release. This is in a fairly deep part of the codebase, so apologies in advance for the detailed description. TL;DR: cloud-init constructs mirror URLs and then sanitises them by replacing invalid characters with hyphens. With the fix for this bug, urlsplit silently removes (some of) those characters before we can replace them, modifying the output of our sanitisation code, and therefore meaning cloud-init will, albeit in fairly specific corner cases, configure different mirrors if run with a Python including this fix vs. one that precedes it. cloud-init constructs mirror URLs based on applying cloud metadata to user-configured (or default) templates. As we're responsible for constructing these URLs, we also sanitise them before configuring the package manager to use them: specifically, we urlsplit to get the hostname, IDNA-encode (to handle non-ASCII input), replace any invalid URL characters with a "-", and then strip "-" off each part of the hostname (to handle leading/trailing invalid characters), then recombine the URL. The most common case for this is a cloud which specifies values for the variables used in the template with an underscore: http://my_openstack_region.cloud.archive.ubuntu.com/ubuntu causes Apache mirrors with the default "HTTPProtocolOptions Strict" configuration to reject all requests to them (as that's an invalid hostname). In contrast, http://my-openstack-region.cloud.archive.ubuntu.com/ubuntu *is* accepted, so is preferable. (This is important because *.cloud.archive.ubuntu.com exists so that l ocal cloud admins can DNS "hijack" subdomains of it to point at internal servers: even though the Ubuntu mirrors don't reject underscored domains (any longer), this is a landmine waiting for any admins running their own mirrors.) For more background, see the bug where we figured this all out: https://bugs.launchpad.net/cloud-init/+bug/1868232 So, more concretely: if we consider a post-templated URL of http://my\topenstack\tregion.mirror.internal/ubuntu, cloud-init changes from rewriting that to my-openstack-region.mirror.internal (on < 3.9.5) to myopenstackregion.mirror.internal (on 3.9.5+): if, in this notional deployment, an apt mirror is running at (exactly) my-openstack-region.mirror.internal, then new instance deployments will start failing: they won't be able to install packages. This is the sort of breakage that we aim to avoid in cloud-init (because you just _know_ that everyone who deployed this cloud left NotionalCorp years ago, so fixing the configuration to remove these obviously-incorrect tabs is not necessarily trivial). Given the current state of the fix here, it's not clear to me how we could (cleanly) achieve our desired behaviour. We could perform replacement of these characters before invoking `urlsplit` but that would then substitute these characters outside of only the hostname: this is also a change in behaviour. We could substitute those characters with magic strings, perform the split, and then replace them in the non-hostname parts with the original character and in the hostname with hyphens: we've obviously left "cleanly" behind at this point. Another option would be to monkeypatch _UNSAFE_URL_BYTES_TO_REMOVE to an empty list: again, not a solution I'd want to have to support across Python versions! One solution that presents itself to me: add a `strip_insecure_characters: bool = True` parameter. We, in cloud-init, would pass this in as `False`, knowing that we're going to handle those ourselves. Of course, this does leave the door open for API users to keep the current insecure behaviour: if library code (either public or project-internal) were to default to `False`, then the situation is no better than today. For our use case, at least, I think a more restricted solution would work: `url_replacement_char: str = ""`. We'd call `urlsplit(..., url_replacement_char="-")` and the rest of our code would work as it does today: from its POV, there were never these invalid chars in the first place. Thanks once again for the work (and apologies for the wall of text)! Dan -- nosy: +odd_bloke -ned.deily ___ Python tracker <https://bugs.python.org/issue43882> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43798] Add position metadata to alias AST type
Change by daniel hahler : -- versions: -Python 3.9 ___ Python tracker <https://bugs.python.org/issue43798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43798] Add position metadata to alias AST type
daniel hahler added the comment: @Pablo: the problem (referenced in the pytest issue) is that `ast.alias` has new required arguments now (see also the adjusted test in https://github.com/python/cpython/commit/75a06f067bd0a2687312e5f8e78f9075be76ad3a#diff-3f516b60719dd445d33225e4f316b36e85c9c51a843a0147349d11a005c55937R1060-R1061). That's expected/wanted though (assuming that it should not have "defaults" for B/C), and there's a patch for the issue at https://github.com/pytest-dev/pytest/pull/8540. -- nosy: +blueyed ___ Python tracker <https://bugs.python.org/issue43798> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43677] Descriptor.rst - Reference to Py_MethodType is not up to date
Change by Daniel Torres : -- title: Descriptor Documentation - Reference to Py_MethodType is not up to date -> Descriptor.rst - Reference to Py_MethodType is not up to date ___ Python tracker <https://bugs.python.org/issue43677> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43677] Descriptor Documentation - Reference to Py_MethodType is not up to date
Change by Daniel Torres : -- title: Documentation -> Descriptor Documentation - Reference to Py_MethodType is not up to date ___ Python tracker <https://bugs.python.org/issue43677> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43677] Documentation
New submission from Daniel Torres : https://github.com/python/cpython/blob/master/Doc/howto/descriptor.rst Section 'Functions and methods' The provided example contains comment 'Emulate Py_MethodType in Objects/classobject.c' But Py_MethodType is nowhere to be found under 'Objects/classobject.c' -- assignee: docs@python components: Documentation messages: 389878 nosy: danielcft, docs@python priority: normal severity: normal status: open title: Documentation type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43677> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Change by Daniel Moisset : -- pull_requests: +23450 pull_request: https://github.com/python/cpython/pull/24664 ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43335] _ctypes/callbacks.c cannot be compiled by gcc 4.4.7 (RHEL6)
DANIEL VILLENEUVE added the comment: I'll let you do so if it's ok for you, since I'm not equipped with Python dev tools. Regards -- ___ Python tracker <https://bugs.python.org/issue43335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Daniel Moisset added the comment: Thanks @xtreak, I'll make some changes in these sections too. The docs are coming along well, there's still some refinement to do in the compound statements section (it's there, but looks to big), I'll submit a draft PR during the weekend so interested people can review -- ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43335] _ctypes/callbacks.c cannot be compiled by gcc 4.4.7 (RHEL6)
New submission from DANIEL VILLENEUVE : gcc 4.4.7 does not accept #pragma's inside functions. I made the following patch, inspired by code found in other Python source files. diff -r U/Python-3.9.2/Modules/_ctypes/callbacks.c V/Python-3.9.2/Modules/_ctypes/callbacks.c 433c433 < #if defined(__GNUC__) --- > #if defined(__GNUC__) && ((__GNUC__ > 4) || ((__GNUC__ == 4) && > (__GNUC_MINOR__ > 5))) 442c442 < #if defined(__GNUC__) --- > #if defined(__GNUC__) && ((__GNUC__ > 4) || ((__GNUC__ == 4) && > (__GNUC_MINOR__ > 5))) Regards -- components: ctypes messages: 387755 nosy: dvilleneuve priority: normal severity: normal status: open title: _ctypes/callbacks.c cannot be compiled by gcc 4.4.7 (RHEL6) type: compile error versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43335> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Daniel Moisset added the comment: Note: I will NOT write something for "What's New in 3.10" seeing that's in progress -- ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Daniel Moisset added the comment: Fyi, I'm looking into the documentation. I think I can have something basic for the middle of this week, so it should be ok for March 1st. -- ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
Daniel Engel added the comment: It reproduced on a windows machine -- ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
New submission from Daniel Engel : Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] Type 'copyright', 'credits' or 'license' for more information IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help. In [1]: import socket ...: s1, s2 = socket.socketpair() ...: import asyncio ...: async def test(): ...: r1, w1 = await asyncio.open_connection(sock=s1) ...: r2, w2 = await asyncio.open_connection(sock=s2) ...: s1.close() ...: asyncio.run(test()) Exception in callback _ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None)) handle: Traceback (most recent call last): File "c:\python39\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "c:\python39\lib\asyncio\proactor_events.py", line 162, in _call_connection_lost self._sock.shutdown(socket.SHUT_RDWR) OSError: [WinError 10038] An operation was attempted on something that is not a socket -- components: asyncio messages: 387228 nosy: asvetlov, danielen1337, yselivanov priority: normal severity: normal status: open title: asyncio open_connection fails when a socket is explicitly closed versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Daniel Moisset added the comment: @Ken: I've invited you to the repo. For tracking the WIP, https://github.com/dmoisset/cpython/tree/patma-docs Do you want to see if you can base the reference/compound_stmts.rst changes in PEP 634? I'll try to fit a variant of the Appendix A of PEP 636 into the tutorial and make something up for the reference/lexical_analysis.rst (which needs a section of soft keywords which are not in the PEP trilogy. Perhaps I can rescue something from PEP 622) -- ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42128] Structural Pattern Matching (PEP 634)
Daniel Moisset added the comment: Hey Ken, I was about to take on this myself, but I wouldn't mind a bit of help. Your list seems ok, I would add to it a section about PM in the python tutorial (which was explicitly mentioned in the SC acceptance notice). Regarding the grammar, I believe that one is autogenerated from the parser, so it's likely that no changes are required there. Correct me if I'm wrong here. I'll set up a branch in github and add you as a collaborator so we can prepare a PR. I'll base the branch of Brandt's implementation PR. If you give me your github id I can add you as collaborator so we can work there. -- nosy: +Daniel Moisset ___ Python tracker <https://bugs.python.org/issue42128> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43092] Python installation fails to install pip3 if pip, setuptools in ~/.local
New submission from Daniel Colascione : The ensurepip module in the Python distribution bootstraps the pip package manager. The Python build system runs this module during installation so that the resulting Python installation is ready to install packages. Right now, the pip command that ensurepip installs the pip and setuptools packages only if it doesn't find them installed already. The problem is that the pip run by ensurepip checks for its bootstrap packages in ~/.local, and if it finds them there, it concludes it doesn't need to install these packages as part of bootstrap. But in various situations (e.g., installing Python into an isolated environment), these packages need to be installed anyway. ensurepip should ignore packages in ~/.local, at least when as part of the Python build system -- components: Installation messages: 386079 nosy: quotemstr priority: normal severity: normal status: open title: Python installation fails to install pip3 if pip, setuptools in ~/.local versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43092> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43041] copying WeakValueDictionary is not iteration safe
Daniel Romberg added the comment: Sorry, I meant an exception. I changed the type to "behavior", thanks for the hint. -- type: crash -> behavior ___ Python tracker <https://bugs.python.org/issue43041> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43041] copying WeakValueDictionary is not iteration safe
New submission from Daniel Romberg : The copy operation is not safe to use during iteration. The following test case raises a "RuntimeError: dictionary changed size during iteration": import weakref class Class: pass def TEST_weakValue(): d = weakref.WeakValueDictionary() a = Class() b = Class() d["a"] = a d["b"] = b e = d.copy() for key in e: a = None c = e.copy() TEST_weakValue() This is related to https://bugs.python.org/issue35615 where I commented as well, but I couldn't find a way to reopen this issue, which is why I open this one. We experience a lot fewer crashes in weakref than before https://bugs.python.org/issue35615 had been fixed, however, there are recursive situations in which copy() is invoked while iterating the WeakValueDictionary (e.g., in our case it is a signal/slot implementation where the slots are stored in a WeakValueDictionary). _commit_removals(), which is called at the beginning of the copy operation, might change the dictionary if there are items that are to be removed. If there is an ongoing iteration, the corresponding RuntimeError is raised. I haven't thought that through entirely, but I wonder whether the copy (and also deepcopy) operation could just blindly copy everything without "committing removals". After the copy, both instances would do their _commit_removals on their own upon access. -- components: Library (Lib) messages: 385779 nosy: djromberg priority: normal severity: normal status: open title: copying WeakValueDictionary is not iteration safe type: crash versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43041> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35615] "RuntimeError: Dictionary changed size during iteration" when copying a WeakValueDictionary
Daniel Romberg added the comment: I think this issue needs to be reopened. The problem has not been solved completely. We experience a lot fewer crashes in weakref than before this fix, however, there are recursive situations in which copy() is invoked while iterating the WeakValueDictionary (e.g., in our case it is a signal/slot implementation where the slots are stored in a WeakValueDictionary). _commit_removals(), which is called at the beginning of the copy operation, might change the dictionary if there are items that are to be removed. If there is an ongoing iteration, the corresponding RuntimeError is raised. I haven't thought that through entirely, but I wonder whether the copy (and also deepcopy) operation could just blindly copy everything without "committing removals". After the copy, both instances would do their _commit_removals on their own upon access. -- nosy: +djromberg ___ Python tracker <https://bugs.python.org/issue35615> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35829] datetime: parse "Z" timezone suffix in fromisoformat()
Daniel Lenski added the comment:
Like many others here, I've run into this issue because I'm trying to parse
timestamps from JSON.
(Specifically, I'm trying to parse timestamps from JSON serialization of Java
POJOs and/or Kotlin data classes, as serialized by the Jackson serialization
library for JVM languages, in conjunction with JavaTimeModule.
https://fasterxml.github.io/jackson-modules-java8/javadoc/datetime/2.9/com/fasterxml/jackson/datatype/jsr310/JavaTimeModule.html)
In order to "be lenient in what I accept" (adhering to the robustness
principal), I need to add a special case for deserialization of strings ending
with 'Z'. This gets pretty tricky and pretty subtle quickly.
Here is my Python 3.7+ code path (the strptime-based code path for earlier
versions is much, much uglier).
from numbers import Number
from datetime import datetime, timezone
def number_or_iso8601_to_dt(ts, t=datetime):
if isinstance(ts, Number):
return datetime.utcfromtimestamp(ts).replace(tzinfo=timezone.utc)
elif ts.endswith('Z'):
# This is not strictly correct, since it would accept a string with
# two timezone specifications (e.g. ending with +01:00Z) and
# silently pass that erroneous representation:
#
# return
datetime.fromisoformat(ts[:-1]).replace(tzinfo=timezone.utc)
#
# This version is better:
d = datetime.fromisoformat(ts[:-1])
if d.tzinfo is not None:
raise ValueError(f"time data '{ts}' contains multiple timezone
suffixes")
return d.replace(tzinfo=timezone.utc)
else:
return datetime.fromisoformat(ts)
I don't really understand why .fromisoformat() must be *strictly* the inverse
of .isoformat(). As @mehaase points out, the transformation isn't strictly
reversible as is.
There are other functions where the Python standard library has special-cased
options for extremely common use cases. For example, `str.split(None)`, which
is certainly not the inverse of the non-existent `None.join()`.
This feels to me like a case where the standard library should simply just
accept an extremely-common real-world variant in the interests of
interoperability.
I would also be in favor of @p-ganssle's proposal (3), wherein
`datetime.isoformat` would also output the 'Z' suffix for the UTC timezone.
--
___
Python tracker
<https://bugs.python.org/issue35829>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35829] datetime: parse "Z" timezone suffix in fromisoformat()
Change by Daniel Lenski : -- nosy: +dlenski ___ Python tracker <https://bugs.python.org/issue35829> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42968] multiprocessing handle leak on Windows when child process is killed during startup/unpickling
Daniel Grunwald added the comment: Fix idea: get_spawning_popen().pid could be used to directly copy the handle into the child process, thus avoiding the temporary copy in the main process. This would help at least in our case (where we pass all connections during startup). I don't know if the general case is solvable -- in general we don't know which process will unpickle the data, and "child process is killed" isn't the only possible reason why the call to rebuild_pipe_connection() might not happen (e.g. exception while unpickling an earlier part of the same message). -- ___ Python tracker <https://bugs.python.org/issue42968> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
