[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Inada Naoki]

Inada Naoki  added the comment:

Memory allocation pattern is:

alloc 24  # float
alloc 24
alloc 24
alloc 64  # temporary tuple
alloc 72

free 64  # free temporary tuples
free 64
free 64


This cause some sort of fragmentation.  Some pools in arenas are unused.
This prevents pymalloc to return arenas to OS.
(Note that pymalloc manages memory as arena (256KiB) > pool (4KiB) > blocks 
(requested sizes <= 512).  pymalloc can return the memory to OS only when arena 
is clean)

But this is not too bad because many pools is free.  Any allocation which
size < 512 can reuse the free pools.
If you run some code after unpickle, the pools will be reused efficiently.

(In case of very bad fragmentation, many pools are dirty: some blocks in pools 
are used while
many blocks in the pool is free.  So only same size alloc request can use the 
pool.)


There are two approach to fix this problem.

1. Investigate why temporary tuple is not freed until last stage of unpickle.
2. When there are too many free pools, return some by MADV_FREE or 
MADV_DONTNEED.

I think (1) should be considered first.  But (2) is more general solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Inada Naoki]

Change by Inada Naoki :


--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Sorry, I will toggle back the issue status. Not sure why bpo didn't warn in 
this case.

--
assignee: gregory.p.smith -> 
stage: patch review -> backport needed
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

STINNER Victor  added the comment:


New changeset db7197543112954b0912e3d46e39fefcb1c3b950 by Victor Stinner in 
branch 'master':
bpo-36763: Rework _PyInitError API (GH-13031)
https://github.com/python/cpython/commit/db7197543112954b0912e3d46e39fefcb1c3b950


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This causes buildbot failure (AMD64 FreeBSD 10-STABLE Non-Debug 3.x and AMD64 
Debian root 3.x). I tried debugging and it's reproducible on my mac machine 
that has python not built with ssl and not reproducible on Ubuntu machine built 
with ssl. 

The failed tests use https scheme and as I can see from the file there is one 
another test (test_cafile_and_context) which is skipped and has skip test if 
ssl is absent @unittest.skipUnless(ssl, "ssl module required"). It seems 
perhaps wrapping these machines don't have ssl built with skip test might help. 
Since primary CI has ssl built it would have been caught.

On trying to add a print statement for lookup variable at 
https://github.com/python/cpython/blob/c4e671eec20dfcb29b18596a89ef075f826c9f96/Lib/urllib/request.py#L485
 I can see the below output where httpshandler was not defined for machines 
without built ssl. HTTPSConnection was not present as part of http.client due 
to import ssl causing ImportError.

Ubuntu with ssl

{'unknown': [], 'http': 
[], 'ftp': 
[], 'file': 
[], 'data': 
[], 'https': 
[]}

Mac without ssl (https handler missing causing unknown to be taken up for the 
test)

{'unknown': [], 'http': 
[], 'ftp': 
[], 'file': 
[], 'data': 
[]}

Gregory, I can create a PR with below patch if my analysis right to see if it 
helps or you can try a buildbot-custom branch to see if this works with 
buildbots since my PR won't have any effect on primary CI which have ssl built 
version of Python. I am not sure I have privileges to trigger a custom buildbot 
run. I tested it on my Mac without ssl and it has no failures since the tests 
are skipped.

diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py
index e87c85b928..c5b23f935b 100644
--- a/Lib/test/test_urllib.py
+++ b/Lib/test/test_urllib.py
@@ -329,6 +329,7 @@ class urlopen_HttpTests(unittest.TestCase, FakeHTTPMixin, 
FakeFTPMixin):
 finally:
 self.unfakehttp()

+@unittest.skipUnless(ssl, "ssl module required")
 def test_url_with_control_char_rejected(self):
 for char_no in list(range(0, 0x21)) + [0x7f]:
 char = chr(char_no)
@@ -354,6 +355,7 @@ class urlopen_HttpTests(unittest.TestCase, FakeHTTPMixin, 
FakeFTPMixin):
 finally:
 self.unfakehttp()

+@unittest.skipUnless(ssl, "ssl module required")
 def test_url_with_newline_header_injection_rejected(self):
 self.fakehttp(b"HTTP/1.1 200 OK\r\n\r\nHello.")
 host = "localhost:?a=1 HTTP/1.1\r\nX-injected: header\r\nTEST: 123"

--
assignee:  -> gregory.p.smith
stage: backport needed -> patch review
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[Gregory P. Smith]

Change by Gregory P. Smith :


--
assignee: gregory.p.smith -> 
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[Gregory P. Smith]

Gregory P. Smith  added the comment:

backports to older releases will need to be done manually and take care 
depending on how much of a concern tightening the existing abusive lenient 
behavior of the http.client API to enforce what characters are allowed in URLs 
is to stable releases.

I question if this is _really_ worthy of a "security" tag and a CVE (thus its 
non-high ranking)... it is a bug in the calling program if it blindly uses 
untrusted data as a URL.  What this issue addresses is that we catch that more 
often and raise an error; a good thing to do for sure, but the stdlib should be 
the last line of defense.

--
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30458] [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection (follow-up of CVE-2016-5699)

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset c4e671eec20dfcb29b18596a89ef075f826c9f96 by Gregory P. Smith in 
branch 'master':
bpo-30458: Disallow control chars in http URLs. (GH-12755)
https://github.com/python/cpython/commit/c4e671eec20dfcb29b18596a89ef075f826c9f96


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35181] Doc: Namespace Packages: Inconsistent documentation of __loader__ being None

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I've approved both PRs, however I am rather uncomfortable about the code 
snippet in import.rst.  Géry's change is a good one AFAICT, and thanks for the 
contribution!  I would feel much better about the long term correctness of this 
code snippet if we had a doctest to run over it.  But that's for another time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12952

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5f38b8407b071acd96da2c8cde411d0e26967735 by Victor Stinner in 
branch 'master':
bpo-36763: Add _PyCoreConfig_SetArgv() (GH-13030)
https://github.com/python/cpython/commit/5f38b8407b071acd96da2c8cde411d0e26967735


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[Chihiro Ito]

Chihiro Ito  added the comment:

I have confirmed that all of my app's test cases have passed.

What I've done:
1. Installed Python 3.7.3.
2. Replaced urllib/parse.py with the one from 781ffb1.
3. Ran my app's test cases.

Thank you for the quick fix!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26493] Bad formatting in WinError 193 when using subprocess.check_call

[Eryk Sun]

Eryk Sun  added the comment:

>> A new issue should be raised to fix the FormatMessage calls in the 
>> standard library that mistakenly leave out 
>> FORMAT_MESSAGE_IGNORE_INSERTS.
>
> Do you suggest to modify OSError constructor to modify the call to
> FormatMessageW(): don't pass the FORMAT_MESSAGE_IGNORE_INSERTS flag?
> I prefer "%1 is not a valid Win32 application" message than 
> "".

I suggested creating a new issue to fix the calls that omit this flag. I think 
it's just two modules: Modules/overlapped.c and Modules/_ctypes/callproc.c. If 
there are more inserts than arguments (i.e. any inserts in our case since we 
pass no arguments), then FormatMessageW fails and the above modules use a 
default message. For example:

>>> _overlapped.FormatMessage(193)
'unknown error code 193'
>>> _ctypes.FormatError(193)
''

> There is no need to re-raise the exception: the "strerror" attribute 
> contains the error message and it can be modified.

I meant that Popen._execute_child would handle the exception by modifying and 
reraising it. In general for OSError exceptions, we could set `filename` to 
either `executable`, if it's defined, or else parse it out of the commandline. 
For ERROR_BAD_EXE_FORMAT (193), we could also change `strerror` to something 
like "Invalid executable format" instead of "%1 is not a valid Win32 
application". This is more consistent with how we append ": {filename}" to the 
message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36760] subprocess.run fails with capture_output=True and stderr=STDOUT

[Martin Panter]

Martin Panter  added the comment:

Python 3.7 added the "capture_output" parameter, for Issue 32102. Before that 
change, you could use "subprocess.PIPE":

https://docs.python.org/3.6/library/subprocess.html#subprocess.run
“To [capture output], pass PIPE for the ‘stdout’ and/or ‘stderr’ arguments”

Since "capture_output" was added, it looks like you can still pass 
"subprocess.PIPE" on your own, but the documentation now only gives subtle 
hints that this might be supported. This was also brought up in Issue 33319.

--
nosy: +bbayles, martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12951

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 463b82a3efe8a6a9f3924a5b37482e961dffe3b8 by Victor Stinner in 
branch 'master':
bpo-36763: Fix Py_SetStandardStreamEncoding() (GH-13028)
https://github.com/python/cpython/commit/463b82a3efe8a6a9f3924a5b37482e961dffe3b8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14905] zipimport needs to support namespace packages when no 'directory' entry exists

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36740] zipimporter misses namespace packages for implicit dirs

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12950
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] PEP 587: Rework initialization API to prepare second version of the PEP

[STINNER Victor]

New submission from STINNER Victor :

I'm working on changes to complete the PEP 587, Python initiaization API.

--
components: Interpreter Core
messages: 341167
nosy: vstinner
priority: normal
severity: normal
status: open
title: PEP 587: Rework initialization API to prepare second version of the PEP
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36762] Teach "import *" to warn when overwriting globals or builtins

[Raymond Hettinger]

New submission from Raymond Hettinger :

One reason we usually suggest that people don't use star imports is that it is 
too easy to shadow a builtin or overwrite an existing global.  Momma Gump 
always used to say, "import star is like a box of chocolates, you never know 
what you've going to get".

>>> from os import *
Warning (from warnings module):
  File "__main__", line 1
ImportWarning: The 'open' variable in the 'os' module shadows a variable in the 
'builtins' module

>>> alpha = 2.0
>>> beta = 3.0
>>> gamma = 4.5
>>> delta = 5.5
>>> from math import *
>>> from os import *
Warning (from warnings module):
  File "__main__", line 8
ImportWarning: The 'gamma' variable in the 'math' overwrites an existing 
variable in the globals.

--
components: Interpreter Core
messages: 341166
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Teach "import *" to warn when overwriting globals or builtins
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[STINNER Victor]

STINNER Victor  added the comment:

Python 3.7 was also affected: I fixed it as well. Thanks for your bug report.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[miss-islington]

miss-islington  added the comment:


New changeset 5f5b187bfa17254f5ae55593820fc938c45c2b32 by Miss Islington (bot) 
in branch '3.7':
bpo-36734: Fix compilation of faulthandler.c on HP-UX (GH-12970)
https://github.com/python/cpython/commit/5f5b187bfa17254f5ae55593820fc938c45c2b32


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[miss-islington]

Change by miss-islington :


--
pull_requests: +12949

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36329] use the right python "make -C Doc/ serve"

[Berker Peksag]

Berker Peksag  added the comment:

Personally, I'd prefer removing the 'serve' target completely. make -C Doc 
htmlview should already cover most of its use cases. There is no deprecation 
period needed and there is already a replacement (and IMO better) for it.

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36345] Deprecate Tools/scripts/serve.py in favour of python -m http.server -d

[Berker Peksag]

Berker Peksag  added the comment:

Please revert 360e1e4c519cfc139de707bcdd1e6c871eec79ee. It's not a good example 
to put into the documentation.

It uses different naming convention. It would only confuse users relatively new 
to the wsgiref module and WSGI protocol.

FileWrapper was supposed to support __getitem__ and __iter__ protocols for 
compatibility with older Python versions, but its __getiem__ implementation is 
buggy and is already deprecated. It has no use case in modern Python code.

It has zero exception handling and will return a cryptic traceback if mimetype 
cannot detect type of the file.

The examples in the wsgiref documentation already cover most of the cases. 
There is no need to add more similar ones.

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14546] lll.py can't handle multiple parameters correctly

[Zackery Spytz]

Zackery Spytz  added the comment:

I've created a PR for this issue (with a test).

--
nosy: +ZackerySpytz
versions: +Python 3.7, Python 3.8 -Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14546] lll.py can't handle multiple parameters correctly

[Zackery Spytz]

Change by Zackery Spytz :


--
pull_requests: +12948
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36761] Extended slice assignment + iterable unpacking

[wim glenn]

New submission from wim glenn :

Could cases like these be made to work? *Should* cases like these be made to 
work?

L = [0, 1, 2]
L[::2], *rest = "abcdef"
# ValueError: attempt to assign sequence of size 1 to extended slice of size 2

a, L[::2] = "abc"
# ValueError: too many values to unpack (expected 2)

The list slice knows exactly how many slots need to be filled, so I can't 
immediately think of any obvious ambiguity. Maybe there are some implementation 
complications with supporting e.g. generators on the RHS (because RHS must be 
evaluated before LHS - 
https://docs.python.org/3/reference/expressions.html#evaluation-order).

--
components: Interpreter Core
messages: 341160
nosy: wim.glenn
priority: normal
severity: normal
status: open
title: Extended slice assignment + iterable unpacking
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36760] subprocess.run fails with capture_output=True and stderr=STDOUT

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Are you using something like below? This exception was added with 
ce0f33d04528fcafc673a8707871f8430d8f7ce8 (issue32102)

>>> subprocess.run('ls', stdout=subprocess.PIPE, capture_output=True)
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/subprocess.py", 
line 469, in run
raise ValueError('stdout and stderr arguments may not be used '
ValueError: stdout and stderr arguments may not be used with capture_output.

There is a note on the docs and a discussion on the issue that capture_output 
and stdout/stderr cannot be used at the same time.

https://docs.python.org/3/library/subprocess.html#subprocess.run

> If capture_output is true, stdout and stderr will be captured. When used, the 
> internal Popen object is automatically created with stdout=PIPE and 
> stderr=PIPE. The stdout and stderr arguments may not be supplied at the same 
> time as capture_output.

--
nosy: +gregory.p.smith, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36760] subprocess.run fails with capture_output=True and stderr=STDOUT

[Joe Borg]

New submission from Joe Borg :

Reading from 
https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess

"""
If you ran the process with stderr=subprocess.STDOUT, stdout and stderr will be 
combined in this attribute, and stderr will be None.
"""

But, if you run `run()` with `capture_output=True`, you get the following 
exception:

"""
ValueError: stdout and stderr arguments may not be used with capture_output.
"""

So, it seems impossible to get the combined outputs of stdout and stderr with 
`run()`.

--
components: Library (Lib)
messages: 341158
nosy: Joe.Borg
priority: normal
severity: normal
status: open
title: subprocess.run fails with capture_output=True and stderr=STDOUT
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] datetime: astimezone() results in OSError: [Errno 22] Invalid argument

[SilentGhost]

SilentGhost  added the comment:

This seems like a duplicate (or at least very similar) to the issue 29097. 
Could you try a newer version of Python (that issue was fixed in 3.6.7) to make 
sure it's not a duplicate?

That was specifically a Windows bug, Windson, make sure that what you're seeing 
is not fixed elsewhere if you only have macOS available.

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33012] Invalid function cast warnings with gcc 8 for METH_NOARGS

[Andreas Schneider]

Andreas Schneider  added the comment:

And how do you deal with METH_VARARGS|METH_KEYWORDS functions which have 3 
arguments?

PyObject* myfunc(PyObject *py_obj, PyObject *args, PyObject *kwargs)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33012] Invalid function cast warnings with gcc 8 for METH_NOARGS

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Yes, and this particular case was fixed in 
adfffc7343ce7ebc88ec734a803d3247ba8927fb.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36757] uuid constructor accept invalid strings (extra dash)

[Josh Rosenberg]

Josh Rosenberg  added the comment:

The documentation does describe a fairly flexible parser. Perhaps it's a little 
too flexible on stuff like URN prefixes, but I don't think we could start 
enforcing a stricter class of hyphen separations without potentially breaking 
existing code.

Is there are reason your validator doesn't use uuid.UUID to normalize the 
value? That is, whatever the customer provides, why not use the result of 
stringifying the resulting UUID, rather than just convert to UUID to validate, 
then throwing it away? As long as the result is compatible with your sql 
connector, and logically equivalent to what the customer provided, that seems a 
valid solution.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33012] Invalid function cast warnings with gcc 8 for METH_NOARGS

[Andreas Schneider]

Andreas Schneider  added the comment:

Looking at:

https://github.com/python/cpython/commit/359a2f3daba49fde0d3a07fb3c7a8b051c450d08

This is not fixing the underlying issue but hiding it. The right fix would be 
to use a union for ml_meth providing members for the 3 different function. So 
the developer could assign them correctly and the compiler would warn if he 
would do something wrong. Casting to (void *) is just hiding the problem not 
fixing it!

--
nosy: +asn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] datetime: astimezone() results in OSError: [Errno 22] Invalid argument

[Windson Yang]

Windson Yang  added the comment:

on macOS 10.14.4, I got `ValueError: offset must be a timedelta representing a 
whole number of minutes, not datetime.timedelta(0, 29143).` I will do some 
research to see why this happen.

--
nosy: +Windson Yang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[miss-islington]

miss-islington  added the comment:


New changeset 4d723e76e1ad17e9e7d5e828e59bb47e76f2174b by Miss Islington (bot) 
in branch '3.7':
bpo-36742: Fixes handling of pre-normalization characters in urlsplit() 
(GH-13017)
https://github.com/python/cpython/commit/4d723e76e1ad17e9e7d5e828e59bb47e76f2174b


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +12947

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[miss-islington]

Change by miss-islington :


--
pull_requests: +12946

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[Steve Dower]

Steve Dower  added the comment:


New changeset d537ab0ff9767ef024f26246899728f0116b1ec3 by Steve Dower in branch 
'master':
bpo-36742: Fixes handling of pre-normalization characters in urlsplit() 
(GH-13017)
https://github.com/python/cpython/commit/d537ab0ff9767ef024f26246899728f0116b1ec3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36742] urlsplit doesn't accept a NFKD hostname with a port number

[miss-islington]

Change by miss-islington :


--
pull_requests: +12945

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] datetime: astimezone() results in OSError: [Errno 22] Invalid argument

[Snidhi Sofpro]

New submission from Snidhi Sofpro :

With: Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit 
(Intel)] on win32

import datetime;

d_Time = datetime.datetime.strptime('03:30 PM', '%I:%M %p');
d_Time = d_Time.astimezone(datetime.timezone.utc);
# RESULTS IN OSError: [Errno 22] Invalid argument

# WHEREAS the foll. does not have the issue!
d_Time = datetime.datetime(year= d_Time.year,
   month  = d_Time.month,
   day = d_Time.day,
   hour= d_Time.hour,
   minute = d_Time.minute,
   second  = d_Time.second,
   tzinfo  = datetime.timezone.utc);

print(d_Time);

--
components: Library (Lib)
messages: 341149
nosy: Snidhi
priority: normal
severity: normal
status: open
title: datetime: astimezone() results in OSError: [Errno 22] Invalid argument
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36758] configured libdir not correctly passed to Python executable

[Michael Osipov]

New submission from Michael Osipov <1983-01...@gmx.net>:

I do compile Python from master on HP-UX with aCC:

# echo $LDFLAGS $CPPFLAGS
-L/opt/ports/lib/hpux32 -I/opt/ports/include
UNIX_STD=1998 LDFLAGS="$LDFLAGS -lreadline" CPPFLAGS="-I$PREFIX/include/ncurses 
$CPPFLAGS" ./configure --prefix=/opt/python \
  --libdir=/opt/python/lib/hpux32 --with-system-expat 
--with-openssl=/opt/openssl

having libs in hpux32 or hpux64 is a convention on this platform. When Python 
is installed the following happens:

$ /opt/python/bin/python3
Could not find platform dependent libraries 
Consider setting $PYTHONHOME to [:]
Python 3.8.0a3+ (default, Apr 30 2019, 12:09:29) [C] on hp-ux11
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/opt/python/lib/python38.zip', '/opt/python/lib/python3.8', 
'/opt/python/lib/lib-dynload', 
'/net/home/osipovmi/.local/lib/python3.8/site-packages', 
'/opt/python/lib/python3.8/site-packages']

I don't see hpux32 anywhere here. Though all shared objects are there.

Reconfiguring:

# echo $LDFLAGS $CPPFLAGS
-L/opt/ports/lib/hpux32 -I/opt/ports/include
UNIX_STD=1998 LDFLAGS="$LDFLAGS -lreadline" CPPFLAGS="-I$PREFIX/include/ncurses 
$CPPFLAGS" ./configure --prefix=/opt/python \
  --with-system-expat --with-openssl=/opt/openssl

gives me the expected result:
$ /opt/python/bin/python3
Python 3.8.0a3+ (default, Apr 30 2019, 12:21:15) [C] on hp-ux11
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/opt/python/lib/python38.zip', '/opt/python/lib/python3.8', 
'/opt/python/lib/python3.8/lib-dynload', 
'/net/home/osipovmi/.local/lib/python3.8/site-packages', 
'/opt/python/lib/python3.8/site-packages']
>>>

It pretty much seems like '--libdir' is not correctly passed down to the 
installation.

I can see that at least this is wrong:
./Lib/distutils/tests/test_install.py:59:libdir = 
os.path.join(destination, "lib", "python")
./Lib/distutils/command/install.py:32:'purelib': 
'$base/lib/python$py_version_short/site-packages',
./Lib/distutils/command/install.py:33:'platlib': 
'$platbase/lib/python$py_version_short/site-packages',
./Lib/distutils/command/install.py:39:'purelib': '$base/lib/python',
./Lib/distutils/command/install.py:40:'platlib': '$base/lib/python',
./Lib/sysconfig.py:23:'stdlib': 
'{installed_base}/lib/python{py_version_short}',
./Lib/sysconfig.py:24:'platstdlib': 
'{platbase}/lib/python{py_version_short}',
./Lib/sysconfig.py:25:'purelib': 
'{base}/lib/python{py_version_short}/site-packages',
./Lib/sysconfig.py:26:'platlib': 
'{platbase}/lib/python{py_version_short}/site-packages',
./Lib/sysconfig.py:35:'stdlib': '{installed_base}/lib/python',
./Lib/sysconfig.py:36:'platstdlib': '{base}/lib/python',
./Lib/sysconfig.py:37:'purelib': '{base}/lib/python',
./Lib/sysconfig.py:38:'platlib': '{base}/lib/python',
./Lib/sysconfig.py:65:'stdlib': 
'{userbase}/lib/python{py_version_short}',
./Lib/sysconfig.py:66:'platstdlib': 
'{userbase}/lib/python{py_version_short}',
./Lib/sysconfig.py:67:'purelib': 
'{userbase}/lib/python{py_version_short}/site-packages',
./Lib/sysconfig.py:68:'platlib': 
'{userbase}/lib/python{py_version_short}/site-packages',
./Lib/sysconfig.py:74:'stdlib': '{userbase}/lib/python',
./Lib/sysconfig.py:75:'platstdlib': '{userbase}/lib/python',
./Lib/sysconfig.py:76:'purelib': '{userbase}/lib/python/site-packages',
./Lib/sysconfig.py:77:'platlib': '{userbase}/lib/python/site-packages',
./configure.ac:4653:  
LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
./Misc/python-config.sh.in:50:LIBDEST=${prefix_real}/lib/python${VERSION}
./Modules/getpath.c:129:wchar_t *lib_python;   /* 
"lib/pythonX.Y" */
./Modules/getpath.c:131:wchar_t zip_path[MAXPATHLEN+1];/* 
".../lib/pythonXY.zip" */
./Modules/getpath.c:520: * e.g. /usr/local/lib/python1.5 is reduced to 
/usr/local.
./Modules/getpath.c:1018:err = joinpath(calculate->zip_path, 
L"lib/python00.zip", zip_path_len);
./Modules/getpath.c:1147:calculate->lib_python = 
Py_DecodeLocale("lib/python" VERSION, );
./Python/coreconfig.c:103:#  define PYTHONHOMEHELP "/lib/pythonX.X"


I have changed those files manually by adding '/hpux32' everywhere and ran 
configure with custom libdir:
no avail. 'lib/python' will still be used. If this cannot be changed, a warning 
should be issued with ./configure that custom libdir will lead to loading 
issues.

--
components: Build, Installation
messages: 341148
nosy: michael-o, vstinner
priority: normal
severity: normal
status: open
title: configured libdir not correctly passed to Python executable
versions: Python 3.8

___
Python tracker 

___

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[Michael Osipov]

Michael Osipov <1983-01...@gmx.net> added the comment:

I do not know because I haven't really tested that branch. My HP-UX PRs 
(https://github.com/python/cpython/pulls/michael-o) are still open and apply to 
master currently. We (you and me) agreed some time ago, that we go master first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26493] Bad formatting in WinError 193 when using subprocess.check_call

[Raúl Núñez de Arenas]

Raúl Núñez de Arenas  added the comment:

That's the kind of patch I assumed was created for the Windows part, Berker, 
that's why I reopened the issue.

Thanks for the information.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6584] gzip module has no custom exception

[Zackery Spytz]

Zackery Spytz  added the comment:

I'd like to see this issue move forward, so I've created a PR.

--
nosy: +ZackerySpytz
versions: +Python 3.8 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6584] gzip module has no custom exception

[Zackery Spytz]

Change by Zackery Spytz :


--
pull_requests: +12944
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for testing, I merged my PR. Does Python 3.7 have the same issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b84cb70880a0acfcbbaca7bcda405af08f94d269 by Victor Stinner in 
branch 'master':
bpo-36734: Fix compilation of faulthandler.c on HP-UX (GH-12970)
https://github.com/python/cpython/commit/b84cb70880a0acfcbbaca7bcda405af08f94d269


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36734] Modules/faulthandler.c does not compile on HP-UX due to bpo-35214/1584a0081500d35dc93ff88e5836df35faf3e3e2

[Michael Osipov]

Michael Osipov <1983-01...@gmx.net> added the comment:

The memset() works as expected and compiles for me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36756] tkinter tk.createcommand memory leak

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +gpolo, serhiy.storchaka
title: createcommand memory leak -> tkinter tk.createcommand memory leak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36757] uuid constructor accept invalid strings (extra dash)

[Cédric Cabessa]

New submission from Cédric Cabessa :

UUID constructor accept string with too many dashes or keyword like urn: / uuid:

For eg, this code do not raise

```
>>> import uuid
>>> uuid.UUID('0be--468urn:urn:urn:urn:54-4bf9-41--d4-9697-41d735uuid:4fbe85uuid:')
UUID('0be46854-4bf9-41d4-9697-41d7354fbe85')
```

For the context, we use a validator based on `uuid.UUID` for an API.
Some customer send string with a UUID followed by extra `-`, the validator let 
it pass but the sql connector raise an exception

We workaround this in our validator, but UUID constructor should not accept 
string like the one in exemple

--
components: Library (Lib)
messages: 341141
nosy: Cédric Cabessa
priority: normal
severity: normal
status: open
title: uuid constructor accept invalid strings (extra dash)
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36756] createcommand memory leak

[Wolfram Kraus]

New submission from Wolfram Kraus :

When using tk.createcommand you get a memory leak when you don't explicitly 
call tk.deletecommand to remove this command.
See attached file: __del__ never get's called due to the memory leak and 
because of that calling tk.deletecommand inside __del__ has no effect. If you 
remove the tk.createcommand everything works fine.

--
components: Tkinter
files: tclmem_bug.py
messages: 341140
nosy: WKraus
priority: normal
severity: normal
status: open
title: createcommand memory leak
type: resource usage
versions: Python 2.7, Python 3.7
Added file: https://bugs.python.org/file48292/tclmem_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32223] distutils doesn't correctly read UTF-8 content from config files

[Eduardo Suarez-Santana]

Eduardo Suarez-Santana  added the comment:

In my opinion, 'setup.cfg' is setuptools stuff and not part of python project. 
Anyway I think it is a good idea to read it as UTF-8.

Setuptools seems to be defaulting to UTF-8: 
https://github.com/pypa/setuptools/pull/1735

--
nosy: +esuarezsantana

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25430] speed up ipaddress __contain__ method

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 3bbcc92577f8e616bc94c679040043bacd00ebf1 by Inada Naoki 
(gescheit) in branch 'master':
bpo-25430: improve performance of IPNetwork.__contains__ (GH-1785)
https://github.com/python/cpython/commit/3bbcc92577f8e616bc94c679040043bacd00ebf1


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36018] Add a Normal Distribution class to the statistics module

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset b0a2c0fa83f9b79616ccf451687096542de1e6f8 by Raymond Hettinger in 
branch 'master':
bpo-36018: Test idempotence. Test two methods against one-another. (GH-13021)
https://github.com/python/cpython/commit/b0a2c0fa83f9b79616ccf451687096542de1e6f8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36018] Add a Normal Distribution class to the statistics module

[Raymond Hettinger]

Change by Raymond Hettinger :


--
pull_requests: +12943

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36750] test_socket leaks file descriptors on macOS

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

The test failure is reproducible on the PR 12271 merging master branch on 
Ubuntu with -R 3:3 test_socket. Sorry, I tried it initially on master thinking 
it was a master branch problem. I don't see C related code in the PR. There was 
a merge commit [0] after which the failure is consistently reproducible. I 
tried reverting parts of the PR to narrow down the failure and happened to come 
across the below statement in the diff where the test object and elapsed test 
for time are stored to be printed later. Commenting out the call makes tests 
pass for test_socket and much more baffling is that changing the code to use 
list instead of tuple causes the error to go away. Also I tried changing append 
to use extend or just to append test object instead of a tuple of test object 
and elapsed time which all pass. Instead of using test object in tuple just 
using (object(), elapsed) as a tuple also causes failure. I guess there is 
something with tuples here but it doesn't make much sense.

On trying to bisect the merge commit I have narrowed down the below commits 
where the PR passes with tuple itself and then fails. In between the success 
and failure commit there were some changes made to hunterleaks and 
socket.create_server utility was added.

 f66e336f45 (success)
 58721a9030 (success)
 2b00db68554422ec37faba2a80179a0172df6349 (fails)

# Sample failure on PR's HEAD on Ubuntu 

⋊> ~/cpython on pr_12271  ./python -m test --fail-env-changed -R 3:3 
test_socket 
   05:45:52
Run tests sequentially
0:00:00 load avg: 0.22 [1/1] test_socket
beginning 6 repetitions
123456
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 
5)
Dangling thread: 
Dangling thread: 
Dangling thread: 
Dangling thread: 
Dangling thread: <_MainThread(MainThread, started 140152731227904)>
..
test_socket failed (env changed) in 3 min 6 sec

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
test_socket

Total duration: 3 min 6 sec
Tests result: ENV CHANGED

The below patch on the PR to use list causes the tests to pass but I am highly 
confused over how changing to tuple to list fixes this and this might 
potentially be hiding a bug.

$ git diff
diff --git a/Lib/unittest/result.py b/Lib/unittest/result.py
index 273ca3beef..b8fd714679 100644
--- a/Lib/unittest/result.py
+++ b/Lib/unittest/result.py
@@ -160,7 +160,7 @@ class TestResult(object):

 def addDuration(self, test, elapsed):
 """Called when a test finished to run, regardless of its outcome."""
-self.collectedDurations.append((test, elapsed))
+self.collectedDurations.append([test, elapsed])

 def wasSuccessful(self):
 """Tells whether or not this result was a success."""

[0] 
https://github.com/python/cpython/commit/3c4af91c1e7c02561efa752885011ff642725bd8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com