[issue40291] socket library support for CAN_J1939

[Karl Ding]

Change by Karl Ding :


--
keywords: +patch
pull_requests: +18886
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19538

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38583] The activate script in Windows is not correct for venvs created in git-bash

[Mark Mikofski]

Mark Mikofski  added the comment:

Would you consider just handling activate for windows directly in the 
lib/venv/__init__.py method "install_scripts(self, context, path)"
https://github.com/python/cpython/blob/4f98f465f14e7258c5b18a62c5aa114dbe1174d8/Lib/venv/__init__.py#L382

if not srcfile.endswith(('.exe', '.pdb')):

# handle activate for Windows (ignore WSL)
if srcfile == "activate":
# from docs: on unix drive is always empty
d, p = os.path.splitdrive(context.env_dir)
d = d.replace(':', '')
p = p.replace('\\', '/')
if d:
p = '/' + d + p
data = data.decode('utf-8')
data = data.replace('__VENV_DIR__', p)
data = data.encode('utf-8')

try:
data = data.decode('utf-8')
data = self.replace_variables(data, context)
data = data.encode('utf-8')
except UnicodeError as e:

IMHO I don't think the use of windows python in WSL is a realistic use-case, my 
preference would be to just make this fail. In fact I tried to use it, and I 
could not make it work.
1. /mnt/c/path/to/python -m venv venv
Error: [WinError 5] Access is denied: 'C:\\WINDOWS\\system32\\venv'
2. /mnt/c/path/to/python -m venv -m venv /mnt/c/some/path/to/venv
fails silently, appears to do nothing, venv is not created
3. /mnt/c/path/to/python -m venv -m venv 'C:/some/path/to/venv'
makes directories at C:\some\path\to\venv, but can't be activated in WSL, that 
I can figure out
source /mnt/c/some/path/to/venv/Scripts/activate
: command not found
-bash: /mnt/c/some/path/to/venv/Scripts/activate: line 4: syntax error near 
unexpected token `$'{\r''
'bash: /mnt/c/some/path/to/venv/Scripts/activate: line 4: `deactivate () {

I guess I don't really understand why it would be useful to use the windows 
python in WSL, and if that's the only thing holding a quick fix for this, I 
guess, I would prefer to just handle windows python in windows in git-bash, and 
ignore WSL. Would you be open to that?

If so, I'm happy to submit a PR

thanks!

--
nosy: +bwanamarko

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40291] socket library support for CAN_J1939

[Karl Ding]

New submission from Karl Ding :

It would be nice to have support J1939 sockets.

Support for J1939 landed as part of the SAE J1939 SocketCAN patches, which are 
available after the Linux 5.4rc1 kernel. This is another protocol family for 
SocketCAN, much like the existing support for ISOTP and BCM.

Effectively, supporting this would hand off as much to the kernel as possible, 
which gives Python programs the ability deal with J1939 without having to 
implement the logic existing in the kernel in userspace.

This is provided to userspace [0] via the  header.

I'd be happy to work on this and provide a PR.

[0] https://www.kernel.org/doc/html/latest/networking/j1939.html

--
components: Library (Lib)
messages: 366487
nosy: karlding
priority: normal
severity: normal
status: open
title: socket library support for CAN_J1939
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Glenn Linderman]

Glenn Linderman  added the comment:

Here's what I have.

Maybe it would be better if parse and dump were under or dunder names, although 
I think parse was in the original implementation I found.

Is this the derived from the same original as PyPI dotable? Not sure.

--
Added file: https://bugs.python.org/file49061/Dotable.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Glenn Linderman]

Glenn Linderman  added the comment:

Yes, I laud this effort. I don't care if it is called SimpleNamespace (which I 
didn't use because it was insufficient), or anything else, but it would be 
extremely handy to have this battery.

I eventually found one called Dotable (or did I rename it to that?) which after 
a fair bit of study I finally understood, and then was able to add a few tweaks 
to make it work the way that was most convenient for me.

While I agree with Guido that these are not standard Python semantics, they 
are, as pointed out by Raymond, far more convenient to use for nested 
structures.

And as far as using CoffeeScript, I don't know of a CoffeeScript to Python 
conversion: the rest of the semantics of Python are more preferable than 
Javascript. I just wish Brendan Eich had consulted with Guido before inventing 
Javascript.

--
nosy: +v+python

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40290] Add z_score to statistics.NormalDist

[Raymond Hettinger]

New submission from Raymond Hettinger :

I've had a couple of requests for a z-score method, once to produce an actual 
z-score for output and another as a way of normalizing gaussian inputs for 
machine learning.

Proposed:

>>> iq = NormalDist(100, 15)
>>> iq.zscore(142)
2.8

Same result as:

>>> (142 - iq.mean) / iq.stdev
2.8

There is some question about whether to name it zscore or z_score.  Numpy uses 
zscore but numpy tends to scrunch names where we would tend to spell them out 
or use an underscore for readability.

See: https://en.wikipedia.org/wiki/Standard_score

--
assignee: rhettinger
components: Library (Lib)
messages: 366484
nosy: rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: Add z_score to statistics.NormalDist
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I checked and all current buildbot failures are related to the refleak in 
test_threading so I think this issue is fixed. I will close the issue, please 
reopen of I missed something or you would like to address something else :)

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I think that this was fixes by PR19525 and PR19523. For instance, the buildbot 
you mentioned is green when building those PRs:

https://buildbot.python.org/all/#/builders/500/builds/289

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28002] ast.unparse can't roundtrip some f-strings

[Shantanu]

Shantanu  added the comment:

Now that `ast.unparse` is being added to stdlib, I thought I'd bump this 
thread. The third party library `astunparse` (which attempts to support 
multiple Python versions while staying very close to Tools/parser/unparse.py) 
has an implementation that works for complex cases.
If that sounds good, I can submit a PR and inform the original author 
(astunparse is license is based on PSF's, so should be okay if the original 
author doesn't respond).

https://github.com/simonpercivall/astunparse/blob/master/lib/astunparse/unparser.py#L461

--
nosy: +hauntsaninja

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40289] "highlighting" how to get Python's Scripts directory in the documentation

[thautwarm]

Change by thautwarm :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
versions: +Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40289] "highlighting" how to get Python's Scripts directory in the documentation

[thautwarm]

New submission from thautwarm :

Currently it's barely impossible for us to search about "How to get where 
Scripts directory/folder is".

If we google this, we can only get answers about '__file__', which is far from 
the expectation.

The correct information lies on
- https://github.com/python/cpython/blob/master/Doc/install/index.rst
- https://github.com/python/cpython/blob/master/Lib/sysconfig.py

It's also not the first time I want to google this but this is the first time I 
got the answer, by dipping into the source code and searching in the codebase.

I'd hope we can directly mention the phrase "Python Scripts directory" in the 
documentation of 'sysconfig' module.

--
messages: 366480
nosy: thautwarm
priority: normal
severity: normal
status: open
title: "highlighting" how to get Python's Scripts directory in the documentation

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Kyle Stanley]

Kyle Stanley  added the comment:

> Agreed, and that is precisely what I suggested in my previous comment. The 
> attached PR already deals with the executor-specific part of the wait, but it 
> would be straightforward to have it also affect the executor to create daemon 
> threads, and the flag moved to the constructor.

Ah, I see. That should be fine then, but I would like to wait on additional 
feedback to make sure that re-adding daemon threads (even as a opt-in flag) is 
something that we want to do. I think you do have a good point regarding the 
practical utility in situations where it may not be feasible to 
programmatically prevent a thread from hanging indefinitely, but the main 
questions are: 

Is it something that ought to be provided by ThreadPoolExecutor? 
If so, is this the best way to implement it?

> It is quite easy to check that a hanging thread (emulated by a simple sleep) 
> is not joined by the executor with the appropriate flag set.

That would certainly be part of it, but we also have to verify a few other 
things (this is off the top of my head, so I'm likely missing some points):

1) All of the threads are created as daemon threads, instead of regular threads 
when the flag is set (easy)
2) The hanging thread itself was able to finalize properly (without any leaked 
references or resource issues)
3) The rest of the executor resources were able to finalize properly, despite 
the hanging thread
4) The interpreter was able to finalize properly, despite the hanging thread

It's not so much writing the tests that will be especially involved, it's more 
so the time investment required to get them all to pass. Which could 
potentially be on the first try, or it could take many different 
implementations until it works across supported platforms. In my experience so 
far from working with the executors, it's more likely to be the latter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40277] Improve repr for _tuplegetter objects

[Ammar Askar]

Change by Ammar Askar :


--
keywords: +patch
nosy: +ammar2
nosy_count: 1.0 -> 2.0
pull_requests: +18885
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19537

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40288] atexit module should not be loaded more than once per interpreter

[STINNER Victor]

New submission from STINNER Victor :

Since Python 3.7, it's possible to load the atexit module more than once:

commit 776407fe893fd42972c7e3f71423d9d86741d07c
Author: Marcel Plch 
Date:   Wed Dec 20 11:17:58 2017 +0100

bpo-31901: atexit callbacks should be run at subinterpreter shutdown (#4611)

Change atexit behavior and PEP-489 multiphase init support.

Each new import executes the module which overrides 
PyInterpreterState.pyexitfunc with _Py_PyAtExit().

Example:
---
import sys

atexit1 = sys.modules.pop('atexit', None)
if atexit1 is None:
import atexit as atexit1
del sys.modules['atexit']

import atexit as atexit2

atexit1.register(print, "atexit1 callback")
atexit2.register(print, "atexit2 callback")
---

Output:
---
atexit2 callback
---

Either PyInterpreterState should support a list of exit functions, or atexit 
should raise an exception if it's loaded more than once.

call_ll_exitfuncs() calls a list of functions: _PyRuntimeState.exitfuncs. But 
these functions are called at the end of Py_Finalize(), whereas atexit 
functions are called after calling threading._shutdown() in Py_Finalize() and 
Py_EndInterpreter().

--
components: Library (Lib)
messages: 366478
nosy: corona10, vstinner
priority: normal
severity: normal
status: open
title: atexit module should not be loaded more than once per interpreter
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18884
pull_request: https://github.com/python/cpython/pull/19536

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4f98f465f14e7258c5b18a62c5aa114dbe1174d8 by Victor Stinner in 
branch 'master':
bpo-40268: Remove unused imports in pylifecycle.c (GH-19533)
https://github.com/python/cpython/commit/4f98f465f14e7258c5b18a62c5aa114dbe1174d8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks!
-- 
--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40287] SpooledTemporaryFile.seek returns None

[amcinnes]

New submission from amcinnes :

The documentation says SpooledTemporaryFile "operates exactly as 
TemporaryFile() does".

seek() would be expected to return the new absolute position; this is what it 
does for TemporaryFile, and is the documented behaviour of seek() in IOBase. 
But for SpooledTemporaryFile it returns None.

Probably trivial to fix by sticking a "return" on 
https://github.com/python/cpython/blob/0361556537686f857f1025ead75e6af4ca7cc94a/Lib/tempfile.py#L741

Python 3.8.2 (default, Apr  8 2020, 14:31:25) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> t = tempfile.TemporaryFile()
>>> print(t.seek(0))
0
>>> u = tempfile.SpooledTemporaryFile()
>>> print(u.seek(0))
None
>>>

--
components: Library (Lib)
messages: 366475
nosy: amcinnes
priority: normal
severity: normal
status: open
title: SpooledTemporaryFile.seek returns None
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:

On python-dev, Ronald Oussoren asked to add support for the buffer protocol in 
PyType_FromSpec() and PyTypeSpec API:

Ronald:
> BTW. This will require growing the PyTypeSpec ABI a little, there are 
> features you cannot implement using that API for example the buffer protocol.

https://mail.python.org/archives/list/python-...@python.org/message/PGKRW7S2IUOWVRX6F7RT6VAWD3ZPUDYS/

See also PyType_FromSpec() issue with opaque PyObject:
https://bugs.python.org/issue39573#msg366473

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39573] [C API] Make PyObject an opaque structure in the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

PyType_FromSpec() and PyType_Spec API are not currently compatible with opaque 
PyObject.

Example:
---
#define PyObject_HEADPyObject ob_base;

typedef struct {
PyObject_HEAD
...
} MyObject;

static PyType_Spec type_spec = {
.name = "MyObject",
.basicsize = sizeof(MyObject),
...
};

... = PyType_FromSpec(_spec);
---

sizeof(MyObject) requires to compute sizeof(PyObject).

Issue reported by Ronald Oussoren on python-dev:
https://mail.python.org/archives/list/python-...@python.org/message/PGKRW7S2IUOWVRX6F7RT6VAWD3ZPUDYS/

--
title: Make PyObject an opaque structure in the limited C API -> [C API] Make 
PyObject an opaque structure in the limited C API

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18883
pull_request: https://github.com/python/cpython/pull/19533

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 361dcdcefc80f5729ed18ac0ef73327794fbf400 by Victor Stinner in 
branch 'master':
bpo-40268: Remove unused osdefs.h includes (GH-19532)
https://github.com/python/cpython/commit/361dcdcefc80f5729ed18ac0ef73327794fbf400


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[STINNER Victor]

STINNER Victor  added the comment:

Many buildbots failed. Example:

https://buildbot.python.org/all/#/builders/500/builds/288

==
FAIL: test_annotations (test.test_future.AnnotationsFutureTestCase)
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/test/test_future.py",
 line 156, in test_annotations
eq("u'some_string'")
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/test/test_future.py",
 line 150, in assertAnnotationEqual
self.assertEqual(actual, expected)
AssertionError: "'some_string'" != "u'some_string'"
- 'some_string'
+ u'some_string'
? +


--

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[STINNER Victor]

STINNER Victor  added the comment:

commit aade1cc453698e1bc48861b16955c2c2219ec521
Author: Batuhan Taşkaya 
Date:   Tue Apr 14 21:55:01 2020 +0300

bpo-395222: Correctly unparse unicode prefix in ast_unparse.c (GH-19512)

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18882
pull_request: https://github.com/python/cpython/pull/19532

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d9ea5cae1d07e1ee8b03540a9367c26205e0e360 by Victor Stinner in 
branch 'master':
bpo-40268: Remove unused pycore_pymem.h includes (GH-19531)
https://github.com/python/cpython/commit/d9ea5cae1d07e1ee8b03540a9367c26205e0e360


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Ethan Smith  added the comment:

I went through the list I generated and it seems that the
ipaddress._BaseNetwork and mmap.mmap cases are the only one I saw that
shouldn't be generic. I will submit a PR to revert those. The only item
left after that which I know of is _lru_cache_wrapper.

On Tue, Apr 14, 2020 at 4:14 PM Guido van Rossum 
wrote:

>
> Guido van Rossum  added the comment:
>
>
> New changeset d01628e411752ee6849f862cae66a1c69fe512b7 by Ethan Smith in
> branch 'master':
> bpo-39481: PEP 585 for dataclasses, mailbox, contextvars (GH-19425)
>
> https://github.com/python/cpython/commit/d01628e411752ee6849f862cae66a1c69fe512b7
>
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18881
pull_request: https://github.com/python/cpython/pull/19531

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4a21e57fe55076c77b0ee454e1994ca544d09dc0 by Victor Stinner in 
branch 'master':
bpo-40268: Remove unused structmember.h includes (GH-19530)
https://github.com/python/cpython/commit/4a21e57fe55076c77b0ee454e1994ca544d09dc0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18880
pull_request: https://github.com/python/cpython/pull/19530

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 62183b8d6d49e59c6a98bbdaa65b7ea1415abb7f by Victor Stinner in 
branch 'master':
bpo-40268: Remove explicit pythread.h includes (#19529)
https://github.com/python/cpython/commit/62183b8d6d49e59c6a98bbdaa65b7ea1415abb7f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-2897: Deprecate structmember.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40268] Reorganize pycore_pystate.h header

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18879
pull_request: https://github.com/python/cpython/pull/19529

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40270] activate (or include) json1 extension in sqlite

[Ammar Askar]

Change by Ammar Askar :


--
keywords: +patch
nosy: +ammar2
nosy_count: 5.0 -> 6.0
pull_requests: +18878
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/19528

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset d01628e411752ee6849f862cae66a1c69fe512b7 by Ethan Smith in branch 
'master':
bpo-39481: PEP 585 for dataclasses, mailbox, contextvars (GH-19425)
https://github.com/python/cpython/commit/d01628e411752ee6849f862cae66a1c69fe512b7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I'm not saying that there is no need for such tool. 
> I am just asking to leave SimpleNamespace unchanged.

I really don't see the downside. It doesn't impair SimpleNamespace in any way.  
Why would we add another type with substantially the same capabilities as 
SimpleNamespace?  That would be a complete waste.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] regrtest: win_utils decodes typeperf output from the wrong encoding (test suite broken due to cpu usage feature on win 10/ german)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37531] Fix regrtest timeout for subprocesses: regrtest -jN --timeout=SECONDS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38502] regrtest: use process groups

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36842] Implement PEP 578

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40286] Add getrandbytes() method to random.Random

[Raymond Hettinger]

Raymond Hettinger  added the comment:

If we have to have this, the method name should be differentiated from 
getrandbits() because the latter returns an integer.  I suggest just 
random.bytes(n), the same as numpy.

> Python already has three functions to generate random bytes:

Now, there will be four ;-)

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[STINNER Victor]

STINNER Victor  added the comment:

This feature seems to be driven by Instagram use case. Are there other users 
who would benefit of that? Is it a common use case to load big data and then 
fork to use preloaded data?

PR 19474 has a maintenance cost:

* new configuration option
* new macro
* need a buildbot to check that the option is not broken
* document the change
* etc.

There is even now a question about using a different ABI flag.

It's not the common case to build a custom Python manually for a specific use 
case. Most users use a binary shipped by their operating system.

I'm not sure that it's worth it for Python to maintain such special build.

Maybe bpo-39511 would be a better motivation to support immortable objects.

But I'm also concerned by the huge impact on performance :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: To apply pca for a large csv

[Oscar Benjamin]

On Tue, 14 Apr 2020 at 12:42, Rahul Gupta  wrote:
>
> Hello all, i have a csv of 1 gb which consists of 25000 columns and 2 
> rows. I want to apply pca so i have seen sciki-learn had inbuilt 
> fucntionality to use that. But i have seen to do eo you have to load data in 
> data frame. But my machine is i5 with 8 gb of ram which fails to load all 
> this data in data frame and shows memory error. Is there any alternative way 
> that still i could aaply PCA on the same machine to the same rata set

Do you know how to compute a covariance matrix "manually"? If so then
it can be done while reading the data line by line without reading all
of the data into memory at once. The problem though is that your 25000
columns mean that the matrix itself will fill most of your memory
(25000**2*8 bytes == 5 GB using double precision floating point).

You can make life much easier for yourself by choosing a subset of the
columns that you are likely to be interested in and reducing the size
of your dataset before you begin.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40282] random.getrandbits(0) should succeed

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-40286: Add getrandbytes() method to random.Random.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40286] Add getrandbytes() method to random.Random

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18877
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19527

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40286] Add getrandbytes() method to random.Random

[STINNER Victor]

New submission from STINNER Victor :

The random module lacks a getrandbytes() method which leads developers to be 
creative how to generate bytes:
https://stackoverflow.com/questions/5495492/random-byte-string-in-python

It's a common use request:

* bpo-13396 in 2011
* bpo-27096 in 2016
* https://bugs.python.org/issue40282#msg366444 in 2020

Python already has three functions to generate random bytes:

* os.getrandom(): specific to Linux, not portable
* os.urandom()
* secrets.token_bytes()

These 3 functions are based on system entropy and they block on Linux until the 
kernel collected enough entropy: PEP 524.

While many users are fine with these functions, there are also use cases for 
simulation where the security doesn't matter, and it's more about being able to 
get reproducible experience from a seed. That's what random.Random is about.

The numpy module provides numpy.random.bytes(length) function for such use case:
https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.random.bytes.html

One example can be to generate UUID4 with the ability to reproduce the random 
UUID from a seed for testing purpose, or to get reproducible behavior.

Attached PR implements the getrandbytes() method.

--
components: Library (Lib)
messages: 366454
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add getrandbytes() method to random.Random
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35569] OSX: Enable IPV6_RECVPKTINFO

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
pull_requests: +18876
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19526

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[STINNER Victor]

STINNER Victor  added the comment:

> Providing this tool would instantly benefit a broad class of json users.

I'm not saying that there is no need for such tool. I am just asking to leave 
SimpleNamespace unchanged.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[STINNER Victor]

STINNER Victor  added the comment:

> That is feature creep and no user has requested it.

Python already provides such function in the secrets module, so I'm not sure if 
what you mean that "no users has requested it". secrets.token_bytes() exists 
because there is a need for such function.

secrets.token_bytes() is more designed for security, but random.Random() is 
more designed for simulations. And such users also exists, that's why numpy 
provides numpy.random.bytes(length) function:
https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.random.bytes.html

To be honest, I never understood where there is such "hole" in the random 
module API. Especially for SystemRandom since its source os.urandom() generates 
bytes.

A concrete use case is to generate manually a UUID4 from 16 random bytes. For 
testing, you may want to get "deterministic random" UUID4. Using getrandbits() 
for thta sounds unnatural to me. 

Another use case is to create a secret token: well, that's basically that 
secrets.token_bytes() does. That's used in multiprocessing but also in urllib 
(AbstractDigestAuthHandler.get_cnonce()).

So yeah, it sounds perfectly reasonable to add such simple function. I don't 
see how add such obvious function would be a "feature creep".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I would prefer that SimpleNamespace remains as simple as it is

This doesn't affect the simplicity of the current API at all.  If you don't 
need the feature, you won't even notice the extension.

> If you want to add the mapping protocol, I suggest you to create
> your own subclass and add the methods that you want

I do that for myself.  However, users would greatly benefit from having this an 
option.

We don't ask users to write their own defaultdicts from scratch even though 
that is simple dict subclass.  Providing this tool would instantly benefit a 
broad class of json users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[STINNER Victor]

STINNER Victor  added the comment:

I would prefer that SimpleNamespace remains as simple as it is:

https://docs.python.org/dev/library/types.html#types.SimpleNamespace
"A simple object subclass that provides attribute access to its namespace"

If you want to add the mapping protocol, I suggest you to create your own 
subclass and add the methods that you want.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Only the magic methods need to be added:  __getitem__, __setitem__, and 
__delitem__, __contains__, __len__, and __iter__.

The non-dunder names risk incursion into user-space names.

>>> SimpleNamespace(username1='value1', username2='value2')
namespace(username1='value1', username2='value2')
>>> dir(_)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', 
'__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', 
'__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', 
'__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', 
'__subclasshook__', 'username1', 'username2']

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40284] Add mapping methods to types.SimpleNamespace

[Raymond Hettinger]

New submission from Raymond Hettinger :

types.SimpleNamespace() would be much more usable if it were more substitutable 
for dictionaries.   In particular, it would be wonderful to be able to use 
object_hook=SimpleNamespace in json.load():

Current:

 catalog = json.load(f)
 print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity'])

Proposed:

 catalog = json.load(f, object_hook=SimpleNamespace)
 print(catalog.clothing.mens.shoes.extra_wide.quantity])

--
components: Library (Lib)
messages: 366447
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Add mapping methods to types.SimpleNamespace
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I agree I don't *need* it per se.  However, I suspect that for non-exports it 
would be easier than
`getrandbits(nbytes * 8).to_bytes(nbytes, 'endian')`.

As for `secrets.token_bytes()`, it's not really adequate for regular 
pseudo-random data generation when you want to use a fixed seed.  And I'm not 
sure what its performance characteristics are when you pass a large size.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: i am want to read data from the csv that i wrote using python csv module but apart from filed names and row count i am unable to read rest of the data

[sjeik_appie]

   On 12 Apr 2020 12:30, Peter Otten <__pete...@web.de> wrote:

 Rahul Gupta wrote:

 > for line in enumerate(csv_reader):
 > print(line[csv_reader.fieldnames[1]])

 enumerate() generates (index, line) tuples that you need to unpack:

     for index, line in enumerate(csv_reader):
     print(line[csv_reader.fieldnames[1]])

   ==》 Shouldn't that be, e.g:
   print( line[csv_reader.fieldnames[1].index()] )
   Or maybe:
   cols = csv_reader.fieldnames
   print( [[val for val, col in zip(record, cols) if col in ['somecol']] for
   record in csv_reader])
   ?
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40282] random.getrandbits(0) should succeed

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Why not use secrets.token_bytes() or randrange(2**b).to_bytes()?   Do you 
really need an API extension?

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Antoine Pitrou]

Change by Antoine Pitrou :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Antoine Pitrou]

Change by Antoine Pitrou :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Antoine Pitrou]

Antoine Pitrou  added the comment:

About a hypothetical getrandbytes(), probably 90% of my uses of getrandbits() 
have been to generate random bytestrings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I do not want to open an issue if I know that the idea will be rejected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 33986465bde2a2188537c4ef6cdb6055e348f31f by Pablo Galindo in 
branch 'master':
bpo-39522: Always initialise kind attribute in constant ast nodes (GH-19525)
https://github.com/python/cpython/commit/33986465bde2a2188537c4ef6cdb6055e348f31f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 43aeefa41915e4d3b0e68bbd4268c1c378a72dce by Batuhan Taşkaya in 
branch 'master':
bpo-39522: Use _PyUnicodeWriter_WriteStr instead of PyUnicode_AS_DATA (GH-19523)
https://github.com/python/cpython/commit/43aeefa41915e4d3b0e68bbd4268c1c378a72dce


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +pablogsal
nosy_count: 1.0 -> 2.0
pull_requests: +18875
pull_request: https://github.com/python/cpython/pull/19525

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

speech reconition

[shubham gupta]

speech recognition is working really slowly in my vs code what will i do 
can someone please help me
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: To apply pca for a large csv

[Rahul Gupta]

64 bit version
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40282] random.getrandbits(0) should succeed

[Raymond Hettinger]

Raymond Hettinger  added the comment:

+0 for having getrandbits(0) return 0.  Conceptually, it is reasonable.  
Practically, it is a bit inconvenient because the ValueError may have to be 
moved upstream to the _randbelow() methods.

-1 for getrandbytes().  That is feature creep and no user has requested it.  
Also, the name leads to a confusing API with getrandbits() returning arbitrary 
sized python ints and getrandbytes() returning bytes.  Lastly, it mostly 
duplicates functionality already supplied by secrets.token_bytes().  If you 
really want this, open another tracker issue and don't derail the issue at hand.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Hrvoje Nikšić]

Hrvoje Nikšić  added the comment:

> The only way I could see this to work as intended without making any changes 
> to threading would be to optionally use daemon threads and avoid joining the 
> threads in `executor.shutdown()` if `wait_at_exit` is set to False in the 
> constructor for the executor.

Agreed, and that is precisely what I suggested in my previous comment. The 
attached PR already deals with the executor-specific part of the wait, but it 
would be straightforward to have it also affect the executor to create daemon 
threads, and the flag moved to the constructor.

> IMO, it also would require some fairly extensive testing to make sure it 
> works as intended, which the patch currently lacks.

It is quite easy to check that a hanging thread (emulated by a simple sleep) is 
not joined by the executor with the appropriate flag set.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40249] __import__ doesn't honour globals

[Stefan Seefeld]

Stefan Seefeld  added the comment:

I'm not entirely sure, but have to admit that the sentence

"The function imports the module name, potentially using the given globals and 
locals to determine how to interpret the name in a package context."

is a bit obscure. What does "determine how to interpret the name" actually mean 
? Is the algorithm described anywhere in detail ? In that case, a simple 
reference might be enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Kyle Stanley]

Kyle Stanley  added the comment:

> ThreadPoolExecutor introduces additional waiting of its own, and it is this 
> wait the PR adds an option to disable.

[next post]

> Thanks for the clarification, I didn't know about the change to non-daemon 
> threads.

> I still think this patch is useful, and won't harm general use because it is 
> opt-in, just like daemon threads themselves. I suggest to update the PR to 
> specify non-waiting pool at the point of creation rather than in shutdown(). 
> (That will also make it more acceptable for the same option not being 
> supported for process pools - it is ok for constructor signatures to differ.)

Yes, but if you simply make the ThreadPoolExecutor forget about the thread, 
then all that it does is divert the wait to occur at a later time. In this 
case, it would get stuck at `threading._shutdown()` (where all non-daemon 
threads are joined) instead of `executor.shutdown()` since they no longer use 
daemon threads.

The only way I could see this to work as intended without making any changes to 
threading would be to optionally use daemon threads and avoid joining the 
threads in `executor.shutdown()` if `wait_at_exit` is set to False in the 
constructor for the executor. But at the least, users would have to be made 
aware in the documentation that this could lead to significant resource 
finalization issues, and potential incompatibility with subinterpreters. 

Otherwise, they may end up accidentally shooting themselves in the foot (which 
we certainly want to avoid). I'm also somewhat concerned that it may end up 
getting used to cover up actual bugs. 

IMO, it also would require some fairly extensive testing to make sure it works 
as intended, which the patch currently lacks. So in order to implement this 
properly, it would require a significant amount of additional time investment 
beyond what has been put into the existing patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[miss-islington]

Change by miss-islington :


--
pull_requests: +18874
pull_request: https://github.com/python/cpython/pull/19524

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[miss-islington]

miss-islington  added the comment:


New changeset 4c0a31fb08407ba043688ad1c21102dd4cb99146 by Miro Hrončok in 
branch 'master':
bpo-9216: Nobody expects the geohashing FIPS inquisition (GH-19520)
https://github.com/python/cpython/commit/4c0a31fb08407ba043688ad1c21102dd4cb99146


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40249] __import__ doesn't honour globals

[Brett Cannon]

Brett Cannon  added the comment:

How would you propose changing the wording found at 
https://docs.python.org/3/library/functions.html?highlight=__import__#__import__?

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40260] modulefinder traceback regression starting on Windows

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40260] modulefinder traceback regression starting on Windows

[miss-islington]

miss-islington  added the comment:


New changeset 59047fab0ef37f583c9e7c3a48d67792fd10ff91 by Miss Islington (bot) 
in branch '3.8':
bpo-40260: Update modulefinder to use io.open_code() and respect coding 
comments (GH-19488)
https://github.com/python/cpython/commit/59047fab0ef37f583c9e7c3a48d67792fd10ff91


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39522] AST Unparser with unicode kinded constants

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +18873
pull_request: https://github.com/python/cpython/pull/19523

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23082] pathlib relative_to() can give confusing error message

[Steve Dower]

Steve Dower  added the comment:

I agree it's worth improving the error message (and probably the docs too).

It's never made clear that relative_to only looks deeper (won't ever generate 
leading ".." parts) - the closest hint in the docs is that os.path.relpath is 
different (and that isn't even in the relative_to() section).


Tagging this easy/newcomer friendly. If you'd like to work on it, just post 
here - I'm happy to help get it merged.

--
keywords: +easy, newcomer friendly
nosy: +steve.dower
versions: +Python 3.8, Python 3.9 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40260] modulefinder traceback regression starting on Windows

[Steve Dower]

Steve Dower  added the comment:


New changeset d42e5820631cd66ee1eab8f610d4b58f3dfdd81c by Barry in branch 
'master':
bpo-40260: Update modulefinder to use io.open_code() and respect coding 
comments (GH-19488)
https://github.com/python/cpython/commit/d42e5820631cd66ee1eab8f610d4b58f3dfdd81c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40260] modulefinder traceback regression starting on Windows

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 7.0 -> 8.0
pull_requests: +18872
pull_request: https://github.com/python/cpython/pull/19522

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Kyle Stanley]

Kyle Stanley  added the comment:

> I don't think there's much ThreadPoolExecutor can do.  If you drop the 
> references to the threads, they still exist and they still be waited upon at 
> interpreter exit.
> 
> The solution is for you to avoid having hanging threads.  In the particular 
> case of TCP connections, I'd recommend using a dedicated framework such as 
> asyncio (or Twisted, Tornado, etc.) instead of home-baked networking code.

As far as I'm aware, I don't think there's a _safe_ way to force a hanging 
thread to immediately exit without causing resource-related problems (an unsafe 
way would be something like releasing the internal thread state lock). But in 
general, I think that hanging threads indicate a fundamental issue in the 
implementation of the function the thread is targeting. For any operation that 
can potentially stall or run indefinitely, there should be some form of fail 
safe or timeout in place to break out of it. So I agree with Antoine that it's 
ultimately the responsibility of the thread itself to not hang. There's not 
much of anything that ThreadPoolExecutor can safely do to resolve a hanging 
thread.

> Also, note that Python sockets have a feature called *timeouts*.

There's also of course timeouts implemented at a higher level in many 
networking frameworks, if the developer doesn't want to do so at the socket 
level. For example, see asyncio.wait_for(): 
https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for.

> These threads were daemon threads in 3.8, so your patch indeed works there, 
> but we've made them non-daemon in 3.9, for two reasons:
1) daemon threads are fragile and can crash the interpreter at shutdown
2) they are not supported on subinterpreters

Just to briefly clarify on (2), Victor recently opened a PR to revert daemon 
threads being entirely unsupported in subinterpreters: PR-19456. From reading 
over the attached bpo issue, it looks like the plan is to deprecate daemon 
threads in subinterpreters, but to not immediately drop support because users 
of mod_wsgi and various monitoring tools were reliant upon it (through the 
C-API).

It seems like the current plan is for it to undergo a deprecation process. 
Either way though, I still think the change to make executors no longer reliant 
upon daemon threads was a significant stability improvement and will mean we 
don't have to make the change later when they are fully unsupported in 
subinterpreters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40267] Error message differs when an expression is in an fstring

[Lysandros Nikolaou]

Change by Lysandros Nikolaou :


--
keywords: +patch
pull_requests: +18871
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19521

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Mark Dickinson]

Mark Dickinson  added the comment:

This was discussed previously in #37000.

I agree that `getrandbits(0)` should succeed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40282] random.getrandbits(0) should succeed

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Seconded.

And I wish to add the getrandbytes() method.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I'm not sure what you mean here by "balanced ref count" or by "work" :) What 
> will happen anytime an immortal object gets into the GC, for any reason, is 
> that the GC will "subtract" cyclic references and see that the immortal 
> object still has a large refcount even after that adjustment, and so it will 
> keep the immortal object and any cycle it is part of alive. This behavior is 
> correct and should be fully expected; nothing breaks. It doesn't matter at 
> all to the GC that this large refcount is "fictional," and it doesn't break 
> the GC algorithm, it results only in the desired behavior of maintaining 
> immortality of immortal objects.


Yep, that is right. I think there was a race condition between my previous 
message and yours :)

I think what was confusing me in this line of reasoning is that I was not 
taking into account that the immortal bit is a very high one, making the 
refcount gigantic. I was treating it mentally like a flag without factoring the 
implications of such big reference count.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Miro Hrončok]

Change by Miro Hrončok :


--
nosy: +hroncok
nosy_count: 17.0 -> 18.0
pull_requests: +18870
pull_request: https://github.com/python/cpython/pull/19520

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40260] modulefinder traceback regression starting on Windows

[Steve Dower]

Steve Dower  added the comment:

Thanks, Barry!

I tweaked your NEWS entry a little, so once CI completes I'll merge and 
backport.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[miss-islington]

miss-islington  added the comment:


New changeset 8821200d85657ef3bbec78dcb43694449c05e896 by Miss Islington (bot) 
in branch '3.7':
bpo-32033: Fix test_pwd failures on Android (GH-19502)
https://github.com/python/cpython/commit/8821200d85657ef3bbec78dcb43694449c05e896


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Hrvoje Nikšić]

Hrvoje Nikšić  added the comment:

Thanks for the clarification, I didn't know about the change to non-daemon 
threads.

I still think this patch is useful, and won't harm general use because it is 
opt-in, just like daemon threads themselves. I suggest to update the PR to 
specify non-waiting pool at the point of creation rather than in shutdown(). 
(That will also make it more acceptable for the same option not being supported 
for process pools - it is ok for constructor signatures to differ.)

If there is interest, someone should take over the PR, as I have changed jobs 
and no longer have time to actively pursue this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[miss-islington]

miss-islington  added the comment:


New changeset 1e1dbdf23f7a18f53a3257badc3541973831f2c4 by Miss Islington (bot) 
in branch '3.8':
bpo-32033: Fix test_pwd failures on Android (GH-19502)
https://github.com/python/cpython/commit/1e1dbdf23f7a18f53a3257badc3541973831f2c4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Carl Meyer]

Carl Meyer  added the comment:

I think the concerns about "perfect" behavior in corner cases are in general 
irrelevant here.

In the scenarios where this optimization matters, there is no quantitative 
change that occurs at 100% coverage. Preventing 99% of CoW is 99% as good as 
preventing 100% :) So the fact that a few objects here and there in special 
cases could still trigger CoW just doesn't matter; it's still a massive 
improvement over the status quo. (That said, I wouldn't _mind_ improving the 
coverage, e.g. if you can suggest a better way to find all heap objects instead 
of using the GC.)

And similarly, gps is right that the concern that immortal objects can keep 
other objects alive (even via references added after immortalization) is a 
non-issue in practice. There really is no other behavior one could prefer or 
expect instead.

> if said objects (isolated and untracked before and now tracked) acquire 
> strong references to immortal objects, those objects will be visited when the 
> gc starts calculating the isolated cycles and that requires a balanced 
> reference count to work.

I'm not sure what you mean here by "balanced ref count" or by "work" :) What 
will happen anytime an immortal object gets into the GC, for any reason, is 
that the GC will "subtract" cyclic references and see that the immortal object 
still has a large refcount even after that adjustment, and so it will keep the 
immortal object and any cycle it is part of alive. This behavior is correct and 
should be fully expected; nothing breaks. It doesn't matter at all to the GC 
that this large refcount is "fictional," and it doesn't break the GC algorithm, 
it results only in the desired behavior of maintaining immortality of immortal 
objects.

It is perhaps slightly weird that this behavior falls out of the immortal bit 
being a high bit rather than being more explicit. I did do some experimentation 
with trying to explicitly prevent immortal instances from ever entering GC, but 
it turned out to be hard to do that in an efficient way. And motivation to do 
it is low, because there's nothing wrong with the behavior in the existing PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> But if said objects (isolated and untracked before and now tracked) acquire 
> strong references to immortal objects, those objects will be visited when the 
> gc starts calculating the isolated cycles and that requires a balanced 
> reference count to work.

I was thinking about this a bit more and because the refcount bit is too high, 
the GC algorithm will be "fine" in the sense that won't break. It will just 
treat immortal objects as referenced from outside any cycle isolate (because 
the refcount hopefully will never reach 0), which is consistent with the fact 
that they are in the permanent generation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think there's a misunderstanding: "wait_at_exit" will make the *executor* 
forget about the threads, but Python itself still knows about them, and it 
waits for them to end at interpreter shutdown.

These threads were daemon threads in 3.8, so your patch indeed works there, but 
we've made them non-daemon in 3.9, for two reasons:
1) daemon threads are fragile and can crash the interpreter at shutdown
2) they are not supported on subinterpreters

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[STINNER Victor]

STINNER Victor  added the comment:

It's now fixed in master and backports to 3.7 and 3.8 will be merged as soon as 
the CI pass.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[miss-islington]

Change by miss-islington :


--
pull_requests: +18869
pull_request: https://github.com/python/cpython/pull/19519

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

[Hrvoje Nikšić]

Hrvoje Nikšić  added the comment:

> I don't think there's much ThreadPoolExecutor can do.  If you drop the 
> references to the threads, they still exist and they still be waited upon at 
> interpreter exit.

ThreadPoolExecutor introduces additional waiting of its own, and it is this 
wait the PR adds an option to disable.

> The solution is for you to avoid having hanging threads.

In some cases that is not possible. For example, the file IO on network file 
systems can take an arbitrary amount of time, and there is no way to implement 
a timeout. DNS lookups have been notorious for not supporting timeouts. Many 
third-party libraries, such as database drivers, still don't support timeouts.

This is a real issue that bit many users in production systems, it's not about 
a toy program using "home-baked networking code".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +18868
pull_request: https://github.com/python/cpython/pull/19518

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32033] The pwd module implementation incorrectly sets some attributes to None

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 96515e9f6785328c52ebc5d4ce60e0087a9adc2d by Zackery Spytz in 
branch 'master':
bpo-32033: Fix test_pwd failures on Android (GH-19502)
https://github.com/python/cpython/commit/96515e9f6785328c52ebc5d4ce60e0087a9adc2d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 675d9a3d7afc767a2818c84da7ba4bf4181dcf26 by Hai Shi in branch 
'master':
bpo-40170: Convert PyObject_IS_GC() macro to a function (GH-19464)
https://github.com/python/cpython/commit/675d9a3d7afc767a2818c84da7ba4bf4181dcf26


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Marking everything as immortal/eternal after you've loaded your common code & 
data but before you do your forking is thenorm for this kind of serving system.

You already presumably have a defined lifetime for the processes so they'll 
likely be set to die within N hours or days.  Memory leaks of too many things 
persisting is a non-issue in this kind of system.

The alternative of trying to pick and choose exactly what (and anything they 
reference) sticks around is a maintenance nightmare exercise in futility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com