[issue33026] Fix jumping out of "with" block

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +5789
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33026] Fix jumping out of "with" block

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

The proposed PR fixes jumping from "with" block. Currently the exit function is 
left on the stack.

This fix is for 3.8 only. 3.7 and older versions are affected by this bug, but 
since the code was significantly changed in 3.8, I'm not sure it will be so 
easy to fix it in older versions.

--
components: Interpreter Core
messages: 313426
nosy: pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Fix jumping out of "with" block
type: crash
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Benjamin Peterson]

Benjamin Peterson  added the comment:

It's useful if you want to hide the fact that a command is implemented in 
Python and don't want it to malfunction if the user has PYTHONPATH set for some 
reason.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33025] urlencode produces bad output from ssl.CERT_NONE and friends that chokes decoders

[Vitaly Kruglikov]

New submission from Vitaly Kruglikov :

```
In [9]: from urllib.parse import urlencode, parse_qs

In [10]: import ast, ssl

In [11]: d = dict(cert_reqs=ssl.CERT_NONE)

In [12]: urlencode(d)
Out[12]: 'cert_reqs=VerifyMode.CERT_NONE'

In [25]: parse_qs('cert_reqs=VerifyMode.CERT_NONE')
Out[25]: {'cert_reqs': ['VerifyMode.CERT_NONE']}
In [29]: ast.literal_eval('VerifyMode.CERT_NONE')
Traceback (most recent call last)
...
ValueError: malformed node or string: <_ast.Attribute object at 0x105c22358>
```

This used to work fine and produce `'cert_reqs=0'` on Python 2.7, allowing it 
to be decoded properly downstream. However, `'cert_reqs=VerifyMode.CERT_NONE'` 
can't be decoded generically. So, something it's that used to work in prior 
python versions that is breaking now.

Additional information. json.dumps() actually dumps that value as a number 
instead of 'VerifyMode.CERT_NONE'. 

It appears that urlencode doesn't work properly with enums, where I would 
expect it to emit the numeric value of the enum.

--
assignee: christian.heimes
components: Library (Lib), SSL
messages: 313424
nosy: christian.heimes, vitaly.krug
priority: normal
severity: normal
status: open
title: urlencode produces bad output from ssl.CERT_NONE and friends that chokes 
decoders
type: crash
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33024] asyncio.WriteTransport.set_write_buffer_limits orders its args unintuitively and inconsistently with its companion function's return value

[Vitaly Kruglikov]

New submission from Vitaly Kruglikov :

`asyncio.WriteTransport.set_write_buffer_limits()` uses an unintuitive order of 
the args (high, low). I would expect `low` to be the first arg, especially 
since `asyncio.WriteTransport.get_write_buffer_limits()` returns them in the 
opposite order. This ordering and inconsistency with the companion function's 
return value is error-prone. See 
https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.set_write_buffer_limits

--
components: asyncio
messages: 313423
nosy: asvetlov, vitaly.krug, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.WriteTransport.set_write_buffer_limits orders its args 
unintuitively and inconsistently with its companion function's return value
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33023] Unable to copy ssl.SSLContext

[Vitaly Kruglikov]

Change by Vitaly Kruglikov :


--
assignee:  -> christian.heimes
components: +SSL
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33023] Unable to copy ssl.SSLContext

[Vitaly Kruglikov]

New submission from Vitaly Kruglikov :

```
import copy
import ssl

copy.copy(ssl.create_default_context())
```
results in 

`TypeError: can't pickle SSLContext objects`

This prevents me from being able to `copy.deepcopy()` an object that references 
`ssl.SSLContext`.

The apparent root cause is apparently that `ssl.SSLContext` passes an extra arg 
to its `__new__` method, but doesn't implement the method `__getnewargs__` that 
would let `copy` extract the extra arg.

--
messages: 313422
nosy: vitaly.krug
priority: normal
severity: normal
status: open
title: Unable to copy ssl.SSLContext
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33023] Unable to copy ssl.SSLContext

[Vitaly Kruglikov]

Change by Vitaly Kruglikov :


--
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33022] Floating Point Arithmetic Inconsistency (internal off-by-one)

[Tim Peters]

Tim Peters  added the comment:

What did you expect?  The precision of Python ints is limited only by the 
amount of memory you have, but Python floats are IEEE-754 double precision 
numbers, and have only 53 bits of precision.

2**53 + 1 simply can't be represented exactly as a float:  it requires 54 
significant bits.  It must be rounded back to 53 significant bits.  Because the 
exact value of 2**53 + 1 is exactly half-way between the adjacent representable 
floats 2**53 and 2**53+2, the IEEE "round to nearest/even" rule requires 
rounding to the closest representable value whose 53rd (the least) significant 
bit is 0, which is 2**53.

Note that the 53rd bit of 9007199254740994.0 (2**53 + 2) is odd (1):

>>> (9007199254740994.0).hex()
'0x1.1p+53'
 ^

That's why the nearest/even rule _must_ pick 2**53 instead.

You'll see the same behavior in every other language (C, C++, Java, ...) 
supporting IEEE-754 double precision too.

Since this really has nothing to do with Python, and is working as intended, 
I'll close this.

--
nosy: +tim.peters
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33022] Floating Point Arithmetic Inconsistency (internal off-by-one)

[Dylan Dmitri Gray]

New submission from Dylan Dmitri Gray :

```
>>> for i in (1,2,3,1.0,2.0,3.0): print(i, i+9007199254740991)
...
1 9007199254740992
2 9007199254740993
3 9007199254740994
1.0 9007199254740992.0
2.0 9007199254740992.0   # <-- !!!
3.0 9007199254740994.0

```

Notably 9007199254740991 = 2**53 -1

Probably an internal off by one?

--
messages: 313420
nosy: ddg
priority: normal
severity: normal
status: open
title: Floating Point Arithmetic Inconsistency (internal off-by-one)
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26701] Documentation for int constructor mentions __int__ but not __trunc__

[Eric Appelt]

Change by Eric Appelt :


--
keywords: +patch
pull_requests: +5788
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[pmpp]

pmpp  added the comment:

josh.r. i think you are right, i was worried if a nfs sillyrename is in 
progress, for eg a lock file ,then server hangs but thread lift the GIL and 
allow another thread to try to start to fstat the same path.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18802] ipaddress documentation errors

[Xiang Zhang]

Xiang Zhang  added the comment:

Ahh, I also find some errors here in ipaddress doc and opened 
https://github.com/python/cpython/pull/6021 to fix them.

--
assignee:  -> docs@python
components: +Documentation -Library (Lib)
nosy: +docs@python, xiang.zhang
versions: +Python 3.7, Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22822] IPv6Network constructor docs incorrect about valid input

[Xiang Zhang]

Change by Xiang Zhang :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> ipaddress documentation errors

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[Josh Rosenberg]

Josh Rosenberg  added the comment:

fstat is async signal safe, and I suspect it's thread safe in general, though 
usage does rely on the file descriptor remaining valid. If the fd in question 
is closed in another thread after the GIL is released, fstat would fail; if a 
new file is opened and assigned the same fd, it would give erroneous results. 
But I don't think any of that would lead to fundamentally incorrect behavior 
(after all, you could do the same thing manually by caching an fd then closing 
the file).

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32972] unittest.TestCase coroutine support

[Zachary Ware]

Zachary Ware  added the comment:

Why is a metaclass required?  Playing with a modified version of John's PR5938, 
it doesn't seem necessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33001] Buffer overflow vulnerability in os.symlink on Windows (CVE-2018-1000117)

[Steve Dower]

Steve Dower  added the comment:

Thanks, Victor!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32972] unittest.TestCase coroutine support

[Yury Selivanov]

Yury Selivanov  added the comment:

> I'm really not seeing what a separate class buys you.

I already mentioned in my previous comments that adding async support to 
unittest.TestCase would require us to add a metaclass to it, which is 
potentially a backwards incompatible change.

Moreover, I'm not even sure that `AsyncioTestCase` should be in the unittest 
module and not in asyncio.  Any non-trivial async package I've worked with 
usually requires quite a complicated setup to prepare an async test 
environment.  Usually a test case needs to test against different event loop 
policies or different event loops.  Which means that we need to support 
asynchronous versions of setUp, setUpClass, etc.  And I'm not sure we should 
just detect when they are asynchronous, perhaps we should make a set of new 
methods: asyncSetUp, asyncSetUpClass, etc.  This is something that needs more 
discussion.

So far I see that two core asyncio devs are against the proposed idea of 
changing unittest.TestCase to support async transparently.  IMO it gives almost 
no benefits but will complicate the implementation of TestCase and TestRunner 
(which are already quite hairy).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32972] unittest.TestCase coroutine support

[R. David Murray]

R. David Murray  added the comment:

I agree with what Zach said in msg313352.  I don't see how having a separate 
class affects the problem of what to use to run an async def test case.  If "an 
AsyncTestCase should be enough" but "other frameworks will require a different 
test tool", why wouldn't support in TestCase for running the test under asyncio 
also be enough?  I'm really not seeing what a separate class buys you.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33018] Improve issubclass() error checking and message

[INADA Naoki]

INADA Naoki  added the comment:

Hmm, normal class doesn't support issubclass(class-like. class).

```
Python 3.8.0a0 (heads/master:fc7df0e664, Mar  8 2018, 09:00:43)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing
>>> issubclass(typing.MutableMapping, object)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: issubclass() arg 1 must be a class
>>> issubclass(typing.MutableMapping, type)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: issubclass() arg 1 must be a class
>>> issubclass(typing.MutableMapping, typing.Mapping)
True
```

OK, problem is ABC should support it or not.

--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33018] Improve issubclass() error checking and message

[INADA Naoki]

INADA Naoki  added the comment:

issubclass(class-like, class-like) is allowed.
I don't think raising type error for issubclass(class-like, ABC) is good idea.  
It should return False.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33018] Improve issubclass() error checking and message

[INADA Naoki]

INADA Naoki  added the comment:

https://bugs.python.org/msg313396

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[pmpp]

pmpp  added the comment:

is fstat thread safe ?

--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[Nir Soffer]

Change by Nir Soffer :


--
pull_requests: +5787

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Matt Harvey]

Matt Harvey  added the comment:

@njs your sketch in msg313406 looks good.  Probably better to go with 
OMP_NUM_THREADS than NCPUS. 
M

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[Nir Soffer]

Change by Nir Soffer :


--
keywords: +patch
pull_requests: +5786
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33021] Some fstat() calls do not release the GIL, possibly hanging all threads

[Nir Soffer]

New submission from Nir Soffer :

If the file descriptor is on a non-responsive NFS server, calling
fstat() can block for long time, hanging all threads. Most of the fstat()
calls release the GIL around the call, but some calls seems to be
forgotten.

In python 3, the calls are handled now by _py_fstat(), releasing the GIL
internally, but some calls use _py_fstat_noraise() which does not release
the GIL. Most of the calls to _py_fstat_noraise() release the GIL around
the call, except these 2 calls, affecting users of:

- mmap.mmap()
- os.urandom()
- random.seed()

In python there are more fstat() calls to fix, affecting users of:

- imp.load_dynamic()
- imp.load_source()
- mmap.mmap()
- mmapobject.size()
- os.fdopen()
- os.urandom()
- random.seed()

--
components: Library (Lib)
messages: 313407
nosy: brett.cannon, eric.snow, ncoghlan, nirs, serhiy.storchaka, twouters, 
vstinner, yselivanov
priority: normal
severity: normal
status: open
title: Some fstat() calls do not release the GIL, possibly hanging all threads
type: performance
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Nathaniel Smith]

Nathaniel Smith  added the comment:

> You mean duplicating "nproc"'s logic in Python?

Yeah.

> If someone wants to do the grunt work of implementing/testing it...

Well, that's true of any bug fix / improvement :-). The logic isn't terribly 
complicated though, something roughly like:

def parse_omp_envvar(env_value):
return int(env_value.strip().split(",")[0])

def estimate_cpus():
limit = float("inf")
if "OMP_THREAD_LIMIT" in os.environ:
limit = parse_omp_envvar(os.environ["OMP_THREAD_LIMIT"])

if "OMP_NUM_THREADS" in os.environ:
cpus = parse_omp_envvar(os.environ["OMP_NUM_THREADS"])
else:
try:
cpus = len(os.sched_getaffinity(os.getpid()))
except AttributeError, OSError:
cpus = os.cpu_count()

return min(cpus, limit)

> There's also the question of how that affects non-scientific workloads. 
> People can use thread pools or process pools for other purposes, such as 
> distributing (blocking) I/O.

We already have some heuristics for this: IIRC the thread pool executor 
defaults to cpu_count() * 5 threads (b/c Python threads are really intended for 
I/O-bound workloads), and the process pool executor and multiprocessing.Pool 
defaults to cpu_count() processes (b/c processes are better suited to CPU-bound 
workloads). Neither of these heuristics is perfect. But inasmuch as it makes 
sense at all to use the cpu count as part of the heuristic, it surely will work 
better to use a more accurate estimate of the available cpus.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I may be mistaken, what's the use of -E if not for security?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I don't think -E is a security feature. Even if the stdlib was fixed, there's 
tons of 3rdparty Python code that consumes os.environ.

It seems like if you really cared about not letting the environment influence a 
Python application, you'd just wrap python in a script that cleans out the 
environment before execing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Nathaniel Smith]

Nathaniel Smith  added the comment:

I can't find any evidence that NPROCS is used by other batch schedulers (I 
looked at SLURM, Torque, and SGE). @M J Harvey, do you have any other examples 
of systems that use it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> So this seems like a reasonable heuristic approach to me.

You mean duplicating "nproc"'s logic in Python?  If someone wants to do the 
grunt work of implementing/testing it...

There's also the question of how that affects non-scientific workloads. People 
can use thread pools or process pools for other purposes, such as distributing 
(blocking) I/O.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Nathaniel Smith]

Nathaniel Smith  added the comment:

That stackoverflow thread points to the GNU coreutils 'nproc', which is an 
interesting compendium of knowledge about this problem.

It looks like their algorithm is roughly:

1. Determine how many CPUs *could* this program access, by going down a list of 
possible options and using the first that works: pthread_getaffinity_np -> 
sched_getaffinity -> GetProcessAffinityMask -> sysconf(_SC_NUMPROCESSORS_ONLN) 
-> some arcane stuff specific to HP-UX, IRIX, etc.

2. Parse the OMP_NUM_THREADS and OMP_THREAD_LIMIT envvars (this is not quite 
trivial, there's some handling of whitespace and commas and references to the 
OMP spec)

3. If OMP_NUM_THREADS is set, return min(OMP_NUM_THREADS, OMP_THREAD_LIMIT). 
Otherwise, return min(available_processors, OMP_THREAD_LIMIT).

Step (1) handles both the old affinity APIs, and also the cpuset system that 
docker uses. (From cpuset(7): "Cpusets are integrated with the 
sched_setaffinity(2) scheduling  affinity  mechanism".) Step (2) relies on the 
quasi-standard OMP_* envvars to let you choose something explicitly.

The PBS Pro docs say that they set both NPROCS and OMP_NUM_THREADS. See section 
6.1.7 of https://pbsworks.com/pdfs/PBSUserGuide14.2.pdf

So this seems like a reasonable heuristic approach to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32972] unittest.TestCase coroutine support

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I'm with Yuri: subclassing is under better control.
The need for a new subclass is relative rare, for the whole asyncio world 
AsyncioTestCase should be enough.
Another async framework most likely require a different test tool with another 
game rules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33020] Tkinter old style classes

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It is not surprising that some classes in Python 2 are old style classes. This 
is not a bug. If you need new style classes, the solution is upgrading to 
Python 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33020] Tkinter old style classes

[Ned Deily]

Change by Ned Deily :


--
nosy: +gpolo, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33001] Buffer overflow vulnerability in os.symlink on Windows (CVE-2018-1000117)

[STINNER Victor]

STINNER Victor  added the comment:

FYI I added this vulnerability to:
http://python-security.readthedocs.io/vuln/cve-2018-1000117_buffer_overflow_vulnerability_in_os.symlink_on_windows.html
https://github.com/vstinner/python-security/commit/349588e8265099341801b20aa18f87a42176f7df

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33020] Tkinter old style classes

[Ben Kirshenbaum]

New submission from Ben Kirshenbaum :

Tkinter objects cannot handle the super() function, and probably other 
functions (I only found a problem with super())

--
components: Tkinter
messages: 313397
nosy: benkir07
priority: normal
severity: normal
status: open
title: Tkinter old style classes
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32999] issubclass(obj, abc.ABC) causes a segfault

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

Serhiy, for example `issubclass(typing.MutableMapping, typing.Mapping)` returns 
`True` while neither of those two are actual class objects. These relationships 
are kept mostly so that `typing.*` can be used as a drop-in replacement for 
`collections.abc.*` (plus backwards compatibility).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Christian Heimes]

Christian Heimes  added the comment:

External libraries like sqlite may also use env vars. I know for sure OpenSSL 
uses SSL_CERT_FILE and SSL_CERT_DIR to override default verify locations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33000] IDLE Doc: Text consumes unlimited RAM, consoles likely not

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I don't know whether memory difference between us is due to IDLEX or Windows 
version or something else, but I don't really care.  The IDLE doc has a section 
"IDLE-console differences".  It should gain a paragraph explain the difference 
between memory-limited tk.Text-based Shell and limited-lines consoles.  For 
Command Prompt, the buffer defaults to a pathetic 300 lines, which is not 
enough for the CPython test suite output.  The max is  for those who 
discover how to enlarge it.  No limit is an advantage for most uses, but not 
for one who fills up memory.

I will also think about a sentence to add to the introductory section about 
development versus production runs.

--
components: +Documentation
stage:  -> needs patch
title: IDLEX GUI consumes all RAM for scrollback buffer, uses 161Bytes / 
character stored -> IDLE Doc: Text consumes unlimited RAM, consoles likely not
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33019] Review usage of environment variables in the stdlib

[Antoine Pitrou]

New submission from Antoine Pitrou :

Python supports a mode where the interpreter ignores environment variables such 
as PYTHONPATH, etc.

However, there are places in the stdlib where environment-sensitive decisions 
are made, without regard for the ignore-environment flag.

Examples include:
- ssl.get_default_verify_paths() queries SSL_CERT_FILE and SSL_CERT_DIR
- shutil.which() queries PATH
- the tempfile module queries TMPDIR, TEMP, TMP to select the defaut directory 
for temporary files

Do you think those need to be sanitized?

--
components: Library (Lib)
messages: 313393
nosy: alex, christian.heimes, pitrou
priority: normal
severity: normal
status: open
title: Review usage of environment variables in the stdlib
type: security
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Note that to avoid any kind of environment variable-driven Denial of Service, 
we should probably ignore NCPUS when sys.flags.ignore_environment is set.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29708] support reproducible Python builds

[Alexandru Ardelean]

Alexandru Ardelean  added the comment:

PYTHONHASHSEED does help on 3.6.4
I'll use it during build.

Thanks for help

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22674] RFE: Add signal.strsignal(): string describing a signal

[Antoine Pietri]

Antoine Pietri  added the comment:

I updated Vajrasky's patch to rebase it onto master, use the clinic argument 
parser and improve the docs. I made a PR on GitHub so the review can be easier 
than a patch.

I left a Co-Authored-By field so I'm not stealing the ownership of this patch 
:-P

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22674] RFE: Add signal.strsignal(): string describing a signal

[Antoine Pietri]

Change by Antoine Pietri :


--
pull_requests: +5784
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32642] add support for path-like objects in sys.path

[Jay Yin]

Jay Yin  added the comment:

I've been stuck on "test_poplib" for over 149661 sec now, that's about 41 
hours... I don't think this is working correctly, although I'm unsure  what 
test_poplib is doing that has been affected by what I've changed here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27683] ipaddress subnet slicing iterator malfunction

[Xiang Zhang]

Change by Xiang Zhang :


--
pull_requests: +5783

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Matt Harvey]

Matt Harvey  added the comment:

Hi,

No, using the affinity's not useful to us as, in the general case, the batch 
system (PBS Pro in our case) isn't using cgroups or cpusets (it does control 
ave cpu use by monitoring rusage of the process group). 

Several other batch system I've worked with either set NCPUS directly or have a 
method for site-specific customisation of the job's environment.

That doesn't preclude using the affinity as an alternative to os.cpu_count()

As @pitrou correctly observes, probably better to have a simple, 
well-sign-posted way for the sysadmins to influence the pool default than try 
to overload multiprocessing with complex heuristics.

--
nosy: +Matt Harvey

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't think we want to hardcode special cases for each resource-limiting 
framework out there: some people care about Docker, others about cgroups, CPU 
affinity settings, etc.  NCPUS has the nice property that each of those 
frameworks can set it, and users or sysadmins can also override it easily.  
It's also trivially queried from Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Matthew Rocklin]

Matthew Rocklin  added the comment:

I agree that this is a common issue.  We see it both when people use batch 
schedulers as well as when people use Docker containers.  I don't have enough 
experience with batch schedulers to verify that NCPUS is commonly set.  I would 
encourage people to also look at what Docker uses.  

After a quick (two minute) web search I couldn't find the answer, but I suspect 
that one exists.  I've raised a question on Stack Overflow here: 
https://stackoverflow.com/questions/49151296/how-many-cpus-does-my-docker-container-have

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29708] support reproducible Python builds

[INADA Naoki]

INADA Naoki  added the comment:

3e 02 00 00 00 is frozenset(size=2)
72 b6/b5 00 00 00 is reference to b5 or b6

So it seems set order changed. (or items in the set is appearance order is 
changed.)
Did you set PYTHONHASHSEED?

Anyway, I think Python 3.7 can't guarantee "reproducible" compile because 
marshal uses reference count.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Antoine Pitrou]

Change by Antoine Pitrou :


--
type: behavior -> enhancement
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Nathaniel Smith]

Nathaniel Smith  added the comment:

This is a duplicate of bpo-26692 and bpo-23530, and possibly others.

My impression is that len(os.sched_getaffinity(os.getpid())) is the Right 
Guess. Currently sched_getaffinity isn't implemented on Windows, but bpo-23530 
has some example code for how it could/should be implemented.

@M J Harvey: does this return the right thing for your batch jobs?

--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32986] multiprocessing, default assumption of Pool size unhelpful

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Matt, what do you think about this proposal? Is NCPUS the right thing to look 
at?

--
nosy: +Matthew Rocklin
type:  -> behavior
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33006] docstring of filter function is incorrect

[Anthony Flury]

Change by Anthony Flury :


--
keywords: +patch
pull_requests: +5782
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33018] Improve issubclass() error checking and message

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

ABC.register() has an explicit check, and it is mentioned in PEP 3119. The 
point here is not to change issubclass(), but to change 
ABC.__subclasscheck__(). It may conceivably have stricter requirements than 
issubclass() has. But certainly an advice from actual ABC users would be nice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33018] Improve issubclass() error checking and message

[INADA Naoki]

INADA Naoki  added the comment:

Why `issubclass()` doesn't check it?  Maybe, non-type class is supported by 
Python.  But I'm not sure.  I'm not meta programming expert.

But we use "duck typing".  In this case, if the object (a) supports weakref and 
(2) has __mro__ and it is tuple, we treat the object as a class.

And when raising TypeError, information about which assumption was broken may 
be useful.

So I'm -1 on adding such strong check or removing internal information without 
meta programming expert advice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32999] issubclass(obj, abc.ABC) causes a segfault

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

Sorry for status update, this was due to a concurrent modification.

--
nosy: +jab
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32999] issubclass(obj, abc.ABC) causes a segfault

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

@inada.naoki: I don't question your change. My point is the same as in #33018 
(I've discovered that PR only after I commented). The error message is 
misleading, and it's just a coincidence that a TypeError and not some other 
error is raised when abc attempts to add a non-type object to a WeakSet.

@serhiy.storchaka: Note that an example that you requested is unlikely to be 
related to ABC and probably is more like my artificial __subclasscheck__ 
example. So, making changes just for ABC as suggested in #33018 might make 
sense.

--
nosy:  -jab
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33014] Clarify doc string for str.isidentifier()

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com