[issue33089] Add multi-dimensional Euclidean distance function to the math module

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

With this implementation

>>> hypot(*range(15), 3)
32.01

The exact result is 32.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33100] dataclasses and __slots__ - non-default argument (member_descriptor)

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> eric.smith
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33100] dataclasses and __slots__ - non-default argument (member_descriptor)

[Adrian Stachlewski]

New submission from Adrian Stachlewski :

I've tried to declare two classes

@dataclass
class Base:
__slots__ = ('x',)
x: Any


@dataclass
class Derived(Base):
x: int
y: int

As long as I correctly understood PEP 557 (inheritance part), changing type of 
variable is possible. This code produce error:

TypeError: non-default argument 'y' follows default argument

'x' variable in Derived class has changed default from MISSING to 
member_descriptor and that's the reason of the exception.

--
components: Library (Lib)
messages: 314077
nosy: stachel
priority: normal
severity: normal
status: open
title: dataclasses and __slots__ - non-default argument (member_descriptor)
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26917] unicodedata.normalize(): bug in Hangul Composition

[Ma Lin]

Ma Lin  added the comment:

> Victor's patch is correct.

I'm afraid you are wrong.
Please see PR 1958 in issue29456, IMO this PR can be merged.

--
nosy: +Ma Lin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33094] dataclasses: ClassVar attributes are not working properly

[Adrian Stachlewski]

Adrian Stachlewski  added the comment:

Thanks for explaining. I was trying to do something like

@dataclass
class A:
  x: ClassVar = set()

and thanks to you I know it should be 

@dataclass
class A:
  x: ClassVar[Set] = set()

If you are looking for improved error message, it's probably should be somehow 
connected to not proper usage of annotation. I've ended with default_factory 
because x: ClassVar = set() wasn't working and there was no error with 
default_factory and without slots.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33089] Add multi-dimensional Euclidean distance function to the math module

[Tim Peters]

Tim Peters  added the comment:

I'd be +1 on generalizing math.hypot to accept an arbitrary number of 
arguments.  It's the natural building block for computing distance, but the 
reverse is strained.  Both are useful.

Here's scaling code translated from the Fortran implementation of SNRM2 in 
LAPACK.  It only looks at each element once, so would work almost as-is for an 
argument that's an arbitrary iterable (just change the formal argument from 
`*xs` to `xs`):

# http://www.netlib.org/lapack/explore-html/d7/df1/snrm2_8f_source.html
def hypot(*xs):
import math
scale = 0.0
sumsq = 1.0
for x in xs:
if x:
absx = abs(x)
if scale < absx:
sumsq = 1.0 + sumsq * (scale / absx)**2
scale = absx
else:
sumsq += (absx / scale)**2
return scale * math.sqrt(sumsq)

I believe it does the right with infinities by magic (return math.inf).  It 
returns 0.0 if no arguments are passed, which makes sense.  For one argument, 
it's a convoluted way to return its absolute value.  The "if x:" test is 
necessary to prevent division by 0.0 if the first argument is 0.0.  It would 
need another blob of code to give up (and return math.nan) if one of the 
arguments is a NaN (there's no guessing what various C compilers would do 
without special-casing NaN).

I doubt `fsum()` would add much value here:  all the addends have the same 
sign, so cancellation is impossible.

To get really gonzo, a single-rounding dot product would be the best building 
block (the vector norm is the square root of the dot product of a vector with 
itself).  `fsum()` is a special case of that (the dot product of a vector with 
a same-length vector of all ones).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32960] dataclasses: disallow inheritance between frozen and non-frozen classes

[Eric V. Smith]

Eric V. Smith  added the comment:

Resolved by issue 32953.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33094] dataclasses: ClassVar attributes are not working properly

[Eric V. Smith]

Change by Eric V. Smith :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33094] dataclasses: ClassVar attributes are not working properly

[Eric V. Smith]

Eric V. Smith  added the comment:

There are a couple of problems here. You're using ClassVar incorrectly. It 
should be:

>>> @dataclass
... class C:
...   __slots__=()
...   x: ClassVar[int] = field(default=3)
... 
>>> C()
C()
>>> C.x
3


And you cannot have a ClassVar with a default_factory, since it would never get 
called:

>>> @dataclass
... class C:
...   __slots__=()
...   x: ClassVar[int] = field(default_factory=set)
... 

raise TypeError(f'field {f.name} cannot have a '
TypeError: field x cannot have a default factory

Although this error message could be improved. Neither InitVars or ClassVars 
can have default factories.

I'm not exactly sure how to improve the error message that you're seeing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32953] Dataclasses: frozen should not be inherited for non-dataclass derived classes

[Eric V. Smith]

Change by Eric V. Smith :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32953] Dataclasses: frozen should not be inherited for non-dataclass derived classes

[Eric V. Smith]

Eric V. Smith  added the comment:


New changeset 45648312e540cda3b10109b6a808cbf6955c84eb by Eric V. Smith (Miss 
Islington (bot)) in branch '3.7':
bpo-32953: Dataclasses: frozen should not be inherited for non-dataclass 
derived classes (GH-6147) (GH-6148)
https://github.com/python/cpython/commit/45648312e540cda3b10109b6a808cbf6955c84eb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33083] math.factorial accepts non-integral Decimal instances

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +5907
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32953] Dataclasses: frozen should not be inherited for non-dataclass derived classes

[miss-islington]

Change by miss-islington :


--
pull_requests: +5906

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32953] Dataclasses: frozen should not be inherited for non-dataclass derived classes

[Eric V. Smith]

Eric V. Smith  added the comment:


New changeset f199bc655eb50c28e94010714629b376bbbd077b by Eric V. Smith in 
branch 'master':
bpo-32953: Dataclasses: frozen should not be inherited for non-dataclass 
derived classes (#6147)
https://github.com/python/cpython/commit/f199bc655eb50c28e94010714629b376bbbd077b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33098] add implicit conversion for random.choice() on a dict

[Tim Peters]

Tim Peters  added the comment:

This won't be changed.  The dict type doesn't support efficient random choice 
(neither do sets, by the way), and it's been repeatedly decided that it would 
do a disservice to users to hide that.  As you know, you can materialize the 
keys in a list (or tuple) first if you _want_ to pay that cost.  Otherwise you 
should use a different data structure.

Note that there's really no differnce between Pythons 2 and 3 here.  If you 
_happen_ to have a dict that uses little integers as keys, then it can _appear_ 
to work, when a random integer picked from range(len(the_dict)) happens to be 
one of the keys.  But then you get back the associated dict value, not the key. 
 For example, here under Python 2.7.11:

>>> import random
>>> random.choice({0: "a", 1: "b"})
'b'
>>> random.choice({0: "a", 1: "b"})
'b'
>>> random.choice({0: "a", 1: "b"})
'a'

But if the keys don't happen to be little integers, it always fails:

>>> random.choice({"a": 1, "b": 2})
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\random.py", line 275, in choice
return seq[int(self.random() * len(seq))]  # raises IndexError if seq is 
empty
KeyError: 1

--
nosy: +tim.peters
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The garbage collector in Python does not work like that. If an object reaches 
zero references is destroyed immediately. The only problem is when circular 
references exist and is in this case when object deletion is delayed until the 
garbage collector runs the algorithm to detect circular references and delete 
them. This time is longer depending on the generation in which the object is 
placed.

Although this is true, there might be a problem in the lines you explain 
because is not guaranteed to collect garbage containing circular references 
(see Data Model in the documentation). In this case is important as you state 
to have an interface to ensure releasing the resources.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26917] unicodedata.normalize(): bug in Hangul Composition

[Ronan Lamy]

Ronan Lamy  added the comment:

Victor's patch is correct. I implemented the same fix in PyPy in 
https://bitbucket.org/pypy/pypy/commits/92b4fb5b9e58

--
nosy: +Ronan.Lamy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Henrique Andrade]

Henrique Andrade  added the comment:

You're a missing the fact that in C++, there is no garbage collector. The
destructor both releases resources and deallocates memory.

There is no window between releasing resources and object destruction.

Python, while not exactly like Java, is similar to Java in the sense that
there is a window of time between an object no longer having a reference
and its being reaped by the garbage collector. During that window,
resources can be held even if no longer in use.

In extreme cases, a lot of these resources can be held (think hundreds of
Queues being created and closed in succession without an intervening GC
run), even if not used. Sure, at some point, they will be reaped, but it
might be a while.

And that's one of the reasons Python and Java have mechanisms to
acquire/release resources in a more timely fashion. Context managers in the
former and try-with-resources in the latter.

The mere presence of a proper close/shutdown method can make this work in
an improved way in the case of a Queue, allowing OS resources (pipes) to be
released in a more timely fashion.

But, sure, let's hear what the community thinks.

On Sun, Mar 18, 2018, 14:01 Pablo Galindo Salgado 
wrote:

>
> Pablo Galindo Salgado  added the comment:
>
> > RAII in C++ ensures that, on object destruction, resources that have
> been acquired will be closed and deallocated.
>
> Which is exactly what is happening here. When the queue gets destroyed
> (because the reference count reaches 0 or because of the garbage collector)
> resources that have been acquired by the queue will be closed an
> deallocated.
>
> Sadly, I don't think I have anything different to apport to this
> discussion, so let's see what other people opinions are on this.
>
> Of course, feel free to start a thread on python-dev or python-ideas on
> how to improve the design. :)
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32642] add support for path-like objects in sys.path

[Jay Yin]

Jay Yin  added the comment:

srry I opened another issue bpo-33099

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33099] test_poplib hangs with the changes done in PR

[Jay Yin]

New submission from Jay Yin :

my test hangs locally on my computer with the changes I've done in bpo-32642 
but doesn't hang on TravisCI, anyone able to help with checking what's wrong 
here (sounds like another edge case with my env but I could be wrong)

the trace for the command https://pastebin.com/q4FKnPZH

--
components: Tests
messages: 314064
nosy: jayyin11043
priority: normal
severity: normal
status: open
title: test_poplib hangs with the changes done in PR
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32867] argparse assertion failure with multiline metavars

[paul j3]

paul j3  added the comment:

I haven't seen anyone try to use \n in a metavar before, but other special 
characters like [] and () produce this assertion error.  At this point the code 
is trying split the usage into 2 or more lines, because it's too long for one.  
It creates a usage for optionals and positionals separately.

In a clumsy way, it formats the whole usage string, and tries to split it into 
pieces so it can decide to split long lines.  The 'assert' is used to make sure 
it has split the line into meaningful blocks.  It splits with

r'\(.*?\)+|\[.*?\]+|\S+'

basically white space (including nl) and [] and () which are used to mark 
optional arguments and groups.  So including any of these characters in text 
via metavar will screwup this split.

We could try to refine this splitting expression, but that feels like a never 
ending task as users become more inventive.

I suggested a major rewrite of this section, one that keeps the pieces a list, 
and joins them after deciding how many can fit on a line.

No one has, to my knowledge, come up with a comprehensive list of characters 
that will cause problems here.

argparse does provide a backup - a user provided usage string.  That's not as 
nice as a automatically generated one, but if you have to have something 
special, that's the way to go.  In the long run there's only so much that 
general purpose parser can do to accommodate special needs.

--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33098] add implicit conversion for random.choice() on a dict

[Aristide Grange]

New submission from Aristide Grange :

In Python 3, the expression:

```python
random.choice(d)
```

where `d` is a `dict`, raises this error:

```
~/anaconda3/lib/python3.6/random.py in choice(self, seq)
256 except ValueError:
257 raise IndexError('Cannot choose from an empty sequence') 
from None
--> 258 return seq[i]
259 
260 def shuffle(self, x, random=None):

KeyError: 2
```

Converting `d` into a list restores the Python 2's behavior:

```python
random.choice(list(d))
```

I am aware that the keys of a dict have now their own type. But IMHO the error 
message is rather uninformative, and above all, couldn't this conversion be 
made implicitely under the hood?

--
messages: 314062
nosy: Aristide Grange
priority: normal
severity: normal
status: open
title: add implicit conversion for random.choice() on a dict
type: enhancement
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33017] Special set-cookie setting will bypass Cookielib

[R. David Murray]

R. David Murray  added the comment:

Can you explain what you think the problem is?  I don't know what your "POC" 
snippets are trying to demonstrate.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32949] Simplify "with"-related opcodes

[Mark Shannon]

Mark Shannon  added the comment:

It is fiddly to get the frame-setlineno code right for duplicated catch blocks, 
but it is far from impossible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32953] Dataclasses: frozen should not be inherited for non-dataclass derived classes

[Eric V. Smith]

Change by Eric V. Smith :


--
keywords: +patch
pull_requests: +5905
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32949] Simplify "with"-related opcodes

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There are problems with the f_lineno setter when duplicate a finally body. The 
duplication works for "with" only because the cleanup code for "with" doesn't 
correspond any line number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32949] Simplify "with"-related opcodes

[Mark Shannon]

Mark Shannon  added the comment:

I intend to reuse RERAISE to implement the exceptional case for a finally 
block. Something like:

SETUP_FINALLY final
body
finalbody
JUMP exit
final:
 finalbody
 RERAISE
exit:

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29673] Some gdb macros are broken in 3.6

[Skip Montanaro]

Change by Skip Montanaro :


--
nosy:  -skip.montanaro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[miss-islington]

miss-islington  added the comment:


New changeset 3c0a5a7c7ba8fbbc95dd1fe76cd7a1c0ce167371 by Miss Islington (bot) 
in branch '3.7':
bpo-32056: Improve exceptions in aifc, wave and sunau. (GH-5951)
https://github.com/python/cpython/commit/3c0a5a7c7ba8fbbc95dd1fe76cd7a1c0ce167371


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32911] Doc strings no longer stored in body of AST

[Ned Deily]

Ned Deily  added the comment:

Thanks everyone for your inputs.  Let's go with the status quo -> closing this 
issue.

--
priority: deferred blocker -> 
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[Ned Deily]

Ned Deily  added the comment:

I'm OK with merging this for 3.7.0b3.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Ned Deily]

Change by Ned Deily :


--
nosy:  -ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19417] Bdb: add a unittest file (test.test_bdb)

[miss-islington]

miss-islington  added the comment:


New changeset fdd8e8b4ffb68a4e8749bdc3b130fea7bbbf821e by Miss Islington (bot) 
in branch '3.7':
bpo-19417: Add test_bdb.py (GH-5217)
https://github.com/python/cpython/commit/fdd8e8b4ffb68a4e8749bdc3b130fea7bbbf821e


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19417] Bdb: add a unittest file (test.test_bdb)

[miss-islington]

Change by miss-islington :


--
pull_requests: +5904

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19417] Bdb: add a unittest file (test.test_bdb)

[Mariatta Wijaya]

Mariatta Wijaya  added the comment:


New changeset 3fe33043ee83d19e15551094fc1e0984617ded3c by Mariatta (xdegaye) in 
branch 'master':
bpo-19417: Add test_bdb.py (GH-5217)
https://github.com/python/cpython/commit/3fe33043ee83d19e15551094fc1e0984617ded3c


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27645] Supporting native backup facility of SQLite

[Berker Peksag]

Berker Peksag  added the comment:

Buildbots look happy, closing this one as 'fixed'. Thanks, Aviv!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30406] async and await should be keywords in 3.7

[miss-islington]

miss-islington  added the comment:


New changeset a90df5085b51e8bb9de1c04c408bbef42ce6cbc3 by Miss Islington (bot) 
in branch '3.7':
Revert "bpo-30406: Make async and await proper keywords (GH-1669)" (GH-6143)
https://github.com/python/cpython/commit/a90df5085b51e8bb9de1c04c408bbef42ce6cbc3


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> RAII in C++ ensures that, on object destruction, resources that have been 
> acquired will be closed and deallocated.

Which is exactly what is happening here. When the queue gets destroyed (because 
the reference count reaches 0 or because of the garbage collector) resources 
that have been acquired by the queue will be closed an deallocated. 

Sadly, I don't think I have anything different to apport to this discussion, so 
let's see what other people opinions are on this.

Of course, feel free to start a thread on python-dev or python-ideas on how to 
improve the design. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Henrique Andrade]

Henrique Andrade  added the comment:

Your comparison is not correct.

RAII in C++ ensures that, on object destruction, resources that have been
acquired will be closed and deallocated.

The closest analogy in Python is the use of a context manager, which, btw,
a Queue does not provide.

Indeed, such a design (with a context manager) would have been cleaner
because, on exit, both pipes would have been closed and file descriptors
would not hang around.

And, yes, that is what I'd prefer too - but one can't have everything. :)

With the current design, which is more akin to Java, one is exposed to the
vagaries of the garbage collector. Note that, even in Java,
try-with-resources and the auto-closeable interface would also take care of
this. In fact, most the Java classes that require external resources have
migrated to this model.

For these reasons, I think the design could be substantially improved
(i.e., with a context manager or with the provision of a method that really
terminates the queue, so resources are properly closed immediately).

On Sun, Mar 18, 2018 at 1:12 PM, Pablo Galindo Salgado <
rep...@bugs.python.org> wrote:

>
> Pablo Galindo Salgado  added the comment:
>
> >"I want to terminate the queue and make sure that no resources are leaked.
>
> Then you don't need to do anything special, those will be cleared on
> object destruction. This is not an unusual pattern even in other languages.
> For example, RAII in C++ is one of the most used patterns for acquiring
> resources and that works cleaning those resources on object destruction.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

-- 

Henrique Andrade | +1-530-426-2123 | h...@unscrambl.com |
https://unscrambl.com

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

>"I want to terminate the queue and make sure that no resources are leaked.

Then you don't need to do anything special, those will be cleared on object 
destruction. This is not an unusual pattern even in other languages. For 
example, RAII in C++ is one of the most used patterns for acquiring resources 
and that works cleaning those resources on object destruction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Henrique Andrade]

Henrique Andrade  added the comment:

I don't want to "close the pipes but maintain the queue alive" - I want to
terminate the queue and make sure that no resources are leaked. It's that
simple.

When one closes a file or a socket, there is no underlying OS resource
being held. That's what I would like to have here too.

Apparently the design does not support that and, if that's the case, it's
fine, it's just that it goes against most of the norm afaict.

On Sun, Mar 18, 2018 at 12:46 PM, Pablo Galindo Salgado <
rep...@bugs.python.org> wrote:

>
> Pablo Galindo Salgado  added the comment:
>
> Notice that the documentation for close says:
>
> > Indicate that no more data will be put on this queue by the current
> process. The background thread will quit once it has flushed all buffered
> data to the pipe. This is called automatically when the queue is garbage
> collected.
>
> The method does not promise to close any pipe, just "Indicate that no more
> data will be put on this queue by the current process". Closing prematurely
> the writer side could lead to issues. I still do not understand why you
> would want to close the pipes but maintain the queue alive.
>
> I could be missing something, so let's see if other people think
> differently about this.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

-- 

Henrique Andrade | +1-530-426-2123 | h...@unscrambl.com |
https://unscrambl.com

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33097] concurrent futures Executors accept tasks after interpreter shutdown

[Mark Nemec]

Change by Mark Nemec :


--
title: concurrent.futures executors accept tasks after interpreter shutdown -> 
concurrent futures Executors accept tasks after interpreter shutdown

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30406] async and await should be keywords in 3.7

[miss-islington]

Change by miss-islington :


--
pull_requests: +5903

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30406] async and await should be keywords in 3.7

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset f64aae46da292f71f6be750026cd052362e066bc by Łukasz Langa (Jelle 
Zijlstra) in branch 'master':
Revert "bpo-30406: Make async and await proper keywords (#1669)" (GH-6143)
https://github.com/python/cpython/commit/f64aae46da292f71f6be750026cd052362e066bc


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Notice that the documentation for close says:

> Indicate that no more data will be put on this queue by the current process. 
> The background thread will quit once it has flushed all buffered data to the 
> pipe. This is called automatically when the queue is garbage collected.

The method does not promise to close any pipe, just "Indicate that no more data 
will be put on this queue by the current process". Closing prematurely the 
writer side could lead to issues. I still do not understand why you would want 
to close the pipes but maintain the queue alive.

I could be missing something, so let's see if other people think differently 
about this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33097] concurrent.futures executors accept tasks after interpreter shutdown

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +5901
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33097] concurrent.futures executors accept tasks after interpreter shutdown

[Mark Nemec]

New submission from Mark Nemec :

Currently, one can submit a task to an executor (both ThreadPoolExecutor and 
ProcessPoolExecutor) during interpreter shutdown. One way to do this is to 
register function fun with atexit as below.

@atexit.register
def fun():
   pool.submit(print, "apple")

The future is accepted and goes into PENDING state. However, this can cause 
issues if the _python_exit function (located in concurrent/futures/thread.py 
and/or concurrent/futures/process.py) executes before function fun.

Function _python_exit will shutdown the running workers in the pool and hence 
there will be no workers running by the time fun is executed so the future will 
be left in PENDING state forever.

The solution submitted here is to instead raise a RuntimeException when a task 
is submitted during interpreter shutdown. This is the same behaviour as when 
the shutdown method of an executor is called explicitly.

--
components: Library (Lib)
messages: 314044
nosy: Mark Nemec
priority: normal
severity: normal
status: open
title: concurrent.futures executors accept tasks after interpreter shutdown
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30406] async and await should be keywords in 3.7

[Jelle Zijlstra]

Change by Jelle Zijlstra :


--
pull_requests: +5900

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Henrique Andrade]

Henrique Andrade  added the comment:

@pablo: I am using Python 2.7.12 (distributed with Ubuntu 16), what are you 
using? This might explain the difference between what we see.

Yet, irrespective of this difference, imho, it would be a better design to have 
"close" actually closing the underlying resources.

In general, if one has to delete and/or invoke the garbage collector on an 
object, it's an indication that the design needs a bit of polish. Just picture 
the small scenario I described amplified to a situation where a large number of 
queues is used, which is perhaps an artificial scenario, but one would end up 
with a bunch of file descriptors hanging around for no reason.

This is what files and sockets, for example, would do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33093] Fatal error on SSL transport

[Matt Eaton]

Matt Eaton  added the comment:

Eric, what is needed to try and reproduce this issue accurately?  Would it be 
installing websockets and setting up your client to ping to the server forever 
(12 hours in this case)?  If you are able to provide a client I would be 
willing to setup a test case on my end.

--
nosy: +agnosticdev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '-p'/'--path0' command line option to override sys.path[0] initialisation

[Nick Coghlan]

Nick Coghlan  added the comment:

This question recently came up again over in 
https://bugs.python.org/issue33053#msg314040.

With the assorted startup refactorings that were landed for 3.7, it likely 
makes sense to take another run at this for 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33053] Avoid adding an empty directory to sys.path when running a module with `-m`

[Nick Coghlan]

Nick Coghlan  added the comment:

https://bugs.python.org/issue13475 is the existing enhancement request to 
expose sys.path[0] management independently of the other execution isolation 
features.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32949] Simplify "with"-related opcodes

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your PR Mark.

The main difference between PR 5883 and PR 5112 is that in PR 5883 the pair of 
old WITH_CLEANUP_FINISH and END_FINALLY are replaced with a single new 
WITH_CLEANUP_FINISH, and in PR 5112 it is replaced with a sequence of 7 opcodes 
including a new opcode RERAISE.

POP_JUMP_IF_TRUE  L
RERAISE
L:
POP_TOP
POP_TOP
POP_TOP
POP_EXCEPT
POP_TOP

This doesn't affect a performance in normal case because this code is executed 
only when an exception has been raised (an in that case the performance is less 
important). And seems this doesn't introduce new race conditions.

The number of opcodes is the same in both PRs. The implementation of RERAISE in 
ceval.c in PR 5112 is a tiny bit simpler than the implementation of 
WITH_CLEANUP_FINISH in PR 5883. But the generated bytecode and the compiler are 
a tiny bit simpler in PR 5883. If RERAISE be used for other purposes besides 
implementing a "with" statement, it would be a great advantage.

For now both approaches look to me not having significant advantages or 
disadvantages against the other one. Does anybody have preferences?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 6142 fixes issues 2 and 4. It adds a new opcode END_ASYNC_FOR and therefore 
can't be backported. END_ASYNC_FOR combines a number of other opcodes and 
guaranties using the true StopAsyncIteration. The new opcode is neccessary also 
for detecting an "async for" loop when jump. Besides introducing the new opcode 
the code of an "async for" loop was changed in a way that allowed to disallow 
jumping into an "async for" loop (this is the only part that can be 
backported). The final bytecode is much simpler. The compiler has been cleaned 
up and its code is now much simpler too. I expect also a performance boost, but 
don't know how to benchmark this.

Perhaps only the half of issue 4 (disallowing jumps into an "async for" loop) 
can be solved in 3.7 and 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33081] multiprocessing Queue leaks a file descriptor associated with the pipe writer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

When I run your script I do not see any file descriptor associated with the 
queue when 

subprocess.call(["ls", "-la", "/proc/%d/fd" % os.getpid()])

is executed.

This is my output if I just execute your program:

starting  3728
Starting 'external_application'
exit status 10

the 'external_application' application finished with exit status '10'...

Note the file descriptor #4 below
total 0
dr-x-- 2 pablogsal pablogsal  0 Mar 18 12:17 .
dr-xr-xr-x 9 pablogsal pablogsal  0 Mar 18 12:17 ..
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:17 0 -> /dev/pts/1
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:17 1 -> /dev/pts/1
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:17 2 -> /dev/pts/1
lr-x-- 1 pablogsal pablogsal 64 Mar 18 12:17 5 -> /dev/urandom

This is my output if I remove the call to "del queue":

starting  3892
Starting 'external_application'
exit status 10

the 'external_application' application finished with exit status '10'...

Note the file descriptor #4 below
total 0
dr-x-- 2 pablogsal pablogsal  0 Mar 18 12:18 .
dr-xr-xr-x 9 pablogsal pablogsal  0 Mar 18 12:18 ..
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:18 0 -> /dev/pts/1
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:18 1 -> /dev/pts/1
lrwx-- 1 pablogsal pablogsal 64 Mar 18 12:18 2 -> /dev/pts/1
l-wx-- 1 pablogsal pablogsal 64 Mar 18 12:18 4 -> 'pipe:[104568]'
lr-x-- 1 pablogsal pablogsal 64 Mar 18 12:18 5 -> /dev/urandom

So at least on my side "del queue" is having the desired effect.

Notice that calling

> del queue

does not destroy the object but just decrement in 1 the number of references to 
the object. To force collection of any possible cycles once you have destroyed 
all references on your side you can call

> import gc
> gc.collect()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +5899

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33053] Avoid adding an empty directory to sys.path when running a module with `-m`

[Jakub Wilk]

Jakub Wilk  added the comment:

-I implies -s, which is not something I want.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33095] Cross-reference isolated mode from relevant locations

[Jakub Wilk]

Change by Jakub Wilk :


--
nosy:  -jwilk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 193760f00aacb122698674ed15327dba412653a5 by Serhiy Storchaka in 
branch '3.6':
[3.6] bpo-33041: Add tests for jumps in/out of 'async with' blocks. (GH-6110). 
(GH-6141)
https://github.com/python/cpython/commit/193760f00aacb122698674ed15327dba412653a5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 773573e9ac654d4b5c6682e70360f75864acd80e by Serhiy Storchaka in 
branch '3.7':
[3.7] bpo-33041: Add tests for jumps in/out of 'async with' blocks. (GH-6110). 
(GH-6140)
https://github.com/python/cpython/commit/773573e9ac654d4b5c6682e70360f75864acd80e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32489] Allow 'continue' in 'finally' clause

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +5898

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +5897

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33096] ttk.Treeview.insert() does not allow to insert item with "False" iid

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
versions: +Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Now open() in modules aifc, sunau and wave will raise only EOFError (if the 
file was truncated) or corresponding module error exception on corrupted files. 
aifc.open() can raise also OverflowError, but this is a different issue32978. 
And of course OSError, MemoryError, RecursionError and KeyboardInterrupt can be 
raised for causes not related to the correctness of the file.

I withdraw the part of my claim in msg313084. I have tested -- backporting this 
is safe, because some error always was raised in open(). But it can help 
existing user code which was not aware about wider set of exceptions (like in 
the original report). It can work in common case, and handle most corrupted 
files well, but fail in cases of specially created malicious data. Thus I think 
it is worth to backport this change at least to 3.7. What are your thoughts Ned?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33096] ttk.Treeview.insert() does not allow to insert item with "False" iid

[Igor Yakovchenko]

New submission from Igor Yakovchenko :

ttk.Treeview.insert(... iid=None, ...) method has a check:
if iid:
res = self.tk.call(self._w, "insert", parent, index,
"-id", iid, *opts)
else:
res = self.tk.call(self._w, "insert", parent, index, *opts)

Documentation says that "If iid is specified, it is used as the item 
identifier", but as you can see from code, iid is used only if it's "True". It 
means that you cannot use iids like 0, 0.0 etc.

--
components: Tkinter
messages: 314032
nosy: gpolo, serhiy.storchaka, truestarecat
priority: normal
severity: normal
status: open
title: ttk.Treeview.insert() does not allow to insert item with "False" iid
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[miss-islington]

Change by miss-islington :


--
pull_requests: +5896

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32489] Allow 'continue' in 'finally' clause

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset fe2bbb1869b4a3f331a3dfb8b304a19a5819 by Serhiy Storchaka in 
branch 'master':
bpo-32489: Allow 'continue' in 'finally' clause. (GH-5822)
https://github.com/python/cpython/commit/fe2bbb1869b4a3f331a3dfb8b304a19a5819


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32056] Improve exceptions in aifc, sunau and wave

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 134cb01cda50f02725575808130b05d2d776693f by Serhiy Storchaka in 
branch 'master':
bpo-32056: Improve exceptions in aifc, wave and sunau. (GH-5951)
https://github.com/python/cpython/commit/134cb01cda50f02725575808130b05d2d776693f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33041] Issues with "async for"

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset bc300ce205f99acb1ef92c37de06dc76147e073b by Serhiy Storchaka in 
branch 'master':
bpo-33041: Add tests for jumps in/out of 'async with' blocks. (#6110)
https://github.com/python/cpython/commit/bc300ce205f99acb1ef92c37de06dc76147e073b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33053] Avoid adding an empty directory to sys.path when running a module with `-m`

[Nick Coghlan]

Nick Coghlan  added the comment:

"python -m mypkg.myscript" does the right thing as far as local packages are 
concerned, whereas "python -m mypkg/myscript.py" will set you up for 
double-import bugs.

Note that you can almost always trigger arbitrary non-obvious code execution 
just by writing sitecustomize.py to the current directory, and any package you 
install can add a "/arbitrary-code.pth" or 
"/arbitrary-code.pth" file that gets run at startup 
(setuptools has long relied on this to implement various features).

Opting in to isolated mode turns *all* of those features off by saying "I'm 
expecting to run system code only here, not custom user code".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com