[issue26934] android: test_faulthandler fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

New patch.

--
Added file: http://bugs.python.org/file45404/buggy_raise_4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

On my computer:

Importing empty module: 160 us
Creating empty class: 30 us
Creating empty function: 0.16 us
Creating empty Enum/IntEnum: 125/150 us
Creating Enum/IntEnum member: 25/27 us
Creating empty namedtuple: 600 us
Creating namedtuple member: 50 us
Importing the itertools module: 40 us
Importing the io module: 900 us
Importing the os module: 1600 us
Importing the functools module: 2100 us
Importing the re module (with all sre submodules): 3300 us
Python startup time: 43000 us

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26934] android: test_faulthandler fails

[STINNER Victor]

STINNER Victor added the comment:

buggy_raise_4.patch LGTM. Thanks for your patience :-)

Maybe later you can also modify existing tests to use the new
@requires_android_level?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28644] Document recen changes in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Thanks Mariatta, here is a new version of patch with indentation fixed.

--
Added file: http://bugs.python.org/file45405/recent-typing-docs-v3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[STINNER Victor]

STINNER Victor added the comment:

About the re module, there is still a regression on the performance of
the import time: see my msg280271.

We may also revert the changeset 88110cfbf4dc (issue #28193) if we
want to keep "import re" as fast as Python 3.5.

The question is if we consider that the re module is part of the
Python core modules which must be blasing fast to import.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader losing rows with big files and tab delimiter

[Marc Garcia]

Marc Garcia added the comment:

I could research a bit more on the problem. This is a minimal code that 
reproduces what happened:

from io import StringIO
import csv

csv_file = StringIO('''1\t"A
2\tB''')

reader = csv.reader(csv_file, delimiter='\t')
for i, row in enumerate(reader):
pass

print(reader.line_num)  # 2
print(i + 1)# 1

The reason to return the right number of rows with the default delimiter, is 
because the quote needs to be immediately after the delimiter to be considered 
the opening of a quoted text.

If the file contains an opening quote, and the EOF is reached without its 
closing quote, the reader considers all the text until EOF to be that field.

This would work as expected in a line like:

1,"well quoted text","this one has a missing quote

But it'd fail silently with unexpected results in all other cases. I'd expect 
csv to raise an exception, more than the current behavior.

Do you agree? Should I create another issue to address this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader losing rows with big files and tab delimiter

[SilentGhost]

SilentGhost added the comment:

No, the module works exactly as advertised. The default value of quoting 
parameter might not be suitable for this file, but it suits majority of files 
out there. The fields in csv can contain line feed, so the line in your example 
does not have a "missing" quote, it's that you're using wrong (the default) 
quoting parameter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28642] csv reader losing rows with big files and tab delimiter

[Marc Garcia]

Marc Garcia added the comment:

I agree that for my case, I was using the wrong quoting parameter, and if I 
specify that my file has no quotes, it works as expected.

But I still think that in a different case, when a file do have quotes, but 
they are not paired, it'd be better to raise an exception, than to ignore the 
error and assume there is just a missing quote at the end.

>From the Zen of Python: "Errors should never pass silently", and I think it's 
>clear that there is an error in the file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26934] android: test_faulthandler fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

> Thanks for your patience :-)

No problem, your suggestions and code reviews are always very much welcome :)
I will push this patch with the other Android patches after 3.6 is released. 
The patch at issue 26936 will also use @requires_android_level.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26929] android: test_strptime fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Patch attached.

--
assignee:  -> xdegaye
components:  -Cross-Build
keywords: +patch
stage:  -> patch review
versions: +Python 3.7
Added file: http://bugs.python.org/file45406/exclude_ymd.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26929] android: test_strptime fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Entered new issue https://code.google.com/p/android/issues/detail?id=227388 on 
the AOSP issue tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[Armin Rigo]

New submission from Armin Rigo:

``python3.5 --help`` gives this information:

-u : unbuffered binary stdout and stderr, stdin always buffered

However, stdout and stderr are actually always opened in text mode, and print() 
always expects a string and never a bytes object.  This usage of "binary" in 
the --help is in contradiction with the usage of "binary" in the description of 
files (e.g. ``help(open)``).

--
components: Interpreter Core
messages: 280390
nosy: arigo
priority: normal
severity: normal
status: open
title: python --help: -u is misdocumented as binary mode
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[STINNER Victor]

STINNER Victor added the comment:

Right. Moreover, "unbuffered" is wrong. It's line buffered: sys.stdout.buffer 
is directly a io.FileIO object, but TextIOWrapper only calls buffer.write() 
when the message contains a newline character. It's not (currently) possible to 
have a fully unbuffered stdout.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[Eryk Sun]

Eryk Sun added the comment:

> It's not (currently) possible to have a fully unbuffered stdout.

Why doesn't create_stdio also pass `write_through = Py_True` when 
Py_UnbufferedStdioFlag is set? This would immediately pass writes through to 
the FileIO object, even without containing a newline (i.e. it sets 
text_needflush in _io_TextIOWrapper_write_impl).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[STINNER Victor]

STINNER Victor added the comment:

I don't recall, I would have to search in old issues for the rationale. But
the explanation is probably performance, reduce the number of syscalls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The write_through argument was added only in 3.3. I think this is a good idea 
to pass `write_through = Py_True` when Py_UnbufferedStdioFlag is set. Using -u 
means that performance is not important (that is why this is not default 
behavior).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Yury Selivanov]

Yury Selivanov added the comment:

The patch is causing refleaks:

test_ast leaked [98, 98, 98] references, sum=294
test_code leaked [2, 2, 2] references, sum=6

Please review the attached patch fixing that.  Please review.

--
nosy: +benjamin.peterson, larry, ned.deily, yselivanov
priority: normal -> release blocker
resolution: fixed -> 
status: closed -> open
Added file: http://bugs.python.org/file45407/refleak.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28641] Describe PEP 495 features in "What's New in Python 3.6" document

[Paul G]

Paul G added the comment:

I've never written a "What's New" before, but here are the main things I took 
away from implementing a PEP 495-compliant tzinfo class:

- The `fold` attribute is the SECOND occurrence of the time, not the first 
occurrence of the time. In my first pass solution of this, I had this inverted, 
so I would say it's worth making it very clear which is which, maybe with an 
example (I'm using full zone names here like those returned by 
`dateutil.tz.tzwin`):

>>> datetime(2011, 11, 6, 1, 30, fold=0, tzinfo=US_EASTERN).tzname()
'Eastern Daylight Time'
>>> datetime(2011, 11, 6, 1, 30, fold=1, tzinfo=US_EASTERN).tzname()
'Eastern Standard Time'

- Because the default for "fold" is 0, the default for "unspecified" fold has 
changed from being on the STD side to being on the DST side:

>>> datetime(2011, 11, 6, 1, 30, tzinfo=US_EASTERN_NO495).tzname()
'EST'
>>> datetime(2011, 11, 6, 1, 30, tzinfo=US_EASTERN_PEP495).tzname()
'EDT'

- It is now best practices to implement your own `fromutc()` (I got the 
impression that in the past it was encouraged that you generally just wrote a 
`utcoffset()` and `dst()` function.

- Comparisons of datetimes have changed such that inter-zone comparisons 
representing the same wall time and offset will now fail during ambiguous times 
(depending on the resolution of #28601, the exact semantics of what is an 
inter-zone comparison should be clarified):

>>> dt1 = datetime(2016, 11, 6, 1, 30, fold=1, tzinfo=US_EASTERN)
>>> dt2 = datetime(2016, 11, 6, 1, 30, fold=0, tzinfo=US_CENTRAL)
>>> dt1 == dt2
False

I think the remainder of my insights would have to do with 
backwards-compatibility and I'm not sure how much relevance they have to 
"What's New".

With regards to the documentation, here are my notes:

- One thing that should be made more clear is that arithmetic operations wipe 
out the `fold` attribute - this behavior and the reasoning behind it should 
probably be clear in the section on datetime operations, because I think people 
will be confused by this:

   >>> from datetime import datetime, timedelta
   >>> from dateutil import tz
   >>> dt = datetime(2011, 11, 6, 1, 30, fold=1, tz=gettz('US/Eastern')
   >>> dt.tzname()
   'EST'
   >>> (dt + timedelta(minutes=1)).tzname()
   'EDT'

- Not related to the changes in 3.6, but it might also be worth clarifying that 
`datetime.astimezone()` with no arguments returns a datetime with a fixed 
offset zone, NOT the local zone (it's somewhat clear, but it could be more 
explicit):

   >>> dt.astimezone().tzname()
   2011-11-06 01:30:00-5:00
   >>> dt.astimezone() - timedelta(hours=5)
   2011-11-05 20:30:00-05:00
   >>> dt.astimezone(tz.tzlocal())
   2011-11-06 01:30:00-5:00
   >>> dt.astimezone(tz.tzlocal()) - timedelta(hours=5)
   2011-11-05 20:30:00-04:00

- In the section on "working with datetime objects", GMT1 and GMT2 objects 
don't support the `fold` attribute. That might be confusing (though there are 
fold-aware tzinfo objects later in the documentation).

- In the section on time.tzname(), the example defines its own GMT1 time zone. 
It might be clearer to just define GMT1 = datetime.timezone(timedelta(hours=1), 
"Europe/Prague') rather than create a new fixed offset tzinfo.

- The "most implementations of dst() will probably look like one of these two" 
section should be updated to reflect `fold`, maybe something like this:

def dst(self, dt):
# Code to set dston and dstoff to the time zone's DST
# transition times based on the input dt.year, and expressed
# in standard local time.

dt_naive = dt.replace(tzinfo=None)
if dston <= dt_naive < dstoff:
if dt_naive < dston + timedelta(hours=1) and dt.fold:
return timedelta(0)
else:
return timedelta(hours=1)
else:
return timedelta(0)

- The tzinfo.fromutc(dt) documentation suggests that this method is capable of 
handling non-fixed offset zones.  Per #28602, it seems that it should be made 
clear that all zones implementing `fold` support should implement their own 
`fromutc` method, as there is a deliberate bug in the algorithm when folds are 
supported.

- This may be slightly confusing: "Note that the datetime instances that differ 
only by the value of the fold attribute are considered equal in comparisons.", 
because when I first read this, I assumed it meant that datetime(2011, 11, 6, 
1, 30, tzinfo=US_EASTERN) == datetime(2011, 11, 6, 1, 30, fold=1, 
tzinfo=US_EASTERN), but in fact it either means that *unambiguous* datetimes 
that differ only by their `fold` attribute are equal, ambiguous datetimes with 
a fold-aware tzinfo attached will return a different utcoffset() and thus not 
compare equal.

- I found this slightly confusing, under the "dateutil.tz" heading: "The 
standard library has timezone class for handling arbitrary fixed offsets from 
UTC and timezone.utc as UTC timez

[issue28641] Describe PEP 495 features in "What's New in Python 3.6" document

[Yury Selivanov]

Yury Selivanov added the comment:

Thanks, Paul. I'm adding Elvis to the nosy list, he's the main editor of What's 
New in 3.6.

--
nosy: +Elvis.Pranskevichus, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Yury Selivanov]

Yury Selivanov added the comment:

Looks like the attached patch also fixes

   test_trace leaked [12, 12, 12] references, sum=36

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28635] Update What's New for 3.6

[Yury Selivanov]

Yury Selivanov added the comment:

Elvis, please take a look at http://bugs.python.org/issue28641

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Good catch! The patch LGTM, thanks Yury.

Interesting, what tests in test_ast leak? I expected that this branch is 
executed in rare circumstances.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Yury Selivanov]

Yury Selivanov added the comment:

> Interesting, what tests in test_ast leak? I expected that this branch is 
> executed in rare circumstances.

The test that tests it all :) test_stdlib_validates

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Yury Selivanov]

Yury Selivanov added the comment:

Thanks for the review, Serhiy!

--
priority: release blocker -> normal
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Roundup Robot]

Roundup Robot added the comment:

New changeset 41613bb27f80 by Yury Selivanov in branch '2.7':
Issue #27942: Fix memory leak in codeobject.c
https://hg.python.org/cpython/rev/41613bb27f80

New changeset 2c6825c9ecfd by Yury Selivanov in branch '3.5':
ssue #27942: Fix memory leak in codeobject.c
https://hg.python.org/cpython/rev/2c6825c9ecfd

New changeset b671ac7ae620 by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #27942)
https://hg.python.org/cpython/rev/b671ac7ae620

New changeset c27269c0d619 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #27942)
https://hg.python.org/cpython/rev/c27269c0d619

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27942] Default value identity regression

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> The test that tests it all :) test_stdlib_validates

:)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

[STINNER Victor]

STINNER Victor added the comment:

Would it make sense to have two modes: line buferred and unbuffered,
as the C function setvbuf() for stdout?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28648] False assert in _Py_DecodeUTF8_surrogateescape

[Xiang Zhang]

New submission from Xiang Zhang:

The assert statement `assert(Py_UNICODE_IS_SURROGATE(ch));` in 
_Py_DecodeUTF8_surrogateescape is wrong. Code points > 0x could reach it 
and fail.

--
files: false_assert.patch
keywords: patch
messages: 280406
nosy: serhiy.storchaka, xiang.zhang
priority: normal
severity: normal
stage: patch review
status: open
title: False assert in _Py_DecodeUTF8_surrogateescape
type: crash
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45408/false_assert.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

New submission from Yury Selivanov:

Looks like we have a refleak somewhere in the system that typing.py is exposing.

The following code exposes the refleak:

def test_refleak(self):
T = typing.TypeVar('T')
class C(typing.Generic[T], typing.Mapping[int, str]): ...

The change that first triggered the leak is 8f0df4db2b06.

I'm not an expert in typing.py, but I suspect that the real bug must be 
somewhere in CPython core.

Guido, do you have any idea how 8f0df4db2b06 could trigger this?

--
components: Library (Lib)
messages: 280407
nosy: gvanrossum, ned.deily, serhiy.storchaka, yselivanov
priority: release blocker
severity: normal
stage: needs patch
status: open
title: refleak in typing.py
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27854] Installed 2.7: IDLE Help disabled because help.html is missing

[Steve Dower]

Steve Dower added the comment:

Tools/msi/msi.py has a whole lot of code to explicitly include certain files. 
Probably just needs a line (or a glob) added into the idlelib section

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28637] Python startup performance regression

[Guido van Rossum]

Guido van Rossum added the comment:

Honestly now the basic startup time is under control I am fine with
restoring the re enum. We can optimize that later.

--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28650] concurrent.futures.ThreadPoolExecutor: tasks in queue not marked as done

[pyfm]

New submission from pyfm:

Hi,

I just realized that the ThreadPoolExecutor's workers do not call task_done() 
on the work_queue from which they took their task.
Or is there a reason for not calling task_done that I am missing?

Calling task_done decrements the queue's unfinished_tasks counter which could 
be used to improve the reuse of idle threads (see bquinlan's comment, 
Lib/concurrent/futures/thread.py:l124).

I am thinking of something like
if self._work_queue.unfinished_tasks > len(self._threads) and 
len(self._threads) < self._max_workers:

One could still construct cases in which threads are created unnecessarily but 
it should improve the situation in most cases.

(proposed patch is based on Python-3.6.0b3)

--
files: thread.py.patch
keywords: patch
messages: 280410
nosy: pyfm
priority: normal
severity: normal
status: open
title: concurrent.futures.ThreadPoolExecutor: tasks in queue not marked as done
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file45409/thread.py.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could it be related to leaks in objects with empty __slots__? See issue24379 
for details.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

> Could it be related to leaks in objects with empty __slots__? See issue24379 
> for details.

Not sure... Do we have a test/patch for objects-with-empty-slots bug?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

Ivan, this was an early typing.py update (merged into CPython on Sept. 27). But 
the refleak is still with us. Do you have any intuition on what could be going 
on here?

My own intuition here is lacking, other than thinking there's a lot of 
metaclass magic going on in this code...

--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

>>> import gc
>>> gc.collect()
23
>>> gc.collect()
0
>>> class A:
... __slots__ = ()
... 
>>> del A
>>> gc.collect()
2
^ leak

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also https://mail.python.org/pipermail/python-dev/2015-October/141993.html.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24329] __qualname__ and __slots__

[Yury Selivanov]

Yury Selivanov added the comment:

Serhiy, maybe you know how to fix this?

--
nosy: +serhiy.storchaka
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28643] Broken makefile depends for profile-opt target

[Brett Cannon]

Changes by Brett Cannon :


--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28643] Broken makefile depends for profile-opt target

[Brett Cannon]

Changes by Brett Cannon :


--
nosy: +brett.cannon, gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

Serhiy, why is `gc.collect()` returning 2 after `del A` proof of a leak?
Yes, there's a cycle, but it's being GC'ed -- isn't that fine? Without the
slots the collect() call returns a larger number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24329] __qualname__ and __slots__

[Xiang Zhang]

Xiang Zhang added the comment:

Why should it work Yury?

__qualname__ and __doc__(if exists) are inserted into the dict when creating a 
class.

>>> class Foo:
... """bar"""
... __slots__ = ('__doc__',)
... 
Traceback (most recent call last):
  File "", line 1, in 
ValueError: '__doc__' in __slots__ conflicts with class variable

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24329] __qualname__ and __slots__

[Yury Selivanov]

Yury Selivanov added the comment:

Because the following works without a problem:

  class F:
__slots__ = ('__name__', )

  f = F()
  f.__name__ = ''

This bug with __qualname__ is why _GeneratorWrapper in types.py doesn't have 
__slots__.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Attached patch (slots_gc.patch) makes objects with empty slots tracked by GC:

   class A: __slots__ = ()
   gc.is_tracked(A()) == False  # before
   gc.is_tracked(A()) == True   # with the patch

This doesn't fix the refleak though.

--
keywords: +patch
Added file: http://bugs.python.org/file45410/slots_gc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

I removed all slots from typing.py, the refleak is still there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27584] New addition of vSockets to the python socket module

[Cathy Avery]

Cathy Avery added the comment:

Please forgive the long delay in providing this update. I got a little 
sidetracked. Attached is the patch for Python 3.7. It includes fixes suggested 
in rev 1 plus VSOCK tests in test_socket.py.

Thanks,

Cathy

--
versions: +Python 3.7 -Python 3.6
Added file: http://bugs.python.org/file45411/vsock_rev2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Wow. In default CPython branch, with no modifications, test_typing.py simply 
crashes in debug mode:

yury@ysmbp ~/dev/py/cpython $ ./python.exe -m test -R3:3 test_typing
Run tests sequentially
0:00:00 [1/1] test_typing
beginning 6 repetitions
123456
.test test_typing failed -- Traceback (most recent call last):
  File "/Users/yury/dev/py/cpython/Lib/test/test_typing.py", line 739, in 
test_generic_forvard_ref
self.assertEqual(get_type_hints(foobar, globals(), locals()), {'x': 
List[List[T]]})
AssertionError: {'x': typing.List[typing.List[~T]]} != {'x': 
typing.List[typing.List[~T]]}
  {'x': typing.List[typing.List[~T]]}

test_typing failed

1 test failed:
test_typing

Total duration: 1 sec
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

The problem is that functools.lru_cache is used in _tp_cache.  If I remove type 
caching, the original "refleak" is fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Yury Selivanov]

Yury Selivanov added the comment:

Even more reduced test case:

def test_refleak(self):
T = typing.TypeVar('T')
typing.Generic[T]

--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Josh Rosenberg]

Josh Rosenberg added the comment:

Serhiy: I think you forgot to make a global instance of the type for your 
demonstration. You mentioned in msg253899 that the loop is:

global instance -> type -> method -> module globals -> global instance

The example you gave doesn't instantiate class A, and it's the instance of A 
that doesn't participate in GC and causes the problem, right?

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28651] Make objects with empty __slots__ GC types

[Yury Selivanov]

New submission from Yury Selivanov:

The attached patch does the trick.  I'm not sure if this should be fixed in 3.6.

--
components: Interpreter Core
files: slots_gc.patch
keywords: patch
messages: 280427
nosy: gvanrossum, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Make objects with empty __slots__ GC types
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45412/slots_gc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Josh, Serhiy,

I've created an issue for tracking empty slots types: #28651.

This issue is about typing.py using functools.lru_cache for internal cache That 
is what causing the refleaks, and inability of test_typing.py to be run in 
refleak hunting mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Steve Dower]

Steve Dower added the comment:

Anyone have any major concerns with add_non_strict_resolve_pathlib_v4.patch?

I'd be quite happy without adding the extra parameter to Path.resolve(), but 
I'm not strongly offended.

>From Guido's email we should default to strict=False (i.e. don't throw if the 
>file doesn't exist) rather than True.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

I could use a hint on a complete program that establishes whether there is
or is not a refleak.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

Good sleuthing! So the bug is in _lru_cache?

On Wed, Nov 9, 2016 at 10:37 AM, Guido van Rossum 
wrote:

>
> Guido van Rossum added the comment:
>
> I could use a hint on a complete program that establishes whether there is
> or is not a refleak.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Guido van Rossum]

Guido van Rossum added the comment:

I'd be very happy if that landed in 3.6 with the default strict=False.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23839] Clear caches after every test

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Synchronized with current code and added clearing typing caches.

--
Added file: http://bugs.python.org/file45413/regrtest_clear_caches_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Serhiy, why is `gc.collect()` returning 2 after `del A` proof of a leak?

Sorry, it was bad example. I don't remember all details.

> The problem is that functools.lru_cache is used in _tp_cache.  If I remove 
> type caching, the original "refleak" is fixed.

Proposed patch clears caches when search for reference leaks. This decreases 
the number of leaks.

Unpatched:
test_typing leaked [3125, 3089, 2897] references, sum=9111
test_typing leaked [1189, 1179, 1103] memory blocks, sum=3471

Patched:
test_typing leaked [125, 125, 125] references, sum=375
test_typing leaked [49, 51, 51] memory blocks, sum=151

See also issue23839.

--
Added file: http://bugs.python.org/file45414/typing-clear-caches.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28651] Make objects with empty __slots__ GC types

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Needed tests. If an example with typing causes a leak (msg280422) this should 
be fixed in all versions that have typing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

I have no objections against typing-clear-caches.patch (though we'll have to 
make sure to apply it to the GitHub upstream as well -- I can take care of that 
once you merge it to CPython).

Does that patch fix the refleak completely or only partially?

I have no opinion on empty slots.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

> Good sleuthing! So the bug is in _lru_cache?

I don't think it's a bug of lru_cache. It stores strong references for 
arguments and return value of the decorated function (which btw isn't 
documented).  I don't think it's possible to write efficient generic lru_cache 
that would just use weakrefs.

The problem is that because _tp_cache helper uses it, the lifetime of some 
typing classes becomes indefinite.  Which shouldn't be a problem for a typical 
user of typing, since most of the time it is used on the module level.

We have two potential solutions to this problem:

1. Add a weak-ref type cache and refactor _tp_cache to use it;

2. Cleanup lru-cache before/adter running each typing test.

[1] is the ideal solution, [2] should work good enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Yury, Guido, I think I understand why test_typing crashes in refleak hunting 
mode. The test relies on caching, namely type variables are compared by id, so 
that if cache is cleared between those two lines, then test fails. I think we 
should just update those test to use a usual class in forward reference. The 
patch is attached.

(Patch fixes the crash in refleak hunting mode, but the refleak is still there)

--
Added file: http://bugs.python.org/file45415/fix-typing-test.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

> Does that patch fix the refleak completely or only partially?

I tried to empty the cache in test_typing.BaseTestCase.setUp and tearDown; this 
doesn't fix all refleaks.

Do we have some other caches in typing.py?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Does that patch fix the refleak completely or only partially?

It fixes 96% of refleaks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Yury, _tp_cache should be the only one (of course most generic classes have 
_abc_cache since GenericMeta inherits from ABCMeta)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

FYI I've opened https://github.com/python/typing/issues/323 to track this 
upstream.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

> It fixes 96% of refleaks.

In typing.py? Or generally?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

> Yury, _tp_cache should be the only one (of course most generic classes have 
> _abc_cache since GenericMeta inherits from ABCMeta)

Then there is at least one more bug not related to lru_cache (or any kind of 
cache in typing).

And it's not classes-with-empty-slots bug either.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28651] Make objects with empty __slots__ GC types

[Yury Selivanov]

Yury Selivanov added the comment:

So this patch has no impact on typing.py.  Moreover, it's incomplete and fails 
test_gc tests, which can be fixed but not in 3.6.

--
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Serhiy's patch typing-clear-caches.patch looks good, we should apply it (fixing 
the code style a little bit).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Here is the corrected patch to avoid crash.

--
Added file: http://bugs.python.org/file45416/fix-typing-test-v2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28652] Make loop methods reject socket kinds they do not support.

[Yury Selivanov]

New submission from Yury Selivanov:

Proxy for https://github.com/python/asyncio/pull/453

--
assignee: yselivanov
components: asyncio
messages: 280448
nosy: gvanrossum, yselivanov
priority: normal
severity: normal
stage: resolved
status: open
title: Make loop methods reject socket kinds they do not support.
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28652] Make loop methods reject socket kinds they do not support.

[Yury Selivanov]

Changes by Yury Selivanov :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28652] Make loop methods reject socket kinds they do not support.

[Roundup Robot]

Roundup Robot added the comment:

New changeset 85022978d900 by Yury Selivanov in branch '3.5':
Issue #28652: Make loop methods reject socket kinds they do not support.
https://hg.python.org/cpython/rev/85022978d900

New changeset 1273f1a3ddf7 by Yury Selivanov in branch '3.6':
Merge 3.5 (issue #28652)
https://hg.python.org/cpython/rev/1273f1a3ddf7

New changeset 719da54652c5 by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28652)
https://hg.python.org/cpython/rev/719da54652c5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Yury Selivanov]

Changes by Yury Selivanov :


--
Removed message: http://bugs.python.org/msg280422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Steve Dower]

Steve Dower added the comment:

Applied. I changed the default for the parameter and updated the docs, but the 
rest is as in the patch.

--
assignee:  -> steve.dower
resolution:  -> fixed
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Roundup Robot]

Roundup Robot added the comment:

New changeset 03bbee2b0d28 by Steve Dower in branch '3.6':
Issue #19717: Makes Path.resolve() succeed on paths that do not exist (patch by 
Vajrasky Kok)
https://hg.python.org/cpython/rev/03bbee2b0d28

New changeset 445415e402be by Steve Dower in branch 'default':
Issue #19717: Makes Path.resolve() succeed on paths that do not exist (patch by 
Vajrasky Kok)
https://hg.python.org/cpython/rev/445415e402be

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19717] resolve() fails when the path doesn't exist

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28556] typing.py upgrades

[Roundup Robot]

Roundup Robot added the comment:

New changeset 9c0df5f51baa by Guido van Rossum in branch '3.5':
Issue #28556: More typing.py updates from upstream.
https://hg.python.org/cpython/rev/9c0df5f51baa

New changeset 9e65bc305a24 by Guido van Rossum in branch '3.6':
Issue #28556: More typing.py updates from upstream. (3.5->3.6)
https://hg.python.org/cpython/rev/9e65bc305a24

New changeset 00e386ac7b95 by Guido van Rossum in branch 'default':
Issue #28556: More typing.py updates from upstream. (3.6->3.7)
https://hg.python.org/cpython/rev/00e386ac7b95

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset d790078797bd by Guido van Rossum in branch '3.5':
Issue #28649: fix-typing-test-v2.diff
https://hg.python.org/cpython/rev/d790078797bd

New changeset 43be7891b1f5 by Guido van Rossum in branch '3.6':
Issue #28649: fix-typing-test-v2.diff (3.5->3.6)
https://hg.python.org/cpython/rev/43be7891b1f5

New changeset fd47a9d791b9 by Guido van Rossum in branch 'default':
Issue #28649: fix-typing-test-v2.diff (3.6->3.7)
https://hg.python.org/cpython/rev/fd47a9d791b9

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset d920bfa5a71a by Guido van Rossum in branch '3.5':
Issue #28649: typing-clear-caches.patch
https://hg.python.org/cpython/rev/d920bfa5a71a

New changeset bd2ec9965f47 by Guido van Rossum in branch '3.6':
Issue #28649: typing-clear-caches.patch (3.5->3.6)
https://hg.python.org/cpython/rev/bd2ec9965f47

New changeset 08f76f89d199 by Guido van Rossum in branch 'default':
Issue #28649: typing-clear-caches.patch (3.6->3.7)
https://hg.python.org/cpython/rev/08f76f89d199

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24379] Add operator.subscript as a convenience for creating slices

[Josh Rosenberg]

Josh Rosenberg added the comment:

#28651 opened for the general problem with empty __slots__ and gc failures.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Guido van Rossum]

Guido van Rossum added the comment:

Serhiy can you apply the non-typing.py part of typing-clear-caches.patch? The 
diff I applied only has the typing.py part (because I've done this upstream 
first).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Roundup Robot]

Roundup Robot added the comment:

New changeset d926b484d33a by Serhiy Storchaka in branch '3.5':
Issue #28649: Clear the typing module caches when search for reference leaks.
https://hg.python.org/cpython/rev/d926b484d33a

New changeset caf3ceb93307 by Serhiy Storchaka in branch '3.6':
Issue #28649: Clear the typing module caches when search for reference leaks.
https://hg.python.org/cpython/rev/caf3ceb93307

New changeset 437564294e6c by Serhiy Storchaka in branch 'default':
Issue #28649: Clear the typing module caches when search for reference leaks.
https://hg.python.org/cpython/rev/437564294e6c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

With all commits pushed, test_typing still seems to leak.  Ivan, do you have 
time to see what's going on?

test_typing leaked [125, 125, 125] references, sum=375
test_typing leaked [49, 49, 49] memory blocks, sum=147

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

The specific test that leaks is test_extended_generic_rules_eq

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Yury, yes I will take a look at this now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Ivan, this is the part of the test that leaks:

def test_extended_generic_rules_eq(self):
T = TypeVar('T')
U = TypeVar('U')
with self.assertRaises(TypeError):
Callable[[T], U][[], int]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Oh. This is a bug in C implementation of functools.lru_cache. I'll take a look.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

My impression is like something wrong happens when TypeError is raised by a 
cached function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

A very simple repro:

with self.assertRaises(TypeError):
Union[[]]

It looks like the problem happens when a non-hashable argument is passed to 
cached function

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

reflesk with only functools:

def test_lru_type_error(self):
@functools.lru_cache(maxsize=None)
def foo(o):
raise TypeError

with self.assertRaises(TypeError):
foo([])

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

Found it, will open a separate issue.

--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Yury Selivanov]

New submission from Yury Selivanov:

The attached patch fixes a refleak in functools.lru_cache.

--
components: Library (Lib)
files: refleak.patch
keywords: patch
messages: 280468
nosy: gvanrossum, larry, ned.deily, serhiy.storchaka, yselivanov
priority: release blocker
severity: normal
stage: patch review
status: open
title: refleak in functools.lru_cache
type: resource usage
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file45417/refleak.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28649] refleak in typing.py

[Yury Selivanov]

Yury Selivanov added the comment:

See http://bugs.python.org/issue28653 for details.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

--
assignee:  -> yselivanov
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

FWIW, if I comment out this code

try:
from _functools import _lru_cache_wrapper
except ImportError:
pass

in functools.py to use the pure Python version, then I get much larger numbers:

$ ./python -m test -R : test_typing
Run tests sequentially
0:00:00 [1/1] test_typing
beginning 9 repetitions
123456789
.
test_typing leaked [25003, 25003, 25003, 25003] references, sum=100012
test_typing leaked [9350, 9352, 9352, 9352] memory blocks, sum=37406
test_typing failed

--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Roundup Robot]

Roundup Robot added the comment:

New changeset ba59f3328032 by Yury Selivanov in branch '3.5':
Issue #28653: Fix a refleak in functools.lru_cache.
https://hg.python.org/cpython/rev/ba59f3328032

New changeset 5b253d641826 by Yury Selivanov in branch '3.6':
Merge 3.6 (issue #28653)
https://hg.python.org/cpython/rev/5b253d641826

New changeset 784fea019cab by Yury Selivanov in branch 'default':
Merge 3.6 (issue #28653)
https://hg.python.org/cpython/rev/784fea019cab

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Yury Selivanov]

Yury Selivanov added the comment:

> FWIW, if I comment out this code

Oh boy... This is something else. Please re-open #28649 explaining that it now 
leaks with pure-Python lru_cache.  Could you please take a look at it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28653] refleak in functools.lru_cache

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

It seems to be unrelated to typing.py
With your test from test_functools:

def test_lru_type_error(self):
@functools.lru_cache(maxsize=None)
def infinite_cache(o):
pass
with self.assertRaises(TypeError):
infinite_cache([])

I get [2, 2, 2, 2] for unpatched C version (this is now fixed)
and [24764, 24764, 24764, 24764] for Python version.

Should I maybe open a separate issue for this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com