[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset a5af6e1af77ee0f9294c5776478a9c24d9fbab94 by Serhiy Storchaka in 
branch 'master':
bpo-25455: Fixed crashes in repr of recursive buffered file-like objects. (#514)
https://github.com/python/cpython/commit/a5af6e1af77ee0f9294c5776478a9c24d9fbab94


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset fca705d533970011e50b3f278aab81cead39b00d by Serhiy Storchaka in 
branch '3.6':
bpo-25455: Fixed crashes in repr of recursive buffered file-like objects. 
(#514) (#722)
https://github.com/python/cpython/commit/fca705d533970011e50b3f278aab81cead39b00d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 08612ed6a91003cdba7617dee7787c26495c50d9 by Serhiy Storchaka in 
branch '3.5':
bpo-25455: Fixed crashes in repr of recursive buffered file-like objects. 
(#514) (#727)
https://github.com/python/cpython/commit/08612ed6a91003cdba7617dee7787c26495c50d9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +642

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +637

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
versions: +Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +423

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Roundup Robot]

Roundup Robot added the comment:

New changeset ea1edf1bf362 by Benjamin Peterson in branch '2.7':
when you enter repr, you must leave, too (#25455)
https://hg.python.org/cpython/rev/ea1edf1bf362

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Antoine, are you fine with io_recursive_repr2.patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0323b33894f2 by Serhiy Storchaka in branch '3.5':
Issue #25455: Clean up reference loops created in tests for recursive
https://hg.python.org/cpython/rev/0323b33894f2

New changeset 688edc946ab9 by Serhiy Storchaka in branch '2.7':
Issue #25455: Clean up reference loops created in tests for recursive
https://hg.python.org/cpython/rev/688edc946ab9

New changeset 818a10534e44 by Serhiy Storchaka in branch 'default':
Issue #25455: Clean up reference loops created in tests for recursive
https://hg.python.org/cpython/rev/818a10534e44

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Indeed, tests for recursive partial objects create reference loops and don't 
clean them. Thank you Berker. I'll fix this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7859742826b2 by Serhiy Storchaka in branch '2.7':
Issue #25455: Backported tests for pickling recursive functools.partial objects.
https://hg.python.org/cpython/rev/7859742826b2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch for io classes needed an update.

--
Added file: http://bugs.python.org/file43360/io_recursive_repr2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Berker Peksag]

Berker Peksag added the comment:

I'm not sure if 17e78918f608 is relevant but test_gc is failing on several 
buildbots:

* http://buildbot.python.org/all/builders/PPC64%20Fedora%203.5/builds/716
  
http://buildbot.python.org/all/builders/PPC64%20Fedora%203.5/builds/716/steps/test/logs/stdio

* http://buildbot.python.org/all/builders/s390x%20RHEL%203.5/builds/775
  
http://buildbot.python.org/all/builders/s390x%20RHEL%203.5/builds/775/steps/test/logs/stdio

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Roundup Robot]

Roundup Robot added the comment:

New changeset c071da010053 by Serhiy Storchaka in branch '2.7':
Issue #25455: Fixed a crash in repr of cElementTree.Element with recursive tag.
https://hg.python.org/cpython/rev/c071da010053

New changeset 17e78918f608 by Serhiy Storchaka in branch '3.5':
Issue #25455: Fixed a crash in repr of recursive functools.partial objects.
https://hg.python.org/cpython/rev/17e78918f608

New changeset 86959c696ab7 by Serhiy Storchaka in branch 'default':
Issue #25455: Fixed a crash in repr of recursive functools.partial objects.
https://hg.python.org/cpython/rev/86959c696ab7

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Roundup Robot]

Roundup Robot added the comment:

New changeset e44bd1259bda by Serhiy Storchaka in branch '3.5':
Issue #25455: Fixed a crash in repr of ElementTree.Element with recursive tag.
https://hg.python.org/cpython/rev/e44bd1259bda

New changeset e3671a684ea0 by Serhiy Storchaka in branch 'default':
Issue #25455: Fixed a crash in repr of ElementTree.Element with recursive tag.
https://hg.python.org/cpython/rev/e3671a684ea0

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Stefan Behnel]

Stefan Behnel added the comment:

Etree patch looks straight forward to me, feel free to apply it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ping.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Eli Bendersky]

Changes by Eli Bendersky :


--
nosy:  -eli.bendersky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Eli Bendersky]

Eli Bendersky added the comment:

As I've mentioned elsewhere, I'll have to temporarily take myself off these 
issues as I don't have the time to work on them (even review patches). I think 
Raymond may have gotten his Stefans mixed up and meant Stefan Behnel, who's 
also been looking at etree patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Eli Bendersky]

Changes by Eli Bendersky :


--
nosy:  -eli.bendersky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Stefan Krah]

Stefan Krah added the comment:

I think you may have meant Eli Bendersky -- I'm not an elementree
expert (Eli, I'm adding you back just to clear this up).

--
nosy: +eli.bendersky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Raymond, could you please make a review of the first patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Raymond Hettinger]

Raymond Hettinger added the comment:

>Raymond, could you please make a review of the first patch?
Will do.

Also, we should ask Antoine Pitrou to look at the TextIO patch and ask Stephan 
Krah to look at the ElementTree patch.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file40972/etree_recursive_repr.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here are patches for io classes and for ElementTree.

--
stage: needs patch -> patch review
versions: +Python 2.7, Python 3.4
Added file: http://bugs.python.org/file40971/io_recursive_repr.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There is also a crash with Python implementation of TextIOWrapper.

>>> import _pyio as io
>>> t = io.TextIOWrapper(io.BytesIO())
>>> t.mode = t
>>> t
Fatal Python error: Cannot recover from stack overflow.

Current thread 0xb74a9700 (most recent call first):
  File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1982 in __repr__
  File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1992 in __repr__
[...]
  File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1992 in __repr__
  File "/home/serhiy/py/cpython/Lib/_pyio.py", line 1992 in __repr__
  ...
Aborted (core dumped)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added ElementTree and io modules experts to the nosy list.

--
nosy: +benjamin.peterson, eli.bendersky, pitrou, scoder, stutzbach

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Recursive partial objects are legitimate. Here is a patch that makes partial's 
repr to support recursive partial objects. Also added a test for pickling.

Cases for Element and file-like objects are questionable. Recursive Element.tag 
and TextIOWrapper.name don't make a sense, and I don't think we should special 
support (and encourage) these cases. To avoid stack overflow we can add a 
restriction for tag to be str or None, but file's name attribute can be 
dynamic. We can omit name from repr if it is not None, str, bytes or int.

--
keywords: +patch
Added file: http://bugs.python.org/file40932/partial_recursive_repr.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Raymond Hettinger]

Raymond Hettinger added the comment:

Changing PyObject_Repr is too course; it affects a broad class of objects other 
than containers, and it risks unknown impacts to larges swaths of third-party 
code use this venerable API.  It is also a break with the long established 
history of recursion detection being a responsibility of the individual types 
(i.e. the code in sets, lists, dicts, etc.)

The three cases listed here should be fixed individually.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yet one example:

>>> import io
>>> class BR(io.BufferedReader):
... @property
... def name(self):
... return self
... 
>>> repr(BR(io.BytesIO()))
Segmentation fault

The same is for other file-like objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The general solution is to make PyObject_Repr to detect recursive calls (as 
reprlib.recursive_repr does).

The straightforward way is to use thread local identity set. It can be 
implemented as a dict that maps id(obj) -> obj (creates an int object for key 
for every call, requires about 40-80 bytes for recurse level), or as 
specialized hash table (see Modules/hashtable.c) (faster, requires about 12-24 
bytes for recurse level).

The fastest solution would be to set special flag inside proceeded object. For 
now general object has no place for such flag, but we can add it to GC head. On 
64-bit this would increase the size of GC head from 24 to 32 bytes, on 32-bit 
there is a free place in 16-bytes GC head.

However the performance can be not critical here, because in any case repr() 
creates new object (resulting string). Using thread local hash table can be 
enough. In any case the patch will be enough complex to target it 3.6 only.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
priority: normal -> high
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25455] Some repr implementations don't check for self-referential structures

[Evgeny Kapun]

New submission from Evgeny Kapun:

Implementations of repr for some of the types in the standard library doesn't 
check for self-referential structures. As a result, when calling repr() on such 
objects, Python crashes due to infinite recursion.

Example:
>>> import functools
>>> x = functools.partial(min)
>>> x.__setstate__((x, (), {}, {}))
>>> repr(x)
Segmentation fault

Another example:
>>> import xml.etree.ElementTree
>>> x = xml.etree.ElementTree.Element('')
>>> x.tag = x
>>> repr(x)
Segmentation fault

One more example:
>>> import io
>>> class X(io.TextIOWrapper): __slots__ = 'name'
>>> x = X(open('/dev/null'))
>>> x.name = x
>>> repr(x)
Segmentation fault

--
components: Extension Modules
messages: 253309
nosy: abacabadabacaba
priority: normal
severity: normal
status: open
title: Some repr implementations don't check for self-referential structures
type: crash
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com