[issue20766] reference leaks in pdb

[daniel hahler]

daniel hahler  added the comment:

Please see https://bugs.python.org/issue36667 for a followup.

It does not look like moving it to `interaction` is relevant for fixing the 
leak, is it?

I think it should be restored in both places.

--
nosy: +blueyed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Donald Stufft]

Changes by Donald Stufft :


--
pull_requests: +858

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Jack Liu]

Jack Liu added the comment:

Good to know the fix will be included Python official builds. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Roundup Robot]

Roundup Robot added the comment:

New changeset 31a2d270c0c3 by Xavier de Gaye in branch '3.5':
Issue #20766: Fix references leaked by pdb in the handling of SIGINT handlers.
https://hg.python.org/cpython/rev/31a2d270c0c3

New changeset 86a1905ea28d by Xavier de Gaye in branch '3.6':
Issue #20766: Merge with 3.5.
https://hg.python.org/cpython/rev/86a1905ea28d

New changeset 8bb426d386a5 by Xavier de Gaye in branch 'default':
Issue #20766: Merge with 3.6.
https://hg.python.org/cpython/rev/8bb426d386a5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

Uploaded refleak_5.patch with a simpler test case.
With the patch applied, './python -m test -R 3:3 test_pdb' runs with success.

The '_previous_sigint_handler' is reset at the Pdb prompt - instead of when the 
signal is triggered - to handle the case where pdb stops at a new set_trace() 
hard breakpoint after it has been run with 'continue'.  As a side effect, this 
also fixes issue 22502.

--
assignee:  -> xdegaye
stage:  -> patch review
versions: +Python 3.6, Python 3.7 -Python 3.4
Added file: http://bugs.python.org/file44936/refleak_5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

> And indeed, this changeset has removed entirely the doctests from the test 
> suite :(

Entered new issue 28338.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

Oh, test_pdb does not fail anymore in refleak mode as expected (see msg212510 
for the change to apply to test_pdb.py in order to reproduce that). 'hg bisect' 
says that this is happening since:

  The first bad revision is:
  changeset:   103552:ee0bbfd972de
  user:Łukasz Langa 
  date:Fri Sep 09 22:21:17 2016 -0700
  summary: Issue #18401: pdb tests don't read ~/.pdbrc anymore

And indeed, this changeset has removed entirely the doctests from the test 
suite :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

The problem may still be reproduced by running the attached pdb_refleak.py 
script. Enter three times 'c' (short for the pdb 'continue' command) and the 
following line is printed:

pdb 3 -> pdb 2 -> pdb 1

This shows that the third pdb instance 'pdb 3', owns indirectly a reference to 
the second and first pdb instances. As the _signal extension module owns a 
reference to 'pdb 3' through its sigint_handler() method, then it also owns 
indirectly a reference to 'pdb 1'.

--
Added file: http://bugs.python.org/file44925/pdb_refleak.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Jack Liu]

Jack Liu added the comment:

Now, I have to set nosigint to True to fix the reference leaks issue for my app.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Jack Liu]

Jack Liu added the comment:

Will the issue be fixed in Python formal build?
I still meet the same issue with Python 3.5.1. It cost me a bit time to track 
down this issue.

I'm using pdb to debug a Python script, there are global variables in the 
script. Duo the leak of sigint_handler, calling gc.collect() doesn't release 
the global variables until calling PyOS_FiniInterrupts().

--
nosy: +Jack Liu, draghuram, gregory.p.smith, isandler, r.david.murray
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

After refleak_3.patch, test_pdb_next_command_in_generator_for_loop is the only 
remaining test that sets a breakpoint without reinitializing the breakpoints 
numbering first.
As a result, this test may also fail in the future when tests are reordered by 
the unittest machinery.
refleak_4.patch adds a fix to that (and could be ignored as a problem that 
should be fixed in a new issue of its own).

--
Added file: http://bugs.python.org/file36258/refleak_4.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Changes by Xavier de Gaye xdeg...@gmail.com:


Added file: http://bugs.python.org/file36163/refleak_3.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

This is because breakpoints number are class attributes. With the following 
change, the ./python -m test test_pdb test_pdb is ok:

$ hg diff
diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -614,6 +614,8 @@
 ... test_function_2()
 ... end = 1
 
+ from bdb import Breakpoint; Breakpoint.next = 1
+
  with PdbTestInput(['break test_function_2',
 ...'continue',
 ...'return',

Attached refleak_3.patch fixes this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[STINNER Victor]

STINNER Victor added the comment:

It's not easy to test the patch because running test_pdb fails with the 
following error (I also get the error if the patch is not applied):

$ ./python -m test test_pdb test_pdb
[1/2] test_pdb
[2/2] test_pdb
test test_pdb failed -- Traceback (most recent call last):
  File /home/haypo/prog/python/default/Lib/doctest.py, line 2189, in runTest
test, out=new.write, clear_globs=False)
AssertionError: Failed doctest test for 
test.test_pdb.test_next_until_return_at_return_event
  File /home/haypo/prog/python/default/Lib/test/test_pdb.py, line 603, in 
test_next_until_return_at_return_event

--
File /home/haypo/prog/python/default/Lib/test/test_pdb.py, line 617, in 
test.test_pdb.test_next_until_return_at_return_event
Failed example:
with PdbTestInput(['break test_function_2',
   'continue',
   'return',
   'next',
   'continue',
   'return',
   'until',
   'continue',
   'return',
   'return',
   'continue']):
test_function()
Expected:
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](3)test_function()
- test_function_2()
(Pdb) break test_function_2
Breakpoint 1 at doctest 
test.test_pdb.test_next_until_return_at_return_event[0]:1
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) next
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](4)test_function()
- test_function_2()
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) until
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](5)test_function()
- test_function_2()
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) return
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](6)test_function()
- end = 1
(Pdb) continue
Got:
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](3)test_function()
- test_function_2()
(Pdb) break test_function_2
Breakpoint 7 at doctest 
test.test_pdb.test_next_until_return_at_return_event[0]:1
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) next
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](4)test_function()
- test_function_2()
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) until
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](5)test_function()
- test_function_2()
(Pdb) continue
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](2)test_function_2()
- x = 1
(Pdb) return
--Return--
 doctest 
test.test_pdb.test_next_until_return_at_return_event[0](3)test_function_2()-None
- x = 2
(Pdb) return
 doctest 
test.test_pdb.test_next_until_return_at_return_event[1](6)test_function()
- end = 1
(Pdb) continue

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Antoine Pitrou]

Antoine Pitrou added the comment:

See issue20746 for the test_pdb failure when run multiple times.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

An improved patch with a test case.

--
Added file: http://bugs.python.org/file35349/refleak_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

After applying patch 'regrtest.diff' from issue 20746, the command:

  $ ./python -m test -W -R3:3 test_pdb

succeeds now and shows there are no reference leaks.


However, with the following change in Lib/test/test_pdb.py and patch 
'regrtest.diff' applied:

diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -34,7 +34,7 @@
 This tests the custom displayhook for pdb.
 
  def test_function(foo, bar):
-... import pdb; pdb.Pdb(nosigint=True).set_trace()
+... import pdb; pdb.Pdb().set_trace()
 ... pass
 
  with PdbTestInput([


there are reference leaks:

$ ./python -m test -R 3:3 test_pdb
[1/1] test_pdb
beginning 6 repetitions
123456
..
test_pdb leaked [200, 200, 200] references, sum=600
test_pdb leaked [43, 43, 43] memory blocks, sum=129
1 test failed:
test_pdb


And now with the proposed patch in the previous msg 212171 'refleak.patch' of 
this issue that fixes the sigint_handler leak, and also patch 'regrtest.diff' 
applied:

$ ./python -m test -R 3:3 test_pdb
[1/1] test_pdb
beginning 6 repetitions
123456
..
1 test OK.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

New submission from Xavier de Gaye:

After the pdb 'continue' command, the signal module owns a reference to 
Pdb.sigint_handler. On the next instantiation of pdb, the signal module owns a 
reference to a new sigint_handler method that owns a reference to the previous 
sigint_handler. As a consequence, the first pdb instance is never freed (as 
well as the frames that are referenced by this pdb instance). The following 
test demonstrates the problem that is fixed by the attached patch.

Run the following script:

# START of refleak.py
import sys, gc, pdb

i = 1
while i:
pdb.set_trace()
x = 1
gc.collect(); print(sys.gettotalrefcount())

# END of refleak.py


With the following pdb commands:

$ python refleak.py
 /tmp/test/refleak.py(6)module()
- x = 1
(Pdb) continue
95898
 /home/xavier/tmp/test/refleak.py(5)module()
- pdb.set_trace()
(Pdb) continue
95983
 /tmp/test/refleak.py(6)module()
- x = 1
(Pdb) continue
96068
 /tmp/test/refleak.py(5)module()
- pdb.set_trace()
(Pdb) i = 0
(Pdb) continue
96153

--
components: Library (Lib)
files: refleak.patch
keywords: patch
messages: 212171
nosy: georg.brandl, xdegaye
priority: normal
severity: normal
status: open
title: reference leaks in pdb
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file34220/refleak.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20766] reference leaks in pdb

[Xavier de Gaye]

Xavier de Gaye added the comment:

 the first pdb instance is never freed

The first pdb instance is (and all the other pdb instances) never freed until 
the call to PyOS_FiniInterrupts() in Py_Finalize().

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20766
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com