[issue32542] memory not freed, aka memory leak continues...

[Michael Felt]

Michael Felt  added the comment:

Not an issue in 3.9, so, closing: "not relevant"

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32542] memory not freed, aka memory leak continues...

[STINNER Victor]

STINNER Victor  added the comment:

ctypes test cases tearDown() may always clear _pointer_type_cache. Would it 
help?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32542] memory not freed, aka memory leak continues...

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> Being curious, is there a way to see what the size of the cache is?

len(_pointer_type_cache)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32542] memory not freed, aka memory leak continues...

[Michael Felt]

Michael Felt  added the comment:

Thanks for the clarification. 

Being curious, is there a way to see what the size of the cache is? I want to 
believe, but i do not have the impression memory is being reallocated to later 
users. My gut feeling is that the code change permits repeated calls to a 
function in a way that it reuses memory it received before (rather than a 
memory leak of repeatedly allocating a brand new buffer). What I expected, more 
accurately hoped for, is that cache was either freed and/or made available for 
reuse. 

Since I am on my phone atm I am less successful at finding documentation. I 
would be very grateful for a link to the relevant information and I’ll look 
further, time permitting. Thx again,

Michael 

Sent from my iPhone

> On 12 Jan 2018, at 23:48, Serhiy Storchaka  wrote:
> 
> 
> Serhiy Storchaka  added the comment:
> 
> Issue25582 fixed a memory leak. If run the tests repeatedly every iteration 
> leaked 100 MB of memory. If you get a MemoryError with a single iteration, 
> this means that your machine just doesn't have enough memory for tests. Tests 
> require around 500-600MB of memory on 32-bit platform.
> 
> --
> nosy: +serhiy.storchaka
> 
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32542] memory not freed, aka memory leak continues...

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Issue25582 fixed a memory leak. If run the tests repeatedly every iteration 
leaked 100 MB of memory. If you get a MemoryError with a single iteration, this 
means that your machine just doesn't have enough memory for tests. Tests 
require around 500-600MB of memory on 32-bit platform.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32542] memory not freed, aka memory leak continues...

[Michael Felt]

New submission from Michael Felt :

in issue25582 - the issue is not (yet) resolved. Perhaps this one can be closed 
and issue25582 reopened.

Both from python2-2.7.14 and "git master" I am getting:

michael@x071:[/data/prj/python/git/gcc-python3-3.7]./python -m unittest -v 
ctypes.test.test_pointers  <
test_abstract (ctypes.test.test_pointers.PointersTestCase) ... ok
test_basic (ctypes.test.test_pointers.PointersTestCase) ... ok
test_basics (ctypes.test.test_pointers.PointersTestCase) ... ok
test_bug_1467852 (ctypes.test.test_pointers.PointersTestCase) ... ok
test_c_void_p (ctypes.test.test_pointers.PointersTestCase) ... ok
test_callbacks_with_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok
test_change_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok
test_charpp (ctypes.test.test_pointers.PointersTestCase)
Test that a character pointer-to-pointer is correctly passed ... ok
test_from_address (ctypes.test.test_pointers.PointersTestCase) ... ok
test_other (ctypes.test.test_pointers.PointersTestCase) ... ok
test_pass_pointers (ctypes.test.test_pointers.PointersTestCase) ... ok
test_pointer_crash (ctypes.test.test_pointers.PointersTestCase) ... ok
test_pointer_type_name (ctypes.test.test_pointers.PointersTestCase) ... ok
test_pointer_type_str_name (ctypes.test.test_pointers.PointersTestCase) ... 
ERROR
test_pointers_bool (ctypes.test.test_pointers.PointersTestCase) ... ok

==
ERROR: test_pointer_type_str_name (ctypes.test.test_pointers.PointersTestCase)
--
Traceback (most recent call last):
  File "/data/prj/python/git/gcc-python3-3.7/Lib/ctypes/test/test_pointers.py", 
line 208, in test_pointer_type_str_name
large_string = 'T' * 2 ** 25
MemoryError

--
Ran 15 tests in 0.319s

FAILED (errors=1)

+
Looking at the test source:

  +196  def test_pointer_type_name(self):
  +197  LargeNamedType = type('T' * 2 ** 25, (Structure,), {})
  +198  self.assertTrue(POINTER(LargeNamedType))
  +199
  +200  # to not leak references, we must clean _pointer_type_cache
  +201  from ctypes import _pointer_type_cache
  +202  del _pointer_type_cache[LargeNamedType]
  +203
  +204  def test_pointer_type_str_name(self):
  +205  large_string = 'T' * 2 ** 25
  +206  P = POINTER(large_string)
  +207  self.assertTrue(P)
  +208
  +209  # to not leak references, we must clean _pointer_type_cache
  +210  from ctypes import _pointer_type_cache
  +211  del _pointer_type_cache[id(P)]


After changing the exponent (** 25) to "** 23" on either line 197 OR 205 
- ALL test succeed -
After changing the exponent to " ** 24" on BOTH lines 197 and 205, all tests 
pass.


My concern is that the "patch" from issue 25582 the 
"del _pointer_type_cache[]" statement is not freeing memory.

+
What can I add to the test to debug!
+
p.s. - results are the same on AIX (5.3 and 6.1) Python2-2.7.14 and 
Pyhton3-3.7.X (git master). Compiler does not seem to matter (both xlc and gcc).
32-bit, default memory model (256 MByte max - aka (2 ** 28) for data and 
malloc. I could also try changing the memory model - but will do that only if 
the test, by definition, is not "free()ing" the memory used, when done.

HTH
Michael

--
components: Tests
messages: 309869
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: memory not freed, aka memory leak continues...
type: behavior
versions: Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com