On Oct 28, 10:33 am, Robert Bradshaw <[EMAIL PROTECTED]>
wrote:
> On Oct 28, 2007, at 2:17 AM, mabshoff wrote:
>
Hi Robert,
>
>
>
>
> > On Oct 28, 9:48 am, Robert Bradshaw <[EMAIL PROTECTED]>
> > wrote:
> >> On Oct 28, 2007, at 1:29 AM, mabshoff wrote:
>
> > Hi Robert,
>
> >>> For 2.8.10.alpha1+Craig's fix and python compiled using "--without-
> >>> pymalloc" I get with the default cleanup level 1:
>
> >>> ==6569== LEAK SUMMARY:
> >>> ==6569== definitely lost: 181,030 bytes in 2,926 blocks.
> >>> ==6569== possibly lost: 266,925 bytes in 747 blocks.
> >>> ==6569== still reachable: 29,184,388 bytes in 179,707 blocks.
> >>> ==6569== suppressed: 0 bytes in 0 blocks.
>
> >>> With cleanup level 3 in general and cleanup level 1 just for
> >>> integer.pyx I get
>
> >>> ==23818== LEAK SUMMARY:
> >>> ==23818== definitely lost: 181,117 bytes in 2,927 blocks.
> >>> ==23818== possibly lost: 266,838 bytes in 746 blocks.
> >>> ==23818== still reachable: 29,157,453 bytes in 179,383 blocks.
> >>> ==23818== suppressed: 0 bytes in 0 blocks.
>
> > With the patch valgrind says:
>
> > ==3482== LEAK SUMMARY:
> > ==3482== definitely lost: 181,235 bytes in 2,929 blocks.
> > ==3482== possibly lost: 266,720 bytes in 744 blocks.
> > ==3482== still reachable: 29,157,648 bytes in 179,395 blocks.
> > ==3482== suppressed: 0 bytes in 0 blocks.
>
> > So the number are slightly worst.
>
> Hmm... strange. I wonder if the dummy integer was getting decrefed
> somewhere else. Also, how deterministic are these numbers?
The amount of memory leaked can vary a little, but the number of
blocks is usually constant.
>
> >>> Hopefully once we can compile integer.pyx with cleanup level 3
> >>> and not crash
>
> >> Try the below patch--this will prevent the double-free from the
> >> global dummy integer. There might be a better fix, as this will make
> >> it so that this integer object is never freed.
>
> > with the patch everything compiles and runs fine with cleanup level 3.
>
> Good, though it looks like not all issues are resolved (as seen below).
>
Well, it is certainly a step by step process and we are making
progress, so I am happy.
> >>> the amount of still reachable memory should decrease
> >>> tremendously because integer.pyx should just about be referenced in
> >>> every extension we have.
>
> >> It doesn't matter how many things reference integer.pyx, only how
> >> many things are referenced by integer.pyx (which isn't too much, but
> >> may include such things as element.pyx).
>
> > Ok, but I plan to actually play with this using only a couple small
> > modules independent of Sage in order to understand the problem better.
>
> That sound like a good idea.
Yep, all I now need is the time to actually do it.
> > The only valgrind complaint I get is the following:
>
> > ==3482== Invalid free() / delete / delete[]
> > ==3482== at 0x4A1B74A: free (vg_replace_malloc.c:320)
> > ==3482== by 0x43FE9A: dict_dealloc (dictobject.c:847)
> > ==3482== by 0x43FE9A: dict_dealloc (dictobject.c:847)
> > ==3482== by 0x499E5B: _PyImport_Fini (import.c:229)
> > ==3482== by 0x4A5D66: Py_Finalize (pythonrun.c:419)
> > ==3482== by 0x4A58AA: handle_system_exit (pythonrun.c:1616)
> > ==3482== by 0x4A5AA8: PyErr_PrintEx (pythonrun.c:1062)
> > ==3482== by 0x4A62B6: PyRun_SimpleFileExFlags (pythonrun.c:976)
> > ==3482== by 0x412319: Py_Main (main.c:134)
> > ==3482== by 0x4FD94C9: (below main) (in /lib/libc-2.3.6.so)
> > ==3482== Address 0xb2de9f8 is 32 bytes inside a block of size 80
> > alloc'd
> > ==3482== at 0x4A1BB35: malloc (vg_replace_malloc.c:207)
> > ==3482== by 0x4B0079: _PyObject_GC_Malloc (gcmodule.c:1321)
> > ==3482== by 0x454C29: PyType_GenericAlloc (typeobject.c:454)
> > ==3482== by 0x975D160:
> > __pyx_tp_new_4sage_9structure_7element_Element (element.c:14429)
> > ==3482== by 0xAC89AAA: __pyx_tp_new_4sage_5rings_7integer_Integer
> > (integer.c:14228)
> > ==3482== by 0x458D92: type_call (typeobject.c:422)
> > ==3482== by 0x415542: PyObject_Call (abstract.c:1860)
> > ==3482== by 0x47C480: PyEval_CallObjectWithKeywords (ceval.c:3433)
> > ==3482== by 0xAC85823: initinteger (integer.c:15100)
> > ==3482== by 0x49E17D: _PyImport_LoadDynamicModule (importdl.c:53)
> > ==3482== by 0x49C08D: import_submodule (import.c:2394)
> > ==3482== by 0x49C550: load_next (import.c:2214)
>
> This looks like it has something to do with the very first integer
> created, though I'm not totally sure how to read the log.
>
>
>
> > It might be related. I also get a lot of the following:
>
> > ==3482== 45 bytes in 1 blocks are definitely lost in loss record 525
> > of 10,255
> > ==3482== at 0x4A1BB35: malloc (vg_replace_malloc.c:207)
> > ==3482== by 0x44D68B: PyString_FromString (stringobject.c:130)
> > ==3482== by 0x13B4DF15: __Pyx_ImportType (real_double_vector.c:
> > 3786)
> > ==3482== by 0x13B52202: initreal_double_vector
> > (real_double_vector.c:3255)
> > ==3482== by 0x49E17D: _PyImport_LoadDynamicModule (importdl.c:53)
> > ==3482== by 0x49C08D: import_submodule (import.c:2394)
> > ==3482== by 0x49C550: load_next (import.c:2214)
> > ==3482== by 0x49C7AD: import_module_level (import.c:2002)
> > ==3482== by 0x49CBF4: PyImport_ImportModuleLevel (import.c:2066)
> > ==3482== by 0x47BEE8: builtin___import__ (bltinmodule.c:47)
> > ==3482== by 0x415542: PyObject_Call (abstract.c:1860)
> > ==3482== by 0x47C480: PyEval_CallObjectWithKeywords (ceval.c:3433)
>
> I see what's going wrong here, try applying the attached patch. This
> is probably all over the place.
>
Yep, the patch fixes that nicely. The valgrind log shrank from 8.2MB
to 5.8MB and we get:
==6724== LEAK SUMMARY:
==6724== definitely lost: 0 bytes in 0 blocks.
==6724== possibly lost: 266,534 bytes in 741 blocks.
==6724== still reachable: 29,157,606 bytes in 179,398 blocks.
==6724== suppressed: 0 bytes in 0 blocks.
So: very nice catch. I will try to run the code with a python using "--
with-pymalloc" and see what happens there.
> - Robert
>
Cheers,
Michael
> __Pyx_ImportModule-decref-fix.patch
> 1KDownload
>
>
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/
-~----------~----~----~----~------~----~------~--~---
