On Oct 28, 11:26 am, mabshoff <[EMAIL PROTECTED]
dortmund.de> wrote:
> On Oct 28, 10:33 am, Robert Bradshaw <[EMAIL PROTECTED]>
> wrote:
>
> > On Oct 28, 2007, at 2:17 AM, mabshoff wrote:
>
> Hi Robert,
>
>
>
>
>
> > > On Oct 28, 9:48 am, Robert Bradshaw <[EMAIL PROTECTED]>
> > > wrote:
> > >> On Oct 28, 2007, at 1:29 AM, mabshoff wrote:
>
> > > Hi Robert,
>
> > >>> For 2.8.10.alpha1+Craig's fix and python compiled using "--without-
> > >>> pymalloc" I get with the default cleanup level 1:
>
> > >>> ==6569== LEAK SUMMARY:
> > >>> ==6569== definitely lost: 181,030 bytes in 2,926 blocks.
> > >>> ==6569== possibly lost: 266,925 bytes in 747 blocks.
> > >>> ==6569== still reachable: 29,184,388 bytes in 179,707 blocks.
> > >>> ==6569== suppressed: 0 bytes in 0 blocks.
>
> > >>> With cleanup level 3 in general and cleanup level 1 just for
> > >>> integer.pyx I get
>
> > >>> ==23818== LEAK SUMMARY:
> > >>> ==23818== definitely lost: 181,117 bytes in 2,927 blocks.
> > >>> ==23818== possibly lost: 266,838 bytes in 746 blocks.
> > >>> ==23818== still reachable: 29,157,453 bytes in 179,383 blocks.
> > >>> ==23818== suppressed: 0 bytes in 0 blocks.
>
> > > With the patch valgrind says:
>
> > > ==3482== LEAK SUMMARY:
> > > ==3482== definitely lost: 181,235 bytes in 2,929 blocks.
> > > ==3482== possibly lost: 266,720 bytes in 744 blocks.
> > > ==3482== still reachable: 29,157,648 bytes in 179,395 blocks.
> > > ==3482== suppressed: 0 bytes in 0 blocks.
>
> > > So the number are slightly worst.
>
> > Hmm... strange. I wonder if the dummy integer was getting decrefed
> > somewhere else. Also, how deterministic are these numbers?
>
> The amount of memory leaked can vary a little, but the number of
> blocks is usually constant.
>
>
>
> > >>> Hopefully once we can compile integer.pyx with cleanup level 3
> > >>> and not crash
>
> > >> Try the below patch--this will prevent the double-free from the
> > >> global dummy integer. There might be a better fix, as this will make
> > >> it so that this integer object is never freed.
>
> > > with the patch everything compiles and runs fine with cleanup level 3.
>
> > Good, though it looks like not all issues are resolved (as seen below).
>
> Well, it is certainly a step by step process and we are making
> progress, so I am happy.
>
> > >>> the amount of still reachable memory should decrease
> > >>> tremendously because integer.pyx should just about be referenced in
> > >>> every extension we have.
>
> > >> It doesn't matter how many things reference integer.pyx, only how
> > >> many things are referenced by integer.pyx (which isn't too much, but
> > >> may include such things as element.pyx).
>
> > > Ok, but I plan to actually play with this using only a couple small
> > > modules independent of Sage in order to understand the problem better.
>
> > That sound like a good idea.
>
> Yep, all I now need is the time to actually do it.
>
>
>
> > > The only valgrind complaint I get is the following:
>
> > > ==3482== Invalid free() / delete / delete[]
> > > ==3482== at 0x4A1B74A: free (vg_replace_malloc.c:320)
> > > ==3482== by 0x43FE9A: dict_dealloc (dictobject.c:847)
> > > ==3482== by 0x43FE9A: dict_dealloc (dictobject.c:847)
> > > ==3482== by 0x499E5B: _PyImport_Fini (import.c:229)
> > > ==3482== by 0x4A5D66: Py_Finalize (pythonrun.c:419)
> > > ==3482== by 0x4A58AA: handle_system_exit (pythonrun.c:1616)
> > > ==3482== by 0x4A5AA8: PyErr_PrintEx (pythonrun.c:1062)
> > > ==3482== by 0x4A62B6: PyRun_SimpleFileExFlags (pythonrun.c:976)
> > > ==3482== by 0x412319: Py_Main (main.c:134)
> > > ==3482== by 0x4FD94C9: (below main) (in /lib/libc-2.3.6.so)
> > > ==3482== Address 0xb2de9f8 is 32 bytes inside a block of size 80
> > > alloc'd
> > > ==3482== at 0x4A1BB35: malloc (vg_replace_malloc.c:207)
> > > ==3482== by 0x4B0079: _PyObject_GC_Malloc (gcmodule.c:1321)
> > > ==3482== by 0x454C29: PyType_GenericAlloc (typeobject.c:454)
> > > ==3482== by 0x975D160:
> > > __pyx_tp_new_4sage_9structure_7element_Element (element.c:14429)
> > > ==3482== by 0xAC89AAA: __pyx_tp_new_4sage_5rings_7integer_Integer
> > > (integer.c:14228)
> > > ==3482== by 0x458D92: type_call (typeobject.c:422)
> > > ==3482== by 0x415542: PyObject_Call (abstract.c:1860)
> > > ==3482== by 0x47C480: PyEval_CallObjectWithKeywords (ceval.c:3433)
> > > ==3482== by 0xAC85823: initinteger (integer.c:15100)
> > > ==3482== by 0x49E17D: _PyImport_LoadDynamicModule (importdl.c:53)
> > > ==3482== by 0x49C08D: import_submodule (import.c:2394)
> > > ==3482== by 0x49C550: load_next (import.c:2214)
>
> > This looks like it has something to do with the very first integer
> > created, though I'm not totally sure how to read the log.
>
> > > It might be related. I also get a lot of the following:
>
> > > ==3482== 45 bytes in 1 blocks are definitely lost in loss record 525
> > > of 10,255
> > > ==3482== at 0x4A1BB35: malloc (vg_replace_malloc.c:207)
> > > ==3482== by 0x44D68B: PyString_FromString (stringobject.c:130)
> > > ==3482== by 0x13B4DF15: __Pyx_ImportType (real_double_vector.c:
> > > 3786)
> > > ==3482== by 0x13B52202: initreal_double_vector
> > > (real_double_vector.c:3255)
> > > ==3482== by 0x49E17D: _PyImport_LoadDynamicModule (importdl.c:53)
> > > ==3482== by 0x49C08D: import_submodule (import.c:2394)
> > > ==3482== by 0x49C550: load_next (import.c:2214)
> > > ==3482== by 0x49C7AD: import_module_level (import.c:2002)
> > > ==3482== by 0x49CBF4: PyImport_ImportModuleLevel (import.c:2066)
> > > ==3482== by 0x47BEE8: builtin___import__ (bltinmodule.c:47)
> > > ==3482== by 0x415542: PyObject_Call (abstract.c:1860)
> > > ==3482== by 0x47C480: PyEval_CallObjectWithKeywords (ceval.c:3433)
>
> > I see what's going wrong here, try applying the attached patch. This
> > is probably all over the place.
>
> Yep, the patch fixes that nicely. The valgrind log shrank from 8.2MB
> to 5.8MB and we get:
>
> ==6724== LEAK SUMMARY:
> ==6724== definitely lost: 0 bytes in 0 blocks.
> ==6724== possibly lost: 266,534 bytes in 741 blocks.
> ==6724== still reachable: 29,157,606 bytes in 179,398 blocks.
> ==6724== suppressed: 0 bytes in 0 blocks.
>
> So: very nice catch. I will try to run the code with a python using "--
> with-pymalloc" and see what happens there.
>
Okay, this doesn't go too well:
----------------------------------------------------------------------
| SAGE Version 2.8.10.alpha1, Release Date: 2007-10-27 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage:
Exiting SAGE (CPU time 0m0.05s, Wall time 0m1.51s).
*** glibc detected *** free(): invalid pointer: 0x00002b21905bd6e0 ***
Program received signal SIGABRT, Aborted.
[Switching to Thread 47423054237536 (LWP 29300)]
0x00002b218a2c607b in raise () from /lib/libc.so.6
(gdb) bt
#0 0x00002b218a2c607b in raise () from /lib/libc.so.6
#1 0x00002b218a2c784e in abort () from /lib/libc.so.6
#2 0x00002b218a2fc6e9 in __libc_message () from /lib/libc.so.6
#3 0x00002b218a303253 in _int_free () from /lib/libc.so.6
#4 0x00002b218a3032de in free () from /lib/libc.so.6
#5 0x000000000043fc2b in dict_dealloc (mp=0x9a1c20) at Objects/
dictobject.c:847
#6 0x000000000043fc2b in dict_dealloc (mp=0x65b160) at Objects/
dictobject.c:847
#7 0x000000000049a22c in _PyImport_Fini () at Python/import.c:229
#8 0x00000000004a6137 in Py_Finalize () at Python/pythonrun.c:419
#9 0x00000000004a5c7b in handle_system_exit () at Python/pythonrun.c:
1616
#10 0x00000000004a5e79 in PyErr_PrintEx (set_sys_last_vars=1) at
Python/pythonrun.c:1062
#11 0x00000000004a6687 in PyRun_SimpleFileExFlags (fp=0x0,
filename=<value optimized out>, closeit=0, flags=0x7fff20dc5d60)
at Python/pythonrun.c:976
#12 0x000000000041231a in Py_Main (argc=<value optimized out>,
argv=<value optimized out>) at Modules/main.c:134
#13 0x00002b218a2b34ca in __libc_start_main () from /lib/libc.so.6
#14 0x000000000041163a in _start () at ../sysdeps/x86_64/elf/start.S:
113
I am recompiling with lower cleanup level and see if it makes a
difference.
> > - Robert
Cheers,
Michael
> > __Pyx_ImportModule-decref-fix.patch
> > 1KDownload
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/
-~----------~----~----~----~------~----~------~--~---
