Hello,
I did some valgrinding of the following code:
def Supercomp():
p=ZZ(10**4).next_prime()
szfilename = "timings100k.txt"
mem_szfilename = "memory100k.txt"
for I in range(5):
t = cputime()
M = get_memory_usage()
X = SupersingularModule(p)
X.hecke_matrix(2)
f = open(szfilename, 'a')
f.write(str([p, cputime(t)]) + ", ")
f.close()
g = open(mem_szfilename, 'a')
g.write(str([p, get_memory_usage()-M]) + ", ")
g.close()
X.save('X' + str(p))
p = ZZ(p).next_prime()
Supercomp()
I reduced the number of rounds to 5 and also lowered the starting
prime by an order of magnitude.
For the full log see
http://sage.math.washington.edu/home/mabshoff/sage-2.8.2-ticket_501.valgrind
(2.1MB)
Overall:
==18345== LEAK SUMMARY:
==18345== definitely lost: 794,602 bytes in 65,456 blocks.
==18345== possibly lost: 329,752 bytes in 816 blocks.
==18345== still reachable: 138,195,057 bytes in 19,586 blocks.
==18345== suppressed: 0 bytes in 0 blocks.
Interesting bits:
Trouble in the NTL wrapper:
==18345== Mismatched free() / delete / delete []
==18345== at 0x4A0541E: free (vg_replace_malloc.c:233)
==18345== by 0x176C71E3: __pyx_f_3ntl_string (ntl.c:908)
==18345== by 0x176C9031: __pyx_f_3ntl_9ntl_ZZ_pX___getitem__ (ntl.c:
6514)
==18345== by 0x481188: PyEval_EvalFrameEx (ceval.c:1192)
==18345== by 0x4865EF: PyEval_EvalCodeEx (ceval.c:2831)
==18345== by 0x4CFF37: function_call (funcobject.c:517)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x41BE0C: instancemethod_call (classobject.c:2497)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x461F33: call_method (typeobject.c:929)
==18345== by 0x186FC84C:
__pyx_f_18polynomial_element_10Polynomial___hash__
(polynomial_element.c:3141)
==18345== by 0x47BB7B: builtin_hash (bltinmodule.c:1039)
Another one: buffer too small?
==18345== Address 0x4C89020 is 4 bytes after a block of size 12
alloc'd
==18345== at 0x4A05CB9: operator new[](unsigned long)
(vg_replace_malloc.c:199)
==18345== by 0x9282247: ZZ_pX_repr (in /tmp/Work2/sage-2.8.2/local/
lib/libcsage.so.0.0.0)
==18345== by 0x176CAD57: __pyx_f_3ntl_9ntl_ZZ_pX___repr__ (ntl.c:
6216)
==18345== by 0x443C61: _PyObject_Str (object.c:406)
==18345== by 0x443D0A: PyObject_Str (object.c:426)
==18345== by 0x44EA8F: string_new (stringobject.c:3892)
==18345== by 0x45A272: type_call (typeobject.c:422)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x480783: PyEval_EvalFrameEx (ceval.c:3775)
==18345== by 0x485025: PyEval_EvalFrameEx (ceval.c:3650)
==18345== by 0x4865EF: PyEval_EvalCodeEx (ceval.c:2831)
==18345== by 0x4CFF37: function_call (funcobject.c:517)
Handling of ZZs in Cython:
==18345== 791,438 bytes in 65,402 blocks are definitely lost in loss
record 2,401 of 2,410
==18345== at 0x4A05CB9: operator new[](unsigned long)
(vg_replace_malloc.c:199)
==18345== by 0x9282247: ZZ_pX_repr (in /tmp/Work2/sage-2.8.2/local/
lib/libcsage.so.0.0.0)
==18345== by 0x176CAD57: __pyx_f_3ntl_9ntl_ZZ_pX___repr__ (ntl.c:
6216)
==18345== by 0x443C61: _PyObject_Str (object.c:406)
==18345== by 0x443D0A: PyObject_Str (object.c:426)
==18345== by 0x44EA8F: string_new (stringobject.c:3892)
==18345== by 0x45A272: type_call (typeobject.c:422)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x480783: PyEval_EvalFrameEx (ceval.c:3775)
==18345== by 0x485025: PyEval_EvalFrameEx (ceval.c:3650)
==18345== by 0x4865EF: PyEval_EvalCodeEx (ceval.c:2831)
==18345== by 0x4CFF37: function_call (funcobject.c:517)
libpari, this one happens once during each round:
==18345== 524,288 bytes in 1 blocks are still reachable in loss record
2,391 of 2,410
==18345== at 0x4A05809: malloc (vg_replace_malloc.c:149)
==18345== by 0xF697B2A: gpmalloc (in /tmp/Work2/sage-2.8.2/local/
lib/libpari-gmp.so.2)
==18345== by 0xF698BF6: pari_init_opts (in /tmp/Work2/sage-2.8.2/
local/lib/libpari-gmp.so.2)
==18345== by 0xFC0CC31: __pyx_f_3gen_12PariInstance___init__ (gen.c:
20390)
==18345== by 0x45A321: type_call (typeobject.c:436)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x47DB71: PyEval_CallObjectWithKeywords (ceval.c:3433)
==18345== by 0xFC0EFA8: initgen (gen.c:27086)
==18345== by 0x49F762: _PyImport_LoadDynamicModule (importdl.c:53)
==18345== by 0x49D63E: import_submodule (import.c:2394)
==18345== by 0x49DB11: load_next (import.c:2214)
==18345== by 0x49DD6E: import_module_level (import.c:2002)
Another common problem is that we do not release dictionaries (there
are easily 50 of those - I didn't count) - I am not sure if we can do
that or if that is a problem in python itself:
==18345== 3,072 bytes in 1 blocks are still reachable in loss record
2,054 of 2,410
==18345== at 0x4A05809: malloc (vg_replace_malloc.c:149)
==18345== by 0x43F179: dictresize (dictobject.c:513)
==18345== by 0x4409DA: PyDict_SetItemString (dictobject.c:2178)
==18345== by 0x45C8DE: PyType_Ready (typeobject.c:4044)
==18345== by 0x1C89C312: initpower_series_poly (power_series_poly.c:
3399)
==18345== by 0x49F762: _PyImport_LoadDynamicModule (importdl.c:53)
==18345== by 0x49D63E: import_submodule (import.c:2394)
==18345== by 0x49DB11: load_next (import.c:2214)
==18345== by 0x49DD33: import_module_level (import.c:1995)
==18345== by 0x49E1A4: PyImport_ImportModuleLevel (import.c:2066)
==18345== by 0x47D5D8: builtin___import__ (bltinmodule.c:47)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
I haven't seen the following one before:
==18345== 524,280 bytes in 16 blocks are still reachable in loss
record 2,383 of 2,410
==18345== at 0x4A04B32: calloc (vg_replace_malloc.c:279)
==18345== by 0x1FD9417D: buildAllCodes (grayflex.c:90)
==18345== by 0x1FD8FFC2: initmatrix_mod2_dense (matrix_mod2_dense.c:
800)
==18345== by 0x49F762: _PyImport_LoadDynamicModule (importdl.c:53)
==18345== by 0x49D63E: import_submodule (import.c:2394)
==18345== by 0x49DB11: load_next (import.c:2214)
==18345== by 0x49DD33: import_module_level (import.c:1995)
==18345== by 0x49E1A4: PyImport_ImportModuleLevel (import.c:2066)
==18345== by 0x47D5D8: builtin___import__ (bltinmodule.c:47)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x47DB71: PyEval_CallObjectWithKeywords (ceval.c:3433)
==18345== by 0x480BD3: PyEval_EvalFrameEx (ceval.c:2063)
I don't have a clue about the following, but it seems huge:
==18345== 524,288 bytes in 2 blocks are still reachable in loss record
2,385 of 2,410
==18345== at 0x4A05809: malloc (vg_replace_malloc.c:149)
==18345== by 0x445FBC: PyObject_Malloc (obmalloc.c:563)
==18345== by 0x4B1938: _PyObject_GC_NewVar (gcmodule.c:1322)
==18345== by 0x4540F4: PyTuple_New (tupleobject.c:68)
==18345== by 0x16781AF8: __pyx_f_7integer_7Integer___reduce__
(integer.c:2905)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0x47DB71: PyEval_CallObjectWithKeywords (ceval.c:3433)
==18345== by 0x459AB1: object_reduce_ex (typeobject.c:2786)
==18345== by 0x4156A2: PyObject_Call (abstract.c:1860)
==18345== by 0xAFF62C6: save (cPickle.c:2495)
==18345== by 0xAFF7937: batch_dict (cPickle.c:1712)
==18345== by 0xAFF6DDB: save (cPickle.c:1782)
Some issue libSingular (there are about 5 or 6 of those) - this is
caused by static initialization:
==18345== 600 bytes in 10 blocks are possibly lost in loss record
1,542 of 2,410
==18345== at 0x4A05CB9: operator new[](unsigned long)
(vg_replace_malloc.c:199)
==18345== by 0x1BF554B7:
Array<CanonicalForm>::operator=(Array<CanonicalForm> const&) (in /tmp/
Work2/sage-2.8.2/local/lib
/libsingular.so)
==18345== by 0x1BC5B032: initPT() (in /tmp/Work2/sage-2.8.2/devel/
sage-main/build/sage/rings/polynomial/multi_polynomial_
libsingular.so)
==18345== by 0x1BC553C4: initCanonicalForm() (in /tmp/Work2/
sage-2.8.2/devel/sage-main/build/sage/rings/polynomial/multi_
polynomial_libsingular.so)
==18345== by 0x1C1137E4:
__static_initialization_and_destruction_0(int, int) (in /tmp/Work2/
sage-2.8.2/local/lib/libsingu
lar.so)
==18345== by 0x1C11B195: (within /tmp/Work2/sage-2.8.2/local/lib/
libsingular.so)
==18345== by 0x1BF313AA: (within /tmp/Work2/sage-2.8.2/local/lib/
libsingular.so)
And two of the following:
==18345== 65,534 bytes in 1 blocks are still reachable in loss record
2,323 of 2,410
==18345== at 0x4A05CB9: operator new[](unsigned long)
(vg_replace_malloc.c:199)
==18345== by 0x1BC7E154:
__static_initialization_and_destruction_0(int, int) (in /tmp/Work2/
sage-2.8.2/devel/sage-main/bu
ild/sage/rings/polynomial/multi_polynomial_libsingular.so)
==18345== by 0x1BC956E5: (within /tmp/Work2/sage-2.8.2/devel/sage-
main/build/sage/rings/polynomial/multi_polynomial_libsi
ngular.so)
==18345== by 0x1BC1F26A: (within /tmp/Work2/sage-2.8.2/devel/sage-
main/build/sage/rings/polynomial/multi_polynomial_libsi
ngular.so)
Cheers,
Michael
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/
-~----------~----~----~----~------~----~------~--~---
