On Aug 29, 4:22 pm, "Joel B. Mohler" <[EMAIL PROTECTED]> wrote:
> On Wednesday 29 August 2007 07:19, mabshoff wrote:
>
> > ==22784== 791,674 bytes in 65,421 blocks are definitely lost in loss
> > record 2,472 of 2,481
> > ==22784== at 0x4A05CB9: operator new[](unsigned long)
> > (vg_replace_malloc.c:199)
> > ==22784== by 0x9280247: ZZ_pX_repr (in /tmp/Work2/sage-2.8.3.alpha2/
> > local/lib/libcsage.so.0.0.0)
>
> I think you also have a similar memory leak in number fields. I'm going to
> fix that one as soon as the c_lib gets integrated in to the main tree and the
> new code for the sage c_lib gets added which provides support for the proper
> fix.
I think that there is lots of the above: In devel/sage do a
[EMAIL PROTECTED] sage]$ grep -r "return\ str(" * | grep --invert-match
".c" | wc -l
62
To be exact (the 2 I "fixed" badly in the ntl wrapper removed"):
build/sage/matrix/strassen.pyx: return str(self._intervals)
build/sage/ext/multi_modular.pyx: return str(type(self))
+str(self.list())
build/sage/groups/perm_gps/permgroup_element.py: return
str(self)
build/sage/groups/matrix_gps/matrix_group_element.py: return
str(self.__mat)
build/sage/algebras/algebra_order_element.py: return
str(self.__ambient_algebra_element)
build/sage/libs/pari/gen.pyx: return
str(type_name(typ(self.g)))
build/sage/libs/pari/gen.pyx: return str(PARIVERSION)
build/sage/modular/modform/element.py: return
str(self.q_expansion())
build/sage/modular/modsym/element.py: return
str(self.element())
build/sage/gsl/interpolation.pyx: return str(self.v)
build/sage/gsl/fft.pyx: return str(list(self))
build/sage/gsl/gsl_array.pyx: return str(list(self))
build/sage/rings/pari_ring.py: return str(self.__x)
build/sage/rings/real_double.pyx: return str(self._value)
build/sage/rings/integer.pyx: return str(self)
build/sage/rings/real_mpfi.pyx: return str(self)
build/sage/rings/integer_mod.pyx: return str(self.lift())
build/sage/rings/integer_mod.pyx: return str(self)
build/sage/rings/finite_field_givaro.pyx: return
str(int(e.element))
build/sage/rings/finite_field_givaro.pyx: return str(int(e))
build/sage/rings/finite_field_givaro.pyx: return str(self)
build/sage/rings/sparse_poly.pyx: return str(self.list())
build/sage/rings/finite_field_element.py: return
str(self.__value)
build/sage/rings/rational.pyx: return str(self.numer())
build/sage/rings/quotient_ring_element.py: return
str(self.__rep)
build/lib.linux-x86_64-2.5/sage/groups/perm_gps/
permgroup_element.py: return str(self)
build/lib.linux-x86_64-2.5/sage/groups/matrix_gps/
matrix_group_element.py: return str(self.__mat)
build/lib.linux-x86_64-2.5/sage/algebras/
algebra_order_element.py: return
str(self.__ambient_algebra_element)
build/lib.linux-x86_64-2.5/sage/modular/modform/element.py:
return str(self.q_expansion())
build/lib.linux-x86_64-2.5/sage/modular/modsym/element.py:
return str(self.element())
build/lib.linux-x86_64-2.5/sage/rings/pari_ring.py: return
str(self.__x)
build/lib.linux-x86_64-2.5/sage/rings/finite_field_element.py:
return str(self.__value)
build/lib.linux-x86_64-2.5/sage/rings/
quotient_ring_element.py: return str(self.__rep)
sage/matrix/strassen.pyx: return str(self._intervals)
sage/ext/multi_modular.pyx: return str(type(self))
+str(self.list())
sage/groups/perm_gps/permgroup_element.py: return str(self)
sage/groups/matrix_gps/matrix_group_element.py: return
str(self.__mat)
sage/algebras/algebra_order_element.py: return
str(self.__ambient_algebra_element)
sage/libs/pari/_py_pari_orig.pyx: return str(PARIVERSION)
sage/libs/pari/_py_pari_orig.pyx: return
str(type_name(typ(self.g)))
sage/libs/pari/gen.pyx: return str(type_name(typ(self.g)))
sage/libs/pari/gen.pyx: return str(PARIVERSION)
sage/modular/modform/element.py: return str(self.q_expansion())
sage/modular/modsym/element.py: return str(self.element())
sage/gsl/interpolation.pyx: return str(self.v)
sage/gsl/fft.pyx: return str(list(self))
sage/gsl/gsl_array.pyx: return str(list(self))
sage/rings/pari_ring.py: return str(self.__x)
sage/rings/real_double.pyx: return str(self._value)
sage/rings/integer.pyx: return str(self)
sage/rings/real_mpfi.pyx: return str(self)
sage/rings/integer_mod.pyx: return str(self.lift())
sage/rings/integer_mod.pyx: return str(self)
sage/rings/finite_field_givaro.pyx: return str(int(e.element))
sage/rings/finite_field_givaro.pyx: return str(int(e))
sage/rings/finite_field_givaro.pyx: return str(self)
sage/rings/sparse_poly.pyx: return str(self.list())
sage/rings/finite_field_element.py: return str(self.__value)
sage/rings/rational.pyx: return str(self.numer())
sage/rings/quotient_ring_element.py: return str(self.__rep)
I believe that all of the above ought to be audited for memory leaks.
It is a pattern that got copy & pasted all over the code and while any
of those failure usually leak only a couple of bytes the example above
showed that those leaks add up quickly. I believe that an OpenBSD
style "find the issue, then exterminate the issue across the whole
tree" approach is the right way to tackle this. I consider it too late
for 2.8.3, but hopefully some time can be spend on this before 2.9
even if one or two new features don't make it into the tree. Stability
should always come before new features. It might be painful in the
short term.
Besides the "How to write fast Cython guide" we also need something
like "How to avoid memory leaks in Cython (with illustrated
examples ;))". Also code review should consider potential leaks and 64
bit portability. I am looking at some pointer issue in the m4ri
library, but nothing definitive there, yet.
I already have a similar issue about dictionaries/hash tables in my
cross hairs, but I do not understand the issue 100% yet, so stay tuned
for that.
>
> The trac for the c_lib integration
> ishttp://trac.sagemath.org/sage_trac/ticket/411
> Perhaps this should be in milestone 2.8.3
>
That definitely cuts it pretty tight, I expect that William just wants
to get the fixed and a couple minor annoyances fixed and then 2.8.3 is
done. I am not even sure the ntl leak fix will make it because it is
rather ugly. But hopefully the c_lib integration can go in right
afterwards for 2.9 (before bug day 2 that is).
> --
> Joel
Cheers,
Michael
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/
-~----------~----~----~----~------~----~------~--~---
