[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
On Wednesday, November 14, 2012 8:58:45 PM UTC-6, Jean-Pierre Flori wrote:
>
>
>
> On Wednesday, November 14, 2012 5:42:27 PM UTC-6, Nils Bruin wrote:
>>
>> However, in an effort to make memory errors during testing a little
>> more reproducible I made this little edit to local/bin/sagedoctest.py
>> to ensure the garbage collector is run before every doctested line:
>>
>>
>> diff --git a/sagedoctest.py b/sagedoctest.py
>> --- a/sagedoctest.py
>> +++ b/sagedoctest.py
>> @@ -1,7 +1,9 @@
>> from __future__ import with_statement
>>
>> import ncadoctest
>> +import gc
>> import sage.misc.randstate as randstate
>> +import sys
>>
>> OrigDocTestRunner = ncadoctest.DocTestRunner
>> class SageDocTestRunner(OrigDocTestRunner):
>> @@ -35,6 +37,8 @@ class SageDocTestRunner(OrigDocTestRunne
>> except Exception, e:
>> self._timeit_stats[key] = e
>> # otherwise, just run the example
>> +sys.stderr.write('testing example %s\n'%example)
>> +gc.collect()
>> OrigDocTestRunner.run_one_example(self, test, example,
>> filename, compileflags)
>>
>> def save_timeit_stats_to_file_named(self, output_filename):
>>
>>
>> (i.e., just add a gc.collect() to run_one_example)
>>
>> and it causes a reliable failure in crypto/mq/mpolynomialsystem.py:
>>
>> Trying:
>> C[Integer(0)].groebner_basis()###line 84:_sage_sage:
>> C[0].groebner_basis()
>> Expecting:
>> Polynomial Sequence with 26 Polynomials in 16 Variables
>> testing example
>> ok
>> Trying:
>> A,v = mq.MPolynomialSystem(r2).coefficient_matrix()###line
>> 87:_sage_sage: A,v = mq.MPolynomialSystem(r2).coefficient_matrix()
>> Expecting nothing
>> testing example
>> *** glibc detected *** python: double free or corruption (out):
>> 0x075c58c0 ***
>> === Backtrace: =
>> /lib64/libc.so.6[0x31cfe7da76]
>> /lib64/libc.so.6[0x31cfe7ed5e]
>> /usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
>> polynomial/pbori.so(+0x880aa)[0x7fa5eba7e0aa]
>> /usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
>> polynomial/pbori.so(+0x1d993)[0x7fa5eba13993]
>> ...
>>
>> Running it under sage -t --gdb gives:
>>
>> (gdb) bt
>> #0 0x0031cfe36285 in raise () from /lib64/libc.so.6
>> #1 0x0031cfe37b9b in abort () from /lib64/libc.so.6
>> #2 0x0031cfe7774e in __libc_message () from /lib64/libc.so.6
>> #3 0x0031cfe7da76 in malloc_printerr () from /lib64/libc.so.6
>> #4 0x0031cfe7ed5e in _int_free () from /lib64/libc.so.6
>> #5 0x7fffce5cb0aa in
>> Delete (mem=0x547db30)
>> at /usr/local/sage/5.5b2/local/include/csage/ccobject.h:77
>> #6
>> __pyx_pf_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_2__dealloc__
>>
>> (__pyx_v_self=)
>> at sage/rings/polynomial/pbori.cpp:37868
>> #7
>> __pyx_pw_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_3__dealloc__
>>
>> (__pyx_v_self=0x54bf390)
>> at sage/rings/polynomial/pbori.cpp:37834
>> #8
>> __pyx_tp_dealloc_4sage_5rings_10polynomial_5pbori_ReductionStrategy
>> (o=0x54bf390) at sage/rings/polynomial/pbori.cpp:52283
>> #9 0x7fffce560993 in
>> __pyx_tp_clear_4sage_5rings_10polynomial_5pbori_GroebnerStrategy
>> (o=0x54baeb0)
>> at sage/rings/polynomial/pbori.cpp:52545
>> #10 0x77d4b637 in delete_garbage (old=0x77fe19e0,
>> collectable=0x7fffbb60) at Modules/gcmodule.c:769
>> #11 collect (generation=2) at Modules/gcmodule.c:930
>> #12 0x77d4bdc9 in gc_collect (self=,
>> args=, kws=) at Modules/gcmodule.c:1067
>>
>> which should give a pretty good pointer for pbori people to figure out
>> which memory deallocation is actually botched.
>>
> We dealt with something very similar in one of the "memleaks" tickets.
> Not sure it was 715 or 11521, but maybe 12313 (the figures here might be
> wrong...).
> So the fix is potentially not included in 5.5.beta2 if it was in the later.
>
>
Ok, I took the time to check and you actually posted in 13710 that the fix
is included in 12313, so not in 5.5.beta2 if I'm not wrong (nor 5.0 of
course).
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To post to this group, send email to sage-devel@googlegroups.com.
To unsubscribe from this group, send email to
sage-devel+unsubscr...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
On Wednesday, November 14, 2012 5:42:27 PM UTC-6, Nils Bruin wrote:
>
> However, in an effort to make memory errors during testing a little
> more reproducible I made this little edit to local/bin/sagedoctest.py
> to ensure the garbage collector is run before every doctested line:
>
>
> diff --git a/sagedoctest.py b/sagedoctest.py
> --- a/sagedoctest.py
> +++ b/sagedoctest.py
> @@ -1,7 +1,9 @@
> from __future__ import with_statement
>
> import ncadoctest
> +import gc
> import sage.misc.randstate as randstate
> +import sys
>
> OrigDocTestRunner = ncadoctest.DocTestRunner
> class SageDocTestRunner(OrigDocTestRunner):
> @@ -35,6 +37,8 @@ class SageDocTestRunner(OrigDocTestRunne
> except Exception, e:
> self._timeit_stats[key] = e
> # otherwise, just run the example
> +sys.stderr.write('testing example %s\n'%example)
> +gc.collect()
> OrigDocTestRunner.run_one_example(self, test, example,
> filename, compileflags)
>
> def save_timeit_stats_to_file_named(self, output_filename):
>
>
> (i.e., just add a gc.collect() to run_one_example)
>
> and it causes a reliable failure in crypto/mq/mpolynomialsystem.py:
>
> Trying:
> C[Integer(0)].groebner_basis()###line 84:_sage_sage:
> C[0].groebner_basis()
> Expecting:
> Polynomial Sequence with 26 Polynomials in 16 Variables
> testing example
> ok
> Trying:
> A,v = mq.MPolynomialSystem(r2).coefficient_matrix()###line
> 87:_sage_sage: A,v = mq.MPolynomialSystem(r2).coefficient_matrix()
> Expecting nothing
> testing example
> *** glibc detected *** python: double free or corruption (out):
> 0x075c58c0 ***
> === Backtrace: =
> /lib64/libc.so.6[0x31cfe7da76]
> /lib64/libc.so.6[0x31cfe7ed5e]
> /usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
> polynomial/pbori.so(+0x880aa)[0x7fa5eba7e0aa]
> /usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
> polynomial/pbori.so(+0x1d993)[0x7fa5eba13993]
> ...
>
> Running it under sage -t --gdb gives:
>
> (gdb) bt
> #0 0x0031cfe36285 in raise () from /lib64/libc.so.6
> #1 0x0031cfe37b9b in abort () from /lib64/libc.so.6
> #2 0x0031cfe7774e in __libc_message () from /lib64/libc.so.6
> #3 0x0031cfe7da76 in malloc_printerr () from /lib64/libc.so.6
> #4 0x0031cfe7ed5e in _int_free () from /lib64/libc.so.6
> #5 0x7fffce5cb0aa in
> Delete (mem=0x547db30)
> at /usr/local/sage/5.5b2/local/include/csage/ccobject.h:77
> #6
> __pyx_pf_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_2__dealloc__
> (__pyx_v_self=)
> at sage/rings/polynomial/pbori.cpp:37868
> #7
> __pyx_pw_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_3__dealloc__
> (__pyx_v_self=0x54bf390)
> at sage/rings/polynomial/pbori.cpp:37834
> #8
> __pyx_tp_dealloc_4sage_5rings_10polynomial_5pbori_ReductionStrategy
> (o=0x54bf390) at sage/rings/polynomial/pbori.cpp:52283
> #9 0x7fffce560993 in
> __pyx_tp_clear_4sage_5rings_10polynomial_5pbori_GroebnerStrategy
> (o=0x54baeb0)
> at sage/rings/polynomial/pbori.cpp:52545
> #10 0x77d4b637 in delete_garbage (old=0x77fe19e0,
> collectable=0x7fffbb60) at Modules/gcmodule.c:769
> #11 collect (generation=2) at Modules/gcmodule.c:930
> #12 0x77d4bdc9 in gc_collect (self=,
> args=, kws=) at Modules/gcmodule.c:1067
>
> which should give a pretty good pointer for pbori people to figure out
> which memory deallocation is actually botched.
>
We dealt with something very similar in one of the "memleaks" tickets.
Not sure it was 715 or 11521, but maybe 12313 (the figures here might be
wrong...).
So the fix is potentially not included in 5.5.beta2 if it was in the later.
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To post to this group, send email to sage-devel@googlegroups.com.
To unsubscribe from this group, send email to
sage-devel+unsubscr...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
Other consequences from gc.collect() insertions: sage -t -force_lib devel/sage/sage/crypto/mq/mpolynomialsystem.py # Killed/crashed sage -t -force_lib devel/sage/sage/rings/polynomial/ multi_polynomial_sequence.py # Killed/crashed (same problem; reported as above) ** File "/usr/local/sage/5.5b2/devel/sage/sage/modular/abvar/ abvar_ambient_jacobian.py", line 345: sage: J0(33).decomposition(simple=False) Expected: [ Abelian subvariety of dimension 2 of J0(33), Simple abelian subvariety 33a(None,33) of dimension 1 of J0(33) ] Got: [ Abelian subvariety of dimension 2 of J0(33), Abelian subvariety of dimension 1 of J0(33) ] ** sage -t -force_lib devel/sage/sage/modular/abvar/ abvar_ambient_jacobian.py # 1 doctests failed (i.e., doctest is relying on a previous copy of 33a remaining in memory on which additional computations have changed the way it prints. That's a violation of immutability anyway and the doctest shouldn't rely on such behaviour) ** File "/usr/local/sage/5.5b2/devel/sage/sage/modular/abvar/abvar.py", line 2840: sage: J0(33).is_simple(none_if_not_known=True) Expected: False Got nothing ** sage -t -force_lib devel/sage/sage/modular/abvar/abvar.py # 1 doctests failed Same problem! Since J0(33) is freshly constructed, one should not rely on anything being cached on it and the test explicitly asks to not compute anything. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
: This is actually reproducible in plain 5.0. This is now http://trac.sagemath.org/sage_trac/ticket/13710 -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
However, in an effort to make memory errors during testing a little
more reproducible I made this little edit to local/bin/sagedoctest.py
to ensure the garbage collector is run before every doctested line:
diff --git a/sagedoctest.py b/sagedoctest.py
--- a/sagedoctest.py
+++ b/sagedoctest.py
@@ -1,7 +1,9 @@
from __future__ import with_statement
import ncadoctest
+import gc
import sage.misc.randstate as randstate
+import sys
OrigDocTestRunner = ncadoctest.DocTestRunner
class SageDocTestRunner(OrigDocTestRunner):
@@ -35,6 +37,8 @@ class SageDocTestRunner(OrigDocTestRunne
except Exception, e:
self._timeit_stats[key] = e
# otherwise, just run the example
+sys.stderr.write('testing example %s\n'%example)
+gc.collect()
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
def save_timeit_stats_to_file_named(self, output_filename):
(i.e., just add a gc.collect() to run_one_example)
and it causes a reliable failure in crypto/mq/mpolynomialsystem.py:
Trying:
C[Integer(0)].groebner_basis()###line 84:_sage_sage:
C[0].groebner_basis()
Expecting:
Polynomial Sequence with 26 Polynomials in 16 Variables
testing example
ok
Trying:
A,v = mq.MPolynomialSystem(r2).coefficient_matrix()###line
87:_sage_sage: A,v = mq.MPolynomialSystem(r2).coefficient_matrix()
Expecting nothing
testing example
*** glibc detected *** python: double free or corruption (out):
0x075c58c0 ***
=== Backtrace: =
/lib64/libc.so.6[0x31cfe7da76]
/lib64/libc.so.6[0x31cfe7ed5e]
/usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
polynomial/pbori.so(+0x880aa)[0x7fa5eba7e0aa]
/usr/local/sage/5.5b2/local/lib/python/site-packages/sage/rings/
polynomial/pbori.so(+0x1d993)[0x7fa5eba13993]
...
Running it under sage -t --gdb gives:
(gdb) bt
#0 0x0031cfe36285 in raise () from /lib64/libc.so.6
#1 0x0031cfe37b9b in abort () from /lib64/libc.so.6
#2 0x0031cfe7774e in __libc_message () from /lib64/libc.so.6
#3 0x0031cfe7da76 in malloc_printerr () from /lib64/libc.so.6
#4 0x0031cfe7ed5e in _int_free () from /lib64/libc.so.6
#5 0x7fffce5cb0aa in
Delete (mem=0x547db30)
at /usr/local/sage/5.5b2/local/include/csage/ccobject.h:77
#6
__pyx_pf_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_2__dealloc__
(__pyx_v_self=)
at sage/rings/polynomial/pbori.cpp:37868
#7
__pyx_pw_4sage_5rings_10polynomial_5pbori_17ReductionStrategy_3__dealloc__
(__pyx_v_self=0x54bf390)
at sage/rings/polynomial/pbori.cpp:37834
#8
__pyx_tp_dealloc_4sage_5rings_10polynomial_5pbori_ReductionStrategy
(o=0x54bf390) at sage/rings/polynomial/pbori.cpp:52283
#9 0x7fffce560993 in
__pyx_tp_clear_4sage_5rings_10polynomial_5pbori_GroebnerStrategy
(o=0x54baeb0)
at sage/rings/polynomial/pbori.cpp:52545
#10 0x77d4b637 in delete_garbage (old=0x77fe19e0,
collectable=0x7fffbb60) at Modules/gcmodule.c:769
#11 collect (generation=2) at Modules/gcmodule.c:930
#12 0x77d4bdc9 in gc_collect (self=,
args=, kws=) at Modules/gcmodule.c:1067
which should give a pretty good pointer for pbori people to figure out
which memory deallocation is actually botched.
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To post to this group, send email to sage-devel@googlegroups.com.
To unsubscribe from this group, send email to
sage-devel+unsubscr...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Access to SPARC 64
Hi, I am a release manager for the next NumPy release (1.7) and we have one bug that only shows on SPARC 64, see below for details. Is there anyone who would be willing and able to give me an access to such a machine so that I can debug it? Jason recommended to ask on the Sage list. I am BCCing the NumPy list. Any help would be greatly appreciated. Thanks, Ondrej On Thu, Aug 30, 2012 at 8:21 PM, Jason Grout wrote: > On 8/30/12 10:10 PM, Ondřej Čertík wrote: >> Hi, >> >> Does anyone have a SPARC 64 machine that I could have an access to, so >> that I can try to reproduce and fix the following issue? >> >> http://projects.scipy.org/numpy/ticket/2076 >> >> That would be greatly appreciated, as it is currently marked as a >> blocker for 1.7.0. > > > You might ask on sage-devel. They were just talking about SPARC > machines the other day on sage-devel. > > Thanks, > > Jason > > > ___ > NumPy-Discussion mailing list > numpy-discuss...@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
On Nov 14, 11:28 am, Jeroen Demeyer wrote: > FYI: it's a Fedora 16 system with an Intel(R) Pentium(R) 4 CPU 3.60GHz > processor running Linux 3.3.7-1.fc16.x86_64. That sounded convenient because my desktop is similar: Fedora 16 running 3.6.5-2.fc16.x86_64 #1 SMP on Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz No such luck, however: with $ ./sage -v Sage Version 5.5.beta2, Release Date: 2012-11-13 I ran for i in `seq 100`; do echo $i; ./sage -t devel/sage/sage/schemes/elliptic_curves/ ell_number_field.py || echo FAULT AT i is $i done which succeeded all 100 times. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
Re: [sage-devel] Re: How to proceed to reduce Sage's memory leaking?
On 2012-11-14 19:06, Nils Bruin wrote: > I don't think I have a login on "sextus" FYI: it's a Fedora 16 system with an Intel(R) Pentium(R) 4 CPU 3.60GHz processor running Linux 3.3.7-1.fc16.x86_64. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: How to proceed to reduce Sage's memory leaking?
On Nov 14, 8:29 am, Jeroen Demeyer wrote: > It happens also on other systems, including 64-bit. It's easy to > reproduce on the Skynet machine "sextus" (Linux x86_64) where it happens > about 71% of the time. That might be workable. What exact version/patches to reproduce the problem? (I don't think I have a login on "sextus"). I don't promise that I'll actually have time to build/test/track down this problem, but I can see. Other people should definitely look at it too. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
Re: [sage-devel] How to proceed to reduce Sage's memory leaking?
On 2012-11-12 22:47, Jeroen Demeyer wrote: > Bad news again. During a preliminary test of sage-5.5.beta2, I got again > a segmentation fault in > devel/sage/sage/schemes/elliptic_curves/ell_number_field.py > but this time on a different system (arando: Linux i686) and with a > different set of patches as before. And for added fun: this time the > error isn't always reproducible. It happens also on other systems, including 64-bit. It's easy to reproduce on the Skynet machine "sextus" (Linux x86_64) where it happens about 71% of the time. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Re: Sage Days in Bobo-Dioulasso debriefing; Sage in developping countries
On 11/13/12 8:49 AM, Thierry wrote: I built a live Debian Sage USB key wich is able to clone itself on another key, indefinitely. I will make the prototype (and the source code) available when i will be back home with a good internet connection. Other features of the key are : personal data persistence, no personal data is duplicated, possibility for the user to share additional data from her key to the cloned one (pdf lecture notes, worksheets, pictures of the workshop,...), lot of softwares (geogebra, latex, editors, gimp, vlc, libreoffice,... there is no need to be small since the bandwith limit is the size of the key). I'm intrigued by the idea of using this to distribute Sage to students as well. I'm curious: how does the software selection compare to, for example, mathbuntu (http://www.mathbuntu.org/), particularly the math packages? Also, is it possible to easily do Sage development with this key? I keep running into somewhat frustrating problems getting students up to speed with a linux/sage development environment, and having a USB key I can keep up to date and hand them would be wonderful. Thanks, Jason -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
[sage-devel] Faster polynomial composition and evaluation
fast_polynomial ( http://trac.sagemath.org/sage_trac/ticket/13358 ) is a sage package that improves significantly the composition of univariate and multivariate polynomial. In particular, the composition of polynomial can be 10 times faster than the current implementation. Currently, it can be easily installed from sage as an external package. What do you think would be the best way to integrate it fully in sage ? Any feedback is appreciated. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To post to this group, send email to sage-devel@googlegroups.com. To unsubscribe from this group, send email to sage-devel+unsubscr...@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en.
