[sage-support] Re: how to enlarge memory limits

[Dima Pasechnik]

On 2014-08-12, Nils Bruin nbr...@sfu.ca wrote:
 --=_Part_5256_754630738.1407883007654
 Content-Type: text/plain; charset=UTF-8

 On Tuesday, August 12, 2014 7:29:01 AM UTC-7, Daniel Krenn wrote:


 It doesn't look like the results above help (but maybe I just interpret 
 them wrongly) 

 Possible fix now on:  http://trac.sagemath.org/ticket/16809

perhaps if this still doesn't help one can call ECL's garbade collector:
sage.libs.ecl.ecl_eval((system:gc 3))

(3 is the most expensive options, 2 less expensive, 1 is the cheapest, 
according to 
http://ecls.sourceforge.net/ecl/The-garbage-collector.html)

although this might collect too much, as the registration of CL objects we 
talked about
in this thread is not implemented, right?





-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-13 um 08:18 schrieb Dima Pasechnik:
 On 2014-08-12, Nils Bruin nbr...@sfu.ca wrote:
 --=_Part_5256_754630738.1407883007654
 Content-Type: text/plain; charset=UTF-8

 On Tuesday, August 12, 2014 7:29:01 AM UTC-7, Daniel Krenn wrote:


 It doesn't look like the results above help (but maybe I just interpret 
 them wrongly) 

 Possible fix now on:  http://trac.sagemath.org/ticket/16809
 
 perhaps if this still doesn't help one can call ECL's garbade collector:
 sage.libs.ecl.ecl_eval((system:gc 3))

 although this might collect too much, as the registration of CL objects we 
 talked about
 in this thread is not implemented, right?

Tried, but it didn't release the memory (still 3 GB or so blocked)

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-12 um 17:42 schrieb Nils Bruin:
 On Tuesday, August 12, 2014 7:29:01 AM UTC-7, Daniel Krenn wrote:
 (type 'sage.symbolic.expression.Expression', 28257)
 
 
 This could be a large number (although expressions are recursive data
 structures, so one complicated expression can cause a lot of them to
 exist). 

I'm using a lot of symbolic expressions, so this does not worry me.

 Expressions could keep things in ecl alive, but there would have
 to be EclObjects in memory for that and you're finding none. This
 suggests that anything you're sending over to maxima gets sent there via
 the strings-based interface.

Ok.

 (type 'tuple', 38713)
 (type 'sage.rings.rational.Rational', 3555901)
 
 that is a rather huge number. Are you keeping a large number of tuples
 with rational numbers in memory somewhere (roughly 100 numbers per tuple
 on average)? If not, then you may want to look where these are coming from.

Actually, no. I don't use rationals much. I have to check the code very
carefully to see, where they might be hidden.
On the other hand, my calculation uses about 3.5 Gigabytes of Memory, so
this would mean about 1 Kilobyte per rational, which are quite large
numbers (and I'm not sure, I'm using that large numbers). But as I said,
I will check.

 [...] So, my guess is that you're just making a lot of calls to maxima that
 end up using the strings-based interface, rather than the binary
 interface, and that the memory blow-up you're experiencing is the
 accumulation of (unbound) `$sage...` symbols. This is certainly
 something that would always happen, albeit slowly. However, you could of
 course be running into a different leak as well, so it may be worth
 playing around with the above yourself a bit.

I will (answers follow, but maybe only in two weeks or so...)

Thanks for your input.

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-11 um 16:19 schrieb Nils Bruin:
 On Monday, August 11, 2014 4:46:45 AM UTC-7, Daniel Krenn wrote:
 As expected.  If you want to get some idea of what is taking memory at
 the sage side you could do something like
 
 import gc
 from collections import Counter
 gc.collect()
 pre={id(c) for c in gc.get_objects()}
 
 your computation
 
 gc.collect()
 post=Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
 sorted(post.iteritems(),key=lambda t: t[1])

(class 'sage.symbolic.function_factory.NewSymbolicFunction', 1)
(type 'builtin_function_or_method', 1)
(type 'set', 1)
(type 'module', 1)
(type 'sage.structure.coerce_actions.LeftModuleAction', 1)
(type 'sage.misc.function_mangling.ArgumentFixer', 1)
(type 'sage.rings.real_double.ToRDF', 1)
(class 'sage.symbolic.function_factory.NewSymbolicFunction', 1)
(type 'sage.rings.real_mpfr.double_toRR', 2)
(type 'sage.structure.coerce_maps.NamedConvertMap', 2)
(type 'sage.categories.morphism.CallMorphism', 3)
(type 'type', 3)
(type 'frame', 4)
(type 'sage.structure.coerce_maps.DefaultConvertMap_unique', 4)
(type 'getset_descriptor', 4)
(type 'staticmethod', 7)
(class 'sage.structure.dynamic_class.DynamicMetaclass', 7)
(type 'sage.misc.constant_function.ConstantFunction', 12)
(type 'list', 12)
(type 'function', 16)
(type 'dict', 16)
(type 'cell', 24)
(class 'weakref.KeyedRef', 147)
(type 'weakref', 151)
(type 'sage.rings.complex_interval.ComplexIntervalFieldElement', 2175)
(type 'sage.rings.real_mpfi.RealIntervalFieldElement', 8563)
(type 'sage.symbolic.expression.Expression', 28257)
(type 'tuple', 38713)
(type 'sage.rings.rational.Rational', 3555901)

 It'll give you a list of python object types that occur in memory,
 sorted by count. This might give you an indication of what is leaking on
 the python side. It may well be nothing, in which case it's probably
 maxima leaking or ecl keeping freed memory mapped for future use. It's
 likely that the memory behaviour you're observing isn't easily changed.

It doesn't look like the results above help (but maybe I just interpret
them wrongly)

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Tuesday, August 12, 2014 7:29:01 AM UTC-7, Daniel Krenn wrote:

 (type 'sage.rings.complex_interval.ComplexIntervalFieldElement', 2175) 
 (type 'sage.rings.real_mpfi.RealIntervalFieldElement', 8563)


Numbers of course depend on the context, but this doesn't look alarming. 
You can probably check yourself if you expect the number of RIF and CIF 
elements to be of this order. They certainly wouldn't keep anything in ECL 
alive. 

(type 'sage.symbolic.expression.Expression', 28257)


This could be a large number (although expressions are recursive data 
structures, so one complicated expression can cause a lot of them to 
exist). Expressions could keep things in ecl alive, but there would have to 
be EclObjects in memory for that and you're finding none. This suggests 
that anything you're sending over to maxima gets sent there via the 
strings-based interface.
 

 (type 'tuple', 38713) 
 (type 'sage.rings.rational.Rational', 3555901) 


that is a rather huge number. Are you keeping a large number of tuples with 
rational numbers in memory somewhere (roughly 100 numbers per tuple on 
average)? If not, then you may want to look where these are coming from. 

It doesn't look like the results above help (but maybe I just interpret 
 them wrongly) 


Well, it excludes EclObject, so that is telling us that any leaking to 
maxima would have to occur strings-based. The strings-based maxima 
interface uses temporary symbols to tie translated results to in maxima. As 
we know, generating symbols in common lisp almost always causes a memory 
leak in that the symbol names get interned and never forgotten, this would 
be the most likely reason to have leaking behaviour. We can find this out 
by defining:

maxima_calculus(1) # make sure maxima_calculus is defined
from sage.libs.ecl import ecl_eval
 def ecl_symbols():
return set(ecl_eval((let ((lst ())) (do-all-symbols (s lst) (push s 
lst)) lst)))

Now let's do some tests to see if we're collecting any symbols:

A=ecl_symbols()
for i in range(10):
   _= integrate(sin(x),1,10)
   B = ecl_symbols()
   print B.difference(A)
   A= B

The above works fine: we get some symbols that do get defined initially, 
but once they exist, no new symbols arise.

for i in range(10):
   _= sin(x).simplify()
   B = ecl_symbols()
   print B.difference(A)
   A= B

Here we see the culprit! simplify still uses the strings-based interface 
and it is creating a new temporary symbol every time: $sage6, $sage7 etc. 
This leaks.

To plug this, we should either try and reuse these symbols or dive into CL 
to see if we can really unintern them (which is going to be the same 
problem, because we'd have to know when it's safe to unintern, i.e., safe 
to recycle.

Either that or someone sits down and finally removes the strings-based uses 
from our maxima_lib interface.

So, my guess is that you're just making a lot of calls to maxima that end 
up using the strings-based interface, rather than the binary interface, and 
that the memory blow-up you're experiencing is the accumulation of 
(unbound) `$sage...` symbols. This is certainly something that would always 
happen, albeit slowly. However, you could of course be running into a 
different leak as well, so it may be worth playing around with the above 
yourself a bit.

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Tuesday, August 12, 2014 7:29:01 AM UTC-7, Daniel Krenn wrote:


 It doesn't look like the results above help (but maybe I just interpret 
 them wrongly) 

 Possible fix now on:  http://trac.sagemath.org/ticket/16809

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-10 um 15:30 schrieb Dima Pasechnik:
 On 2014-08-08, Nils Bruin nbr...@sfu.ca wrote:
 --=_Part_203_2052776100.1407511132909
 Content-Type: text/plain; charset=UTF-8
 Would at least restarting ecl release the memory?
 (it's not clear how the latter
 can be done; would maxima_calculus.reset() do the job?)

In my case, this (maxima_calculus.reset()) gives the following:

;;;
;;; Detected access to protected memory, also kwown as 'bus or
segmentation fault'.
;;; Jumping to the outermost toplevel prompt
;;;

Traceback (most recent call last):
...
TypeError: ECL says: Detected access to an invalid or protected memory
address.


Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Sunday, August 10, 2014 11:16:19 PM UTC-7, Daniel Krenn wrote:

 In my case, this (maxima_calculus.reset()) gives the following: 


See

http://maxima.sourceforge.net/docs/manual/en/maxima_4.html

for what the reset() command does: not that much. The description doesn't 
make it sound it should help you much with memory leaks. It should also not 
lead to a segfault (although there may be variables that get reset to 
values that sage doesn't expect, so it may well be that sage will act 
funnily when you execute reset). I wasn't able to reproduce the segfault. 
Would you have a concise code fragment that can reliably reproduce the 
segfault? It may be worth investigating, because it might point to a bug 
elsewhere.

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-11 um 09:24 schrieb Nils Bruin:
 On Sunday, August 10, 2014 11:16:19 PM UTC-7, Daniel Krenn wrote:
 
 In my case, this (maxima_calculus.reset()) gives the following:
 
 [...] I wasn't able to
 reproduce the segfault. Would you have a concise code fragment that can
 reliably reproduce the segfault? It may be worth investigating, because
 it might point to a bug elsewhere.

Ok, tried again (at least on a partial calculation) but didn't get the
error again. Now I get

[errormsg,?rhs,?lhs,piece,nolabels,display2d,lispdisp,?tr\\-unique,domai\
n,multiplicities,keepfloat,?genvar,features,?odds]

(Since I played around a bit before, I cannot exclude that I did
something weird before the first .reset())

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-10 um 17:38 schrieb Nils Bruin:
 On Sunday, August 10, 2014 6:31:03 AM UTC-7, Dima Pasechnik wrote:
 (it's not clear how the latter can be done; would
 maxima_calculus.reset() do the job?)

Tried again (since I had troubles before; see other posting), but memory
is not released.

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-support] Re: how to enlarge memory limits

[Daniel Krenn]

Am 2014-08-08 um 17:18 schrieb Nils Bruin:
 On Friday, August 8, 2014 3:02:03 AM UTC-7, Dima Pasechnik wrote:
 so doing
 
 sage: import sage.libs.ecl
 sage: sage.libs.ecl.ecl_eval((ext:set-limit 'ext:heap-size 0))
 
 might solve your problem. 

Many thanks, this solved my original problem; I can run my computation
for the given parameter.

Now I have a new problem: I want to run it for different parameters, but
the memory is eaten up. It is not freed after each run.
I checked my code and nothing except the result (one RIF element) is
stored; I have removed all caches and even ran the garbage collector,
but no effect. There is still a huge amount of memory blocked (about 16 GB).
How can I free this memory or at least find out which object / software
package takes this memory?

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Dima Pasechnik]

On 2014-08-10, Daniel Krenn kr...@aon.at wrote:
 Am 2014-08-08 um 17:18 schrieb Nils Bruin:
 On Friday, August 8, 2014 3:02:03 AM UTC-7, Dima Pasechnik wrote:
 so doing
 
 sage: import sage.libs.ecl
 sage: sage.libs.ecl.ecl_eval((ext:set-limit 'ext:heap-size 0))
 
 might solve your problem. 

 Many thanks, this solved my original problem; I can run my computation
 for the given parameter.

 Now I have a new problem: I want to run it for different parameters, but
 the memory is eaten up. It is not freed after each run.
 I checked my code and nothing except the result (one RIF element) is
 stored; I have removed all caches and even ran the garbage collector,
 but no effect. There is still a huge amount of memory blocked (about 16 GB).
 How can I free this memory or at least find out which object / software
 package takes this memory?

valgrind?
there could well be a memory leak somewhere...


-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Dima Pasechnik]

On 2014-08-08, Nils Bruin nbr...@sfu.ca wrote:
 --=_Part_203_2052776100.1407511132909
 Content-Type: text/plain; charset=UTF-8

 On Friday, August 8, 2014 3:02:03 AM UTC-7, Dima Pasechnik wrote:

 have a look at 
 http://ecls.sourceforge.net/new-manual/re86.html#table.memory.limits 
 and 
 http://trac.sagemath.org/ticket/6772 
 where one of these limits was removed. 
 Perhaps removing other limits will help. 


 Actually, the change there did not make it into the maxima_lib interface 
 that calculus uses:

 sage: from sage.libs.ecl import *
 sage: ecl_eval((ext:get-limit 'ext:heap-size))
ECL: 1073741824

 so doing

 sage: import sage.libs.ecl
 sage: sage.libs.ecl.ecl_eval((ext:set-limit 'ext:heap-size 0))

 might solve your problem. I'm not sure if we should do the same. Since 
 maxima_lib runs in the same process as the rest of sage, it might be 
 problematic to let ecl munch away all memory and starve python (for 
 instance when a ulimit has been set). I don't think the boehm garbage 
 collector will ever give back once allocated memory to the operating 
 system.

if this is the case then how one would to to reclaim the memory?
And why this is not a memory leak?
Would at least restarting ecl release the memory?
(it's not clear how the latter
can be done; would maxima_calculus.reset() do the job?)



-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Sunday, August 10, 2014 6:31:03 AM UTC-7, Dima Pasechnik wrote:

 if this is the case then how one would to to reclaim the memory? 
 And why this is not a memory leak?

 
It's not necessarily  a leak from the perspective of ecl: the memory 
obtained from the operating system may well sit there, ready to be filled 
with new lisp objects. From the python perspective you could call this a 
memory leak, though. But it's a virtually unavoidable one if you place 
multiple memory management systems in the same process that are not 
designed to cooperate: each thinks they have a complete view of the world 
but they don't. It may well be that ecl/boehm does unmap memory pages every 
now and again. But really, in our case the same scenarios can arise with 
pari/gp, libsingular, libgap

Would at least restarting ecl release the memory? 


Possibly.
 

 (it's not clear how the latter can be done; would maxima_calculus.reset() 
 do the job?) 


restarting ecl is currently not a supported operation, because it would be 
a terribly difficult operation to do correctly: all EclObjects in python 
memory would somehow have to be invalidated, a state that currently 
doesn't even exist for them. That also means *finding* those objects (or 
knowing that there are none-which will never happen because we have plenty 
of global EclObjects), which would require an extra expensive facility.

Regardless, of this, symbolics leak anyway:

sage: for i in range(1000): _=SR.var(t%d%i)

will steadily eat memory. Probably the pynac symbol table doesn't get 
garbage collected. Even if that wouldn't leak then

sage: for i in range(1000): _=maxima_calculus(SR.var(t%d%i))

and 

sage: from sage.interfaces.maxima_lib import sr_to_max
sage: for i in range(1000): _=sr_to_max(SR.var(t%d%i))

would leak for sure via slightly different mechanisms, but for the same 
reason: we need to keep symbols alive in python and in maxima, depending on 
the lifetime of objects in the other. To garbage collect these, you'd need 
cycle detection *across the python/ecl boundary*. So currently, symbols 
that have been transported across to maxima have eternal life. It seems 
that symbols in SR proper already have eternal life, so it doesn't really 
matter. Solving this leak would be really quite hard, so we should probably 
call it a feature.

The solution is to restart sage early and often. It has good facilities for 
it, see @parallel.

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Volker Braun]

Afaik all library interfaces (in particular libGAP) have only Sage-Library 
references. So Sage objects can keep library objects alive, but not the 
other way round. Once the Sage objects get collected, the library GC will 
do its part. Of course you can still have memory leaks, either in Sage or 
in the library, but there is no insurmountable obstacle.


-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Sunday, August 10, 2014 1:06:44 PM UTC-7, Dima Pasechnik wrote:

 hmm, AFAIK GAP will call its GC now and then, thus libGAP better be 
 able to cope with this... 


It will garbarge collect just fine. It's a question on the side of the 
memory manager whether unmapping memory pages or keeping them around for 
meeting the next allocation requests is better. Getting pages from and 
releasing pages to the operating system is generally quite expensive, so 
memory managers specialized for handling many (small) allocs/deallocs tend 
to keep an ample amount of memory around (never mind that fragmentation 
might lead to many barely filled pages, none of which *could* be returned. 
There's plenty of memory available to the memory manager, but memory 
allocations via other managers might fail.

This problem is not a memory leak in any one of the memory managers 
individually, but it's a memory page mapping leak, which is only a leak 
when considered on the level of multiple long-lived processes or when you 
put multiple memory managers in the same process.

my understanding is that it is possible to register with ECL external 
 pointers to 
 Lisp objects, so if such a pointer goes away (i.e. the corresponding 
 Python object is gone) 
 then it will become possible for ECL to collect its garbage related to 
 this Lisp 
 object (i.e. calling in ECL (system:gc level) will do the right thing). 


Yes, we do that via a very simple mechanism: All EclObjects register their 
pointer to a lisp pointer to a global double-linked list within ecl/boehm's 
scope, and deregister that link upon their __dealloc__. This is an 
extremely cheap and effective mechanism, which was recommended by the main 
ecl developer at the time. EclObject by itself doesn't leak. The 
corresponding lisp objects have their lifetime bounded below by the 
referring EclObject, but not more than that (and with a mark/sweep 
collector like boehm, that's all you need).
 

 if the registration I talk about above is implemented, I imagine that 
 one can also create a database of EclObjects to maintain. 
 Although this is some extra cost, indeed. 


Do you mean: also have a way to link back from ecl to python objects, 
i.e., let lisp objects impose lower bounds on lifetimes of certain python 
objects? Then cycle detection indeed becomes a problem, and doing that 
across the python/ecl boundary would require very careful coordination of 
*both* garbage collectors. This would entail a significant rewrite of 
*both*.
 
The leak with symbols, however, would not even be solved by that. 
Certainly, currently, symbols in Pynac are immortal anyway, so there are no 
lifetimes to coordinate. Probably, given the age of Maxima, symbols are 
effectively immortal there too (maxima symbols are instantiated as lisp 
symbols, which are normally interned. If you don't manually unintern them, 
they would likely be immortal), so even solving the Pynac side would 
probably not solve the pynac/maxima leak, because maxima would still leak. 
It's rather fundamental: The state of computer algebra systems like 
maxima normally consists of all computations done until this moment (maxima 
has an unlimited history by default--we disable that). That includes all 
instantiated symbols.

Finally, the string-based interface means that we fundamentally cannot tie 
lifetimes of python objects unambiguously to ecl objects: every (interned) 
symbol is alive, because it can be retrieved via its name. You'd have to be 
very careful in deciding when a symbol (and all its assumptions and other 
properties, including its translations to other computer algebra systems) 
can be discarded. Currently there is nothing to manage, because symbols are 
immortal anyway. If they weren't, you'd either need to artificially put 
lower bounds on their existence or be prepared to reinstall things like 
assumptions on them any time you are translating a symbol from one system 
to another.

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.

[sage-support] Re: how to enlarge memory limits

[Nils Bruin]

On Friday, August 8, 2014 3:02:03 AM UTC-7, Dima Pasechnik wrote:

 have a look at 
 http://ecls.sourceforge.net/new-manual/re86.html#table.memory.limits 
 and 
 http://trac.sagemath.org/ticket/6772 
 where one of these limits was removed. 
 Perhaps removing other limits will help. 


Actually, the change there did not make it into the maxima_lib interface 
that calculus uses:

sage: from sage.libs.ecl import *
sage: ecl_eval((ext:get-limit 'ext:heap-size))
ECL: 1073741824

so doing

sage: import sage.libs.ecl
sage: sage.libs.ecl.ecl_eval((ext:set-limit 'ext:heap-size 0))

might solve your problem. I'm not sure if we should do the same. Since 
maxima_lib runs in the same process as the rest of sage, it might be 
problematic to let ecl munch away all memory and starve python (for 
instance when a ulimit has been set). I don't think the boehm garbage 
collector will ever give back once allocated memory to the operating 
system.

-- 
You received this message because you are subscribed to the Google Groups 
sage-support group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.