Re: [sage-devel] Re: Weak references in the coercion model
How difficult is it to create a stack that can control the order of objects being created and destroyed so you prevent leaks? Can you add a new method on your base class that is inherited to all children to track this? Sent from my iPad > On Dec 4, 2018, at 6:54 PM, David Roe wrote: > > > >> On Tue, Dec 4, 2018 at 9:06 PM wrote: >> Would it be advisable to change the base programming language to one that >> does automatic garbage collection instead of having to check to see if a >> class has been properly disposed like it appears from all of these related >> bugs? > > I can't tell if you're being sarcastic, but I'll just say that the problem > isn't the language. Both Python and Cython have garbage collection. The > issue is that we want to cache things for speed reasons, but also need to > prevent unbounded growth in Sage's memory usage. > David > >> >> Sent from my iPhone >> >> > On Dec 4, 2018, at 12:44 PM, Jeroen Demeyer wrote: >> > >> >> On 2018-12-04 18:06, Nils Bruin wrote: >> >> Tripledict does that to some extent (with its keys): if one of the key >> >> parts gets deallocated, the weakref callback removes the strong >> >> reference to the value. >> > >> > Yes, but then we potentially end up again in the situation where things >> > are *only* weakly referenced. Currently, you still need a strong reference >> > in a fixed place and ideally we shouldn't. >> > >> > I have a very preliminary idea at #26811 to "fix" this. >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "sage-devel" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to sage-devel+unsubscr...@googlegroups.com. >> > To post to this group, send email to sage-devel@googlegroups.com. >> > Visit this group at https://groups.google.com/group/sage-devel. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sage-devel" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sage-devel+unsubscr...@googlegroups.com. >> To post to this group, send email to sage-devel@googlegroups.com. >> Visit this group at https://groups.google.com/group/sage-devel. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion modeled
No sarcasm intended. I used to work at a software development firm and we were forced to use c++. Most of the back and forth that I have been observing with the team is around memory leaks and I just wondered if that is due to your development language choice. Hence, my comment. However, if you are using low level languages to get the fastest processing time, then I can see why you are doing what you are doing. I’m just afraid that there may be some low level design flaw leading to all of these leaks. For instance you might consider using messages between different objects to help isolate object disposal, instead of carefully remembering what object has to be disposed in which order. This is difficult to do because of the late binding due to the flexibility of sage. Sent from my iPhone > On Dec 4, 2018, at 6:54 PM, David Roe wrote: > > > >> On Tue, Dec 4, 2018 at 9:06 PM wrote: >> Would it be advisable to change the base programming language to one that >> does automatic garbage collection instead of having to check to see if a >> class has been properly disposed like it appears from all of these related >> bugs? > > I can't tell if you're being sarcastic, but I'll just say that the problem > isn't the language. Both Python and Cython have garbage collection. The > issue is that we want to cache things for speed reasons, but also need to > prevent unbounded growth in Sage's memory usage. > David > >> >> Sent from my iPhone >> >> > On Dec 4, 2018, at 12:44 PM, Jeroen Demeyer wrote: >> > >> >> On 2018-12-04 18:06, Nils Bruin wrote: >> >> Tripledict does that to some extent (with its keys): if one of the key >> >> parts gets deallocated, the weakref callback removes the strong >> >> reference to the value. >> > >> > Yes, but then we potentially end up again in the situation where things >> > are *only* weakly referenced. Currently, you still need a strong reference >> > in a fixed place and ideally we shouldn't. >> > >> > I have a very preliminary idea at #26811 to "fix" this. >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "sage-devel" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to sage-devel+unsubscr...@googlegroups.com. >> > To post to this group, send email to sage-devel@googlegroups.com. >> > Visit this group at https://groups.google.com/group/sage-devel. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sage-devel" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sage-devel+unsubscr...@googlegroups.com. >> To post to this group, send email to sage-devel@googlegroups.com. >> Visit this group at https://groups.google.com/group/sage-devel. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion model
On Tue, Dec 4, 2018 at 9:06 PM wrote: > Would it be advisable to change the base programming language to one that > does automatic garbage collection instead of having to check to see if a > class has been properly disposed like it appears from all of these related > bugs? > I can't tell if you're being sarcastic, but I'll just say that the problem isn't the language. Both Python and Cython have garbage collection. The issue is that we want to cache things for speed reasons, but also need to prevent unbounded growth in Sage's memory usage. David > Sent from my iPhone > > > On Dec 4, 2018, at 12:44 PM, Jeroen Demeyer wrote: > > > >> On 2018-12-04 18:06, Nils Bruin wrote: > >> Tripledict does that to some extent (with its keys): if one of the key > >> parts gets deallocated, the weakref callback removes the strong > >> reference to the value. > > > > Yes, but then we potentially end up again in the situation where things > are *only* weakly referenced. Currently, you still need a strong reference > in a fixed place and ideally we shouldn't. > > > > I have a very preliminary idea at #26811 to "fix" this. > > > > -- > > You received this message because you are subscribed to the Google > Groups "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sage-devel+unsubscr...@googlegroups.com. > > To post to this group, send email to sage-devel@googlegroups.com. > > Visit this group at https://groups.google.com/group/sage-devel. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion model
Would it be advisable to change the base programming language to one that does automatic garbage collection instead of having to check to see if a class has been properly disposed like it appears from all of these related bugs? Sent from my iPhone > On Dec 4, 2018, at 12:44 PM, Jeroen Demeyer wrote: > >> On 2018-12-04 18:06, Nils Bruin wrote: >> Tripledict does that to some extent (with its keys): if one of the key >> parts gets deallocated, the weakref callback removes the strong >> reference to the value. > > Yes, but then we potentially end up again in the situation where things are > *only* weakly referenced. Currently, you still need a strong reference in a > fixed place and ideally we shouldn't. > > I have a very preliminary idea at #26811 to "fix" this. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion model
On 2018-12-04 18:06, Nils Bruin wrote: Tripledict does that to some extent (with its keys): if one of the key parts gets deallocated, the weakref callback removes the strong reference to the value. Yes, but then we potentially end up again in the situation where things are *only* weakly referenced. Currently, you still need a strong reference in a fixed place and ideally we shouldn't. I have a very preliminary idea at #26811 to "fix" this. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion model
On Tuesday, December 4, 2018 at 1:50:19 AM UTC-8, Jeroen Demeyer wrote: > > On 2018-12-03 17:21, Nils Bruin wrote: > > In order to > > make that possible, the coercion map (referenced strongly on the > > codomain -- it needs to be strongly referenced somewhere to keep it > > alive) must not hold a strong ref to the domain. > > I wonder if there is a way to somehow reference an object from a pair of > objects: have A and B reference C in such a way that, if either A or B > get deallocated, also C gets deallocated. > Tripledict does that to some extent (with its keys): if one of the key parts gets deallocated, the weakref callback removes the strong reference to the value. Note that you will never ensure that C gets deallocated: if someone else is keeping a ref to C, it should be kept alive. The best you can hope for is a structure where the existence of both A and B ensures the continued existence of C, but as soon as one of A or B goes, then the existence of C is no longer assured. All the leaks I've seen in settings like this come from loops where A,B,C have relations as above, but somehow C ends up anchoring a reference to A and B. That's the big loophole with globally rooted references that are guarded by a weak reference callback: what would normally be cycles in garbage now are suddenly globally anchored data structures not eligible for collection. As we've seen again and again, that's a very difficult paradigm to program correctly with. Note that the weak ref to the domain on the "coercion maps" shouldn't be a performance issue. It's just there to ensure that it's straightforward to turn it into a full-blown map. In situations where the map is recovered from the coercion system, it's done by looking up via domain and codomain. No need to look it up on the map again. So a micro-optimization would be to short-cut some evaluation steps, if you think the domain is presently being actively recovered via the weakref. See https://trac.sagemath.org/ticket/14711 for very detailed discussions about the how and why of weakly referenced domains in the coercion system. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Re: Weak references in the coercion model
On 2018-12-03 17:21, Nils Bruin wrote: In order to make that possible, the coercion map (referenced strongly on the codomain -- it needs to be strongly referenced somewhere to keep it alive) must not hold a strong ref to the domain. I wonder if there is a way to somehow reference an object from a pair of objects: have A and B reference C in such a way that, if either A or B get deallocated, also C gets deallocated. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
