Rockin'. On Thu, Dec 10, 2009 at 9:10 AM, Rich Hickey <richhic...@gmail.com> wrote:
> One of the objectives of Clojure is to reduce incidental complexity. > And one of the biggest sources of incidental complexity in Clojure was > the retention of the head of a lazy sequence due to its being > referenced by some local (argument or local (let) binding). One might > expect that, if no subsequent code in the body of a function uses that > arg/local, it would be subject to GC. Unfortunately, on the JVM, that > is, in many cases, not true - the local is considered a live reference > and is thus not GCed. This yields the infamous 'holding onto the head' > problem, and subsequent Out Of Memory errors on large data sets. > > I had put in place a workaround, which was the 'clearing/nulling-out' > of locals on the tail call of the function. This helps in many, but > not all, cases. Not all logic flows are amenable to rearrangement to > leverage this cleanup. And there are many cases where the local is not > visible - e.g. when destructuring. > > The full solution is to track, during compilation, the lifetime of all > locals on all branches of the execution path and to emit code that > clears them at the point of last use in any particular branch. > > I'm happy to announce I have implemented this fine-grained locals > clearing in the compiler, in the 'new' branch. It should automatically > cover all cases in which the code doesn't explicitly reuse the head - > including non-tail usage, destructuring etc. In short, such cases > should 'just work' from now on. > > N.B. that this is strictly a lifetime management issue and does not > change the nature of lazy sequences - they are real, linked data > structures, the tail of which might not yet have been created. They > are most emphatically *not* ephemeral streams of values. However, with > fine-grained locals clearing, they are subject to GC 'as you go', > delivering the benefits of both. > > If you've got a pet case of incidental head-retention, please try out > the 'new' branch and let me know how it goes. > > Thanks, > > Rich > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com<clojure%2bunsubscr...@googlegroups.com> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
