On Wed, Sep 2, 2009 at 1:02 PM, tmountain <tinymount...@gmail.com> wrote:
> > Hi all - I've recently encouraged a friend to start learning Clojure, > and he has written some basic Markov chaining code as a learning > exercise. His code works fine with small sets of input data, but > larger inputs have been causing a StackOverflowError. > > I've taken a look at the code and suspect that the error may be caused > by recurrent calls to filter occupying increasing amounts of stack > space due to filter leaving closed-over locals hanging around. That > being said, this is only a suspicion, I'm still learning the language > myself, so I could be totally wrong. Wrapping the filter calls in > doall seems to prevent the problem from happening, but performance is > abysmal in that case. The problem is laziness: you have a loop/recur that layers on successive filters. Then when you go to realize an element, it calls filter, which calls filter, which calls filter ... however deep. Unfortunately, if you apply successive lazy operations to a seq in a loop/recur you sort of lose tail optimization because of this, and there doesn't seem to be a way to make lazy operations that call other lazy operations tail-optimized. At least, not yet. At least for now you will probably have to doall the seq every so many iterations to avoid the stack overflow. If doing it every iteration kills performance, you might try doing it every Nth for some value of N that's reasonably large, but small enough to avoid the overflow. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
