reply = parse ... -- Lazily evaluated
tokens = rTokens reply -- Has some values "immediately"
list = D.toList tokens -- Has some values "immediately"
mapM_ list print -- Start printing "immediately"!
..
reply = parse ... -- Lazily evaluated
result = rResult reply -- Lazy; has value when parsing is done
extra = case result ... -- Lazy; has value when parsing is done
parsed = rTokens reply -- Has some values "immediately"
tokens = D.append parsed extra -- Has some values "immediately"
list = D.toList tokens -- Has some values "immediately"
mapM_ list print -- Starts printing "immediately"!
This *still* starts printing tokens immediately. However, while in the
previous case the GC is smart enough to keep the program in constant
memory size, in the second case it for some reasons starts missing more
and more PAP objects so memory usage goes through the roof.
is that a nail for this hammer, perhaps?-)
http://hackage.haskell.org/trac/ghc/ticket/917
if you don't use rResult reply, reply can be used and freed as it is used.
if you do use rResult reply, you are going to use it late, and that is going to hang
on to reply, which is being expanded by the main thread of activities (rTokens).
i'm just guessing here, but if that is indeed the problem, you would need to
exert
more control over what is evaluated when and shared where:
- evaluate rResult synchronously with rTokens, instead of rResult long after
rTokens has unfolded the reply
- evaluate rResult independent of rTokens, on a separate copy of reply
since you want to use parts of the output before you can be sure whether the
whole input is correct, you might also want local errors instead of global ones
(i've seen a correct chunk of input, here is the corresponding chunk of output;
instead of here is a list of output chunks i've produced so far, i'll tell you later
whether they are worth anything or whether they were based on invalid input).
You'd think... but the fact of the matter is that while the first
version works fine, the second doesn't, UNLESS I add the magic SCC
section:
extra = {-# SCC "magic" #-} case result ...
And compile with '-prof' (no '-O' flags). Then it somehow, finally, "get
the idea" and the program runs perfectly well with constant memory
consumption. Which, as you aptly put it, is very "fishy" indeed...
adding profiling might (another wild guess here..) lose sharing, just as in the
ticket,
i used \()->[..] to avoid sharing of the list. (although that guess wouldn't
necessarily
suggest this particular SCC to be useful, so perhaps it is the wrong track..)
hth,
claus
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
