Send Beginners mailing list submissions to
beginners@haskell.org
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
beginners-requ...@haskell.org
You can reach the person managing the list at
beginners-ow...@haskell.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. Writing huge result of `show` to file results in out of
memory (Mahdi Dibaiee)
----------------------------------------------------------------------
Message: 1
Date: Thu, 27 Oct 2016 04:53:07 -0400
From: Mahdi Dibaiee <mdiba...@aol.com>
To: beginners@haskell.org
Subject: [Haskell-beginners] Writing huge result of `show` to file
results in out of memory
Message-ID: <1580557c3ae-16d1-3...@webstg-a06.mail.aol.com>
Content-Type: text/plain; charset="utf-8"
Hi,
So I have a data instance which contains a few big matrices, and I want to save
my instance to a file so I can `read` it back later
to avoid the long computation every time (I'm training a recurrent neural
network).
First attempt, the simplest method:
writeFile "rnn" (show dataInstance)
It starts to take all of the memory and then bails out with `out-of-memory`
error.
So I wrote a function to write the string chunk by chunk, without buffering,
here is the code:
https://github.com/mdibaiee/sibe/blob/728df02fbdd6f134af107c098f5477094c61ea76/examples/recurrent.hs#L52-L64
Copy/pasted from the link:
saveRecurrent :: FilePath -> String -> Int -> IO ()
saveRecurrent path str chunkSize = do
handle <- openFile path AppendMode
hSetBuffering handle NoBuffering
loop handle str
hClose handle
where
loop _ [] = return ()
loop handle s = do
hPutStr handle $ take chunkSize s
hFlush handle
loop handle $ drop chunkSize s
But it doesn't work either, I still get `out-of-memory` errors. From what I
understand, this should work, but it isn't.
I asked on IRC and someone said "Show is not lazy enough", if that's the case,
I would appreciate an explanation of that.
Thanks,
Mahdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20161027/f2453c5b/attachment-0001.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
------------------------------
End of Beginners Digest, Vol 100, Issue 18
******************************************
