Send Beginners mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. Re: Writing huge result of `show` to file results in out of
memory (Norbert Melzer)
2. Re: Writing huge result of `show` to file results in out of
memory (Imants Cekusins)
----------------------------------------------------------------------
Message: 1
Date: Fri, 28 Oct 2016 07:33:41 +0000
From: Norbert Melzer <[email protected]>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <[email protected]>
Subject: Re: [Haskell-beginners] Writing huge result of `show` to file
results in out of memory
Message-ID:
<ca+bcvsvph8irogbw9mrlie4zm5orfno_ffsqswrvyc_yegb...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I do not think that `show` or `read` are good candidates for serialising
data into a file. I'd suggest to use a dedicated serialisation library, to
write out proper JSON/YAML/XML/binary.
Mahdi Dibaiee <[email protected]> schrieb am Do., 27. Okt. 2016 um 10:53 Uhr:
Hi,
So I have a data instance which contains a few big matrices, and I want to
save my instance to a file so I can `read` it back later
to avoid the long computation every time (I'm training a recurrent neural
network).
First attempt, the simplest method:
writeFile "rnn" (show dataInstance)
It starts to take all of the memory and then bails out with `out-of-memory`
error.
So I wrote a function to write the string chunk by chunk, without
buffering, here is the code:
https://github.com/mdibaiee/sibe/blob/728df02fbdd6f134af107c098f5477094c61ea76/examples/recurrent.hs#L52-L64
Copy/pasted from the link:
saveRecurrent :: FilePath -> String -> Int -> IO ()
saveRecurrent path str chunkSize = do
handle <- openFile path AppendMode
hSetBuffering handle NoBuffering
loop handle str
hClose handle
where
loop _ [] = return ()
loop handle s = do
hPutStr handle $ take chunkSize s
hFlush handle
loop handle $ drop chunkSize s
But it doesn't work either, I still get `out-of-memory` errors. From
what I understand, this should work, but it isn't.
I asked on IRC and someone said "Show is not lazy *enough*", if that's
the case, I would appreciate an explanation of that.
Thanks,
Mahdi
_______________________________________________
Beginners mailing list
[email protected]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20161028/47857015/attachment-0001.html>
------------------------------
Message: 2
Date: Fri, 28 Oct 2016 10:37:41 +0200
From: Imants Cekusins <[email protected]>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <[email protected]>
Subject: Re: [Haskell-beginners] Writing huge result of `show` to file
results in out of memory
Message-ID:
<CAP1qinagaHx=u47uqyiyfvyj_xm7i7l375vmjhtdm93odlr...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
can
https://hackage.haskell.org/package/cereal-streams
help?
I did not use this lib but it may be relevant.
see also:
https://hackage.haskell.org/package/io-streams-1.3.5.0/docs/System-IO-Streams-Tutorial.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20161028/ffa19f75/attachment-0001.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Beginners mailing list
[email protected]
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
------------------------------
End of Beginners Digest, Vol 100, Issue 19
******************************************
