Send Beginners mailing list submissions to
beginners@haskell.org
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
beginners-requ...@haskell.org
You can reach the person managing the list at
beginners-ow...@haskell.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. CAF taking all the memory (Mahdi)
2. Re: audio generation (Dennis Raddle)
3. ByteString unfold that can fail (John Ky)
4. Re: ByteString unfold that can fail (Simon Jakobi)
5. Re: ByteString unfold that can fail (Oliver Charles)
6. Re: CAF taking all the memory (Ben Gamari)
----------------------------------------------------------------------
Message: 1
Date: Mon, 2 May 2016 23:25:20 +0430
From: Mahdi <mdiba...@aol.com>
To: beginners@haskell.org
Subject: [Haskell-beginners] CAF taking all the memory
Message-ID: <16b1c1bb-4565-43e3-a83a-355f7fdd8...@aol.com>
Content-Type: text/plain; charset=utf-8
Hello there,
I?m pretty new to Haskell and I?m trying to write a Neural Network in Haskell
for educational purposes.
So, I have my neural network working, it can learn XOR in 500 iterations, but
the thing is, if I increase the iterations to something like 5 milion times,
the process just drains my RAM until either it?s killed or the OS drowns. Here
is the code: [0]
I searched online and tried to get information on why this is happening, I
profiled the memory usage and found that the memory is taken by CAF, searching
online,
it seems like an optimization done by compiler to reduce runtime computations.
Here is the profile: [1]
I searched online on how to prevent this from happening, couldn?t find anything
helpful.
I found this [2] and tried the workaround, but it didn?t work.
Is there any way to solve this problem? Because when I?m moving to more complex
problems, I have to train with much more iterations, and it just doesn?t work.
[0] https://github.com/mdibaiee/biscuit/blob/master/src/examples/xor.hs
[1] https://gist.github.com/mdibaiee/95878469a80f1c448b8cf0762178720e
[2]
http://stackoverflow.com/questions/6090932/how-to-make-a-caf-not-a-caf-in-haskell
Thank you,
? Mahdi
------------------------------
Message: 2
Date: Mon, 2 May 2016 15:32:21 -0700
From: Dennis Raddle <dennis.rad...@gmail.com>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <beginners@haskell.org>
Subject: Re: [Haskell-beginners] audio generation
Message-ID:
<cakxlvoon_qjopf9y41-wcr1d5vv+mqba5jok+sfcp7f4cgh...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
On Sun, May 1, 2016 at 9:59 PM, Jeremy Shaw <jer...@n-heptane.com> wrote:
> You might consider using the pipes library:
>
> http://hackage.haskell.org/package/pipes-4.1.8/docs/Pipes-Tutorial.html
>
> The pipes library will allow you to generate audio and write it to
> disk with out having to worry if you are going to suck up all the RAM
> accidentally.
>
> It should also help you decompose your pipeline into smaller pieces.
> For example, you would like to be able to decompose your code into
> things like:
>
> 1. the code that generates the audio
> 2. the code that converts the audio into a format like wav/aiff/etc
> 3. the code that writes binary data to disk
>
> And then simply glue those pieces together, while feeling secure that
> you don't create a space leak in the process.
>
> It will also allow you to use StateT or IO (or anything other type
> with a Monad instance) if you need to.
>
> The pipes library is not trivial to learn. But it is well designed.
>
> Without using the pipes library you have two options:
>
> 1. understand how laziness works and then be very careful to make
> sure you never accidentally hold onto data too long and cause a space
> leak.
>
> 2. use strict IO functions and write code that generates the output
> in little chunks at a time. The concept is, perhaps, easy to
> understand. But as your code base grows, the code will become harder
> to understand and to modify.
>
Thanks, Jeremy. At this time the code is experimental and may never go
anywhere after the initial experiments, so if your option #2 is the easiest
to get going, that would be my choice.
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20160502/b209a1ed/attachment-0001.html>
------------------------------
Message: 3
Date: Mon, 02 May 2016 22:49:38 +0000
From: John Ky <newho...@gmail.com>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <Beginners@haskell.org>
Subject: [Haskell-beginners] ByteString unfold that can fail
Message-ID:
<camb4o-chkjpnhnfu1qacn2vya0pvqvlohpu1ckapsy3ft9d...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello,
Does anyone know of anything like Data.ByteString.unfoldr, except that it
can fail with a return value?
Cheers,
-John
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20160502/6ff407e6/attachment-0001.html>
------------------------------
Message: 4
Date: Tue, 3 May 2016 01:34:27 +0200
From: Simon Jakobi <simon.jak...@googlemail.com>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <beginners@haskell.org>
Subject: Re: [Haskell-beginners] ByteString unfold that can fail
Message-ID:
<CAGtp2Sj8TLtg=sjncmxdm88yhuhpprmmvs1tvjyjb+d7-5f...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi John,
what's the exact type you're looking for?
Does hoogle give any useful results for it?
Cheers,
Simon
------------------------------
Message: 5
Date: Tue, 03 May 2016 08:29:10 +0000
From: Oliver Charles <ol...@ocharles.org.uk>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <beginners@haskell.org>
Subject: Re: [Haskell-beginners] ByteString unfold that can fail
Message-ID:
<CAGRp5R=d2c9a1x_yax54dby4ff-8v5hkrhvtjaoxkffmpx3...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
If there is an unfoldM like function then that might do the job, with Maybe
or MaybeT m as your choice of monad.
Ollie
On Mon, 2 May 2016, 11:49 p.m. John Ky, <newho...@gmail.com> wrote:
> Hello,
>
> Does anyone know of anything like Data.ByteString.unfoldr, except that it
> can fail with a return value?
>
> Cheers,
>
> -John
> ?
> _______________________________________________
> Beginners mailing list
> Beginners@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20160503/8933f5f8/attachment-0001.html>
------------------------------
Message: 6
Date: Tue, 03 May 2016 12:06:38 +0200
From: Ben Gamari <b...@smart-cactus.org>
To: Mahdi <mdiba...@aol.com>, beginners@haskell.org
Subject: Re: [Haskell-beginners] CAF taking all the memory
Message-ID: <87h9ef78wh....@smart-cactus.org>
Content-Type: text/plain; charset="utf-8"
Mahdi <mdiba...@aol.com> writes:
> Hello there,
>
Hi!
> I?m pretty new to Haskell and I?m trying to write a Neural Network in
> Haskell for educational purposes.
>
> So, I have my neural network working, it can learn XOR in 500
> iterations, but the thing is, if I increase the iterations to
> something like 5 milion times, the process just drains my RAM until
> either it?s killed or the OS drowns. Here is the code: [0]
>
> I searched online and tried to get information on why this is
> happening, I profiled the memory usage and found that the memory is
> taken by CAF, searching online,
>
Great! The next question you should then ask is "which CAF"? A CAF is
simply a top-level "constant" in your program. Indeed it sounds like you
have not defined any cost-centers, which means that GHC will attribute
the entire cost of your program to the ambiguous cost-center "CAF"
(which in this case really just means "the whole program").
As discussed in the users guide [1] one way to define cost-centers
within your program is to manually annotate expressions with SCC
pragmas. However, in this case we simply want to let GHC do this for us,
for which we can use the `-fprof-auto -fprof-cafs` flags (which
automatically annotate top-level definitions and CAFs with
cost-centers),
$ ghc -O examples/xor.hs -fprof-auto -fprof-cafs
Now your program should give a much more useful profile (run having set
the iteration count to 50000),
$ time examples/xor +RTS -p
[[0.99548584],[4.5138146e-3],[0.9954874],[4.513808e-3]]
real 0m4.019s
user 0m0.004s
sys 0m4.013s
$ cat xor.prof
Tue May 3 11:15 2016 Time and Allocation Profiling Report (Final)
xor +RTS -p -RTS
total time = 1.11 secs (1107 ticks @ 1000 us, 1 processor)
total alloc = 1,984,202,600 bytes (excludes profiling overheads)
COST CENTRE MODULE %time %alloc
matrixMap Utils.Math 21.4 26.8
matadd Utils.Math 20.8 22.7
matmul.\ Utils.Math 10.4 16.0
dot Utils.Math 7.1 13.4
column Utils.Math 7.0 2.9
dot.\ Utils.Math 6.9 1.9
rowPairs Utils.Math 5.8 6.5
sigmoid' NN 4.7 0.8
train.helper Main 4.0 1.3
sigmoid NN 3.3 0.8
matmul Utils.Math 2.2 2.0
hadamard Utils.Math 1.8 2.1
columns Utils.Math 1.2 1.3
individual inherited
COST CENTRE MODULE no. entries %time
%alloc %time %alloc
MAIN MAIN 79 0 0.0
0.0 100.0 100.0
[snip]
CAF:main3 Main 143 0 0.0
0.0 100.0 100.0
(...) Main 162 1 0.0
0.0 100.0 100.0
train Main 163 1 0.0
0.0 100.0 100.0
train.helper Main 164 50001 4.0
1.3 100.0 100.0
train.helper.hweights Main 258 50001 0.5
0.0 0.5 0.0
train.helper.oweights Main 235 50001 0.4
0.0 0.4 0.0
train.helper.oback Main 207 50000 0.3
0.1 19.0 20.9
backward' NN 208 50000 0.3
0.6 18.7 20.8
[snip]
So, here we see that costs are actually spread throughout the program.
Without diving any deeper into this particular program it's hard to give
more guidance however I will say that your lazy list Matrix
representation is very rarely the right choice for even toy linear
algebra problems.
First, consider the fact that even just a list cons constructor requires
three words (an info table pointer, a pointer to payload, and a pointer
to tail) plus the size of the payload (which in the case of an evaluated
`Float` is 2 words: one info table pointer and the floating point value
itself). So, a list of n *evaluated* `Float`s (which would require only
4*n bytes if packed densely) will require 40*n bytes if represented as a
lazy list.
Then, consider the fact that indexing into a lazy list is an O(n)
operation: this means that your `Math.column` operation on an n x m
matrix may be O(n*m). Even worse, `Math.columns`, as used by
`Math.matmul` is O(n * m!).
Finally, consider the fact that whenever you "construct" a lazy list you
aren't actually performing any computation: you are actually
constructing a single thunk which represents the entire result; however,
if you then go to index into the middle of that list you will end up
constructing n cons cells and a thunk for the payload of each. In the
case of primitive linear algebra operations the cost of constructing
this payload thunk can be greater than simply computing the result.
For these reasons I wouldn't recommend that lazy lists are used in this
way. If you have a dense matrix use an array (probably even unboxed;
see, for instance, the `array`, `vector`, and `repa` libraries); if
you have a sparse matrix then use an appropriate sparse representation
(although sadly good sparse linear algebra tools are hard to come by in
Haskell) Not only will the result be significantly more efficient in
space and time but the runtime behavior of the program will be
significantly easier to follow since you can more easily ensure that
evaluation occurs when you expect it to.
Hopefully this helps. Good luck and let us know if there are further
issues.
Cheers,
- Ben
[1]
http://downloads.haskell.org/~ghc/master/users-guide//profiling.html#cost-centres-and-cost-centre-stacks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 472 bytes
Desc: not available
URL:
<http://mail.haskell.org/pipermail/beginners/attachments/20160503/a1138152/attachment.sig>
------------------------------
Subject: Digest Footer
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
------------------------------
End of Beginners Digest, Vol 95, Issue 4
****************************************
