Am 31.12.2014 um 14:05 schrieb Petr Baudis:
Hi!
On Wed, Dec 31, 2014 at 11:16:57AM +0100, Detlef Schmicker wrote:
I am just trying to reproduce the data from page 7 with all features
disabled. I do not reach the accuracy (I stay below 20%).
Now I wonder about a short statement in the paper, I did not really
understand:
On page 4 top right they state "In our experience using the
rectifier function was slightly more effective then using the tanh
function"
Where do they put this functions in? I use caffe, and as far as I
understood it, I would have to add extra layers to get a function
like this. Does this mean: before every layer there should be a tanh
or rectifier layer?
I think this is talking about the non-linear transformation function.
Basically, each neuron output y = f(wx) for weight vector w and input
vector x and transfer function f. Traditionally, f is a sigmoid (the
logistic function 1/(1+e^-x)), but tanh is also popular and with deep
learning, rectifier and such functions are very popular IIRC because
they allow much better propagation of error to deep layers.
Thanks a lot. I was struggling with the "traditionally", and expected
this to be the case for the standard convolutional layers in caffe. This
seems not to be the case, so now I added layers for f(x): Now I reach
>50% accuracy for a small dataset (285000 positions). Of cause this
data set is too small (therefore the number is overestimated), but I
only had 15% on this before introducing f(x) :)
I would be glad to share my sources if somebody is trying the same,
I hope to be able to start dedicating time to this starting the end of
January (when I'll be moving to Japan for three months! I'll be glad to
meet up with fellow Go developers some time, and see you at the UEC if
it's in 2015 too :-).
I would very much appreciate an open source implementation of this
- or rather, I'd rather spend my time using one to do interesting things
rather than building one, I do plan to open source my implementation if
I have to make one and can bring myself to build one from scratch...
oakfoam is open source anyway. In my branch my caffe based
implementation is available. My branch is not so clean as Francois's
one, and we did not merge for quite a time:(
At the moment the CNN part is in a very early state, you have to produce
the database by different scripts...
But I would be happy to assist!
Detlef
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
