-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi,
this is a very difficult questions: I do have 100 batches (each 64) within 1 minute for the big facebook DCNN (384 layers in each of the 9 3x3 kernel two 128 5x5 and two 128 7x7 before that) What facebook calls an epoch is 400 of this (40000 * 64 positions) Now you can have a look at the fig 5 of facebook and you see it is difficult to say where to stop training. My experience at the moment is, that my training is not increasing after 20 of facebook batches, but I am fighting with this at the moment and still am not sure if figure 5 corresponds to KGS or GoGoD database, which makes a huge difference for me. Sorry for the not to good answer, but that is my state:( I trained the same net with all 128 layers within about two weeks in December, but was not happy with the result (49%, but after I read Hiroshi's post I am not sure, if it was not ok anyway :) At the moment I am preparing a net with additional winrate value output: I was fighting with Komi representation last year, now I will try to support 6.5, 7.5 and 0.5 komi (>90% of kgs games 6d+) using 6 layers (3 for b moving and 3 for white moving) Before I tried flexible komi support, but this was not successful enough:( And as google only supports 7.5 :) If somebody else working on this, I would love to share here! Detlef Am 02.02.2016 um 19:38 schrieb David Fotland: > How long does it take to train one of your nets? Is it safe to > assume that training time is roughly proportional to the number of > neurons in the net? > > Thanks, > > David > >> -----Original Message----- From: Computer-go >> [mailto:computer-go-boun...@computer-go.org] On Behalf Of Detlef >> Schmicker Sent: Tuesday, February 02, 2016 10:35 AM To: >> computer-go@computer-go.org Subject: *****SPAM***** Re: >> [Computer-go] What hardware to use to train the DNN >> > Hi David, > > I use Ubuntu 14.04 LTS with a NVIDIA GTX970 Graphic card (and > i7-4970k, but this is not important for training I think) and > installed CUDNN v4 (important, at least a factor 4 in training > speed). > > This Ubuntu version is officially supported by Cuda and I did only > have minor problems if an Ubuntu update updated the graphics > driver: I had 2 times in the last year to reinstall cuda (a little > ugly, as the graphic driver did not work after the update and you > had to boot into command line mode). > > Detlef > > Am 02.02.2016 um 19:25 schrieb David Fotland: >>>> Detlef, Hiroshi, Hideki, and others, >>>> >>>> I have caffelib integrated with Many Faces so I can evaluate >>>> a DNN. Thank you very much Detlef for sample code to set up >>>> the input layer. Building caffe on windows is painful. If >>>> anyone else is doing it and gets stuck I might be able to >>>> help. >>>> >>>> What hardware are you using to train networks? I don t have >>>> a cuda-capable GPU yet, so I'm going to buy a new box. I'd >>>> like some advice. Caffe is not well supported on Windows, so >>>> I plan to use a Linux box for training, but continue to use >>>> Windows for testing and development. For competitions I >>>> could use either windows or linux. >>>> >>>> Thanks in advance, >>>> >>>> David >>>> >>>>> -----Original Message----- From: Computer-go >>>>> [mailto:computer-go-boun...@computer-go.org] On Behalf Of >>>>> Hiroshi Yamashita Sent: Monday, February 01, 2016 11:26 PM >>>>> To: computer-go@computer-go.org Subject: *****SPAM***** >>>>> Re: [Computer-go] DCNN can solve semeai? >>>>> >>>>> Hi Detlef, >>>>> >>>>> My study heavily depends on your information. Especially >>>>> Oakfoam code, lenet.prototxt and >>>>> generate_sample_data_leveldb.py was helpful. Thanks! >>>>> >>>>>> Quite interesting that you do not reach the prediction >>>>>> rate 57% from the facebook paper by far too! I have the >>>>>> same experience with the >>>>> >>>>> I'm trying 12 layers 256 filters, but it is around 49.8%. I >>>>> think 57% is maybe from KGS games. >>>>> >>>>>> Did you strip the games before 1800AD, as mentioned in >>>>>> the FB paper? I did not do it and was thinking my >>>>>> training is not ok, but as you have the same result >>>>>> probably this is the only difference?! >>>>> >>>>> I also did not use before 1800AD. And don't use hadicap >>>>> games. Training positions are 15693570 from 76000 games. >>>>> Test positions are 445693 from 2156 games. All games are >>>>> shuffled in advance. Each position is randomly rotated. And >>>>> memorizing 24000 positions, then shuffle and store to >>>>> LebelDB. At first I did not shuffle games. Then accuracy is >>>>> down each 61000 iteration (one epoch, 256 mini-batch). >>>>> http://www.yss-aya.com/20160108.png It means DCNN >>>>> understands easily the difference 1800AD games and 2015AD >>>>> games. I was surprised DCNN's ability. And maybe 1800AD >>>>> games are also not good for training? >>>>> >>>>> Regards, Hiroshi Yamashita >>>>> >>>>> ----- Original Message ----- From: "Detlef Schmicker" >>>>> <d...@physik.de> To: <computer-go@computer-go.org> Sent: >>>>> Tuesday, February 02, 2016 3:15 PM Subject: Re: >>>>> [Computer-go] DCNN can solve semeai? >>>>> >>>>>> Thanks a lot for sharing this. >>>>>> >>>>>> Quite interesting that you do not reach the prediction >>>>>> rate 57% from the facebook paper by far too! I have the >>>>>> same experience with the GoGoD database. My numbers are >>>>>> nearly the same as yours 49% :) my net is quite simelar, >>>>>> but I use 7,5,5,3,3,.... with 12 layers in total. >>>>>> >>>>>> Did you strip the games before 1800AD, as mentioned in >>>>>> the FB paper? I did not do it and was thinking my >>>>>> training is not ok, but as you have the same result >>>>>> probably this is the only difference?! >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Detlef >>>>> >>>>> _______________________________________________ Computer-go >>>>> mailing list Computer-go@computer-go.org >>>>> http://computer-go.org/mailman/listinfo/computer-go >>>> >>>> _______________________________________________ Computer-go >>>> mailing list Computer-go@computer-go.org >>>> http://computer-go.org/mailman/listinfo/computer-go >>>> >> _______________________________________________ Computer-go >> mailing list Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go > > _______________________________________________ Computer-go mailing > list Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJWsP1LAAoJEInWdHg+Znf4plkP/0iGgTiknky/A71k91/BwtxK BgQPCFhtY5K8Np49zlwXHY+52h4k8673Ny9J4BkDd6ak59qnBhowgEKOQpMnj71e yxRyKPY38W2+VNyZ2Gwg5epUjKYRdb4qhBIlla+aMDWqhP6BiA+x5bMt1wEOsMlL MiuQb+aYG0yNMkFgBJmV48nzhw+uf3JgV7lRTomla0ZtPjgoCSjv0wEPT/K8+KHn 7jccrpwMHM9KlGxqEaXlVjRTY5rFNAi+IUvssXq1W/u94cfQp1MfJ6uwyd8+ixJJ epZRe2SKeC/NKtCBAFXh2gx/1ru4YKdL5Cub3Jbd7GLINYPnh88ro5h2mfvWe+/g ore2nzm+F1Xk2b1FzlmOiJ6SmRSEnCPhlZukfl3GVANtrpPcbWkJhgVTohcutpYn zuXGEw2H0djmht9rrK2+OzdU0GDXiYoMkLNroBwnOQSN/YFmXXUJwrOr197PjC4y nEEuX7PvaoBPRUaWZPrOQUiNO8SiyJRNJ1k3mOHOya82wC9hHKx+RZvZC8rO/yLZ XA4APnY1cLQU28DArXORtQdiDJXCfbQmSQo9MBhgHw58c16ZgtAAgIZr8Ro3EVDq vYn4G8z/T7yzUE6ZTuQZepcpqqirK3fGWRb5QhsqML6VZe85Ao4WlaOx1FRslAER ZLcsE6/+jMm/6Fh17MmK =+mkt -----END PGP SIGNATURE----- _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
