Hi!
On Fri, Aug 30, 2013 at 11:41:12AM +0200, Detlef Schmicker wrote:
> Up to now I always was able to measure oakfoam improvenment by playing
> against gnugo.
> (700 playouts against gnugo level 10 and 300 playouts against gnugo
> level 0)
By nature of the probability distribution, the play-testing
measurements are most sensitive when your program is around the 50%
winrate. Since you want to test with as many playouts as feasible wrt.
time allocated (since that's closest to the real playing conditions),
what I do is use komi to even the game out (fairly big komi at that,
in the order of few tens of points).
> But now we seem to be at a strenght, that makes this not very sensitive
> anymore. I can change parameters,which have significant effects on
> regression tests, but do not change playing strength against gnugo
> anymore. I included fuego and pachi at a playing level playing 50%
> against gnugo, but this only improved the sensibility a little.
>
> How do you handle this problem?
>
> What do you think is the reason?
>
> Thanks a lot for any secrets:)
I'd say that possibly it's not the measurement being less sensitive to
strength changes of your programs, but the absolute strength of your
program less sensitive to bugfixes you make. Our returns are diminishing
and like for human players, the stronger you get the more it takes to
improve further, especially in the way of incremental bugfixing.
--
Petr "Pasky" Baudis
If I had more time, I would have written you a shorter
letter. -- Blaise Pascal
_______________________________________________
Computer-go mailing list
Computer-go@dvandva.org
http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
