This is an interesting problem. It seems to me that the reality is that when you are talking about non-ideal play, ranking systems aren't linear. Program A could beat B which could beat C which could beat A. How would you rank those? Clearly there is going to have to be some degree of arbitrary selection. I propose convenience as the best reason for picking one anchor over another. I think a completely random player is the only other choice from a theoretically perfect player that doesn't have arbitrariness. But, by defining players relative to that anchor, we would really be measuring how effectively a program exploits a weak player rather than how good the program is.
It is my opinion that it is more important to have a relative ranking system than an absolute system. - Nick On 12/28/06, Aloril <[EMAIL PROTECTED]> wrote:
On Wed, 2006-12-27 at 21:34 -0500, Don Dailey wrote: > I'm having an interesting problem - my hope is to set > a random legal move making player (who doesn't fill > 1 point eyes) at ELO zero. > > I feel this would define a nice standard that is > easy to reproduce and verify experimentally and > at least would be a known quantity even 100 years > from now. > > But I'm having a difficult time creating players > who are slightly better than this at 19x19. I need > incrementally better and better players. I suspect this is quite hard problem. On 9x9 we have some of this and I suspect even there "do not fill eyes random" (PythonBrown) has not yet settled (maybe 100-200 ELO overrated). Probably too few weak players ;-) On 19x19 I think problem is much harder and required amount of intermediate players is much bigger. I'm of course interested in hearing your experimentation results. Maybe I'm wrong and it is actually feasible. My vague recollection was that random player is maybe 200 kuy, "do not fill eyes" adds 60 stones, atari detection adds about 20-30 stones, idiotbot is maybe 100 kuy, weakbot50k maybe 50 kuy. However differences between computers tend to be much bigger than when they play against humans! For example GNU Go 2.0 can give Liberty 1.0 easily 9 stones and win more than 50% of games (based on few ha9 test games), but at KGS they are rated at 10k and 14k. Even WeakBot50k is rated at 20k while latest GNU Go rated at 6k can give it numerous handicap stones (much more than 14 stones, I think it's more than 40 stones). Here is my proposal for anchor player: Use GNU Go 3.7.10 (or any enough recent with super-ko support) at level 0 and use well defined randomization on top of moves it returns. Ie. ask all_move_values (lists only moves that gnugo considers positive) and add remaining moves and then apply slight randomization so that it still plays close to original strength but is much more unpredictable than GNU Go. Example program (by blubb and me): http://londerings.cvs.sourceforge.net/londerings/go/gtpTuner/ Reasons: - reasonably strong, no need for huge amount of intermediate players - source code available - well known entity - with some randomization should be unpredictable I suspect that GNU Go without randomization is too predictable. This is very clearly case on 9x9 board and possibly on 19x19 too. -- Aloril <[EMAIL PROTECTED]> _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
