I made a reference bot and I want someone(s) to help me check it out with equivalent data from their own program. There are no guarantees that I have this correct of course.
Doing 1 million play-outs from the opening position I get the following
numbers for various komi:
playouts: 1,000,000
komi: 5.5
moves: 111,030,705
score: 0.445677
playouts: 1,000,000
komi: 6.0
moves: 111,066,273
score: 0.446729
playouts: 1,000,000
komi: 6.5
moves: 111,040,546
score: 0.447138
playouts: 1,000,000
komi: 7.0
moves: 111,029,204
score: 0.4333795
playouts: 1,000,000
komi: 7.5
moves: 111,047,843
score: 0.421281
(I also get a score of 0.524478 for 0.0 komi)
Score is from blacks point of view. Score is not the score of the
best move of course but the combined average score of all 1 million
play-outs using the stated komi and ranges from zero to one.
I am going to build a test harness to compare multiple bots side by
side using gtp commands. I made up two private gtp commands to
facilitate this:
ref-nodes -> return total moves executed in play-outs
(including both pass moves at end of each
play-out.)
ref-score -> return total win fraction for black.
NOTE: both commands report stats from last given genmove search.
I hope to get peoples opinion on the following implementation
specification. I'm definitely not a writer, so I need to know if this
very informal spec is enough at least for experienced MC bot authors
or where there are still some ambiguous points.
I'm using the following implementation specification:
----[ bot implementation specification ]----
This is an informal implementation specification document for
writing a simple Monte Carlo Bot program. The idea is to build a bot
like this in ANY language and test it for performance (and
conformity.) Can be used as a general language benchmark but is as much
about the implementation as the language. This specification assumes
some knowledge of go and Monte Carlo go programs. (If you don't like
it, please write a better one for me!)
1. Must be able to play complete games for comprehensive conformity
testing.
2. In the play-out phase, the moves must be chosen in a "uniformly
random" way between legal moves that do not fill 1 point eyes and
obey the simple-ko restriction.
When a move in the play-out is not possible, a pass is given.
3. Play-outs stop after 2 consecutive pass moves, OR when N*N*3
moves have been completed, except that at least 1 move gets tried
where N is the size of the board. So if the board is 9x9 then
the game is stopped after 9*9*3 = 81*3 = 243 move assuming at
least one move has been tried in the play-outs.
4. A 1 point eye is an empty point surrounded by friendly stones
for the side to move. Additionally, we have 2 cases. If the
stone is NOT on any edge (where the corner is an edge) there
must be no more than one diagonal enemy stone. If the point in
question is on the edge, there must be NO diagonal enemy stones.
5. Scoring is Chinese scoring. When a play-out completes, the
score is taken accounting for komi and statistics are kept.
6. Scoring for game play uses AMAF - all moves as first. In the
play-outs, statistics are taken on moves played during the
play-outs. Statistics are taken only on moves that are played by
the side to move, and only if the move in question is being
played for the first time in the play-out (by either side.) A
win/loss record is kept for these moves.
7. The move with the highest statistical win rate is the one
selected for move in the actual game. In the case of moves with
even scores the choice is randomly made between them.
8. Pass move are never selected as the final move to play unless no
other non-eye filling move is possible.
9. Random number generator is unspecified - your program should
simply pass the "black box" test and possible an optional
additional test which consists of long matches between other
known conforming bots. Your program should score close to 50%
against other "properly implemented" programs.
10. Suicide not allowed in the play-outs or in games it plays.
11. When selecting moves to play in the actual game (not play-outs)
positional superko is checked and forbidden.
12. If stats for a move was never seen in the play-outs, (has a count
of zero) it is ignored for move selection.
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
