Don Dailey wrote:
It would be great if you would provide recommendations for a simple
conversion formula when you are ready based on this study. Also,
if you have any suggestions in general for CGOS ratings the
cgos-developers would be willing to listen to your suggestions.
- Don
My suggestion would be to tell programmers to use a different login each
time they change version or hardware (most do that, already), and use
bayeselo to rank the programs.
This would be best if combined with a mechanism to recognize that two
logins are versions of the same program (for instance, if they use the
same password), and avoid pairing them.
Regarding correspondance with human ranks, and handicap value, I cannot
tell yet. It is very clear to me that the Elo-rating model is very wrong
for the game of Go, because strength is not one-dimensional, especially
when mixing bots and humans. The best way to evaluate a bot in terms of
human rating is to make it play against humans, on KGS for instance.
Unfortunately, there is no 9x9 rating there. I will compute 9x9 ratings
with the KGS data I have.
What I have observed with Crazy Stone is that gaining Elo points against
humans is more difficult than gaining Elo points against GNU Go, which
is more difficult than gaining Elo points against MC programs, which is
more difficult than gaining Elo points against itself. But it is more an
intuition than a scientific study.
Rémi
_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
