Robert Jasiek wrote: > Don Dailey wrote: >> Just a few years ago it was widely held that computers will not reach >> Dan level "in my lifetime" even in 9x9 Go. When it happened in 9x9 >> go, it was not accepted - the day it happened passed us by and nobody >> noticed it. It's probably still not common knowledge and it will >> take time for it to be generally believed. > > What is the basis of the claim that a program has reached a certain > human rank level? > > There should be systematic tests. Let it play against many humans. Let > it enter human tournaments. Use a meaningful evaluation context. Yes, there should be. Right now it's very informal and anecdotal. Strictly speaking it is not possible to evaluate any player - the ELO system is subject to the laws of probability. So no matter how many games you play people can claim it's a fluke and blame the playing conditions or other factors. That's the point I was really trying to make. People will be resistant to the idea no matter what - so the best you can do is supply overwhelming empirical evidence and then if some want to be unreasonable they are easily identified - at least by reasonable people.
> > For some such evaluation, let me refer to a useful handicap system for > 9x9, which has been used in some European 9x9 "Championships": For the > first 10 rank differences (0, 1,.., 9) decrease the komi from 6.5 for > an even game by 1 point per extra rank. (Komi can become negative.) - > OC, I prefer to see even games. OTOH, until the program rank is well > known, it may be suitable to let a simgle human (the more humans the > better) play until the handicap becomes stable. > For 9x9 ELO works better. For 19x19 it's less clear cut. The handicap system appears to be a good system at 19x19 and has the very nice merit of allowing grossly mismatched players to compete. I think the two systems can be married by adding a fixed offset per stone handicap to your ELO. - Don _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
