2009/4/8 Zhiheng Zheng <zhiheng.zh...@gmail.com>: > I think most of test are designed by people who is stronger than best > computer go program. So if MC program fail to pass a test, it is most likely > MC is wrong. MC program is strong in some aspect, but week in other aspect. > And the test suit is too focus on special aspect. We might split the test > into different category, like opening, end game, L&D etc, and for each > category we can set different weight. And when program pass all test, we can > calculate final score adjusted by weight. > > ZZ >
Sure program would be "wrong" but it does not imply weak. It is easily a case that program A would fail on tests and program B would do great on those, but on actual game program A would wipe the carpet with program B. I think L&D id kinda showcase for this. Early MC programs had huge issues with it but they were still pretty strong. And their opening ... did not even resemble opening of humans. But still they are/were pretty strong. Passing test is nice, but it matters only if it REALLY translates to winning games. So I am afraid that for a while playing few hundred games is the only meaningful measurement available. End game is another issue. MC programs only aim on winning, so they endgame is nor perfect in sense human would define it, but perfect enough to win if the game is winnable. Petri -- Petri Pitkänen e-mail: petri.t.pitka...@gmail.com _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
