Peter, many thanks to you and all the other participants in the Computer Go Tournament! It was fun watching and participating! Here's hoping we have a nice strong cluster or two mowing down the opposition next year!
----- Original Message ---- From: Peter Drake <[EMAIL PROTECTED]> To: Computer Go <computer-go@computer-go.org> Sent: Friday, August 8, 2008 10:52:02 AM Subject: [computer-go] Report on 2008 US Go Congress Computer Go Tournament (This is about the computer-computer tournament, not the Kim-MoGo match.) Results of the Computer Go tournament at the 2008 US Go Congress in Portland, OR, USA can temporarily be found at: http://svcs.cs.pdx.edu/cgo2008 I would like to thank: Hierarchical Systems Research Foundation for providing the bulk of the $1250 prize and travel expense money (the rest was donated anonymously); Bart Massey, Kathi Lee, and everyone at PSU for providing a physical venue and helping with technical setup; Bill Shubert and everyone at KGS for providing a virtual venue; and all of the programmers and operators involved. The tournament was a double round-robin tournament, so each program played as W and B against each opponent. All games were 19x19, 45 minutes per side sudden death, Chinese rules, 7.5 komi. Here are the results in traditional round-robin format: 1 2 3 4 5 6 7 | Total ----------------------------------- 1. GNU Go XX 11 01 11 11 11 11 | 11 2. Many Faces 00 XX 11 11 11 11 11 | 10 3. Leela 10 00 XX 11 11 11 11 | 9 4. House Bot 00 00 00 XX 11 01 11 | 5 5. First Go 00 00 00 00 XX 11 11 | 4 6. Orego 00 00 00 10 00 XX 11 | 3 7. Butter Bot 00 00 00 00 00 00 XX | 0 NOTABLE EVENTS: The tournament was surprisingly smooth. Every game but one ended in resignation. SlugGo was not able to attend due to numerous problems encountered while assembling a new hardware cluster. We decided to enter GNU Go (3.7.10, level 12) instead, as GNU Go had only refrained from entering due to the expected presence of SlugGo. The version of ManyFaces used includes Monte Carlo search. In fact, we believe GNU Go was the only non-MC program in the tournament. In ManyFaces-Leela (that is, the game with ManyFaces as white and Leela as black), David Fotland discovered that his T61 laptop was unplugged, so ManyFaces was running at half speed to conserve power. The laptop was properly plugged in mid-game and ManyFaces went on to win the game anyway. In Leela-GNU, KGS reported a win for GNU under Japanese rules, because we had failed to set one of our KgsGtp configuration files to specify Chinese rules. Since both programs believed that they were playing under Chinese rules, we re-scored under Chinese rules and found the game was a win for Leela (as Leela had reported). In FirstGo-GNU, FirstGo played out a ladder. In Leela-FirstGo, FirstGo played out a short ladder. It began to run when caught in a second, much longer broken ladder; 13 moves were played before Leela abandoned the chase. In FirstGo-Orego, Orego resigned after 76 moves -- probably a bit premature. After the event, Fotland had hew new multithreaded version of ManyFaces up and running. It played three games against GNU Go and won all of them. LESSONS FOR NEXT YEAR'S TOURNAMENT: We definitely want to do this again at next year's Go Congress in Washington, DC. Is there anyone in that area willing to direct the tournament? Double round-robin ran extremely smoothly. We were able complete our games in less time than in a tournament with fixed rounds, because a new game could start whenever one ended. In fact, we were able to run some programs on multiple machines, thus completing some games in parallel. If there are a lot of entrants, perhaps they could be filtered by a Swiss tournament or the results of previous tournaments before playing round-robin among the strong programs. The TD's program (Orego) was ineligible to win to avoid any appearance of conflict of interest. In the future, we would do what has been done in the past: appoint a deputy director or committee to make any decisions regarding the TD's program. If the prize money is very large, it may be important to require that source code be made available for inspection by the TD to avoid any question of plagiarism. We should do a better job of making rules explicit, e.g., all results stand, even if it is discovered afterward that a program was running with incorrect parameters. (We discovered that FirstGo was running with the incorrect time setting; the result stood.) It would be good to have some talks by the programmers involved. The expense of travel is a serious impediment to attendance. Some of our prize money was used to defray such costs. It would be better to clarify the distribution of prize money in advance. Perhaps there could be one pool of prize money for everyone (with more money for programs that place higher, of course), and a second pool for those programmers who travel, with appropriate consideration to prevent a trivial program being entered just to collect travel money while the "programmer" just attends the Congress. Our intent is to encourage interesting interactions between programmers. There were useful conversations in the room as the tournament ran. It was great when Anders Kierulf dropped in even though his program SmartGo was not entered. Peter Drake http://www.lclark.edu/~drake/
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
