Hi David, I argue that the matches should be longer, perhaps 30 minutes per side. They should more closely resemble time controls used in a serious competition.
Here is the reason I say that. One could argue that with computers it doesn't matter, they do not need to be constrained as much by our sense of time - they do not feel pressure or get rattled if they play too fast and they don't get bored or lose focus if they play too slow. I've argued that way myself many times. However, the choice of time control, in my estimation, has a good chance of influencing the outcome, especially if we view this as a test of a strong commercial program versus a new experimental technology, which I think it is. Mogo is a program that clearly performs better with more time. I suspect that MFGO is a program that is close to optimal at 10 or 15 minutes. I can't say that for sure, perhaps you can give us your insights on that. In such a case what is "fair" depends on the point of view of the observer. If someone wanted to see Mogo dominate such a match he would consider short time controls "unfair" and the opposite would be true if one wanted to see Many Faces win. Of course I could be wrong, perhaps Many Faces is the one that would benefit more from extra time - but I'm working from the assumption that Mogo would benefit the most based on my own knowledge of how UCT works. Regardless of the time control used another issue is the selection of hardware. Doubling the computer power effectively doubles the programs thinking time. Having considered all of these issues, and also taking into consideration that this is a contest of sorts, it makes sense that we should testing at a level that simulates or at least approaches serious computer chess time-controls. Certainly no faster than 30 minutes per side. These are levels at which most humans will take the results seriously. In addition to this, it makes sense to know what hardware and what time-setting is being used. Many programs on CGOS were set to play very fast, often indicated their level in the name of the program something like "mogo4k" or something similar. So if we set a liberal time control on CGOS 19x19 we could publish the identify of the players and draw conclusion based on that. Mogo could be tested at several levels and/or hardware configurations and so could Many Faces. It's not difficult to set up a rotating script for logging off one bot and starting up another. (By the way, the right way to do this is to select the bot RANDOMLY, not to rotate back and forth.) The server does report the time each side spent calculating in the SGF files, although it's not reported on the web sites, so this is useful information if we are considering the scalability of programs. My feeling is that there is likely to be a crossover point - that MFGO will win at time-controls faster than this and Mogo will win at time-controls slower than this. That point may be beyond what we can test, or it may be testable on the CGOS server soon. By the way, I would probably argue for longer than 30 minutes per side, but for a server like CGOS that would involve a long wait between matches. Anyway, that's my 2 cents. - Don David Fotland wrote: > 10 minutes per side should be enough for Many Faces 11. Version 11 has > fixed search limits, and only does time management if it runs low on time. > It can usually play a game in 10 minutes on the computer I'll use. It will > be slower against Mogo since the games are longer and there might me more > unsettled situations to read. If you do add more time, 15 or 20 minutes per > side should be enough. > > David > > >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Chris Fant >> Sent: Tuesday, October 23, 2007 1:27 PM >> To: computer-go >> Subject: Re: [computer-go] 19x19 CGOS >> >> >> I oppose more time per side. >> >> On 10/23/07, Christoph Birk <[EMAIL PROTECTED]> wrote: >> >>> On Tue, 23 Oct 2007, Olivier Teytaud wrote: >>> >>>> http://www.lri.fr/~teytaud/cgosStandings.html >>>> >>>> If someone wants to test it, the port is 6919 on machine >>>> pc5-120.lri.fr. 10 minutes per side. But only try it if >>>> >> you want to >> >>>> take risks, it is almost surely not stable yet, and the >>>> >> connection >> >>>> might be refused for an unknown reason :-) >>>> >>> Am really curious to see MFGO, Crazystone and Mogo play at >>> >> 19x19. But >> >>> I suggest allowing more time, at least 20 minutes per side. >>> >>> Christoph >>> >>> _______________________________________________ >>> computer-go mailing list >>> computer-go@computer-go.org >>> http://www.computer-go.org/mailman/listinfo/computer-go/ >>> >>> >> _______________________________________________ >> computer-go mailing list >> computer-go@computer-go.org >> http://www.computer-go.org/mailman/listinfo/computer-go/ >> >> > > > _______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
