Thanks for the comments and suggestions. My responses below ... On Tue, 2008-10-14 at 09:00 +0200, Denis fidaali wrote: > > Hi. > First things first : i think the specification is enougth as it is. > I hope that we can end up with something useable by anyone, > even non go people as a final goal. We'll have to get non-go experienced > people to beta-read it (and criticized) for us, i suppose. And we'll probably > have to get a HTML version with some fancy illustration (wich i won't be > helpfull for)
I don't really want to be too involved in drafting this as it's not my forte. However I hope someone else (who is a better writer) will be willing. I recognize that my draft is far from ideal and I just hoped it was enough for those of us experienced to make sense of. Ultimately, I think we have to explain the rules and we could start with John Tromps logical rules contained at http://homepages.cwi.nl/~tromp/go.html and modify them accordingly or just integrate them into the document. Note that the only different between those rules and our rules is the suicide rule. > > I really look forward to be able to get involved non go people easily :) > I'm pretty sure a lot of them accepting this contest would end up > being very valuable for the community :) It's not clear to me whether others will respond, but it would be pretty cool and it would probably grow the computer go community at the same time. > > We'll probably have to get a bit deeper in the gtp-part ultimately. Do you mean the explanation or the actual commands I added? For the explanation we could point them to the excellent GTP draft standard and then explain this as 2 additional commands and then clarify what I wrong. I made only 2 GTP command as I wanted to be careful not to add anything that would likely impose a "barrier" to implementation and this influenced the specification in many places too. I would personally have made slightly different choices about some things which would have complicated the draft and the implementation. But I resisted doing that to keep it as generic as possible. Examples: my own test show that you should stop the AMAF tally a little early. Christopher Birk uses a decaying weight in his tally. I'm sure everyone who has implemented one of these has their own little "enhancement" but we want to keep this real simple. > -------------------------------------------------------------------------- > > === > 5. Scoring is Chinese scoring. When a play-out completes, the > score is taken accounting for komi and statistics are kept. > > I think i would like it if we just gave how it should be done. > Using the eye definition we impose anyway. > > ------------- > I propose : > ------------- > 5. Scoring is done at the end of a game (two consecutive pass) > , in the following way : > each stone on the board gives a point to the player who > owns it. An empty intersection gives a point to the player (if any) who > has a stone on each orthogonal intersection around it. > If black's score is greater than 0.0 then it is scored as a black win. > Otherwise > it is scored as a white win. Yes, or as I mentioned we could borrow from the logical rules. > > === > 1. Must be able to play complete games for comprehensive conformity > testing. > > I do not quite understand the point. But it can't really hurt either .. :) It seems slightly out of place but most of this is focused on how to do the play-outs and I wanted to make it clear that this should be able to play a real game and real games could be used for conformance testing (if you don't score very close to 50% against others, something is wrong.) > > === > 2. In the play-out phase, the moves must be chosen in a "uniformly > random" way between legal moves that do not fill 1 point eyes and > obey the simple-ko restriction. > > When a move in the play-out is not possible, a pass is given. > > I'd like that we got more descriptive on the simple-ko restriction, if > possible. > (i'll try to propose something, but i'm getting low on time right now) Yes, this goes back to my point (and yours) that we really need to explain the rules. > > === > 3. Play-outs stop after 2 consecutive pass moves, OR when N*N*3 > moves have been completed, except that at least 1 move gets tried > where N is the size of the board. So if the board is 9x9 then > the game is stopped after 9*9*3 = 81*3 = 243 move assuming at > least one move has been tried in the play-outs. > > I don't quite get the point of the "except that at least 1 move gets tried" > part Because you don't want play-outs that stop before they begin. In order to get samples of moves you must play at least 1 move in the play-outs even if you are beyond move 243. > > === > ref-nodes -> return total moves executed in play-outs > (including both pass moves at end of each > play-out.) > > ref-score -> return total win fraction for black. > > i do not find ref-nodes that much descriptive for "return total moves > executed in play outs" > Maybe it is quite standard to call that number ref-nodes ? As it's only amaf, > there are no node per-se are there. what about a ref-numberOfMove command ? I'm not too fussy about the name of the command but I would take a hint from the current GTP standard. I don't see them constructing names with mixed case and I would avoid that even though that is exactly how I do variable names in high level languages. I used nodes because I have come to think of "nodes" as positions created from moves and it's concise to me. Do you have an alternative suggestion that is more concise and does not mix case? (Mixed case is ok in my own programs, but it requires people to remember not just the name of the command, but how it's spelled case-wise.) Perhaps the standard isn't case-sensitive, I don't remember. > -------------------------------------------------------------------------- > Here are the data you requested for with the implementation i want > to use as a reference. I'm not able to get value for integer komi. > (my system do no account for draws ..) These numbers appear to be close to mine. I didn't do 0.5 komi but the others agree closely. I'm not going to have standard values, you will just compare to other programs (presumably some standard reference program could be used as the gold standard once we believe in it.) > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Komi 0.5 > mean score =0.5244261847821416 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > komi 5.5 > mean score =0.44674397181685754 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Komi 6.5 > mean score =0.4467712426921182 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Komi 7.5 > mean score =0.42132957622630574 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 87317.15777498983 Playout/sec > Time=11.461321297 > Number of playout=1000770.0 > Mean moves per sim 111.06128680915695 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > It uses the mersene twister for random-generation. > But this is a 4 thread test. I use nanotime as an help > to set up the (per thread) seed, combined with thread number. I want to be very careful not to specify these kinds of details such as which RNG must be used or how uniform random move selection should be done. I want to specify the "what" not the "how." "What" the task is, not "how" it should be done. In this language the "task" is to pass the black box test but I hope that it cannot be passed without following the specification closely. The specification is already constrained enough but hopefully not beyond what is necessary. > I think it's interesting to give the Playout/sec score. An external tester will provide that number. That's what I had in mind but in a contest format you don't ask a program to grade itself. > Then your reference bot "can" be used as a refence benchmark. Any bot can be a reference implementation of the "standard" but none needs to be viewed as a reference performance benchmark. Just for a quick reference using IBM's java I did the 0.5 komi benchmark and also timed it: komi: 0.5 nodes: 111052804 score: 0.524683 PPS: 14492.7 PPS is playouts per second. processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz stepping : 6 cpu MHz : 2667.000 cache size : 4096 KB > That is not perfect of course, but that gives something to chew. > (I get quite a large variation in speed from run to run > with 1000 000 simulations. Ranging from less than 80k/s to close to 90k/s > with 4 threads over my 4 cores. > > My implementation is in java too, and has nothing fancy to it, > so i might as well publish it later on. I probably should clean it > up a bit. And make a few optimisation (by refactoring). I think anyone wishing to participate (officially) should submit their source code. You can post numbers but it cannot be taken seriously without code and verification. Perhaps we could require that AT LEAST the executable be submitted? Right now my implementation is the reference implementation but I would consider a better written one. The criteria for "better" is short, simple and readable, not high performing or feature-rich. - Don > > > > > ---------------------------------- > Don said : > ---------------------------------- > I made a reference bot and I want someone(s) to help me check it out > with equivalent data from their own program. There are no guarantees > that I have this correct of course. > > Doing 1 million play-outs from the opening position I get the following > numbers for various komi: > > playouts: 1,000,000 > komi: 5.5 > moves: 111,030,705 > score: 0.445677 > > playouts: 1,000,000 > komi: 6.0 > moves: 111,066,273 > score: 0.446729 > > playouts: 1,000,000 > komi: 6.5 > moves: 111,040,546 > score: 0.447138 > > playouts: 1,000,000 > komi: 7.0 > moves: 111,029,204 > score: 0.4333795 > > playouts: 1,000,000 > komi: 7.5 > moves: 111,047,843 > score: 0.421281 > > (I also get a score of 0.524478 for 0.0 komi) > _________________________________________________________________ > Installez gratuitement les 20 émôticones Windows Live Messenger les plus fous > ! Cliquez ici ! > http://www.emoticones-messenger.fr/_______________________________________________ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/
signature.asc
Description: This is a digitally signed message part
_______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
