I have to agree with Don. This year I went from knowing nothing about Monte Carlo and UCT to world champion. I tried more than 400 variations on the basic UCT algorithm or playout strategy during 6 months of intensive development. The engine is written in C and tuned from the start for performance. I couldn't have done this many experiments (often several per day), without a very fast engine, because I used 1000 game contests to see if there was improvement, to get statistically significant results. Part of the reason I won is because the basic UCT/MC is so fast that I can incorporate the slow Many Faces knowledge and still get over 10K playouts per second per CPU on 9x9.
David > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:computer-go- > [EMAIL PROTECTED] On Behalf Of Don Dailey > Sent: Thursday, November 27, 2008 8:56 AM > To: computer-go > Subject: Re: [computer-go] Re: hex robot > > On Thu, 2008-11-27 at 13:03 -0200, Mark Boon wrote: > > You say you're going to try to make a prototype first and then when it > > shows promise, move to a more flexible language like Java. What > > language are you intending to use? It seems the wrong way around to > > me. Develop the prototype in a flexible language and when it seems to > > work, move it to a specific langauge that can leverage some specific > > CPU features. Seems to make much more sense to me. > > This is clearly the conventional wisdom, but I think it's a mistake for an > "ultimately" high performance game program, even for an initial > prototype. The supposed benefit is "freedom to experiment" and nail > down your algorithm, but I think this is a myth. > > I know that sounds like blasphemy, but when I've tried this before I > discovered that even though C/C++ is pretty gnarly, it has at least 3 major > advantages that might not matter for many other things, but is very important > for games: > > 1. It's about as fast as you will get. > > 2. It is superbly flexible. > > 3. It is not a memory hog. > > > To summarize my experiences, these wonderful high level languages are full of > limitations and you spent more time pulling your hair out to > work around them. Even the speed limitation is more of an issue that > you think, if you make heavy use of testing. If you don't make heavy use of > testing, you don't know what you are doing anyway. > > For instance, let's say you are experimenting with an algorithm. At some > point you must test that algorithm to see if it's better than what you were > doing, or to compare it with something else. You must play a large number of > games to measure superiority, unless it is overwhelming. > Most changes won't be overwhelming and even if it is you still need a lot of > games to know how overwhelming it is. With a slower language this is > correspondingly a slower process, negating much of the supposed > high level language advantage. I spend more time waiting on tests than > programming, even in C. > > Someone says, but if you have a large bank of workstations ... Well > if you do, it doesn't change the fact that you are wasting them. An > author for a strong playing engine for any game will be able to utilize > as much power as you give him for testing. If I had 100 workstations I > still would not use Ruby (a joy to program in and one of my favorites) because > what a stupid waste of resources it would be to make those 100 workstations > perform like only 2 or 3 workstations. > > > When developing the simple reference bot, and experimenting with ideas > to make it play better with fewer simulations, guess what? I am > performance bound - I cannot test ideas fast enough and this would be worse > with a high level language. > > A word about Java, which will probably produce some anger because I know > some of us here love Java. I have never seen a top level, non-trivial > game playing program in Java. I don't think you could build a strong > chess program in Java. Probably more due to memory issues than performance - > but I think for a top chess program performance would be > an issue too. Yes, it's possible to write some programs in Java that > are almost as fast as C, but probably not a chess program. You > really need system level programming type of flexibility that C provides, not > high level abstractions that the processor doesn't care about, and the > compiler cannot optimize away. > > An ad-hoc poll: What is the strongest Java playing program on CGOS? > > I can't see building a very strong MCTS GO program in Java, because even > in C, the tree does not fit in memory. > > My little reference bot can be done in Java however. The performance > loss is modest and memory isn't an issue. But the code is clearly > larger. Perhaps because I'm not a Java expert, it seems to take more > code to do the same thing in Java. All kinds of scaffolding required > to use the standard data structures. Much more typing. > System.out.printf() is just one example. Seems like a little thing > and I'm nitpicking - I see some value in this kind of anal-ism for big > projects especially, but it's pretty annoying. I cannot see an ease of use > difference between Java and C but I can imagine with large projects > Java has some advantages. With it's strong typing Java seems almost > as low level to me as C. > > To summarize, I have found over the years that just plain CPU/MEMORY > performance is the primary barrier not just to program strength, but > development time. > > You must measure development time 2 ways. One is how many man hours are > spent, and the other how much "calendar" time is spent. With a high > level language you can (in theory, but maybe not in practice) minimize man > hour time, by giving up calendar time and dramatically increasing > testing time (which need not involve the programmer.) But you also > would have to be willing to wait much longer between tests and be willing to > work piece-meal on a project while mostly waiting for tests. > You will also inevitably take more shortcuts in testing because it takes so > long. > > > - Don > > > > > > _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
