[computer-go] programming (languages) light-simulation contest
--------------------------------------------------------------------------------------------
The reason i have been eager to make so sure my implementation was conform
to what i had in mind (this time ..) is that i wanted to be able
to test various (possibly a lot) of different implementations
in different environments (machines, langages, memory etc etc ..).
So far i have developed simple light simulators for those environments :
(all single threaded)
- Java 32 bits.
- C 32 bits linux.
- C 64 bits linux
- Assembly 32 bits on some intel duo with no SSE
- Assembly 32 bits on some intel duo with some SSE
- Assembly 64 bits with and without SSE
- NVIDIA Cuda C
I was quite dissatisfied with the results. The development cycle was just too
long,
there was just too many things to be tuned and tried. I got lost in the
different versions
not remembering how it had been validated, and more than often plainly loosing
where the code
was (i used more than 5 computers, most of witch are not there anymore - with
no clear backup strategy)
, when it had not been changed to death anyway (without backing up - so not
much left to recover).
So i figured out, what i had a need for to begin with, was method.
-------------------------------------------------------------------------------------
I have followed loosly those "hey here is a new interesting language"
conversations.
My conclusion where always the same : it's just a confrontation of random
thoughts.
So i kinda stopped trying to follow them. I think things are started to go in
a good
direction now : it's mostly useless to talk about ideas, or unverifiable data
in a list like this one.
So what i like a lot about how things are going now, is that it seems that
the community
has got here a strong will to get something to talk-about in the first
place. Still i hardly
anticipate this to gives something fruitful in the end. That would be
because everyone's goal
seems different from everyone's else. I guess that there is a goal worth
pursuing underlying
the conversation. I see such a goal for myself. Which would be to answer the
question : "what
does a really good version of light playout looks like". I chose "light
playouts" because
it's so easy now to agree on an implementation : most people seems to use
quite exactly the
same one. That's why i was able to just ask for other to confirm that mine
was correct.
By the way, i think that AMAF implementation based on those light-play-out,
is straight forward
too. That's why i chose those to try to get the most out of everybody's else
experience.
As for me, i'm really NOT interested in knowing "what langage is good for go
programming". That's
simply not a question i can ask myself, nor anyone else. This question
doesn't make any sense for
me. Still if someone can get the "standard light playout" right in less than
10 code line,
and they are very understandable lines. I would be very happy to see it. But
it would never mean for me that,
"this language is BEST". Even if the peformances are optimal there. I think
90% of the
"this language is the best" debate gets it's root in some affective part of
the people
engaged with it.
So .. i'm not interested at all in a langage contest. I really don't care
about that.
Still i find interesting to argue over facts, rather than about random
thoughts.
Now, to really set the frame-rules of this "language contests" seems a
challenge
really interesting in itself. I don't think it'll be easy to do. I think that
it has to be so that most people do agree that it has been set up properly.
And then, it has to be so, that absolutely anybody that has the slightest
interest
in the matter will be able, and will be willing to participate.
I didn't really put a lot of thought in it, but here are what i can think of
right now.
-It should be useful. So let's make it a programming contest, rather than a
pure
language contest. It may be slower to get enough data to be able to compare
different
languages.But then it'll make it so that more people can participate and
find values in it.
I would never try anything for a "language contest" myself. But i may well
want to invest
some time in a programming contest. So when i say "it should be useful" the
question i really
ask is : what information (or benefices) should we hope to get out of this
challenge.
That would both help us to get nice ideas about how to set it, and a base
for discussing if what
we proposed is right or wrong. And then later on, we can compare the real
benefice of it, with
what we expected to get.
- It should allow a lot of freedom. Once again this is based on the
assumption, that
it is most probable that NOBODY will actually join the contest. So we have
to get
it as wide as possible, as interesting as possible for as many people as
possible.
And with all that, it has to looks like there is a frame easy to follow. It
has to look
like something valuable will come out of it. It has to be satisfying for the
participants.
So here is what i would hope for :
- To get a set of data, that everybody can use to judge if it's
implementation is conform.
- To get a set of algorithms easy to use, and that are guaranteed to be
conform
- To get very efficient algorithmist propose unheard of solutions, and push
back
the limits of what seems to be doable. And to be able to realize for
example, that
it is possible to get as much as 1000 000 000 evaluations per processor and
per second
on the processor and machine i use everyday.
- I think that anybody should be able to propose an implementation. And then
anybody else should be able to study the black box submitted. But i disagree
when
we ask for every-participant to give full access to their code. I think that
most
people would do it anyway. But if someone really has interesting ideas, it
could
benefits all to allow him to submit a black-box. Maybe later on, he will
share
the mechanisms of it with the rest of us. AND people could wonder at how the
thing
work. Which could be more beneficial than to get a code that nobody
understand
in the first place ....
Then the point i would like to be discussed are :
- how do we make it easy that people only have to focus on what is relevant
(ie the algorithms)
I think we need both to propose an easy way the measure the behavior of the
black box
AND i really think we should build up tools to help iterative validation of
the
algorithms people are trying to make. In any cases, those are things i
really try
to solve for myself, because i'm trying to set-up such a programming
contest of my own.
With only me involved in it. I just want to know how fast a light
simulation can be.
It'll just take a lot of time and effort.
---------------------------------------
Stuart A. Yeates syeates at gmail.com
said :
---------------------------------------
If we, as a community, could come up with a sufficiently detailed
description of light playouts algorithm (see the current "Light
simulation : Characteristic values" thread), there is no reason that
this algorithm couldn't join them.
I suspect that detailing the algorithm sufficiently for non-go players
to implement may be surprising challenging.
cheers
stuart
---------------------------------------
darren at dcook.org
said :
---------------------------------------
This is an excellent idea. Go for it!
---------------------------------------
Don Dailey drdailey at cox.net
said :
---------------------------------------
I have considered the very same thing and I think it's a great idea!
In fact I implemented the same exact algorithm for a simple go program
in C and D for the exact purpose of evaluating D. I even used the same
data representation and algorithms for doing things so that it was
mostly apples to apples.
I think it's easy to describe and code up a simple algorithm. Uniform
random play-outs. For the purpose of the benchmark we could avoid
using AMAF or else we would have to make sure it was carefully
specified. We could specify the simple eye rule that almost all of us
use and make sure all of these things are precisely defined.
And then we would need a simple test harness to verify that the program
was correct. We could do things like measure the average game length
and average win percentage in one or more specified positions. We
could create a couple extra GTP commands for reporting these numbers.
And we could specify that the programs are ready to play complete games
so that they could be tested in real games.
But I would suggest that this is not just a language contest, but a
programming context too. We should not have just one implementation per
language but users would be free to submit their own entry for any
language because no two implementations would be the same. The only
rule is that it must conform and pass the specified tests and the source
code is published with each entry for external verification.
_________________________________________________________________
Installez gratuitement les 20 émôticones Windows Live Messenger les plus fous !
Cliquez ici !
http://www.emoticones-messenger.fr/_______________________________________________
computer-go mailing list
computer-go@computer-go.org
http://www.computer-go.org/mailman/listinfo/computer-go/
