On Tue, Mar 22, 2016 at 04:00:41PM +0000, Lucas, Simon M wrote:
> With AlphaGo winning 4 games to 1, from a simplistic
> stats point of view (with the prior assumption of a fair
> coin toss) you'd not be able to claim much statistical
> significance, yet most (me included) believe that
> AlphaGo is a genuinely better Go player than Lee Sedol.
>
> From a stats viewpoint you can use this approach:
> http://www.inference.phy.cam.ac.uk/itprnn/book.pdf
> (see section 3.2 on page 51)
>
> but given even priors it won't tell you much.
What complicates things further is that the coin distribution is
non-stationary; definitely from the point of the human performance
against a fixed program (how much AlphaGo with its novel RL component
is fixed is of course another matter). In fact, as anyone watching bots
playing on KGS knows, initially the non-stationarity is actually very
extreme as the human gets "used to" the computer's style and soon are
able to beat even a program that's formally quite stronger than the
human player. At least that's the case for the weaker programs.
I'm not sure if we can say with certainty that AlphaGo is significantly
better Go player than Lee Sedol at this point. What we can say with
certainty is that AlphaGo is in the same ballpark and at least roughly
as strong as Lee Sedol. To me, that's enough to be really huge on its
own accord!
--
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton
_______________________________________________
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go
