Re: [computer-go] 13x13 study. Time and memory
Le jeudi 21 février 2008, Don Dailey a écrit : If you look at the table you will notice that going from level 4 to level 11 (which is 7 doublings and should take 128X longer) only takes 59.43 X longer. So if we plot 9X9 rank vs time, maybe we have a straight line :) ELO vs size of the tree (or memory usage) should show the same ? On 13x13 study it does, but there are not enough data at high level. Alain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study. Time and memory
2008/2/22, Alain Baeckeroot [EMAIL PROTECTED]: Le jeudi 21 février 2008, Don Dailey a écrit : If you look at the table you will notice that going from level 4 to level 11 (which is 7 doublings and should take 128X longer) only takes 59.43 X longer. So if we plot 9X9 rank vs time, maybe we have a straight line :) It would indeed be very interesting to see that plot! Sylvain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] The odd 13x13 graph
I'm going to do a private autotest with just the first 6 levels or so - to see if I get the same bend. I will leave out gnugo for the time being to see what happens.If I don't get a bend, I will try adding gnugo in a fresh test. - Don Harald Korneliussen wrote: Don Dailey wrote: It's also interesting how the graph up to level 11 seems to form 2 very straight lines, almost as if they were connected at an angle. This must be a by-product of how we started the test. We played only the first 4 levels as we were testing the system and that is where the bend point is. Then I added levels gradually. I cannot figure out why this would cause such strange behavior. As we're closing in on 2 games, that trend seems just as apparent: It's not so much a curve as two lines, one below mogo4 and one above... but are we looking at the right axis? I wonder if it may have something to do with the anchor. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
[computer-go] The odd 13x13 graph
Don Dailey wrote: It's also interesting how the graph up to level 11 seems to form 2 very straight lines, almost as if they were connected at an angle. This must be a by-product of how we started the test. We played only the first 4 levels as we were testing the system and that is where the bend point is. Then I added levels gradually. I cannot figure out why this would cause such strange behavior. As we're closing in on 2 games, that trend seems just as apparent: It's not so much a curve as two lines, one below mogo4 and one above... but are we looking at the right axis? I wonder if it may have something to do with the anchor. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study. Time and memory
Le vendredi 22 février 2008, Sylvain Gelly a écrit : 2008/2/22, Alain Baeckeroot [EMAIL PROTECTED]: Le jeudi 21 février 2008, Don Dailey a écrit : If you look at the table you will notice that going from level 4 to level 11 (which is 7 doublings and should take 128X longer) only takes 59.43 X longer. So if we plot 9X9 rank vs time, maybe we have a straight line :) It would indeed be very interesting to see that plot! and if i remember it was Don's initial claim, that doubling thinking time (for humans and scalable bots) will produce a fixed Elo increase, at least until exhaustion of other resources (human fall asleep, bot fill memory ...) Alain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] 13x13 study. Time and memory
Alain Baeckeroot wrote: Le vendredi 22 février 2008, Sylvain Gelly a écrit : 2008/2/22, Alain Baeckeroot [EMAIL PROTECTED]: Le jeudi 21 février 2008, Don Dailey a écrit : If you look at the table you will notice that going from level 4 to level 11 (which is 7 doublings and should take 128X longer) only takes 59.43 X longer. So if we plot 9X9 rank vs time, maybe we have a straight line :) It would indeed be very interesting to see that plot! and if i remember it was Don's initial claim, that doubling thinking time (for humans and scalable bots) will produce a fixed Elo increase, at least until exhaustion of other resources (human fall asleep, bot fill memory ...) I didn't predict perfect linearity, I expect it to gradually curve until it's horizontal as the programs approach perfection ASSUMING the program is completely scalable and isn't hitting internal limits as you say. However, it should appear to us as nearly linear because we are so far from perfect play. Especially at 13x13 and beyond. Since I am no go expert I can't speak for 9x9, but my belief (I'm willing to be wrong here) is that mogo at really high levels of thinking time on fast computers is playing pretty strong - strong enough to bend the curve. This is despite the games that David Fotland looked at and criticized.I do not doubt his analysis that it played bad moves, but my experience has always been that human experts look at games and under-rate computers by enormous amounts based on a few moves that rub them wrong. The typical scenario is that they see one bad move - and from that point on they lose all objectivity. Besides, I don't think even a highly objective human can accurately assign ratings or rankings based on simply looking at moves. There is also the phenomenon that if you look at a LOT of data points, you can perceive the curve more easily when graphed.In chess studies over the years we have typically looked at only a few data points and saw a straight line.But with mogo we are looking at a huge range - from the very weak to the very strong. With 18 doublings for Mogo on the 9x9 study, there is an enormous difference between the best and the worst.We also saw some empirical evidence that mogo was suffering from memory limits and this was throttling it's strength downward at the upper levels. With chess I think the ELO advantage of a doubling has decreased from about 100 ELO to about 50 or 60 - I'm not sure of the exact figure. But that's pretty amazing that we can still keep finding 50 ELO points, already humans cannot compete. I would also mention that it is not difficult to find bad moves - they still make mistakes and sometimes they are ugly - but can you beat it? Nope! - Don Alain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
