[computer-go] Rank on servers at 9x9
Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the "fuego9" account on KGS but it appears to be unranked and only playing free games (*). The "ManyFaces" account appears to play only 19x19. I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
> Do any of the strongest MCTS programs have a rank > at 9x9 on any major server? i'm not sure that a "9x9 rank" is super well-defined, at least in the sense that ranks are generally established by your play on a 19x19 board, not a 9x9 board. s. ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Le 12/02/2010 à 08:43, Darren Cook a écrit : > > Do any of the strongest MCTS programs have a rank at 9x9 on any major > server? I found the "fuego9" account on KGS but it appears to be > unranked and only playing free games (*). The "ManyFaces" account > appears to play only 19x19. > > I know the programs are stronger at 9x9 than 19x19, but I'm trying to > get a figure of just exactly how strong they are against humans of known > rank. Ideally in a statistically meaningful way (not just a short series > of games against one player), and on non-supercomputer hardware. > roughtly the best MC bots are strong dans at 9x9 on kgs on a decent computer, and near pro level when running on big iron. (crazystone, mogo, probably zen too and other strong programs...) my 2 cents Alain ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
In message <4b7506ad.6080...@dcook.org>, Darren Cook writes Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the "fuego9" account on KGS but it appears to be unranked and only playing free games (*). The "ManyFaces" account appears to play only 19x19. On KGS, ratings are calculated from 19x19 games only. A rating there aims to measure strength at 19x9 Go, and nothing else. On OGS ( http://www.online-go.com ), a play-by-move server, 9x9 games are used for rating purposes. The bot player Fuego9x9_1h there has a rating of 5.7 dan. This bot runs on ordinary desktop-type hardware, but thinks for an hour about each move. Its opponents are also allowed to think for an hour, or even a day, about each move, but it is most unlikely that any of them do. I don't know about other servers. Nick I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ -- Nick Weddn...@maproom.co.uk ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
I think we can guess something from CGOS rating. I made a table about CGOS and KGS bot rating. At least in 19x19, CGOS and KGS rank is similar. CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) 1900 5k 2000 4k 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) 2200 2k Aya693a_10k 2k pachi2 2300 1k pachi-c919f1k ManyFaces (30minutes) 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) 2600 3d Fuego-1095-1c 2700 4d Zen-4.9-1c 2800 5d Zengg9-4x4c 2900 6d Hiroshi Yamashita - Original Message - From: "Darren Cook" To: "computer-go" Sent: Friday, February 12, 2010 4:43 PM Subject: [computer-go] Rank on servers at 9x9 Do any of the strongest MCTS programs have a rank at 9x9 on any major server? I found the "fuego9" account on KGS but it appears to be unranked and only playing free games (*). The "ManyFaces" account appears to play only 19x19. I know the programs are stronger at 9x9 than 19x19, but I'm trying to get a figure of just exactly how strong they are against humans of known rank. Ideally in a statistically meaningful way (not just a short series of games against one player), and on non-supercomputer hardware. Thanks in advance, Darren ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Thanks for all the replies! It is much appreciated. Hiroshi Yamashita wrote: > I think we can guess something from CGOS rating. > I made a table about CGOS and KGS bot rating. > At least in 19x19, CGOS and KGS rank is similar. That is a very interesting table. It is curious that Zen is only 2 ranks higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 ranks higher. The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of games, so this should be statistically significant; the ranking charts are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there a way on KGS to get the underlying rating number?) Darren > > > CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 > > 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) > 1900 5k > 2000 4k > 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) > 2200 2k Aya693a_10k 2k pachi2 > 2300 1k pachi-c919f1k ManyFaces (30minutes) > 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) > 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) > 2600 3d Fuego-1095-1c > 2700 4d Zen-4.9-1c > 2800 5d Zengg9-4x4c > 2900 6d > > Hiroshi Yamashita [1]: http://www.gokgs.com/graphPage.jsp?user=Fuego -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
On Fri, Feb 12, 2010 at 10:08:07PM +0900, Darren Cook wrote: > Thanks for all the replies! It is much appreciated. > > Hiroshi Yamashita wrote: > > I think we can guess something from CGOS rating. > > I made a table about CGOS and KGS bot rating. > > At least in 19x19, CGOS and KGS rank is similar. > > That is a very interesting table. It is curious that Zen is only 2 ranks > higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu > on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 > ranks higher. I guess that this is because the higher your rating is, the more difficult it is to get even stronger. ;-) > The difference between ManyFaces blitz and slow is also a lovely gem of > information. > > 2300 1k pachi-c919f1k ManyFaces (30minutes) > > 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) My guess is that this is caused by huge amount of domain-knowledge from "classical ManyFaces" used to prune the tree initially - that gives big constant boost against MCTS bots with equal thinking time, but with additional time this does not contribute anything anymore and MFGo scales similarly to other MCTS bots. At least that's my interpretation, I wonder what David Fotland thinks about it. (Another factor is that 10sec blitz is much more prone for the human opponents to lose on time.) > A quick check shows both accounts are playing a lot of > games, so this should be statistically significant; the ranking charts > are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there > a way on KGS to get the underlying rating number?) In your graphical editor from the rank graph, perhaps. ;-) -- Petr "Pasky" Baudis A great many people think they are thinking when they are merely rearranging their prejudices. -- William James ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
>> The difference between ManyFaces blitz and slow is also a lovely gem of >> information. >>> 2300 1k pachi-c919f1k ManyFaces (30minutes) >>> 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) > > My guess is that this is caused by huge amount of domain-knowledge from > "classical ManyFaces" used to prune the tree initially - that gives big > constant boost against MCTS bots with equal thinking time, but with > additional time this does not contribute anything anymore and MFGo > scales similarly to other MCTS bots. The above stat was from KGS, not CGOS. So, my thought was: MCTS doesn't scale with extra thinking time as well as humans do. Which conveniently matches my hypothesis that MCTS doesn't use extra CPU cycles very efficiently. Though, as you say, those foolish humans foolishly managing their blitz time foolishly does distort things a bit. Darren -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
The difference between ManyFaces blitz and slow is also a lovely gem of information. A quick check shows both accounts are playing a lot of I have a data about this. AyaMC4 and AyaMC play on same condition, but it is not for human. AyaMC4 is blitz. Human is about 1 rank weaker in blitz. AyaMC4 1k (0.1k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) AyaMC 1k (0.8k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) AyaMC2 3k (2.6k) 1 playouts 10minutes + 30sec byoyomi (x5) AyaBot2 4k (3.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) AyaBot4 5k (4.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) These are last year's data, AyaMC4 2k (1.4k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) AyaMC 3k (2.6k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) (about 16 playouts/mov) In your graphical editor from the rank graph, perhaps. ;-) Yes, (rank) is from the graph. Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook wrote: >Hiroshi Yamashita wrote: >> I think we can guess something from CGOS rating. >> I made a table about CGOS and KGS bot rating. >> At least in 19x19, CGOS and KGS rank is similar. > >That is a very interesting table. It is curious that Zen is only 2 ranks >higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu >on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 >ranks higher. Zen's algorithm is getting heavier and heavier. It works well on 19x19 but does not so on 9x9. Another problem is 7.5 komi. I guess it gives very big advantage to white for high-dan level players. 7.0 komi would give better results. -- Yamato ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Darren Cook: <4b7552b7.9060...@dcook.org>: >Thanks for all the replies! It is much appreciated. > >Hiroshi Yamashita wrote: >> I think we can guess something from CGOS rating. >> I made a table about CGOS and KGS bot rating. >> At least in 19x19, CGOS and KGS rank is similar. > >That is a very interesting table. It is curious that Zen is only 2 ranks >higher on 9x9. However Aya is 4 ranks higher and Fuego [1] is about 2kyu >on KGS, so (if KGS and CGOS 19x19 correspond for it too, then) also 4 >ranks higher. Please don't compare the ratings of different versions of programs on different hardware. The rating of Zen (at least version 4.2) on CGOS 9x9 is about 3 ranks higher than KGS, although (:-) only 1 rank higher than CGOS 19x19. Followings are the rating and rank of Zengg-4x4c (CGOS) and Zengg19 (KGS) respectively, though the games for Zengg19 aren't many enough. Zengg-4x4c and Zengg19 share the same code based on Zen-4.2 and hardware (a cluster of 4 quad-core PCs). CGOS 9x9 bayeselo 2919+75-64 CGOS 19x19 bayeselo 2830+80-71 KGS 19x19 3d I don't know why Hirosi didn't use "bayeselo" for the ratings on CGOS, which should give more reliable rating numbers. Below is another table using bayeselo. http://cgos.boardspace.net/9x9/bayes.html http://cgos.boardspace.net/19x19/bayes.html CGOS 9x9CGOS 19x19 KGS 19x19 GNU Go v3.7.10 lvl101805+7-61800 (ref) 6k Aya693_1c 2493+-742175+-443k AyaMC2 (10k po) Fuego svn1095 1c2629+-22 Fuego svn1095 2c2150+31-30 Fuego svn1089 8c2759+86-71 2353+-141 2k Fuego? Zen v4.9 1c 2795+74-68 2716+210-146 Zen v4.9 8c 2d Zen19? (15sec/move) Zengg (v4.2) 4x4c 2919+75-64 2830+80-71 3d Zengg19 (20:00+5x0:30) Note that Zen19 (KGS) runs on an 8-core MacPro while Zen-4.9-1c ran on just one core. I also wonder if Zen19 and Fuego on KGS are v4.9 and svn1089, respectively. Watching the ratings of GNU Go and Zengg on CGOS 19x19 and KGS in above table, they don't scale the same, say, 6k to 3d should be plus 800 Elo but is 1000 Elo on CGOS. Hideki >The difference between ManyFaces blitz and slow is also a lovely gem of >information. A quick check shows both accounts are playing a lot of >games, so this should be statistically significant; the ranking charts >are hard to read, but it looks like 1 to 1.5 ranks difference. (Is there >a way on KGS to get the underlying rating number?) > >Darren > > > >> >> >> CGOS KGS? CGOS 9x9 CGOS 19x19 KGS rank 19x19 >> >> 1800 6k gnugo3.7.10 gnugo3.7.106k GnuGo (postneo, etc) >> 1900 5k >> 2000 4k >> 2100 3k Aya693c_1c 3k AyaMC2 (10k playouts) >> 2200 2k Aya693a_10k 2k pachi2 >> 2300 1k pachi-c919f1k ManyFaces (30minutes) >> 2400 1d mfgo12-610-2c 1d ManyFaces1 (10sec blitz) >> 2500 2d Aya693_1c Zen-4.9-1c 2d Zen19 Zen (15sec/move) >> 2600 3d Fuego-1095-1c >> 2700 4d Zen-4.9-1c >> 2800 5d Zengg9-4x4c >> 2900 6d >> >> Hiroshi Yamashita > > >[1]: http://www.gokgs.com/graphPage.jsp?user=Fuego > > >-- >Darren Cook, Software Researcher/Developer > >Specializing in intelligent search (in multiple languages), discovery >of context, aiding communication, and basically helping people find >and make good use of their data. > >http://dcook.org/gobet/ (Shodan Go Bet - who will win?) >http://dcook.org/mlsn/ (Multilingual open source semantic network) >http://dcook.org/work/ (About me and my work) >http://dcook.org/blogs.html (My blogs and articles) >___ >computer-go mailing list >computer-go@computer-go.org >http://www.computer-go.org/mailman/listinfo/computer-go/ -- g...@nue.ci.i.u-tokyo.ac.jp (Kato) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
>> The difference between ManyFaces blitz and slow is also a lovely gem of >> information. A quick check shows both accounts are playing a lot of > > I have a data about this. > AyaMC4 and AyaMC play on same condition, but it is not for human. > AyaMC4 is blitz. > > Human is about 1 rank weaker in blitz. Hi Hiroshi, That is interesting; can I just check I've not misunderstood? You are saying that humans play about 1 rank (more precisely, 0.7 ranks) weaker when playing blitz against a fixed strength opponent? Going back to the Many Faces blitz (1.8d) and slow (0.6d), suggests that ManyFaces playing blitz against a human playing at slow time levels would be 1.8d - 0.7 = 1.1d. I.e. ManyFaces *loses* half a rank by thinking more :-) That is unlikely to be the case, implying people lose more strength playing blitz? In the Aya experiment I wonder if people playing slowly against a fast opponent tend to speed up and not use all their thinking time? Darren P.S. Using your last year's data, the difference is 1.2 ranks, which exactly matches the ManyFaces data, but still suggests ManyFaces gets no benefit from thinking more. > AyaMC4 1k (0.1k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) > AyaMC 1k (0.8k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) > AyaMC2 3k (2.6k) 1 playouts 10minutes + 30sec byoyomi (x5) > AyaBot2 4k (3.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) > AyaBot4 5k (4.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) > > These are last year's data, > > AyaMC4 2k (1.4k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) > AyaMC 3k (2.6k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) > (about 16 playouts/mov) -- Darren Cook, Software Researcher/Developer Specializing in intelligent search (in multiple languages), discovery of context, aiding communication, and basically helping people find and make good use of their data. http://dcook.org/gobet/ (Shodan Go Bet - who will win?) http://dcook.org/mlsn/ (Multilingual open source semantic network) http://dcook.org/work/ (About me and my work) http://dcook.org/blogs.html (My blogs and articles) ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
On Sat, Feb 13, 2010 at 7:03 PM, Darren Cook wrote: > >> The difference between ManyFaces blitz and slow is also a lovely gem of > >> information. A quick check shows both accounts are playing a lot of > > > > I have a data about this. > > AyaMC4 and AyaMC play on same condition, but it is not for human. > > AyaMC4 is blitz. > > > > Human is about 1 rank weaker in blitz. > > Hi Hiroshi, > That is interesting; can I just check I've not misunderstood? You are > saying that humans play about 1 rank (more precisely, 0.7 ranks) weaker > when playing blitz against a fixed strength opponent? > In chess it's well known that humans play significantly better with more time. So do computers, but the question is who benefits more, computers or humans?In chess it's humans by a pretty large margin, so I personally think it's likely to be the same with GO. One thing that I am aware of is that in chess people do not really perceive that they are playing that much stronger, but they are. Part of the reason is probably because when you get more time on your clock, so does your opponent.So no matter how much time you get you do not feel like you are playing much stronger. Don > Going back to the Many Faces blitz (1.8d) and slow (0.6d), suggests that > ManyFaces playing blitz against a human playing at slow time levels > would be 1.8d - 0.7 = 1.1d. I.e. ManyFaces *loses* half a rank by > thinking more :-) > > That is unlikely to be the case, implying people lose more strength > playing blitz? In the Aya experiment I wonder if people playing slowly > against a fast opponent tend to speed up and not use all their thinking > time? > > Darren > > P.S. Using your last year's data, the difference is 1.2 ranks, which > exactly matches the ManyFaces data, but still suggests ManyFaces gets no > benefit from thinking more. > > > > AyaMC4 1k (0.1k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) > > AyaMC 1k (0.8k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) > > AyaMC2 3k (2.6k) 1 playouts 10minutes + 30sec byoyomi (x5) > > AyaBot2 4k (3.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) > > AyaBot4 5k (4.6k) 5000 playouts 10minutes + 30sec byoyomi (x5) > > > > These are last year's data, > > > > AyaMC4 2k (1.4k) 10sec/mov (8cores) 1minute + 15sec byoyomi (x10) > > AyaMC 3k (2.6k) 10sec/mov (8cores) 10minutes + 30sec byoyomi (x5) > > (about 16 playouts/mov) > > > -- > Darren Cook, Software Researcher/Developer > > Specializing in intelligent search (in multiple languages), discovery > of context, aiding communication, and basically helping people find > and make good use of their data. > > http://dcook.org/gobet/ (Shodan Go Bet - who will win?) > http://dcook.org/mlsn/ (Multilingual open source semantic network) > http://dcook.org/work/ (About me and my work) > http://dcook.org/blogs.html (My blogs and articles) > ___ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ > ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
Re: [computer-go] Rank on servers at 9x9
Hi Darren, saying that humans play about 1 rank (more precisely, 0.7 ranks) weaker when playing blitz against a fixed strength opponent? Yes, I think so. Going back to the Many Faces blitz (1.8d) and slow (0.6d), suggests that I think ManyFaces (30minutes) and ManyFaces1 (10sec blitz) use different time setting. So it seems difficult to compare. playing blitz? In the Aya experiment I wonder if people playing slowly against a fast opponent tend to speed up and not use all their thinking time? I agree with you, when computers play too fast (1sec, or 2sec), human often play fast too, and make blunder. Hiroshi Yamashita ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
RE: [computer-go] Rank on servers at 9x9
Many Faces has the same issue. The pruning and tuning that is required for 19x19 doesn't help 9x9. It seems that now the programs are strong enough that 9x9 requires a good opening book, and I'd rather spend my time making 19x19 stronger. David > > Zen's algorithm is getting heavier and heavier. It works well on 19x19 > but does not so on 9x9. > -- > Yamato > ___ > computer-go mailing list > computer-go@computer-go.org > http://www.computer-go.org/mailman/listinfo/computer-go/ ___ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
