Hi Ingo, I add the comments below
Am Donnerstag, den 19.12.2013, 11:17 +0100 schrieb "Ingo Althöfer": > Hello Detlef, > > thans for the interesting data. > > A few questions: > * on what board size(s)? Sorry, forgot to mention, 13 was the board size, at least with oakfoam-pachi scaling was even worse on 19x19. > * how many games for which pairings? The actual were 457 pairings fuego-pachi, changing the color every game and 208 pairings oakfoam-pachi. I am at the moment adding games, so the statistics gets better. Fitting the results I described before, done with libre office.... > * can you provide the "naked" scores (from which you made the graphics)? oakfoam-pachi W is win for oakfoam, Even games are white for oakfoam, second coulumn is the number of playouts 1000 1146 W 1001 2173 W 1002 52995 L 1003 12706 W 1004 37629 L 1005 2144 L 1006 1987 W 1007 17562 L 1008 16339 W 1009 10154 W 1010 21647 L 1011 17483 L 1012 5729 L 1013 7195 L 1014 4836 L 1015 13651 L 1016 4382 W 1017 62973 L 1018 6093 W 1019 23293 W 1020 2386 W 1021 3521 W 1022 36464 W 1023 80821 W 1024 14715 L 1025 19361 W 1026 40735 W 1027 6681 W 1028 4976 W 1029 1291 W 1030 4623 W 1031 1193 W 1032 2543 W 1033 18015 L 1034 24198 L 1035 6594 W 1036 1135 W 1037 58474 L 1038 9609 W 1039 82610 L 1040 2033 W 1041 1003 W 1042 1606 W 1043 18180 W 1044 37145 L 1045 53345 L 1046 4759 W 1047 9299 W 1048 2635 L 1049 22002 L 1050 1300 W 1051 27921 W 1052 45564 L 1053 58667 L 1054 5405 W 1055 3238 W 1056 10231 W 1057 18805 L 1058 9397 W 1059 2988 W 1060 7027 W 1061 22455 L 1062 17789 W 1063 12295 W 1064 1908 W 1065 47726 W 1066 51372 L 1067 4855 W 1068 33493 W 1069 1843 L 1070 10816 W 1071 8050 W 1072 5633 W 1073 3296 W 1074 2402 L 1075 86156 L 1076 21265 W 1077 69549 W 1078 13320 W 1079 4116 W 1080 32498 W 1081 3544 W 1082 2957 W 1083 1144 W 1084 4255 L 1085 2497 W 1086 18919 W 1087 88948 W 1088 38728 W 1089 56505 L 1090 1319 W 1091 29775 W 1092 1924 W 1093 58396 L 1094 10955 W 1095 27874 W 1096 7861 W 1097 2367 L 1098 28489 L 1099 35940 W 1100 15696 W 1101 3399 W 1102 1098 W 1103 3256 W 1104 25529 L 1105 63268 L 1106 13662 L 1107 14324 W 1108 3448 W 1109 74501 W 1110 23679 L 1111 13075 W 1112 1105 W 1113 19031 W 1114 2418 L 1115 27316 W 1116 3551 W 1117 67785 W 1118 36054 W 1119 2545 W 1120 3473 W 1121 88841 L 1122 17487 W 1123 1093 W 1124 4990 W 1125 17320 W 1126 2743 W 1127 59744 W 1128 1218 W 1129 39773 L 1130 44767 W 1131 35787 L 1132 4786 W 1133 34206 W 1134 4451 W 1135 1130 W 1136 72890 W 1137 1295 W 1138 10179 W 1139 1656 W 1140 1974 L 1141 19602 W 1142 5005 L 1143 5651 W 1144 6618 W 1145 4185 W 1146 2269 W 1147 6983 W 1148 3073 W 1149 2416 W 1150 4932 W 1151 50296 L 1152 9014 W 1153 13061 W 1154 11018 L 1155 11249 W 1156 1055 L 1157 2455 W 1158 5471 L 1159 51279 W 1160 1497 W 1161 25042 W 1162 5986 L 1163 1308 W 1164 1625 L 1165 14384 W 1166 43808 L 1167 25537 W 1168 28335 L 1169 13238 L 1170 2415 W 1171 8218 W 1172 67279 L 1173 1446 W 1174 16462 W 1175 82010 W 1176 2155 L 1177 88587 L 1178 1585 W 1179 59444 W 1180 96765 W 1181 56317 W 1182 43097 W 1183 37124 W 1184 14604 L 1185 6327 W 1186 1318 W 1187 2631 W 1188 1159 L 1189 19976 L 1190 44887 W 1191 39317 L 1192 23881 W 1193 45266 L 1194 32446 W 1195 3926 W 1196 10644 W 1197 5415 W 1198 1798 L 1199 1830 W 1200 11960 W 1201 2030 W 1202 8618 W 1203 26808 L 1204 28126 W 1205 10584 W 1206 19264 L 1207 85863 L fuego-pachi 1000 3188 L 1001 77275 W 1002 2215 W 1003 76779 W 1004 6412 W 1005 7196 L 1006 14910 W 1007 90639 L 1008 5198 L 1009 59585 W 1010 2105 W 1011 22219 W 1012 10013 W 1013 4197 W 1014 78076 W 1015 66195 W 1016 28203 L 1017 4628 W 1018 4550 L 1019 34647 W 1020 10853 W 1021 16555 L 1022 9104 W 1023 1233 W 1024 95992 L 1025 41463 L 1026 6283 L 1027 2103 L 1028 44902 L 1029 16157 W 1030 10827 L 1031 1249 W 1032 2474 L 1033 63669 L 1034 3688 L 1035 44821 L 1036 1122 W 1037 38355 W 1038 8461 W 1039 42244 L 1040 53620 W 1041 1693 W 1042 4340 L 1043 5285 W 1044 8873 W 1045 26856 W 1046 17757 L 1047 30183 W 1048 22967 W 1049 4587 W 1050 26913 L 1051 2878 L 1052 9259 L 1053 1442 W 1054 19724 L 1055 3118 L 1056 53001 L 1057 51968 L 1058 96355 L 1059 16173 W 1060 6225 L 1061 64776 W 1062 1187 L 1063 12966 W 1064 4426 W 1065 30048 W 1066 47668 L 1067 9762 W 1068 63751 W 1069 23109 W 1070 1359 L 1071 20933 W 1072 2129 W 1073 1330 L 1074 1901 L 1075 1152 W 1076 62263 L 1077 13112 L 1078 6556 L 1079 3122 W 1080 52222 L 1081 3604 L 1082 54108 L 1083 15330 L 1084 2178 W 1085 6527 W 1086 97005 L 1087 3711 W 1088 29842 L 1089 1791 W 1090 2329 W 1091 33183 L 1092 77755 W 1093 2194 W 1094 68095 W 1095 64971 W 1096 1100 W 1097 17184 W 1098 5124 W 1099 63030 W 1100 1621 L 1101 4270 L 1102 24781 W 1103 2760 W 1104 26123 L 1105 22776 L 1106 1055 W 1107 13875 W 1108 8762 W 1109 4804 L 1110 47506 L 1111 22316 L 1112 1432 W 1113 9023 L 1114 65074 L 1115 48656 W 1116 1898 L 1117 3160 W 1118 7071 L 1119 19224 L 1120 7049 L 1121 1983 L 1122 11900 W 1123 1145 W 1124 2921 W 1125 15534 W 1126 7682 W 1127 90694 W 1128 72500 W 1129 18451 W 1130 71064 L 1131 4293 L 1132 48123 W 1133 2871 L 1134 17570 L 1135 5936 W 1136 3120 W 1137 16484 W 1138 6040 W 1139 3394 W 1140 1825 W 1141 1970 W 1142 51329 W 1143 13657 W 1144 7296 L 1145 1166 L 1146 5849 W 1147 2193 W 1148 39182 W 1149 66104 L 1150 21400 L 1151 4143 W 1152 6049 L 1153 20611 W 1154 4191 L 1155 45687 L 1156 1457 L 1157 73984 L 1158 37116 L 1159 47559 L 1160 55596 L 1161 83546 L 1162 40934 W 1163 5313 W 1164 13614 L 1165 16238 W 1166 17173 W 1167 1108 L 1168 4813 W 1169 5943 W 1170 1538 W 1171 4578 L 1172 42161 L 1173 12235 W 1174 3293 L 1175 2738 W 1176 38472 W 1177 5722 L 1178 74964 W 1179 13410 L 1180 27755 L 1181 13807 L 1182 38096 W 1183 68805 W 1184 21626 L 1185 27498 L 1186 1974 W 1187 36634 W 1188 2368 L 1189 51907 L 1190 33070 W 1191 15778 W 1192 12705 W 1193 58849 L 1194 37512 L 1195 3991 W 1196 2273 W 1197 11831 W 1198 1732 L 1199 14594 W 2000 4780 W 2001 1317 W 2002 2752 L 2003 18337 W 2004 1691 W 2005 30801 L 2006 79058 L 2007 5167 L 2008 1726 L 2009 5581 L 2010 4991 W 2011 2708 W 2012 11614 L 2013 55600 L 2014 52353 L 2015 1695 W 2016 12695 L 2017 8225 L 2018 2982 W 2019 26172 L 2020 4819 L 2021 13543 L 2022 42352 L 2023 4010 W 2024 75549 L 2025 83678 W 2026 78360 L 2027 7006 L 2028 98193 L 2029 17012 W 2030 1281 W 2031 5122 W 2032 5591 L 2033 34127 L 2034 1538 W 2035 1806 L 2036 57432 L 2037 47882 L 2038 21412 L 2039 2577 L 2040 5649 L 2041 4149 L 2042 4450 W 2043 1121 W 2044 3374 L 2045 32175 L 2046 10856 W 2047 4578 L 2048 57188 L 2049 3239 W 2050 32942 L 2051 1815 W 2052 22957 W 2053 3481 L 2054 14122 L 2055 36858 W 2056 20059 W 2057 10142 W 2058 2177 W 2059 1132 W 2060 73040 L 2061 2380 W 2062 1131 W 2063 3278 W 2064 3435 L 2065 2596 W 2066 4557 L 2067 18975 W 2068 1454 W 2069 7239 W 2070 13625 L 2071 1418 W 2072 2038 W 2073 12488 L 2074 3703 W 2075 66177 W 2076 31846 L 2077 1407 W 2078 1933 L 2079 59339 W 2080 1429 W 2081 3347 L 2082 2972 L 2083 1477 W 2084 91502 L 2085 99569 L 2086 22996 L 2087 35351 W 2088 23285 L 2089 11261 L 2090 5996 W 2091 12818 W 2092 25566 W 2093 13749 W 2094 16386 W 2095 6805 L 2096 4735 W 2097 20071 L 2098 4968 W 2099 1527 W 2100 19434 L 2101 2220 L 2102 9080 L 2103 68644 L 2104 13563 L 2105 2434 L 2106 18380 W 2107 21120 L 2108 1180 W 2109 13811 L 2110 20806 L 2111 78997 L 2112 52654 L 2113 9192 W 2114 6837 L 2115 57705 W 2116 1818 L 2117 77224 L 2118 36342 W 2119 50730 L 2120 8783 L 2121 4180 L 2122 64422 W 2123 19703 L 2124 58120 L 2125 3765 L 2126 23324 L 2127 9039 L 2128 1484 L 2129 9482 W 2130 15017 L 2131 7182 L 2132 13730 W 2133 2988 L 2134 18095 L 2135 13518 L 2136 82135 L 2137 2871 L 2138 3562 L 2139 2007 W 2140 4535 L 2141 32438 W 2142 44806 L 2143 22304 W 2144 2022 W 2145 64963 W 2146 66691 L 2147 26906 L 2148 38424 W 2149 1841 L 2150 35635 L 2151 38285 W 2152 1262 L 2153 11250 L 2154 1938 W 2155 20553 W 2156 1173 L 2157 17339 W 2158 8485 L 2159 84060 L 2160 1854 W 2161 5656 L 2162 1476 W 2163 17272 W 2164 1208 W 2165 17619 L 2166 2389 W 2167 10321 L 2168 7043 W 2169 27586 W 2170 70140 W 2171 54666 L 2172 36326 W 2173 41814 W 2174 25161 W 2175 20189 L 2176 15597 W 2177 1229 L 2178 52471 L 2179 3538 L 2180 31208 W 2181 20759 L 2182 5257 W 2183 24519 W 2184 80626 W 2185 5278 L 2186 5237 L 2187 43739 L 2188 12002 L 2189 76803 W 2190 3042 L 2191 1773 W 2192 70607 W 2193 3430 W 2194 49423 L 2195 85532 W 2196 17739 W 2197 7151 W 2198 7336 L 2199 10543 W 3000 23237 W 3001 3952 W 3002 2789 L 3003 3695 L 3004 1818 W 3005 1161 W 3006 7687 W 3007 59655 W 3008 77758 W 3009 2491 L 3010 2034 W 3011 13376 W 3012 19864 L 3013 15508 W 3014 14030 W 3015 17966 W 3016 44248 L 3017 27424 W 3018 6575 L 3019 15475 W 3020 49380 L 3021 51466 L 3022 6311 L 3023 78221 L 3024 38215 W 3025 68620 L 3026 38899 L 3027 1329 L 3028 21630 L 3029 23605 L 3030 80347 L 3031 1801 L 3032 5395 W 3033 68668 W 3034 34009 L 3035 1725 W 3036 56544 W 3037 1406 L 3038 55244 W 3039 17425 L 3040 4775 L 3041 39531 L 3042 1345 W 3043 3077 W 3044 3084 W 3045 9171 L 3046 1903 L 3047 1596 W 3048 17111 L 3049 1522 W 3050 14269 W 3051 33946 L 3052 6264 W 3053 9926 W 3054 7331 W 3055 33885 L 3056 30333 L > * Do you also have results for "oakfoam vs fuego"? Yes, the results were inbetween. But I did not make new ones with this setting, as I wanted to compare to pachi, as the effect is stronger and I hoped to get improvenment with lower number of games. In case you want to try it your self, I can offer the quick and dirty scripts (one perl and one python script is needed) > > Thanks in advance, Ingo. > > > > Gesendet: Donnerstag, 19. Dezember 2013 um 10:53 Uhr > > Von: "Detlef Schmicker" <d...@physik.de> > > An: computer-go@dvandva.org > > Betreff: Re: [Computer-go] Playouts vs playing strength > > > > Hi, > > > > I want to get more people interested into this scaling, therefore I did > > also some scaling tests fuego against pachi :) > > > > It is not as bad as oakfoam against pachi, but pachi scales a lot better > > than fuego too. (attached file) To avoid additional complications I set > > the number of playouts to the same value for both opponents. ELO is > > again as defined in CGOS from winning rate. > > > > > > I used: > > Pachi version 10.00 (Satsugen) > > > > fuego 1.1 (does not show a more detailed version) > > > > with following configuration > > > > opponent_program2='/home/detlef/fuego-1.1/fuegomain/fuego' > > opponent_settings2='uct_param_player ignore_clock 1\nuct_param_player > > max_games '+str(playouts)+'\nuct_param_player resign_min_games 5000 > > \nuct_param_search number_threads 8\nuct_max_memory 8000000000 > > \nuct_param_player reuse_subtree 1' > > > > opponent_program3='/home/detlef/pachi/pachi -d 0 -t ='+str(playouts)+' > > -r chinese threads=8,max_tree_size=2048,pondering=0,pass_all_alive ' > > opponent_settings3='' > > > > taken from a CLOP like python file. > > > > > > For oakfoam I tried to optimize a number of parameters which I thought > > are relevant to scaling (progressive widening, ucb_c weighting of random > > moves in playouts), but none of them was as relevant as I thought :( > > > > I hope I did not understand the playout number parameters wrong in pachi > > and fuego. > > > > To me it seems there is a lot of potential in scaling, not only for > > oakfoam... > > > > I read fuego and pachi mailing list too, if it is not of too much > > interest here, we might change the mailing list:) > > > > Detlef > > > > > > > > Am Samstag, den 23.11.2013, 11:32 +0100 schrieb Detlef Schmicker: > > > Just to let you know: > > > > > > I did a comparison of the playings strength vs. playouts. > > > > > > This time I used 4 times the oakfoam playouts for pachi > > > (eg. 1000 for oakfoam 4000 for pachi) > > > > > > The graph shows how bad we become (in comparison) with more playouts:(. > > > >From the games the first impression is, that the joseki becomes worse > > > with more playouts e.g. > > > > > > http://www.physik.de/playouts2.pdf > > > The plot is 1050 games fitted with a 5th order polynome. The borders of > > > the plot are not statistical significant! > > > > > > Thanks for every hint :) > > > > > > Detlef > > > > > > > > > Am Montag, den 18.11.2013, 22:45 +0100 schrieb Detlef Schmicker: > > > > Am Montag, den 18.11.2013, 21:11 +0100 schrieb Petr Baudis: > > > > > On Sun, Nov 17, 2013 at 03:11:22PM +0100, Erik van der Werf wrote: > > > > > > make sure Pachi isn't doing any kind of pondering in the > > > > > > background. > > > > > > > > > > Indeed, Pachi will ponder by default. Turn pondering off by passing > > > > > > > > > > pondering=0 > > > > > > > > > > on the commandline. > > > > > > > > Thanks a lot for the hint!!! From the command line documentation I > > > > thought pondering is off by default.and I did not check it:( > > > > > > > > > > > > > > > > > > > pachi -d 0 -t =4000 -r chinese threads=1,max_tree_size=2048 > > > > > > > > > > Also, it may be worth passing pass_all_alive unless you are doing a > > > > > sophisticated scoring procedure, to make sure Pachi captures all dead > > > > > groups at the end of the game. > > > > > > > > > > P.S.: Do your results imply that on 4000 playouts/move, oakfoam is > > > > > quite stronger than Pachi now? I'd love to hear more. :) How does the > > > > > playout speed compare? > > > > > > > > Yes, we play even with 1000 against this settings. But I did not take > > > > pondering into account, as I thought it is turned off. Therefore I do > > > > not know if pachi really played 4000 playouts, as I thought. > > > > > > > > We have a little less than 1000 playouts/core/second. And my main aim is > > > > to get the iPad version strong, therefore the strength with lower > > > > playouts is more important to me. > > > > > > > > I did not optimize parameter against pachi alown, I started running clop > > > > with three opponents gnugo level 10, pachi with this setting and > > > > > > > > /home/detlef/fuego-1.1/fuegomain/fuego > > > > > > > > with setting > > > > uct_param_player ignore_clock 1 > > > > uct_param_player max_games 3000 > > > > uct_param_player resign_min_games 5000 > > > > uct_max_memory 300000000 > > > > > > > > All 4 programs have comparable strenght than. > > > > > > > > Always happy to share any idea:) > > > > > > > > Detlef > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Computer-go mailing list > > > > Computer-go@dvandva.org > > > > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > > > > > > > > > > > > _______________________________________________ > > > Computer-go mailing list > > > Computer-go@dvandva.org > > > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > > > > > > > _______________________________________________ > > Computer-go mailing list > > Computer-go@dvandva.org > > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > _______________________________________________ > Computer-go mailing list > Computer-go@dvandva.org > http://dvandva.org/cgi-bin/mailman/listinfo/computer-go > _______________________________________________ Computer-go mailing list Computer-go@dvandva.org http://dvandva.org/cgi-bin/mailman/listinfo/computer-go
