On Mon, 2009-02-02 at 18:09 +0100, Isaac Deutsch wrote: > "I don't think many people realize that you have to play hundreds of > games just to be within 40 or 50 ELO with much certainty. If you > play > less than 100 games you could easily be off by over 100 ELO." > > Maybe I'm a bit (a lot :) impatient, but I try to make a rough guess > after > about 150-200 games, which is about 1-2 days of letting the program > run on > the server. I think it's possible to say after 150 games that RAVE did > not > give me a 300 ELO boost.
You already reported 75 ELO improvement. Gelly said you should get 200-300. So you are only about 125 off of Gelly's lower bound (which sounded like a rough guess to me.) So you have a lot of uncertainty here and only 150 games. Of course this is plenty enough to suspect a problem and investigate, but it's not enough to conclude that you surely did something wrong. How much did you test the version that doesn't have this change? You have 3 sources of slop here: 1. Gelly's rough estimate of how much you should get. 2. The rating uncertainty of the unmodified program. 3. The rating uncertainty of the modified program. - Don _______________________________________________ computer-go mailing list computer-go@computer-go.org http://www.computer-go.org/mailman/listinfo/computer-go/
