Ian Shaw wrote:
The scaling of the PR values comes historically from Snowie, which used
the sum of both players' moves as the divisor. Gnubg uses only the
player's unforced moves, which naturally means gnubg error rates are at
least double Snowie error rates. When XG was created Xavier calculated
the error rates using the same method as gnubg, but then divided by 2 to
scale them to the match Snowie Error Rate, which is what most people
were familiar with.
XG's definition of PR is rather complicated because of its definition of a
"decision":
http://timothychow.net/cg/www.bgonline.org/forums/197598.html
Since PR has become a de facto standard, it makes sense to try to
replicate it. But replicating PR would require some additional programming
since it's not quite the same as GNU's native error rate calculation.
I'm not in favor of dropping Snowie ER entirely. It has its merits, or
rather, PR has its pathologies. Neil Robins pointed out one surprising
example here:
https://www.bgonline.org/forums/webbbs_config.pl?read=154585
More generally, as I've stated numerous times on rec.games.backgammon and
BGOnline, eliminating forced or obvious moves from the denominator has
some strange consequences that most people don't seem to appreciate. One
reason we divide the total equity lost by the length of the session is so
that errors are weighted according to their *frequency of occurrence in
actual play*. If a very unusual type of decision arises and I botch it,
then that should not count against me as much as a very common type of
decision that I mess up (assuming both types of mistake cost 0.05 each,
say). So far so good.
But now think about what happens if we delete forced moves from the
denominator. That means that errors occurring in games with a lot of
forced moves hurt our PR more than errors occurring in games with no
forced moves. In two separate games, I might make a error of exactly the
same size, but in one game I get unlucky and get closed out. My PR will
probably suffer more in the game where I have bad luck, because I'll be
dividing my equity loss by a smaller number. Is this what we really want
from PR? Maybe, maybe not. It's not obvious to me. A large majority of the
backgammon community has somehow gone along with this way of doing things
without thinking it through, or even recognizing that there is something
to think about here.
Somehow people have come to conceptualize a backgammon session as a
sequence of quiz problems, where the only role of the denominator is the
measure the length of the quiz, but in reality there can be correlations
(or anti-correlations) between the *types* of decisions you're presented
with in a game and the *number* of decisions in the game. By messing with
the denominator in a funny way, PR produces some strange and
hard-to-understand effects. ER has the advantage of keeping things simple:
the denominator is just the number of rolls. That is the most obvious
measure of length, and it has the advantage of being simple to understand.
If GNU stops calculating Snowie ER, then it will be very difficult to
extract this potentially illuminating and instructive statistic from a
backgammon session.
Tim
