Dear GNU Chess developers, I would like to report a bug in GNU Chess 5.07. The bug results in serious time trouble when GNU Chess is playing with esternal book controlled by a GUI, when playing with 40/40 type time control (also with any other X/Y time control). GNU chess does not consider the moves operated by a GUI as part of the time control session. As a result its move counter is lagging behind the real move count. This results in a huge time trouble after about move 50. The exact mexhanism of this bug is explained by H.G. Muller (copied from Winboard Forum: http://www.open-aurec.com/wbforum/viewtopic.php?p=188279#p188279):
" Many WB engines suffer from this when you play from an external opening book. They then receive the go command later in the game, (say at move 5), and start counting the moves only from there. So when they reach move 40, to them it is only move 35. The extra time on their clock that the GUI reports to them then comes as a complete surprise to them, which they happily exploit during the remaining 5 moves, so that they have used up nearly all time at move 45 (which they think is move 40). It then comes again as a complete surprise (this time an unpleasant one) that they do not receive the expected new time quota after this move, so that in fact they have to play the remaining 35 moves of the second session in the few seconds that were left on their clock after move 45 (which they thought to be 40). On move 75/80 the whole thing repeats. They thus play the first 5 moves of any session except the first in the full time for the session, and the other 35 moves in just a few seconds. Of course this means they usually do not survive upto the third session... In principle this behavior also occurs under WinBoard. But the more book moves you have, the less obvious the effect is. This because most engines manage their time such that they normally do leave a safety margin of about 7 moves before any time-control point. This because a move in a difficult position can easily take 5 times as long as an average move. So if you had 24 moves in book, the engine realizes its error only on move 64, (at 40/40), and then have to do the remaining 16 moves before it gets new time in 7 min, which is not extremely much faster than normal, and usually interpreted as that the engine speeds up in the end-game under time pressure. That this is so common is due to a very unclear sn confusing description of the clock issue in the WB protocol document on Tim Mann's website. I tried to clarify it in the protocol definitin here, based on how WB did actually manage the clocks. There is no easy work-around for engines that suffer from this, other than play them with external book at incremental time controls only. " In fact a number of Winboard engines are suffering from this same bug. Some discussions about this: http://www.open-aurec.com/wbforum/viewtopic.php?t=49792 http://www.open-aurec.com/wbforum/viewtopic.php?t=50346 Now I just want to stress the seriousness of the bug. Basically it hugely handicaps an engine in tournaments with external books. Large number of computer chess enthusiasts like to compare engines without own books. There are multiple reasons for this: 1. Some engines include huge opening books created by professionals. These books basically ensure engine's advantage from the opening. For many it is interesting to compare the chess algorithms rather than human-assembled opening knowledge. 2. Many people use engines to analyze their games, to study the openings, study the games of others, etc.. This typically involves using a large game database, and an engine is only used as analysis tool. For such uses engine's own book is totally irrelevant, but engine's analysis quality (which correlates well with the playing strength) is important. So for these users it is important to know how strong is the engine on its own, without own book. 3. As the opening theory develops, new good lines are found, some old moves are refuted, etc.. So this year book is likely to be stronger than the last year one. Some people feel it's not fair to punish the older engines just because their book is outdated. There may be more reasons, in any case there is significant interest in engine performances without own books. However if two engines will play a match without any opening books, they will repeat the same game over and over. (Actually there may be some variations, but anyway a match of 100 games will have many repeated games). The solution is to use a short external book, same for both engines in a match. The opening is taken randomly from the book, then the engines play it out. Often a limit is placed on book length. I use books up to 8 moves deep in my tournament. CCRL limits the book to 12 moves. Some important rating lists and tournaments are played this way (with a short general book shared by all engines): CCRL 40/40: http://computerchess.org.uk/ccrl/4040/ CCRl 40/4: http://computerchess.org.uk/ccrl/404/ CEGT 40/20: http://www.husvankempen.de/nunn/rating.htm CEGT 40/4: http://www.husvankempen.de/nunn/blitz.htm KCEC: http://kirill-kryukov.com/chess/kcec/ Now, what happens when an engine with this bug plays in such tournament. There are two possibilities for such engine to play: 1. Play with external book like all other engines. In this case the engine is handicapped by having to move instantly after about move 50. How much is the performance decrease? It really depends on an engine. Some, like Resp, can't play this way at all - they lose on time every game. Some, like EXchess, seem to be almost OK, because they keep some time reserve. How much GNU Chess is punished by this bug remains to be seen. I may run it both ways and compare the results, but it will take some time (It needs many games). 2. Play without any opening book. In this case the engine is still handicapped since the opponent is using an external book. Book moves are usually better than what an engine can come up with on its own (at least more often better than not). Also book saves some thinking time. My estimation is that a short 8-move opening book (general book, not tuned to particular engine) is worth about 60 Elo points. It's a rough estimate, I did not study this systematically. It also depends on an engine. With a strong engine there is less benefit from a book, with weaker engines the book benefit is probably larger. In both of these two cases the real strength of an engine can not be measured. The ideas, algorithms and knowledge put into the engine can not be evaluated properly because of this subtle protocol issue. I hope it is easy to fix. If I can help with anything, or provide any additional information, please let me know. I don't subscribe to the list, so please use email to contact me. Best wishes, Kirill _______________________________________________ Bug-gnu-chess mailing list Bug-gnu-chess@gnu.org http://lists.gnu.org/mailman/listinfo/bug-gnu-chess
