On 8/03/19 16:14, David Wu wrote: > I suspect Leela Zero would come off as far *less* favorable if one > tried to do such a comparison using their actual existing code rather > than abstracting down to counting neural net evals, because as far as > I know in Leela Zero there is no cross-game batching of neural net > evaluations, which makes a huge difference in the ability to use a > strong GPU efficiently.
We found that the speedup from batching mostly depended on whether: - you're using cuDNN (i.e. NVIDIA hardware) - you're using cards with Tensor Cores - you're using smaller boards Not so coincidentally all of these are true for *you*, i.e. a homogeneous farm of powerful NVIDIA Volta cards where you control the entire software stack, and are mixing in different sized boards in the games. But that's not quite the platform Leela Zero was targeted at, it's almost the exact opposite :-) There are extensive benchmarks in the github issues of cuDNN vs TensorRT vs generic OpenCL performance with varying batch sizes, and you'll see that for the majority of hardware there wasn't that much to gain by adding batching. Once Tensor Core support was added to the OpenCL code, batching immediately made a huge difference (on Volta/RTX at least...) and was thus merged shortly after. > Only in the last couple months or so based on > what I've been seeing in chat and pull requests, Leela Zero > implemented within-search batching of neural net evals, but clients > still only play one game at a time. Playing multiple games at the same time has been supported since very early on (-g switch in AutoGTP or always by running multiple clients), it's batching networks over multiple games that wasn't (and still isn't) implemented in the default client. I do think it's useful to have, it's just that for Leela Zero this wasn't - and probably still isn't - very important compared to everything else. Anyway, I agree and note this is all completely tangential to the question of "extra computation overhead" for the changes, which should be negligible. -- GCP _______________________________________________ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
