Anirudh Subramanian created MXNET-1096:
------------------------------------------
Summary: Improve customer experience with CUDNN Auto Tuning
Key: MXNET-1096
URL: https://issues.apache.org/jira/browse/MXNET-1096
Project: Apache MXNet
Issue Type: Improvement
Components: Apache MXNet Backend
Reporter: Anirudh Subramanian
Look into improvement of Customer Experience for CUDNN_AUTOTUNE_DEFAULT and how
it can be improved for use cases where it may become a bottleneck.
Suggestion from Marco: "One possible mitigation for cases like this where the
parameter distribution is non-uniform(varied input shape, output shape, weight
shape combination), we could have a strategy that doesn't start CuDNN
auto-tuning on the first time but only if that combination has been received
multiple times. The optimization might then happen in the background. If the
background task finishes, the chosen implementation is then atomically
switched. This would give us the benefit of low latency for every single
request as well as even further reduced latency in the long run as soon as the
background optimization finished. We might just have to check back with Nvidia
whether there's a CuDNN autotuning strategy that allows to run in the
background with reduced resource ultilization (to avoid running out of memory
or bottlenecking the main thread). "
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org
For additional commands, e-mail: issues-h...@mxnet.apache.org
