access2rohit commented on a change in pull request #16715: Lamb optimizer update URL: https://github.com/apache/incubator-mxnet/pull/16715#discussion_r350381992
########## File path: src/operator/optimizer_op-inl.h ########## @@ -1563,6 +1563,192 @@ inline void AdamUpdateEx(const nnvm::NodeAttrs& attrs, } } +struct LambUpdatePhaseOneParam : public dmlc::Parameter<LambUpdatePhaseOneParam> { + float beta1; + float beta2; + float epsilon; + float t; Review comment: we are using float here for integer data type. @sxjscience can you explain how we will loses precision for the operation `beta^t` ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services