access2rohit commented on a change in pull request #16715: Lamb optimizer update
URL: https://github.com/apache/incubator-mxnet/pull/16715#discussion_r350381992
##########
File path: src/operator/optimizer_op-inl.h
##########
@@ -1563,6 +1563,192 @@ inline void AdamUpdateEx(const nnvm::NodeAttrs& attrs,
}
}
+struct LambUpdatePhaseOneParam : public
dmlc::Parameter<LambUpdatePhaseOneParam> {
+ float beta1;
+ float beta2;
+ float epsilon;
+ float t;
Review comment:
we are using float here for integer data type. @sxjscience can you explain
how we will loses precision for the operation `beta^t` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
