MoisesHer opened a new pull request #17700: Use FP32 copy of weights for norm (multitensor LAMB optimizer) URL: https://github.com/apache/incubator-mxnet/pull/17700 ## Description ## When using Mixed Precision, use the master copy of weights (FP32) for computing the norm ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: test was already available (tests/python/unittesttest_optimizer:test_multilamb) - [x] Code is well-documented: - [x] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] Modified the code to use the master copy of weights for computing the norm when mixed precision is used
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
