reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-551173088
I'm personally not fond of using the operator `reset_array` for some
specific purpose. First of all, it do
reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-551241818
@ptrendx The the performance overhead in your benchmark really comes from
the FFI and pushing ops to the a
reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-551246891
Another big factor that may contribute to the slowdown of assigning zeros is
through `a[:] = 0` which has
reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-551388401
To avoid too frequent cuda stream synchronization for the arrays to be
zeroed without introducing an opera
reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-551962930
@ptrendx Thanks for the script. I think a large part of overhead for zeroing
ndarrays individually in Pyth
reminisce commented on issue #16716: [Numpy] Fix collect_params().zero_grad()
in gluon numpy interface
URL: https://github.com/apache/incubator-mxnet/pull/16716#issuecomment-552737990
@szha @ptrendx From the discussion, I think we have aligned to keep
`nd.reset_arrays` as is for the legacy
