ptrendx commented on issue #20293: URL: https://github.com/apache/incubator-mxnet/issues/20293#issuecomment-881829759
The graph is exactly the same, whether you have static or dynamic alloc. The problem I believe lies here: https://github.com/apache/incubator-mxnet/blob/3480ba2c6df02bb907d3a975d354efa8697c4e71/src/imperative/cached_op.cc#L986-L1006 especially lines 992 and 1002 - it sets the reqs for `NodeEntries` of the graph outputs based on the user preferences. This is only fine if those `NodeEntries` are not actually used in the other places in the computation, which for most cases is true. However, the gradient of `elementwise_add` is just a split. The `CloneGradient` method does not do any copying actually, it just reuses the `NodeEntry` and passes it to both downstream operators - this is the right behavior. But then when 1 of those downstream places is the output of the graph marked with `req=null`, you get a problem - because of those lines in `StaticBackward` function that edge is marked as `kNullOp` even though it is used in the rest of the computation. The proper fix therefore would be to change the logic in the `StaticBackward` to take into account all usages of those `NodeEntries` and only mark it as null if the graph output is the only usage. There could also be an additional improvement here (although the real world impact of this is probably pretty minimal) to make the `kNullOp` requirement "traverse" the graph - right now if you have a chain of (single input/output) operations A -> B -> C which ultimately produces unnecessary output, then only the C operation will know about this and A and B will be performed (and their outputs allocated) no matter what. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
