altanh opened a new pull request #7731: URL: https://github.com/apache/tvm/pull/7731
This PR introduces two new passes: - `ConcretizeLike`: replaces `*_like` operators with their concrete-shape equivalent when the result shape is concrete. - `EliminateIdentity`: eliminates identity expressions like `x + zeros()`, `ones() * x`, etc. Expressions that broadcast `x` to a new shape are not removed, although we could explicitly replace them with broadcasting ops (not sure of the performance difference for this). This pass also doesn't examine the value of constants, so `x + const(0)` will not be eliminated; if anyone has a datatype-portable solution for this let me know. For this reason, this pass should be run before `SimplifyExpr`. I also refactored the existing DFPatternCallback-based passes slightly to lift out common machinery. I tried an approach of making these pattern rewrites statically initialized (@comaniac please let me know what you think of this approach). Together, these passes should help optimize the generated AD code (and credit to @t-vi for prototyping them in the blog post), by `FirstOrderGradient -> ConcretizeLike -> EliminateIdentity -> ...` cc @tqchen @comaniac @MarisaKirisame @yzhliu (I'll work on a `DeadParameterElimination` pass to complement `ConcretizeLike` as we discussed in the previous PR, but will send as a follow-up.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
