mbrookhart opened a new pull request #6351: URL: https://github.com/apache/incubator-tvm/pull/6351
Hello Friends, Over the last couple of months, @electriclilies and I have been working to add more dynamic support to relay ops, to separate the dynamic implementations into a dyn namespace, and to provide a pass for converting ops back to static forms when possible. The culmination of that work is this PR, which refactors the ONNX importer to directly create dynamic relay graphs instead of using infer_value to make them static in the importer. Longer term, this will allow us to import dynamic models that we can't currently use. We don't want to cause regressions for anyone, so this PR enables the dynamic_to_static pass by default in the graph runtime, we tested the PR against the ONNX model zoo https://github.com/onnx/models and fixed a number of issues in ops that apparently hadn't been tested with dynamic shapes to date. An added benefit of this PR is that it removes a severe bottleneck in the infer_value calls. Models with lots of dynamic ops will import and compile much faster than before, Bert Squad from the ONNX model zoo imports and compiles in ~170s on master vs ~15s with this change. This PR is not yet complete, we're working on adding dynamic upsampling3d and strided slice (#6316) to remove the last two infer value calls. Since we don't want to introduce regressions for anyone, I'd appreciate it if you could test any models you are currently running against this branch and let us know if you run into issues. Thanks! cc @masahi @jwfromm @soiferj @siju-samuel Please tag anyone else you think might be interested ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
