leandron commented on a change in pull request #24: URL: https://github.com/apache/tvm-rfcs/pull/24#discussion_r692785060
########## File path: rfcs/0024-add-oneflow-frontend.md ########## @@ -0,0 +1,133 @@ +- Feature Name: (`add oneflow frontend`) +- Start Date: (2021-8-20) +- RFC PR: [apache/tvm-rfcs#0024](https://github.com/apache/tvm-rfcs/pull/24) +- GitHub Issue: [apache/tvm#8804](https://github.com/apache/tvm/issues/8804) + +# Summary +[summary]: #summary + +To enhance the compatibility of TVM with deep learning frameworks, +we have created a frontend for TVM that targets [oneflow](https://github.com/Oneflow-Inc/oneflow) + +# Motivation +[motivation]: #motivation + +OneFlow, an open source deep learning framework with whole new frame design and the world's leading technology for distributed system. Here are advantages of OneFlow: + +- Perfectly support container platforms(k8s & docker) +- Handle large models easily +- Almost zero runtime overhead & linear speedup +- Support automatic mixed precision +- ... + +We are proud that OneFlow can support basic CNNs as well as very large pre-trained models, such as [GPT3, BERT, etc](https://github.com/Oneflow-Inc/OneFlow-Benchmark/tree/master/LanguageModeling). They are built using an early version of OneFlow, based on lazy mode. + +Currently, oneflow(nightly) supports the conversion of eager models to lazy graphs. We can quickly convert the eager model built by OneFlow to a lazy graph with the following code. + +```python +import oneflow as flow + + +class Graph(flow.nn.Graph): + def __init__(self, module): + super().__init__() + self.m = module + + def build(self, x): + out = self.m(x) + return out + + +# module: eager model built by OneFlow, align with Pytorch's python api. +graph = Graph(module) +``` + +Because of these features we wrote this `from_oneflow` which is based on lazy mode. + +We also note that many developers have converted their models to ONNX format for compatibility with TVM, and this part of the work is ongoing, as you can see [here](https://github.com/Oneflow-Inc/oneflow_convert_tools). + +Based on this background, we proposed this RFC to add a OneFlow frontend for TVM, improving usability for OneFlow users and enhancing the compatibility between OneFlow and TVM. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +We use a simple API to help users convert oneflow to tvm relay. + +```python +relay.frontend.from_oneflow(graph, model_dir_path, freeze_params=True, user_input=None) +``` + +- graph: flow.nn.Graph, contains information about the nodes and edges of the model +- model_dir_path: str, path of parameters +- freeze_params: bool, if this parameter is False, then the user can specify the input of the graph +- user_input: dict, information about the specified input of the model + +> NOTES: +> We prefer to let the user change the model node information by changing the model itself + +The following codes will show how to convert a oneflow model to a tvm relay + +```python +import tvm +import tvm.relay as relay + + +# load eager model(assuming that resnet50 has been built) +res50_module = resnet50() +pretrain_models = flow.load(model_path) +res50_module.load_state_dict(pretrain_models) Review comment: Does OneFlow support any input model from a file? If so, I'd like also to suggest that you also cover `tvmc` on your plans/implementation at some point. Example implementation - https://github.com/apache/tvm/blob/e691c7f83892d7242d0992c78cec1e2f8953a9e3/python/tvm/driver/tvmc/frontends.py#L174-L198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
