alter-xp commented on code in PR #75: URL: https://github.com/apache/tvm-rfcs/pull/75#discussion_r890104292
########## rfcs/0075_RISC-V_CSI-NN2_Intergration.md: ########## @@ -0,0 +1,171 @@ +- Feature Name: [RFC] RISC-V CSI-NN2 Compute Library integration +- Start Date: 2022-5-19 +- RFC PR: https://github.com/apache/tvm-rfcs/pull/75 +- GitHub Issue: [https://github.com/apache/tvm/issues/11506](https://github.com/apache/tvm/issues/11506) + +# Summary + +Introduce CSI-NN2 Compute Library into TVM to accelerate the inference performance of RISC-V CPU with Vector Extension. + +# Motivation + +Recently, in the latest Tiny v0.7 list released by AI benchmark MLPerf. Alibaba’s T-Head XuanTie RISC-V C906 processor has achieved first place in all 4 indicators. So, it’s a good time to support RISC-V CPUs with vector extension in TVM. + +[CSI-NN2 Compute Library](https://github.com/T-head-Semi/csi-nn2)(CSINN2) is an open-source project that provides hand-crafted assembler routines for RISC-V CPUs with vector extension. It is compatible with RISC-V v0.7.1 and v1.0 vector extension instruction standards. This integration will look at how we can accelerate CPU performance for RISC-V devices like XuanTie C906 in TVM using CSINN2. The idea is that by converting operators from a relay graph to CSINN2 we can achieve faster inference times due to these routines. The initial intention is that this will improve performance for FP32 models. Although, with further improvements to the integration this will extend to quantized models and support for a wider range of operators. + +PS: If you are interested in XuanTie C906 processor, [the D1 development board](https://d1.docs.aw-ol.com/en/) is a good choice. + +# Guide-level explanation + +## Build + +- Build with CSI-NN2 support in `build` + + - Set in your config.cmake file + + ```cmake + set(USE_OPENMP gnu) + set(USE_CSINN /path/to/csi-nn2) + set(USE_CSINN_DEVICE_RUNTIME X86) + ``` + + - Execute on the command lin + + ```shell + cmake ..;make -j4 + ``` + +- Cross-compile CSI-NN2 support in `build-rv` + + - Set in your config.cmake file + + ```cmake + set(USE_CPP_RPC ON) + set(USE_LIBBACKTRACE OFF) + set(USE_CSINN /path/to/csi-nn2) + set(USE_CSINN_DEVICE_RUNTIME C906) + ``` + + - Execute on the command lin + + ```shell + cmake ..;make -j4 runtime tvm_rpc + ``` + + After building successfully, we need to copy tvm_rpc and libs which used to device. + +## Run + +- Export binary library + + For a relay graph, following python APIs can be used to generate the binary library. + + ```python + from tvm.relay.op.contrib import csinn + + # API to call CSINN2 partitioning + # Here, module is the relay module + csinn_module = csinn.partition_for_csinn(module) + + # Build the Relay graph. + with tvm.target.Target("llvm -mtriple=riscv64-unknown-linux-gnu -mcpu=sifive-u74 -mabi=lp64d"): + factory = tvm.relay.build(csinn_module) + + # Export the module + lib_path = "lib_csinn2.so" + cross_compile = 'riscv64-unknown-linux-gnu-g++' + lib.export_library(lib_path, cc=cross_compile) + ``` + +- Running RPC service on device. + +- Connect the device and run. Review Comment: We are using the [D1 development board ](https://d1.docs.aw-ol.com/en) and QEMU for testing now. You can use QEMU to test. relevant documents can be viewed [here](https://github.com/apache/tvm/commit/2a051f39a22110a403ba2e44b5384ee0085a534b#diff-330c6f2d08738b9f7e5880b9fe245798559202d161db73243533525100a7d459). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
