This is a follow-up RFC from
https://discuss.tvm.ai/t/rfc-tvm-target-specification/6844
This RFC is discussed with @junrushao1994.
In this RFC, we propose the way to represent a composite target. The motivation
is that now we have multiple codegens contributed by the community members via
BYOC targeting to specialized hardware and libraries, such as ARM ACL, ARM
Ethos-N, TensorRT, and Vitis-AI. The main difference between those codegens and
other TVM backends (e.g., LLVM, CUDA, OpenCL, etc) is that those specialized
codegens may not execute an entire Relay graph, so we need to partition the
graph and only offload the supported subgraphs to the device; while keeping the
rest part on LLVM or CUDA.
Currently, we require users to manually run a list of Relay passes to partition
graphs to let the compile engine dispatch each Relay function to the
corresponding codegen. In general, we should encapsulate the required build
pipeline, including graph partitioning, to the target semantic.
We come up with two proposals to represent a composite target, so that we can
make corresponding changes in the compile engine to invoke the required passes.
**P1: Add an attribute `accelerators` (or whatever name) to all TVM backend
targets.**
Example:
```c
TVM_REGISTER_TARGET_KIND("llvm")
.add_attr_option<Array<String>>("keys")
.add_attr_option<Array<Target>>("accelerators") // Accelerator targets in
order.
.add_attr_option<Array<String>>("libs")
.add_attr_option<String>("mcpu")
.add_attr_option<Array<String>>("mattr")
.add_attr_option<String>("mtriple")
.add_attr_option<String>("mfloat-abi")
.set_default_keys({"cpu"})
.set_device_type(kDLCPU);
```
**P2: Create a separate composite target.**
Example:
```c
TVM_REGISTER_TARGET_KIND("composite")
.add_attr_option<Target>("target_host")
.add_attr_option<Array<Target>>("accelerators") // Accelerator targets in
order.
```
For both proposals, we will also do two things for each new device target:
1. Create a new accelerator target. Note that accelerator targets are not
allowed to be used directly. It has to be in other target's accelerator
attribute.
```c
TVM_REGISTER_ACCEL_TARGET_KIND("arm_acl")
.add_attr_option<...>... // arm_acl specific attributes.
```
2. Create an alias (e.g., `acl` in the above example) so that users do not have
to know anything about the target system. In this example, user can write
`target="acl"` to replace a composite target (taking the second proposal as an
example):
```json
{
kind: composite,
target_host: { ... },
accelerators: [{kind: arm_acl, ...}]
}
```
Comments and suggestions are welcome.
cc @zhiics, @anijain2305, @masahi, @tqchen, @kparzysz @ramana-arm, @matt-arm,
@jtuyls
---
[Visit Topic](https://discuss.tvm.ai/t/rfc-composite-target/7744/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click
here](https://discuss.tvm.ai/email/unsubscribe/ee51963c2ee468c8fce0f39707e8517b398b5cd088d15868d9e266059d873ada).
