Mousius commented on code in PR #12671:
URL: https://github.com/apache/tvm/pull/12671#discussion_r961506187
##########
python/tvm/relay/qnn/op/legalizations.py:
##########
@@ -417,6 +417,16 @@ def is_aarch64_arm():
return "aarch64" in target.attrs.get("mtriple", "")
+def is_cortexm_arm():
+ """Checks whether we are compiling for a Cortex-M target. We want to
preserve
+ int8 operations for these boards, because it halves the number of memory
loads
+ needed for dense layers and convolutions. It comes at a cost of needing
more
+ micro kernels and sign extension instructions to load these values, but
this
+ trade-off seems to be worth it on Cortex-M."""
+ target = tvm.target.Target.current(allow_none=False)
+ return "cortex-m" in target.attrs.get("mcpu", "")
Review Comment:
Can we add this to
https://github.com/apache/tvm/blob/main/src/target/parsers/mprofile.c as
`is_mprofile` ? We're about to remove all of these helpers in
https://github.com/apache/tvm/pull/12454
##########
python/tvm/relay/qnn/op/legalizations.py:
##########
@@ -433,15 +443,15 @@ def _qnn_conv2d_legalize_arm_cpu(attrs, inputs, types):
attrs["groups"],
)
use_int8_on_arm = (not is_depthwise) and is_aarch64_arm() and
attrs["data_layout"] == "NHWC"
- if use_int8_on_arm or is_fast_int8_on_arm():
+ if use_int8_on_arm or is_fast_int8_on_arm() or is_cortexm_arm():
return helper_change_dtypes_to_be_same(attrs, inputs, types,
relay.qnn.op.conv2d)
return helper_no_fast_int8_hw_legalization(attrs, inputs, types,
relay.nn.conv2d)
Review Comment:
I think this is actually the same as an issue awhile back
(https://github.com/apache/tvm/issues/8717#issuecomment-976400930), instead of
special casing with `is_cortexm` you should be able to instead use something
like:
```suggestion
has_asimd = (is_aarch64_arm() or "+neon" in target.mattr)
if has_asimd and not is_fast_int8_on_arm():
return helper_no_fast_int8_hw_legalization(attrs, inputs, types,
relay.nn.conv2d)
return helper_change_dtypes_to_be_same(attrs, inputs, types,
relay.qnn.op.conv2d)
```
Which means it will only do the casting when the specific architecture
features are available (`has_asimd` will become `target.features.has_asimd`
after https://github.com/apache/tvm/pull/12454 so is fine temporarily)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
