[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939481576
##
python/tvm/relay/backend/contrib/uma/api/lower.py:
##
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+"""Lowering base class of the Universal Modular Accelerator Interface
(UMA)."""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+self._operator_strategies: List[
+Tuple[
+str,
+Callable[
+[tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType,
tvm.target.Target],
+tvm.relay.op.op.OpStrategy,
+],
+Optional[int],
+]
+] = []
+self._tir_passes: List[Tuple[PassPhase,
tvm.tir.transform.PrimFuncPass]] = []
+
+def _lower_relay_to_tir(self, relay_prim_func: relay.Function) ->
tvm.tir.PrimFunc:
+"""Lower a Relay primitive function to a S-TIR primitive function.
+
+Parameters
+--
+prim_func : tvm.relay.Function
+The Relay function to lower.
+
+Returns
+---
+out : tvm.tir.PrimFunc
+The lowered schedulable TensorIR primitive function.
+
+"""
+
+def _get_tensors(te_cached_func):
+outputs = list(te_cached_func.outputs)
+stack = []
+visited = set()
+for output_ in outputs:
+if output_ not in visited:
+visited.add(output_)
+stack.append(output_)
+
+args = []
+while len(stack) != 0:
+tensor = stack.pop()
+if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+args.append(tensor)
+elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+inputs = tensor.op.input_tensors
+for input_ in inputs:
+if input_ not in visited:
+visited.add(input_)
+stack.append(input_)
+
+return args + outputs
+
+f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
+te_cached_func = f(relay_prim_func)
+x = _get_tensors(te_cached_func)
+tir_prim_func = te.create_prim_func(x)
+tir_prim_func = tir_prim_func.with_attr(
+"global_symbol", relay_prim_func.attrs["global_symbol"]
+)
+# TODO: The target should probably come from somewhere else instead of
being created here.
+tir_prim_func = tir_prim_func.with_attr("target",
tvm.target.Target(self.target_name))
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939213435
##
python/tvm/relay/backend/contrib/uma/api/lower.py:
##
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+"""Lowering base class of the Universal Modular Accelerator Interface
(UMA)."""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+self._operator_strategies: List[
+Tuple[
+str,
+Callable[
+[tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType,
tvm.target.Target],
+tvm.relay.op.op.OpStrategy,
+],
+Optional[int],
+]
+] = []
+self._tir_passes: List[Tuple[PassPhase,
tvm.tir.transform.PrimFuncPass]] = []
+
+def _lower_relay_to_tir(self, relay_prim_func: relay.Function) ->
tvm.tir.PrimFunc:
+"""Lower a Relay primitive function to a S-TIR primitive function.
+
+Parameters
+--
+prim_func : tvm.relay.Function
+The Relay function to lower.
+
+Returns
+---
+out : tvm.tir.PrimFunc
+The lowered schedulable TensorIR primitive function.
+
+"""
+
+def _get_tensors(te_cached_func):
+outputs = list(te_cached_func.outputs)
+stack = []
+visited = set()
+for output_ in outputs:
+if output_ not in visited:
+visited.add(output_)
+stack.append(output_)
+
+args = []
+while len(stack) != 0:
+tensor = stack.pop()
+if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+args.append(tensor)
+elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+inputs = tensor.op.input_tensors
+for input_ in inputs:
+if input_ not in visited:
+visited.add(input_)
+stack.append(input_)
+
+return args + outputs
+
+f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
Review Comment:
ah, sorry, you are referring to the f. I thought you meant the
relay.backend.LowerToTE.
Good that @cgerum understood what you meant and fixed it :smirk:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939160847 ## apps/uma/_template/conv2dnchw.cc: ## @@ -0,0 +1,95 @@ +/* +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +*/ +#include + +#ifdef __cplusplus +extern "C" +#endif + +/*! + * \brief Conv2D function for mock-accelerator examples. Limited to same-padded Conv2D with + * stride (1,1) and datatype float. \param ifmap Pointer to input feature map data of size + * iw*ih*ic*sizeof(float). \param weights Pointer to weight data of size + * kh*kw*ic**oc*sizeof(float). \param result Pointer to output feature map data of size + * iw*ih*oc*sizeof(float). \param oc Number of channels of output feature map. \param iw Width + * of input feature map, ifmap. \param ih Height of input feature map, ifmap. \param ic Number + * of channels of input feature map. \param kh Height of convolution kernels. \param kw Width of + * convolution kernels. + * + * \return error code + * + */ +int +my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic, Review Comment: I agree that it doesnt look linted. It is what ./docker/lint.sh -i clang_format gives me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939153921
##
python/tvm/relay/backend/contrib/uma/api/lower.py:
##
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+"""Lowering base class of the Universal Modular Accelerator Interface
(UMA)."""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+self._operator_strategies: List[
+Tuple[
+str,
+Callable[
+[tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType,
tvm.target.Target],
+tvm.relay.op.op.OpStrategy,
+],
+Optional[int],
+]
+] = []
+self._tir_passes: List[Tuple[PassPhase,
tvm.tir.transform.PrimFuncPass]] = []
+
+def _lower_relay_to_tir(self, relay_prim_func: relay.Function) ->
tvm.tir.PrimFunc:
+"""Lower a Relay primitive function to a S-TIR primitive function.
+
+Parameters
+--
+prim_func : tvm.relay.Function
+The Relay function to lower.
+
+Returns
+---
+out : tvm.tir.PrimFunc
+The lowered schedulable TensorIR primitive function.
+
+"""
+
+def _get_tensors(te_cached_func):
+outputs = list(te_cached_func.outputs)
+stack = []
+visited = set()
+for output_ in outputs:
+if output_ not in visited:
+visited.add(output_)
+stack.append(output_)
+
+args = []
+while len(stack) != 0:
+tensor = stack.pop()
+if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+args.append(tensor)
+elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+inputs = tensor.op.input_tensors
+for input_ in inputs:
+if input_ not in visited:
+visited.add(input_)
+stack.append(input_)
+
+return args + outputs
+
+f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
+te_cached_func = f(relay_prim_func)
+x = _get_tensors(te_cached_func)
+tir_prim_func = te.create_prim_func(x)
+tir_prim_func = tir_prim_func.with_attr(
+"global_symbol", relay_prim_func.attrs["global_symbol"]
+)
+# TODO: The target should probably come from somewhere else instead of
being created here.
+tir_prim_func = tir_prim_func.with_attr("target",
tvm.target.Target(self.target_name))
Review Comment:
@cgerum can you comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939153521
##
python/tvm/relay/backend/contrib/uma/api/lower.py:
##
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+"""Lowering base class of the Universal Modular Accelerator Interface
(UMA)."""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+self._operator_strategies: List[
+Tuple[
+str,
+Callable[
+[tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType,
tvm.target.Target],
+tvm.relay.op.op.OpStrategy,
+],
+Optional[int],
+]
+] = []
+self._tir_passes: List[Tuple[PassPhase,
tvm.tir.transform.PrimFuncPass]] = []
+
+def _lower_relay_to_tir(self, relay_prim_func: relay.Function) ->
tvm.tir.PrimFunc:
+"""Lower a Relay primitive function to a S-TIR primitive function.
+
+Parameters
+--
+prim_func : tvm.relay.Function
+The Relay function to lower.
+
+Returns
+---
+out : tvm.tir.PrimFunc
+The lowered schedulable TensorIR primitive function.
+
+"""
+
+def _get_tensors(te_cached_func):
+outputs = list(te_cached_func.outputs)
+stack = []
+visited = set()
+for output_ in outputs:
+if output_ not in visited:
+visited.add(output_)
+stack.append(output_)
+
+args = []
+while len(stack) != 0:
+tensor = stack.pop()
+if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+args.append(tensor)
+elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+inputs = tensor.op.input_tensors
+for input_ in inputs:
+if input_ not in visited:
+visited.add(input_)
+stack.append(input_)
+
+return args + outputs
+
+f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
Review Comment:
well, probably a matter of taste :smile: .
We followed the convention/naming used in ethossu/tir/compiler.py
@cgerum: any thoughts here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939147157
##
src/relay/backend/contrib/uma/targets.cc:
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/targets.cc
+ *
+ * \brief this file contains the targets for the Universal Modular Accelerator
Interface (UMA).
+ */
+
+#include
+#include
+
+namespace tvm {
+
+namespace relay {
+namespace contrib {
+namespace uma {
+tvm::transform::Pass RelayToTIR(String target_name);
+runtime::Module TIRToRuntime(IRModule mod, Target target);
+} // namespace uma
+} // namespace contrib
+} // namespace relay
+
+TVM_REGISTER_GLOBAL("relay.backend.contrib.uma.RegisterTarget")
+.set_body_typed([](String target_name, Map
attr_options) -> bool {
+ // @todo(cgerum): We probably should get rid of target.register rather
sooner than later
+ // And use a proper registry for uma backends
+ for (const String registered_target_name :
::tvm::TargetKindRegEntry::ListTargetKinds()) {
+if (registered_target_name == target_name) {
+ return false;
+}
+ }
+
+ auto target_kind =
+ ::tvm::TargetKindRegEntry::RegisterOrGet(target_name)
+ .set_name()
+ .set_device_type(kDLCPU)
+ .add_attr_option>("keys")
+ .add_attr_option("tag")
+ .add_attr_option("device")
+ .add_attr_option("model")
+ .add_attr_option>("libs")
+ .add_attr_option("host")
+ .add_attr_option("from_device")
+ .set_attr(tvm::attr::kRelayToTIR,
+
relay::contrib::uma::RelayToTIR(target_name))
+ .set_attr("TIRToRuntime",
relay::contrib::uma::TIRToRuntime);
+
+ for (auto& attr_option : attr_options) {
+auto option_name = attr_option.first;
+auto default_value = attr_option.second;
+if (default_value->IsInstance()) {
+ target_kind.add_attr_option(option_name,
Downcast(default_value));
+} else if (default_value->IsInstance()) {
+ target_kind.add_attr_option(option_name,
Downcast(default_value));
+} else {
+ LOG(FATAL) << "Attribute option of type " <<
attr_option.second->GetTypeKey()
Review Comment:
added tests in test_target to check these cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939146296
##
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+auto includes_pf =
+tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" +
target_str_);
+ICHECK(includes_pf);
Review Comment:
I agree with your finding for "includes".
It is optional, therefore I removed the ICHECK from the cc code.
I also agree that it is important, therefore I added a separate parameter
instead of kwargs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939145045
##
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Optional
+import tvm
+
+
+class UMACodegen(object):
+"""
+Codegen base class of the Universal Modular Accelerator Interface (UMA)
+"""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+if fmt == "c":
+self._register_c_codegen(**kwargs)
+else:
+raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+def _register_c_codegen(self, includes: Optional[Callable[[], str]] =
None) -> None:
+"""Registration of UMA helper functions, e.g. includes and
replace_call_extern.
+
+Parameters
+--
+includes : OptionalCallable[[], str]]
+user-defined function that adds C-#include statement to UMA C-Code.
+"""
+if includes is not None:
Review Comment:
I agree with your finding for "includes".
It is optional, therefore I removed the ICHECK from the cc code.
I also agree that it is important, therefore I added a separate parameter
instead of kwargs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939142883 ## apps/uma/_template/strategies.py: ## @@ -0,0 +1,17 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Strategies for the my_ai_hw accelerator""" Review Comment: Added a comment with an example. Later examples in the tutorial will have detailed code with explanation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939140952 ## include/tvm/relay/transform.h: ## @@ -509,6 +509,7 @@ TVM_DLL Pass SimplifyExpr(); * * \param config All available targets. * + * Review Comment: nope. removed it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939140465
##
tests/python/contrib/test_uma/test_uma_pipeline.py:
##
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+from tvm.relay import transform, testing
+from tvm.testing.aot import (
+AOTTestModel,
+AOTTestRunner,
+generate_ref_data,
+compile_and_run,
+)
+
+import tvm
+from test_uma_vanilla_accelerator import VanillaAcceleratorBackend
+from tvm import relay
+import numpy as np
+from collections import OrderedDict
+
+from tvm.relay.backend.contrib.uma.api.utils import uma_available
+
+pytestmark = pytest.mark.skipif(not uma_available(), reason="UMA not
available")
+
+
+@pytest.mark.parametrize(
+"interface_api,use_unpacked_api,test_runner,groups,weight_shape",
+[("c", True, AOT_DEFAULT_RUNNER, 1, 32)],
+)
+def test_conv2d(interface_api, use_unpacked_api, test_runner, groups,
weight_shape):
+"""Test a subgraph with a single conv2d operator."""
+mod, inputs, output_list, test_runner = create_conv2d(groups, test_runner,
weight_shape)
+
+uma_backend = VanillaAcceleratorBackend()
+uma_backend.register()
+mod = uma_backend.partition(mod)
+target = tvm.target.Target("vanilla_accelerator",
host=tvm.target.Target("c"))
+
+compile_and_run(
+AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+test_runner,
+interface_api,
+use_unpacked_api,
+target=target,
+)
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+dtype = "float32"
+ishape = (1, 32, 14, 14)
+wshape = (32, weight_shape, 3, 3)
+pass_config = {"tir.usmp.enable": True}
+test_runner = AOTTestRunner(
+makefile=test_runner.makefile,
+prologue=test_runner.prologue,
+epilogue=test_runner.epilogue,
+includes=test_runner.includes,
+parameters=test_runner.parameters,
+pass_config=pass_config,
+)
+data0 = relay.var("data", shape=ishape, dtype=dtype)
+weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1),
groups=groups)
+main_f = relay.Function([data0, weight0], out)
+mod = tvm.IRModule()
+mod["main"] = main_f
+mod = transform.InferType()(mod)
+i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+output_list = generate_ref_data(mod, inputs)
+return mod, inputs, output_list, test_runner
+
+
+def _generate_runtime_data(input_shapes: dict, output_shapes: dict) ->
[OrderedDict, OrderedDict]:
+assert len(input_shapes) == 1
+assert len(output_shapes) == 1
+
+iname = list(input_shapes.keys())[0]
+oname = list(output_shapes.keys())[0]
+ishape = input_shapes[iname]
+oshape = output_shapes[oname]
+i_data = np.random.uniform(0, 1, ishape).astype("float32")
+o_data = np.random.uniform(0, 1, oshape).astype("float32")
+oname = "output" # name set by relay.build in
executor_codegen_metadata.outputs
+inputs = OrderedDict([(iname, i_data)])
+outputs = OrderedDict([(oname, o_data)])
+return inputs, outputs
+
+
+def test_mobilenet():
+"""Full network test with Mobilenet"""
+use_unpacked_api = True
+interface_api = "c"
+test_runner = AOT_DEFAULT_RUNNER
+
+mod, params = testing.mobilenet.get_workload(batch_size=1)
+
+uma_backend = VanillaAcceleratorBackend()
+uma_backend.register()
+target = tvm.target.Target("vanilla_accelerator",
host=tvm.target.Target("c"))
+target_c = tvm.target.Target("c")
+
+data_shape = [int(x) for x in mod["main"].checked_type.arg_types[0].shape]
+data = np.random.uniform(size=data_shape).astype("float32")
+input_list = {"data": data}
+output_list = generate_ref_data(mod, input_list, params)
+mod = uma_backend.partition(mod)
Review Comment:
Added comment
--
This is an automated message from the Apache Git Service.
To respond to the
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939139200 ## apps/uma/_template/conv2dnchw.cc: ## @@ -0,0 +1,95 @@ +/* +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +*/ +#include + +#ifdef __cplusplus +extern "C" +#endif + Review Comment: done ## apps/uma/_template/passes.py: ## @@ -0,0 +1,142 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +_EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw" +_TVM_BLOCK_MATCH_NAME = "conv2d_nchw" + +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@classmethod +def _my_ai_hw_conv2d_pass(cls, func, mod, ctx): +_loops = dict() +_handles = [] +_entry_node = None + +def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool: +""" +Determine of a tir.block with `name` exists in `func` +""" + +def _hb(op): +if isinstance(op, tvm.tir.Block): +_found_blocks.append(op.name_hint) + +_found_blocks = [] +tvm.tir.stmt_functor.post_order_visit(func.body, _hb) +return name in _found_blocks + +def _detect_and_replace_conv2d( +func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +def _replace_conv2d(op): +if op == _entry_node: +irb = tvm.tir.ir_builder.create() +# Collection of buffer address +buffers = [b[1].data for b in _handles] +# extraction of loop offsets +for k, v in _loops.items(): +assert v.min.value == 0 +offset_order = ["co", "w", "h", "ci", "kh", "kw"] +offsets = [_loops[i].extent.value for i in offset_order] +args = buffers + offsets +external_call = tvm.tir.Evaluate( +tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args) +) +ext_calls = tvm.tir.SeqStmt([external_call]) +irb.emit(ext_calls) +irb_result = irb.get() +return irb_result +elif isinstance(op, tvm.tir.SeqStmt): +# Remove that pad block of TOPI's conv2DNCHW by only returning the 2nd statement +return op.seq[1] +return op + +sch = tir.Schedule(func) + +if _has_block(cls._TVM_BLOCK_MATCH_NAME, func): +conv2d_block = sch.get_block(cls._TVM_BLOCK_MATCH_NAME) +rv_loops = sch.get_loops(conv2d_block) +assert len(rv_loops) == 7 +loops = dict( +n=rv_loops[0], +co=rv_loops[1], +h=rv_loops[2], +w=rv_loops[3], +ci=rv_loops[4], +kh=rv_loops[5], +kw=rv_loops[6], +
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939139928 ## apps/uma/_template/passes.py: ## @@ -0,0 +1,142 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +_EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw" +_TVM_BLOCK_MATCH_NAME = "conv2d_nchw" + +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@classmethod +def _my_ai_hw_conv2d_pass(cls, func, mod, ctx): +_loops = dict() +_handles = [] +_entry_node = None + +def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool: +""" +Determine of a tir.block with `name` exists in `func` +""" + +def _hb(op): +if isinstance(op, tvm.tir.Block): +_found_blocks.append(op.name_hint) + +_found_blocks = [] +tvm.tir.stmt_functor.post_order_visit(func.body, _hb) +return name in _found_blocks + +def _detect_and_replace_conv2d( +func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +def _replace_conv2d(op): +if op == _entry_node: +irb = tvm.tir.ir_builder.create() +# Collection of buffer address +buffers = [b[1].data for b in _handles] +# extraction of loop offsets +for k, v in _loops.items(): +assert v.min.value == 0 +offset_order = ["co", "w", "h", "ci", "kh", "kw"] +offsets = [_loops[i].extent.value for i in offset_order] +args = buffers + offsets +external_call = tvm.tir.Evaluate( +tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args) +) +ext_calls = tvm.tir.SeqStmt([external_call]) Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r939138720 ## apps/uma/_template/__init__.py: ## @@ -0,0 +1,23 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +""" + +Template files for UMA tutorial + +Do not import Review Comment: Removed the Do not import comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r936306933 ## python/tvm/relay/backend/contrib/uma/api/partitioner.py: ## @@ -0,0 +1,118 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)""" + +from typing import Callable, Dict, List, Tuple, Optional + +import tvm +from tvm import relay +from tvm.relay.build_module import bind_params_by_name +from tvm.relay.op.contrib.register import register_pattern_table +from .utils import PassPhase + + +PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]] + + +class UMAPartitioner: +"""Partitioner base class of the Universal Modular Accelerator Interface (UMA).""" + +def __init__(self, target_name: str, merge_compiler_regions: bool = True) -> None: +self.target_name = target_name +self.merge_compiler_regions = merge_compiler_regions + +self._relay_passes: List[Tuple[PassPhase, tvm.transform.Pass]] = [] +self._patterns: PatternTable = [] + +def add_pattern( +self, +name: str, +pattern: tvm.relay.dataflow_pattern.DFPattern, +predicate: Optional[Callable] = None, +) -> None: +"""Add pattern to UMA partitioner + +Parameters +-- +name : str +relay name of pattern + +pattern: tvm.relay.dataflow_pattern.DFPattern +pattern description as DFPattern + +predicate: Optional[Callable] +Optional predicate + +""" + +name = self.target_name + "." + name +if predicate: +self._patterns.append((name, pattern, predicate)) +else: +self._patterns.append((name, pattern)) + +def _pattern_table(self) -> PatternTable: +return self._patterns + +def register(self) -> None: +"""Register all relevant relay-to-relay functions.""" +register_pattern_table(self.target_name, self._pattern_table) + +def partition( +self, mod: tvm.IRModule, params: Optional[Dict[str, tvm.runtime.NDArray]] = None +) -> tvm.IRModule: +"""Partition the relay graph in by the NPU supported and unsupported parts. + +Parameters +-- +mod : tvm.IRModule +The relay module to be partitioned. + +params: Optional[Dict[str, tvm.runtime.NDArray]] + +Returns +--- +out : tvm.IRModule +The partitioned relay module. + +""" +if params: +mod["main"] = bind_params_by_name(mod["main"], params) + +mod = relay.transform.InferType()(mod) Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r936302355 ## python/tvm/relay/backend/contrib/uma/_template/backend.py: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936300831
##
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+auto includes_pf =
+tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" +
target_str_);
+ICHECK(includes_pf);
+String includes = (*includes_pf)();
+decl_stream << includes;
+std::unordered_set devices;
+devices.insert(target_str_);
+CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) {
CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+}
+auto replace_call_extern_pf =
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936300311
##
src/relay/backend/contrib/uma/relay_to_tir.cc:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular
Accelerator Interface (UMA).
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const
std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+this->memo_[var] = value;
+ }
+};
+auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+Var var = Downcast(this->VisitExpr(op->var));
+if (var.same_as(op->var) && value.same_as(op->value) &&
body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+} else {
+ this->memo_[expr] = Let(var, value, body);
+}
+ }
+};
+ExpandANormalForm(op, pre_visit, post_visit);
+return memo_[GetRef(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+Call call = Downcast(post);
+if (CanOutlineExpr(call->op)) {
+ Function func = Downcast(call->op);
+ auto gv_name = func->GetAttr("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but
didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+}
+return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+if (!expr->IsInstance()) {
+ return false;
+}
+Function func = Downcast(expr);
+auto compiler = func->GetAttr(attr::kCompiler);
+if (!compiler.defined()) {
+ return false;
+}
+if (compiler != compiler_name_) {
+ return false;
+}
+return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external
functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string&
compiler_name) {
+ runtime::TypedPackedFunc
pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+GlobalVar gv = mod->GetGlobalVar("main");
+Function main_func = Downcast(mod->Lookup("main"));
+auto new_main_body =
+OutlineCompilerFunctionsMutator(mod,
compiler_name).VisitExpr(main_func->body);
+if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params,
new_main_body);
+ mod->Update(gv, new_main_func);
+}
+
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936299201
##
tests/python/contrib/test_uma/test_partition.py:
##
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+partitioner = UMAPartitioner("test_partition")
+assert get_pattern_table("test_partition") is None
+
+partitioner.register()
+
+assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+"workload,backend,merge,expected_partitions",
+[
+("resnet", "dnnl", False, 17),
+("resnet", "dnnl", True, 17),
Review Comment:
@areusch, we removed the hardcoded integers for the expected paritions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930832276 ## python/tvm/relay/backend/contrib/uma/_template/passes.py: ## @@ -0,0 +1,137 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import relay, tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@staticmethod +def _my_ai_hw_conv2d_pass(func, mod, ctx): +_found_blocks = [] +_loops = dict() +_handles = [] +_entry_node = None +_external_function_name = "my_ai_hw_conv2dnchw" Review Comment: The decorator transform.prim_func_pass prevents me from using class-level constant, so I would tend to leave them in the _my_ai_hw_conv2d_pass function -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930006872 ## python/tvm/relay/backend/contrib/uma/_template/passes.py: ## @@ -0,0 +1,137 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import relay, tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@staticmethod +def _my_ai_hw_conv2d_pass(func, mod, ctx): +_found_blocks = [] +_loops = dict() +_handles = [] +_entry_node = None +_external_function_name = "my_ai_hw_conv2dnchw" Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930056748 ## tests/scripts/task_python_uma.sh: ## @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +set -euxo pipefail + +source tests/scripts/setup-pytest-env.sh + +run_pytest ctypes test_uma tests/python/contrib/test_uma +run_pytest cython3 test_uma tests/python/contrib/test_uma Review Comment: we removed this file -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930047570 ## python/tvm/relay/backend/contrib/uma/_template/backend.py: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. Review Comment: We will move it to apps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930044501 ## python/tvm/relay/backend/contrib/uma/_template/run.py: ## @@ -0,0 +1,88 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER + +from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner + +import tvm +from tvm import relay +from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend +from tvm.relay import transform +from collections import OrderedDict + +import numpy as np +import tarfile +from pathlib import Path +import onnx + +from tvm.testing.aot import ( +AOTTestModel, +AOTTestRunner, +generate_ref_data, +compile_and_run, +) Review Comment: @manupa-arm , this is not supposed to be test code. This is example code for the tutorial. We will replace AOTTestRunner by microTVM Project API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930041551 ## python/tvm/relay/backend/contrib/uma/uma_cli.py: ## @@ -0,0 +1,92 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +UMA Command Line Interface (CLI) Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930040959
##
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+auto includes_pf =
+tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" +
target_str_);
+ICHECK(includes_pf);
+String includes = (*includes_pf)();
+decl_stream << includes;
+std::unordered_set devices;
+devices.insert(target_str_);
+CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) {
CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+}
+auto replace_call_extern_pf =
Review Comment:
We discussed this again and reviewed the options.
The conclusion is to remove this and also remove
codegen_c_replace_call_extern_{target_str} from Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930030731
##
src/relay/backend/contrib/uma/relay_to_tir.cc:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular
Accelerator Interface (UMA).
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const
std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+this->memo_[var] = value;
+ }
+};
+auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+Var var = Downcast(this->VisitExpr(op->var));
+if (var.same_as(op->var) && value.same_as(op->value) &&
body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+} else {
+ this->memo_[expr] = Let(var, value, body);
+}
+ }
+};
+ExpandANormalForm(op, pre_visit, post_visit);
+return memo_[GetRef(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+Call call = Downcast(post);
+if (CanOutlineExpr(call->op)) {
+ Function func = Downcast(call->op);
+ auto gv_name = func->GetAttr("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but
didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+}
+return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+if (!expr->IsInstance()) {
+ return false;
+}
+Function func = Downcast(expr);
+auto compiler = func->GetAttr(attr::kCompiler);
+if (!compiler.defined()) {
+ return false;
+}
+if (compiler != compiler_name_) {
+ return false;
+}
+return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external
functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string&
compiler_name) {
+ runtime::TypedPackedFunc
pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+GlobalVar gv = mod->GetGlobalVar("main");
+Function main_func = Downcast(mod->Lookup("main"));
+auto new_main_body =
+OutlineCompilerFunctionsMutator(mod,
compiler_name).VisitExpr(main_func->body);
+if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params,
new_main_body);
+ mod->Update(gv, new_main_func);
+}
+
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930028351
##
src/relay/backend/contrib/uma/relay_to_tir.cc:
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular
Accelerator Interface (UMA).
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
@areusch, should we move this to a seperate PR then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930025468
##
tests/python/contrib/test_uma/test_partition.py:
##
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+partitioner = UMAPartitioner("test_partition")
+assert get_pattern_table("test_partition") is None
+
+partitioner.register()
+
+assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+"workload,backend,merge,expected_partitions",
+[
+("resnet", "dnnl", False, 17),
+("resnet", "dnnl", True, 17),
Review Comment:
@areusch can you elaborate what you mean? We don't fully understand what you
mean here
##
python/tvm/relay/backend/contrib/uma/tutorial.md:
##
@@ -0,0 +1,195 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Making your hardware accelerator TVM-ready with UMA
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930022435
##
tests/python/contrib/test_uma/test_partition.py:
##
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+partitioner = UMAPartitioner("test_partition")
+assert get_pattern_table("test_partition") is None
+
+partitioner.register()
+
+assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+"workload,backend,merge,expected_partitions",
+[
+("resnet", "dnnl", False, 17),
+("resnet", "dnnl", True, 17),
+("mlp", "dnnl", False, 1),
+("resnet", "cutlass", False, 2),
+("resnet", "cutlass", True, 2),
+("mlp", "cutlass", False, 4),
+("mlp", "cutlass", True, 2),
+],
+)
+def test_existing_pattern_tables(workload, backend, merge,
expected_partitions):
+partitioner = UMAPartitioner(backend + "_uma", merge)
+pattern_table = get_pattern_table(backend)
+
+for entry in pattern_table:
+partitioner.add_pattern(*entry)
+
+if workload == "resnet":
+net = resnet.get_net(1, 10)
+elif workload == "mlp":
+net = mlp.get_net(1, 10)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930020506
##
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+"""
+Codegen base class of the Universal Modular Accelerator Interface (UMA)
+"""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+if fmt == "c":
+self._register_c_codegen(**kwargs)
+else:
+raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+def _register_c_codegen(
+self,
+includes: Callable[[], str] = None,
+replace_call_extern: Callable[[tvm.ir.container.Array], str] = None,
+) -> None:
+if includes is not None:
+tvm._ffi.register_func(
+
"relay.ext.uma.codegen_c_includes_{}".format(self.target_name), includes
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930017268
##
python/tvm/relay/backend/contrib/uma/_template/run.py:
##
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+AOTTestModel,
+AOTTestRunner,
+generate_ref_data,
+compile_and_run,
+)
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+dtype = "float32"
+ishape = (1, 32, 14, 14)
+wshape = (32, weight_shape, 3, 3)
+pass_config = {"tir.usmp.enable": True}
+test_runner = AOTTestRunner(
+makefile=test_runner.makefile,
+prologue=test_runner.prologue,
+epilogue=test_runner.epilogue,
+includes=test_runner.includes,
+parameters=test_runner.parameters,
+pass_config=pass_config,
+)
+data0 = relay.var("data", shape=ishape, dtype=dtype)
+weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1),
groups=groups)
+main_f = relay.Function([data0, weight0], out)
+mod = tvm.IRModule()
+mod["main"] = main_f
+mod = transform.InferType()(mod)
+i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+output_list = generate_ref_data(mod, inputs)
+return mod, inputs, output_list, test_runner
+
+
+def main():
+mod, inputs, output_list, test_runner = create_conv2d()
+
+uma_backend = MyAiHwBackend()
+uma_backend.register()
+mod = uma_backend.partition(mod)
+target = tvm.target.Target("my_ai_hw", host=tvm.target.Target("c"))
+
+export_directory = tvm.contrib.utils.tempdir(keep_for_debug=True).path
+print(f"Generated files are in {export_directory}")
+compile_and_run(
+AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+test_runner,
+interface_api="c",
+use_unpacked_api=True,
+target=target,
+test_dir=str(export_directory),
+)
+
+
+if __name__ == "__main__":
+main()
Review Comment:
we would tend to move anything TVMC and CLI related to the next PR
##
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+UMA Command Line Interface (CLI)
+
+Tool to create code skeletons for an easy integration of
+new AI hardware accelerators/libraries into TVM using UMA
+"""
+
+import argparse
+import os
+import shutil
+import sys
+from inflection import camelize, underscore
+
+
+def _parse_args():
+parser = argparse.ArgumentParser(description="UMA Interface command line
interface")
+parser.add_argument(
+"--add_hardware",
+type=str,
+required=True,
+)
+parser.add_argument(
+"--tutorial",
+type=str,
+)
+args = parser.parse_args()
+return args
+
+
+def replace_template_name(
+fi
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930008614 ## python/tvm/relay/backend/contrib/uma/_template/passes.py: ## @@ -0,0 +1,137 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import relay, tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@staticmethod +def _my_ai_hw_conv2d_pass(func, mod, ctx): +_found_blocks = [] +_loops = dict() +_handles = [] +_entry_node = None +_external_function_name = "my_ai_hw_conv2dnchw" +_tvm_block_match_name = "conv2d_nchw" + +def _has_block(name: str, func) -> bool: Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r930006872 ## python/tvm/relay/backend/contrib/uma/_template/passes.py: ## @@ -0,0 +1,137 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import relay, tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@staticmethod +def _my_ai_hw_conv2d_pass(func, mod, ctx): +_found_blocks = [] +_loops = dict() +_handles = [] +_entry_node = None +_external_function_name = "my_ai_hw_conv2dnchw" Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r93397 ## python/tvm/relay/backend/contrib/uma/_template/passes.py: ## @@ -0,0 +1,137 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Transform passes for the my_ai_hw accelerator""" + +import tvm +from tvm import relay, tir +from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block + + +@tvm.tir.transform.prim_func_pass(opt_level=2) +class MyAiHwConv2dPass: +def transform_function( +self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext +) -> tvm.tir.PrimFunc: +return self._my_ai_hw_conv2d_pass(func, mod, ctx) + +@staticmethod Review Comment: `self` is not used here. the Pycharm lint therefore detects a smell and proposes to make it a staticmethod. We could go both ways here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929993795
##
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include
+
+#ifdef __cplusplus
+extern "C"
+#endif
+int
+my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc,
int iw, int ih, int ic,
+int kh, int kw) {
+
+ int kw_low = kw / 2;
+ int kh_low = kh / 2;
+ int kw_high = iw + kw / 2;
+ int kh_high = ih + kh / 2;
+
+ int padded_iw = iw + 2 * kw_low;
+ int padded_ih = ih + 2 * kh_low;
+
+ float* pad_temp = (float*)malloc(
Review Comment:
Not really, this should imitate a driver/low-level interface to an
accelerator.
Therefore, C/C++-only is intended to demonstrate that this part in
user-specific.
We added some additional comment to clarify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r929982909 ## python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc: ## @@ -0,0 +1,76 @@ +/* +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +*/ +#include + +#ifdef __cplusplus +extern "C" +#endif +int Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929982499
##
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+"""UMA backend for the MyAiHw accelerator."""
+
+def __init__(self):
+super().__init__()
+
+###
+# Target configuration
+###
+self._register_target_attr("dimension")
+
+###
+# Relay to Relay function registration
+###
+self._register_pattern("conv2d", conv2d_pattern())
+
+###
+# Relay to TIR function registration
+###
+self._register_tir_pass(PassPhase.TIR_PHASE_0, MyAiHwConv2dPass())
Review Comment:
@areusch, we don't have strong opinion here.
If the way you propose is preferred by the others, too, then we can make
this change
@PaulPalomeroBernardo @cgerum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r929975111 ## python/tvm/relay/backend/contrib/uma/api/utils.py: ## @@ -0,0 +1,52 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Utility methods for the Universal Modular Accelerator Interface (UMA)""" + +from enum import Enum, auto +import uuid +import tvm.tir +from tvm.contrib import utils, clang + + +class PassPhase(Enum): +"""UMA pass phases.""" + +PRE_PARTITIONING = auto() +POST_PARTITIONING_0 = auto() Review Comment: done: see code ``` """ UMA pass phases: PRE_PARTITIONING: prior to UMA partitioning POST_PARTITIONING_0: after UMA partitioning, before Defunctionalization POST_PARTITIONING_1: after UMA partitioning and after Defunctionalization TIR_PHASE_0: Generates the raw IR and loop levels. TIR_PHASE_1: Flattens the array storage. TIR_PHASE_2: Transforms loops, like unroll, vectorization and thread-binding. TIR_PHASE_3: Does some cleanup work. Reference to TIR phases: src/driver/driver_api.c """ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929962126
##
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+"""
+Codegen base class of the Universal Modular Accelerator Interface (UMA)
+"""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+if fmt == "c":
+self._register_c_codegen(**kwargs)
+else:
+raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+def _register_c_codegen(
+self,
+includes: Callable[[], str] = None,
Review Comment:
To generate dynamic strings more easily we would be in favor of passing a
Callable
@PaulPalomeroBernardo @cgerum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r929946409 ## python/tvm/relay/backend/contrib/uma/_template/backend.py: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928586644
##
python/tvm/relay/backend/contrib/uma/backend.py:
##
@@ -0,0 +1,299 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Backend base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from abc import ABC, abstractmethod
+from typing import Union, Dict, Callable, Optional, Any
+
+import tvm
+from tvm.relay.backend.contrib.uma.api.codegen import UMACodegen
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.partitioner import UMAPartitioner
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+class UMABackend(ABC):
+"""Backend base class of the Universal Modular Accelerator Interface
(UMA)"""
+
+def __init__(self, merge_compiler_regions: bool = True) -> None:
+self._target_attrs: Dict = {}
+self._target_preprocessor: Callable[[str], Dict[str, Any]] = None
+self._relay_to_relay = UMAPartitioner(self.target_name,
merge_compiler_regions)
+self._relay_to_tir = UMALower(self.target_name)
+self._tir_to_runtime = UMACodegen(self.target_name)
+
+@property
+@abstractmethod
+def target_name(self) -> str:
+"""Name of the hardware target.
+
+Returns
+---
+out : str
+The hardware target name.
+"""
+...
+
+
+# Target configuration
+
+def _register_target_attr(
+self,
+name: str,
+default: Optional[Union[str, int, bool]] = "",
+) -> None:
+"""Register a target attribute name that can be used during target
instantiation.
+Parameters
+--
+name: str
+ The name of the target attribute.
+
+default: Optional[Union[str, int, bool]]
+A default value for the attribute.
+If none is provided, the attribute will be treated as a string.
+
+Example
+---
+Here is an example of how two attribute options are registered.
+
+.. code-block:: python
+
+self._register_target_attr("attrA", default=0)
+self._register_target_attr("attrB", default=False)
+"""
+self._target_attrs[name] = default
+
+
+# Relay to Relay function registration
+
+def _register_relay_pass(self, phase: PassPhase, relay_pass:
tvm.transform.Pass) -> None:
+"""Registers a relay pass at the given phase in the lowering process.
+
+Parameters
+--
+phase: PassPhase
+ The phase at which the pass is registered.
+
+relay_pass: tvm.transform.Pass
+The relay pass to be registered.
+
+Example
+---
+Here is an example of how two relay passes are registered.
+Passes of the same phase are executed in the order they are registered.
+
+.. code-block:: python
+
+self._register_relay_pass(PassPhase.PRE_PARTITIONING, MyPassA)
+self._register_relay_pass(PassPhase.POST_PARTITIONING, MyPassB)
+
+Where a relay pass can look like this:
+
+.. code-block:: python
+
+@tvm.ir.transform.module_pass(opt_level=0)
+class MyPassA:
+def transform_module(self, mod, ctx):
+# My pass functionality...
+return mod
+"""
+self._relay_to_relay._relay_passes.append((phase, relay_pass))
+
+def _register_pattern(
+self,
+name: str,
+pattern: tvm.relay.dataflow_pattern.DFPattern,
+predicate: Optional[Callable] = None,
+) -> None:
+"""Registers a dataflow pattern that is used to partition the relay
graph.
+
+Parameters
+--
+name: str
+ The name of the pattern.
+
+pattern: tvm.relay.d
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r928579415 ## python/tvm/relay/backend/contrib/uma/_template/backend.py: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""UMA backend for the my_ai_hw accelerator""" +from .passes import MyAiHwConv2dPass +from ..api.utils import PassPhase +from ..backend import UMABackend +from .codegen import gen_includes, gen_replace_call_extern +from .patterns import conv2d_pattern + + +class MyAiHwBackend(UMABackend): +"""UMA backend for the MyAiHw accelerator.""" + +def __init__(self): +super().__init__() + +### Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r928576011 ## cmake/config.cmake: ## @@ -296,9 +296,6 @@ set(USE_VTA_FPGA OFF) # Whether use Thrust set(USE_THRUST OFF) -# Whether use cuRAND Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928574359
##
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+"""
+Codegen base class of the Universal Modular Accelerator Interface (UMA)
+"""
+
+def __init__(self, target_name: str) -> None:
+self.target_name = target_name
+
+def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+if fmt == "c":
+self._register_c_codegen(**kwargs)
+else:
+raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+def _register_c_codegen(
+self,
+includes: Callable[[], str] = None,
Review Comment:
Doc-string added.
@cgerum : can you answer the other question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r926637840 ## tests/python/contrib/test_uma/test_partition.py: ## @@ -0,0 +1,71 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import pytest + Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r926627576 ## python/tvm/relay/backend/contrib/uma/uma_cli.py: ## @@ -0,0 +1,92 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +UMA Command Line Interface (CLI) Review Comment: Just talked to @cgerum. We would tend to move it to apps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r926342146 ## python/tvm/relay/backend/contrib/uma/uma_cli.py: ## @@ -0,0 +1,92 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +UMA Command Line Interface (CLI) Review Comment: @manupa-arm, I agree in general. IMHO uma_cli is more a helper for the tutorial in its current form, therefore I think it could be of use for early users of UMA. A deeper embedding into TVM, e.g. via tvmc is definitely a further discussion that we need to have, e.g. on the RFC level -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r925598695 ## python/tvm/relay/backend/contrib/uma/tutorial.md: ## @@ -0,0 +1,195 @@ + + + + + + + + + + + + + + + + + +Making your hardware accelerator TVM-ready with UMA += + +**Disclaimer**: *This is an early preliminary version of this tutorial. Feel free to aks questions or give feedback via the UMA thread in the TVM +discussion forum [[link](https://discuss.tvm.apache.org/t/rfc-uma-universal-modular-accelerator-interface/12039)].* + + +This tutorial will give you step-by-step guidance how to use UMA to +make your hardware accelerator TVM-ready. +While there is no one-fits-all solution for this problem, UMA targets to provide a stable and Python-only +API to integrate a number of hardware accelerator classes into TVM. + +In this tutorial you will get to know the UMA API in three use cases of increasing complexity. +In these use case the three mock-accelerators +**Vanilla**, **Strawberry** and **Chocolate** are introduced and +integrated into TVM using UMA. + + +Vanilla +=== +**Vanilla** is a simple accelerator consisting of a MAC array and has no internal memory. +It is can ONLY process Conv2D layers, all other layers are executed on a CPU, that also orchestrates **Vanilla**. +Both the CPU and Vanilla use a shared memory. + +For this purpose **Vanilla** has a C interface `vanilla_conv2dnchw`, that accepts pointers to input data *if_map*, +*weights* and *result* data, as well as the parameters of `Conv2D`: `oc`, `iw`, `ih`, `ic`, `kh`, `kw`. +```c +int vanilla_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic, int kh, int kw); +``` + +The script `uma_cli` creates code skeletons with API-calls into the UMA-API for new accelerators. +For **Vanilla** we use it like this: + +``` +cd tvm/python/tvm/relay/backend/contrib/uma +python uma_cli.py --add-accelerator vanilla_accelerator --tutorial vanilla Review Comment: @kslavka , nice catch! Thanks. Fixed it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r925592054 ## python/tvm/relay/backend/contrib/uma/tutorial.md: ## @@ -0,0 +1,195 @@ + + + + + + + + + + + + + + + + + +Making your hardware accelerator TVM-ready with UMA Review Comment: Generally yes. Would be great! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r925590913 ## include/tvm/relay/transform.h: ## @@ -509,6 +509,8 @@ TVM_DLL Pass SimplifyExpr(); * * \param config All available targets. * + * \param config All available targets. Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925589594
##
tests/python/contrib/test_uma/test_uma_lowering_with_umalower.py:
##
@@ -0,0 +1,115 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import pytest
+import pathlib
+
+import tvm
+from tests.python.contrib.test_uma.test_uma_utils import _create_schedule,
_generate_io_arrays
+from tvm import topi
+from tvm.relay.backend.contrib.uma._template.passes import MyAiHwConv2dPass
+import tvm.testing
+from tvm import te
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+def _conv2d_te_definition(shapes: dict) -> list:
+n, w, h, ci, kw, kh, co = (
+shapes["n"],
+shapes["w"],
+shapes["h"],
+shapes["ci"],
+shapes["kw"],
+shapes["kh"],
+shapes["co"],
+)
+ifmap = te.placeholder((n, ci, w, h), dtype="float32", name="ifmap")
+weights = te.placeholder((co, ci, kw, kh), dtype="float32", name="weights")
+result = topi.nn.conv2d_nchw(ifmap, weights, stride=1, padding=[kw // 2,
kh // 2], dilation=1)
+return [ifmap, weights, result]
+
+
+def _pepare_conv2d_schedule(shapes, use_external_conv2d_impl=True):
+placeholders = _conv2d_te_definition(shapes)
+
+uma_path =
pathlib.Path(str(tvm.relay.backend.contrib.uma.__file__)).parent.absolute()
+conv2d_file = uma_path / "_template" / "conv2dnchw.cc"
+
+with conv2d_file.open() as f:
+sch_tir = _create_schedule(
+placeholders, f, use_external_conv2d_impl=use_external_conv2d_impl
+)
+return placeholders, sch_tir
+
+
+def _run_external_conv2d(dut_io_arrays, conv2d_shapes, target):
+# Run conv2d with external function
+placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+
+uma_lower = UMALower("lower_test")
+uma_lower._tir_passes.append((PassPhase.TIR_PHASE_0, MyAiHwConv2dPass()))
+with tvm.transform.PassContext():
+tir_mod = uma_lower._lower_stir_to_nstir(schedule.mod["main"])
+
+ifmap_data, weight_data, result_data = dut_io_arrays
+
+llvm_conv2d_mod = tvm.build(tir_mod, placeholders, target=target,
name="test_external_conv2d")
+llvm_conv2d_mod(ifmap_data, weight_data, result_data)
+
+
+def _run_reference_conv2d(reference_io_arrays, conv2d_shapes, target):
+placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+ref_mod = tvm.build(schedule.mod, placeholders, target=target,
name="test_reference_conv2d")
+ifmap, weights, result = reference_io_arrays
+ref_mod(ifmap, weights, result)
+
+
+def _prepare_io_arrays(conv2d_shapes, dev):
+dut_io_arrays = _generate_io_arrays(conv2d_shapes, dev)
+_, _, ref_result = _generate_io_arrays(conv2d_shapes, dev)
+reference_io_arrays = [dut_io_arrays[0], dut_io_arrays[1], ref_result]
+return dut_io_arrays, reference_io_arrays
+
+
+@pytest.mark.parametrize(
+"n, w, h, ci, kw, kh, co",
+[
+(1, 224, 224, 3, 3, 3, 4),
+(1, 224, 224, 3, 5, 5, 4),
+(1, 224, 224, 3, 7, 7, 4),
+(1, 224, 320, 3, 7, 7, 4),
+(1, 224, 224, 3, 7, 7, 4),
+],
+)
+def test_lower_with_uma(n, w, h, ci, kw, kh, co):
+target = tvm.target.Target(target="llvm", host="llvm")
+dev = tvm.device(target.kind.name, 0)
+conv2d_shapes = dict(n=n, w=w, h=h, ci=ci, kw=kw, kh=kh, co=co)
+
+dut_io_arrays, reference_io_arrays = _prepare_io_arrays(conv2d_shapes, dev)
+
+_run_external_conv2d(dut_io_arrays, conv2d_shapes, target)
+_run_reference_conv2d(reference_io_arrays, conv2d_shapes, target)
+
+# compare results
+dut_results = dut_io_arrays[2].numpy()
+ref_results = reference_io_arrays[2].numpy()
+tvm.testing.assert_allclose(dut_results, ref_results, rtol=1e-5)
+
+
+if __name__ == "__main__":
+test_lower_with_uma(1, 224, 224, 3, 3, 3, 4)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infras
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925574906
##
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+auto includes_pf =
+tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" +
target_str_);
+ICHECK(includes_pf);
+String includes = (*includes_pf)();
+decl_stream << includes;
+std::unordered_set devices;
+devices.insert(target_str_);
+CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) {
CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+}
+auto replace_call_extern_pf =
+
tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_replace_call_extern_" +
target_str_);
+if (replace_call_extern_pf == nullptr) {
+ CodeGenCHost::VisitExpr_(op, os);
+} else {
+ // - funtion type (void) still gets printed before CallNode if extern
call is wrapped in
+ // EvaluateNode
+ // - VarNode arguments might have "wrong" name_hints. The correct
variable name is determined
+ // in C++ through GetVarID
+ String api_string = (*replace_call_extern_pf)(op->args);
+ os << api_string;
+}
+return;
+ }
+};
+
+runtime::Module TIRToRuntime(IRModule mod, Target target) {
+ bool output_ssa = false;
+ bool emit_asserts = false;
+ UMACodegen codegen(target->kind->name);
+ Array function_names;
+ codegen.Init(output_ssa, emit_asserts);
+ for (auto kv : mod->functions) {
+auto prim_func = Downcast(kv.second);
+auto global_symbol = prim_func->GetAttr(tvm::attr::kGlobalSymbol);
+function_names.push_back(global_symbol.value());
+codegen.AddFunction(prim_func);
+ }
+ std::string code = codegen.Finish();
+ return codegen::CSourceModuleCreate(code, "c", function_names);
Review Comment:
I see that UMA can provide this sort of infrastructure e.g. by doing this on
TIR level.
An accelerator-specific TIR pass would split a function into DMA calls for
transporting data and accelerator calls for the actual computation. In the
Strawberry example we plan to demonstrate this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r925569714 ## python/tvm/relay/backend/contrib/uma/_template/backend.py: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: It is more intended to be a directory for "templates" / skeleton code to be customizes with uma_cli.py for new accelerator targets -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925568107
##
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+"""UMA backend for the MyAiHw accelerator."""
+
+def __init__(self):
+super().__init__()
+
+###
+# Target configuration
+###
+self._register_target_attr("dimension")
+
+###
+# Relay to Relay function registration
Review Comment:
I agree. Comment will be adapted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r923152921 ## CMakeLists.txt: ## @@ -494,6 +494,7 @@ include(cmake/modules/contrib/ArmComputeLib.cmake) include(cmake/modules/contrib/TensorRT.cmake) include(cmake/modules/contrib/VitisAI.cmake) include(cmake/modules/contrib/Verilator.cmake) +include(cmake/modules/contrib/UMA.cmake) Review Comment: CC @manupa-arm @areusch @sunggg @Mousius @lhutton1 @cgerum @PaulPalomeroBernardo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
MichaelJKlaiber commented on code in PR #12087: URL: https://github.com/apache/tvm/pull/12087#discussion_r923152921 ## CMakeLists.txt: ## @@ -494,6 +494,7 @@ include(cmake/modules/contrib/ArmComputeLib.cmake) include(cmake/modules/contrib/TensorRT.cmake) include(cmake/modules/contrib/VitisAI.cmake) include(cmake/modules/contrib/Verilator.cmake) +include(cmake/modules/contrib/UMA.cmake) Review Comment: CC @manupa-arm @areusch @sunggg @Mousius @lhutton1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
