[PR] [Web] Add `kv_state` and `rnn_state` to wasm_runtime [tvm]
Hzfengsy opened a new pull request, #16791: URL: https://github.com/apache/tvm/pull/16791 Fix the outdated `wasm_runtime` to include the `kv_state` and `rnn_state` cc @CharlieFRuan @tqchen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch nightly updated (cb08f0d57b -> b2204ae698)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly in repository https://gitbox.apache.org/repos/asf/tvm.git from cb08f0d57b [TIR][Driver] Use `BindTarget` to specify target for FP8 legalization (#16767) add 9899f9cd28 [AOT][Testing] Improve output mismatch information on test failure (#16765) add b3981d2f77 [Fix] Fix numpy dtype map (#16780) add 5a8d928b84 [Relax] Improve malform error msg (#16779) add ef46f4e8d3 Revert "[SLM] Allow modules to define pre-processing of weights" (#16777) add b2204ae698 [IR] Default to empty attributes, instead of NULL (#16745) No new revisions were added by this update. Summary of changes: include/tvm/ir/attrs.h | 7 +- include/tvm/ir/module.h| 3 +- include/tvm/relax/expr.h | 5 +- include/tvm/relay/function.h | 2 +- include/tvm/runtime/object.h | 14 +- include/tvm/script/ir_builder/tir/frame.h | 2 +- include/tvm/tir/function.h | 2 +- python/tvm/_ffi/runtime_ctypes.py | 3 +- python/tvm/contrib/cutlass/build.py| 2 +- python/tvm/contrib/relay_viz/interface.py | 23 +- python/tvm/dlight/base/transform.py| 2 - python/tvm/dlight/gpu/matmul.py| 4 +- python/tvm/driver/build_module.py | 2 +- python/tvm/ir/attrs.py | 4 + python/tvm/meta_schedule/relax_integration.py | 2 +- python/tvm/relax/backend/contrib/cutlass.py| 4 +- python/tvm/relax/frontend/common.py| 2 +- python/tvm/relax/frontend/nn/core.py | 17 +- python/tvm/relax/frontend/nn/exporter.py | 40 +- python/tvm/relax/training/setup_trainer.py | 12 +- .../tvm/relax/transform/lazy_transform_params.py | 4 +- python/tvm/relay/backend/contrib/ethosu/util.py| 2 +- python/tvm/relay/function.py | 3 + .../tvm/relay/quantize/_partition_conversions.py | 4 +- python/tvm/relay/testing/py_converter.py | 6 +- python/tvm/testing/aot.py | 54 ++- python/tvm/tir/function.py | 3 + src/relax/analysis/well_formed.cc | 12 +- src/relay/analysis/type_solver.cc | 2 +- src/relay/backend/vm/lambda_lift.cc| 2 +- src/relay/ir/dataflow_matcher.cc | 2 +- src/relay/ir/function.cc | 4 +- src/relay/transforms/dynamic_to_static.cc | 3 +- src/relay/transforms/to_cps.cc | 4 +- src/script/ir_builder/ir/frame.cc | 2 +- src/script/ir_builder/relax/ir.cc | 21 +- src/script/ir_builder/tir/frame.cc | 16 +- src/script/ir_builder/tir/ir.cc| 26 +- tests/python/contrib/test_coreml_codegen.py| 2 +- .../test_meta_schedule_cpu_dot_product.py | 2 +- tests/python/relax/test_codegen_cutlass.py | 6 +- tests/python/relax/test_frontend_nn_exporter.py| 443 - .../python/relax/test_frontend_nn_extern_module.py | 10 +- tests/python/relax/test_frontend_nn_modules.py | 3 +- tests/python/relax/test_frontend_nn_op.py | 27 +- tests/python/relax/test_frontend_nn_packing.py | 3 +- tests/python/relax/test_frontend_nn_subroutines.py | 13 +- tests/python/relay/aot/test_aot_test_harness.py| 52 ++- tests/python/tir-base/test_tir_nodes.py| 2 +- .../tir-transform/test_tir_transform_helpers.py| 20 +- 50 files changed, 278 insertions(+), 627 deletions(-) delete mode 100644 tests/python/relax/test_frontend_nn_exporter.py
Re: [PR] [BugFix] Fix Crash Cases Caused by "__tvm_meta__ = None" [tvm]
ysh329 commented on PR #16634: URL: https://github.com/apache/tvm/pull/16634#issuecomment-2019299851 > #16770 The sage continues... Hi Steven, I'm little confused. Does these three cases been fixed? I tried main with commit: 240326, b2204ae6 and update status as below. It seems still two cases failed: - **still failed**: tests/python/tir-transform/test_tir_transform_force_narrow_index_to_i32.py::test_thread_axis2 - **passed**: tests/python/tir-transform/test_tir_transform_hoist_if.py::test_hoisting_block_scope_4 - **still failed**: tests/python/tir-transform/test_transform_default_gpu_schedule.py::test_add_on_metal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [SLM] Add unit tests for SLM to Relax exporter [tvm]
slyubomirsky commented on code in PR #16784:
URL: https://github.com/apache/tvm/pull/16784#discussion_r1538501095
##
tests/python/relax/test_frontend_nn_exporter.py:
##
@@ -0,0 +1,632 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+import tvm.testing
+
+from tvm import relax, tir
+from tvm.ir import assert_structural_equal
+from tvm.relax.frontend import nn
+from tvm.script import ir as I, relax as R, tir as T
+
+
+def test_simple():
+"""A module may be exported from nn.Module to Relax"""
+
+slm_mod = nn.modules.ReLU()
+exported_mod, _ = slm_mod.export_tvm(
+spec={"forward": {"x": nn.spec.Tensor((3, 3), "float32")}},
+debug=False,
+)
+
+@I.ir_module
+class Expected:
+@R.function
+def forward(x: R.Tensor([3, 3], dtype="float32")):
+R.func_attr({"num_input": 1})
+with R.dataflow():
+relu = R.nn.relu(x)
+relu = relu
+R.output(relu)
+return relu
+
+assert_structural_equal(exported_mod, Expected)
+
+
+def test_custom_module():
+"""A module may be exported from nn.Module to Relax"""
Review Comment:
Perhaps the comment should differ in soem way from that in the above test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [SLM] Allow modules to define pre-processing of weights [tvm]
slyubomirsky commented on code in PR #16785:
URL: https://github.com/apache/tvm/pull/16785#discussion_r1538475836
##
python/tvm/relax/frontend/nn/exporter.py:
##
@@ -190,34 +207,64 @@ def _convert_input(arg):
def _params(mode: str) -> typing.List[rx.Var]:
inputs: typing.List[rx.Var] = []
-def _get_var(shape_var: tir.Var) -> tir.Var:
-name = shape_var.name
-if name in str2var_params:
-return str2var_params[name]
-var = tir.Var(name, "int64")
-str2var_params[name] = var
-return var
+def _normalize_dim(dim: typing.Union[int, str, tir.Var]) ->
tir.PrimExpr:
+if isinstance(dim, int):
+return tir.IntImm("int64", dim)
+elif isinstance(dim, str):
+if dim in str2var_params:
+return str2var_params[dim]
+else:
+new_var = tir.Var(dim, "int64")
+str2var_params[dim] = new_var
+return new_var
+elif isinstance(dim, tir.Var):
+return dim
+else:
+raise TypeError(
+f"Expected dim to be int, str, or tir.Var, "
+f"but {dim} was of type {type(dim)}."
+)
for name, param in params:
# Make sure the a symbolic shape is not re-registered (same as
_method_spec_to_inputs)
# e.g. we do not see `vocab_size` for `lm_head` and `vocab_size_1`
for `embed_tokens`
-new_shape = [_get_var(x) if isinstance(x, tir.Var) else x for x in
param.shape]
-var = core.Tensor.placeholder(new_shape, param.dtype, name)._expr
+new_shape = [_normalize_dim(dim) for dim in param._shape]
+# var_cls = rx.DataflowVar if mode == "packed" else rx.Var
Review Comment:
Is this line meant to be used at any point?
##
python/tvm/relax/frontend/nn/op.py:
##
@@ -676,12 +676,31 @@ def permute_dims(x: Tensor, axes: Optional[List[int]] =
None, name: str = None)
result : Tensor
The transposed result.
"""
+
+# TODO(Lunderberg): This is a more extensive auto-naming than
+# intended here. Is this still worth it?
Review Comment:
Do we expect these chains of definitions to be deep? If they can be, this
might be undesirable.
##
tests/python/relax/test_frontend_nn_packing.py:
##
@@ -25,7 +25,9 @@ def _iter_binding_names(mod):
"""Helper function to compare the names of relax variables"""
for block in mod["forward"].body.blocks:
for binding in block.bindings:
-yield binding.var.name_hint
+# Relax variable names may contain '.' even though it
+# cannot be expressed in TVMScript.
Review Comment:
I wonder if this is something we should just check for and prohibit.
##
python/tvm/relax/frontend/nn/exporter.py:
##
@@ -135,9 +136,18 @@ def _effects() -> typing.List[typing.Tuple[str,
core.Effect]]:
with self.builder.dataflow():
outputs, inputs = _emit_method(self.builder,
method_spec, params, effects)
self.builder.emit_func_output(outputs, inputs)
+
+# TODO(Lunderberg): Make a `ir.transform.ConvertSSA`,
+# similar to the existing `tir.transform.ConvertSSA`,
+# that converts an entire module to SSA, including TIR
+# variable definitions used in either TIR or Relax.
Review Comment:
What would this conversion do on the Relax side? I thought vars already had
exactly one point of definition in Relax.
##
python/tvm/relax/frontend/nn/core.py:
##
@@ -591,7 +609,22 @@ def wrap_nested(expr: rx.Expr, name: str) -> Union[Tensor,
Sequence[Tensor]]:
The computed result.
"""
if not isinstance(expr, rx.DataflowVar):
-expr = BlockBuilder.current().emit(expr, name)
+block_builder = BlockBuilder.current()
+if block_builder is None:
+# Normalize to make sure we have valid StructInfo, but
+# wait until we are actually building the function to
+# flatten nested expressions.
+#
+# TODO(Lunderberg): Make this easier to call. Infering
+# struct info for a nested expression should be doable in
+# a free function, without requiring an active
+# BlockBuilder and an active FunctionFrame.
Review Comment:
Yeah... we might be able to decouple some of the functionality and allow for
calling things like that separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at
[PR] [Fix] Fix build errors with VS2022 [tvm]
Jiawei-Shao opened a new pull request, #16790: URL: https://github.com/apache/tvm/pull/16790 This patch fixes all the build errors from VS2022. With this patch we can build tvm.dll successfully with VS2022. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [IR][Relax] Improve highlighting in assert_structural_equal [tvm]
slyubomirsky commented on code in PR #16756:
URL: https://github.com/apache/tvm/pull/16756#discussion_r1538419796
##
src/relax/ir/expr.cc:
##
@@ -384,6 +384,33 @@ TVM_REGISTER_GLOBAL("relax.MatchCast")
return MatchCast(var, value, struct_info, span);
});
+bool MatchCastNode::SEqualReduce(const MatchCastNode* other, SEqualReducer
equal) const {
+ if (value->IsInstance()) {
+// Recursive function definitions may reference the bound variable
+// within the value being bound. In these cases, the
+// `DefEqual(var, other->var)` must occur first, to ensure it is
+// defined at point of use.
+return equal.DefEqual(var, other->var) && equal.DefEqual(struct_info,
other->struct_info) &&
Review Comment:
Per the above discussion, this sounds correct and indeed, local functions
are the only time a local var can be used recursively in Relax.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [TIR] Fix segfaults from ordering of Let/Assert in MakePackedAPI [tvm]
slyubomirsky commented on code in PR #16543:
URL: https://github.com/apache/tvm/pull/16543#discussion_r1538390306
##
src/tir/transforms/arg_binder.cc:
##
@@ -186,18 +191,8 @@ void ArgBinder::BindDLTensor(const Buffer& buffer, const
PrimExpr& device_type,
if (!(buffer->dtype == DataType::Int(1) || buffer->dtype == DataType::Int(4)
||
buffer->dtype == DataType::UInt(4))) {
auto type_msg = tvm::tir::StringImm(type_err_msg.str());
-asserts_.emplace_back(AssertStmt(a_ndim == v_ndim, msg, nop));
Review Comment:
Was this just a duplicate?
##
rust/tvm-graph-rt/tests/test_tvm_basic/build.rs:
##
@@ -48,10 +48,6 @@ fn main() -> Result<()> {
obj_file.exists(),
"Could not build tvm lib: {}",
String::from_utf8(output.stderr)?
-.trim()
-.split("\n")
-.last()
-.unwrap_or("")
Review Comment:
Wonder how this came up. Just for readability?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [Bugfix][Cutlass] Remove a typo in cutlass build [tvm]
Lunderberg opened a new pull request, #16789: URL: https://github.com/apache/tvm/pull/16789 Introduced in https://github.com/apache/tvm/pull/16745, should be the string `"Composite"`, not the bytes `b"Composite"`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [TIR] LowerTVMBuiltin may use device_type from PrimFunc annotation [tvm]
slyubomirsky commented on code in PR #16727:
URL: https://github.com/apache/tvm/pull/16727#discussion_r1538357879
##
tests/python/tir-transform/test_tir_transform_lower_tvm_builtin.py:
##
@@ -260,11 +260,13 @@ def expected():
class TestLowerAllocateRequiresDeviceID(tvm.testing.CompareBeforeAfter):
+"""If device id is missing, error."""
+
transform = tvm.tir.transform.LowerTVMBuiltin()
def before():
T.func_attr({"target": T.target("llvm")})
-T.attr("dummy", "device_id", 0)
+T.attr("dummy", "device_type", 2) # kDLCuda
Review Comment:
Is this constant defined anywhere on the Python side? Perhaps it should be,
to avoid such magic numbers (if we expect that to happen again).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Relax] Allow composition of DFPattern replacements [tvm]
slyubomirsky commented on code in PR #16732:
URL: https://github.com/apache/tvm/pull/16732#discussion_r1538340938
##
src/relax/ir/dataflow_matcher.cc:
##
@@ -1140,34 +1071,173 @@ class PatternRewriter : ExprMutator {
return block;
}
- /*! \brief The pattern for rewriting call nodes */
- Optional pattern_;
/*! \brief The pattern constraint contexts for rewriting dataflow blocks */
- Optional ctx_;
+ PatternContext ctx_;
/*!
* \brief The user-provided rewriter function. Its signature and semantics
are:
- * - (Call, Map) -> Call for call node rewriting. Given the
matched
- *call node and the map of patterns and matched expressions, it should
return a new call node
- *to replace the original one or the original matched call node as is.
- * - (Map, Map) -> Map for dataflow
block rewriting.
- *Given the map of patterns and corresponding variables (bound variables
or parameters),
- *it should return a map that specifies new values for matched bound
variables. It can refer
+ *
+ * - (Map, Map) -> Map
+ *
+ *Given the map of patterns and corresponding variables (bound
+ *variables or parameters), it should return a map that
+ *specifies new values for matched bound variables. It can refer
*to the passed bindings to create the replacement expressions.
*/
- PackedFunc rewriter_func_;
- std::unordered_set params_;
+ TypedPackedFunc(Map, Map)>
rewriter_func_;
+};
+
+/*!
+ * \brief Apply pattern matching to each expression, replacing
+ * matches with the output of a user-provided rewriter function.
+ */
+class ExprPatternRewriter : ExprMutator {
+ public:
+ using ExprMutator::VisitBindingBlock_;
+ using ExprMutator::VisitExpr_;
+
+ ExprPatternRewriter(DFPattern pat,
+ TypedPackedFunc)>
rewriter_func)
+ : pattern_(pat), rewriter_func_(rewriter_func) {}
+
+ template
+ static Function Run(PatternType pat,
+ TypedPackedFunc)>
rewriter_func,
+ Function func) {
+ExprPatternRewriter rewriter(pat, rewriter_func);
+func = Downcast(rewriter(func));
+func = Downcast(RemoveAllUnused(func));
+return func;
+ }
+
+ Expr VisitExpr_(const SeqExprNode* seq) override {
+auto cache = bindings_;
+SeqExpr prev = GetRef(seq);
+
+StructuralEqual struct_equal;
+
+while (true) {
+ SeqExpr next =
Downcast(builder_->Normalize(ExprMutator::VisitExpr_(prev.get(;
+ if (struct_equal(prev, next)) {
+return std::move(next);
+ }
+
+ // Canonicalization may result in two previously-different
+ // expressions being recognized as identical. Elimination of
+ // common subexpressions may result in trival var-to-var
+ // bindings that can be canonicalized. Therefore, iterate the
+ // simplification steps until converged.
+ while (true) {
+auto start_of_loop = next;
+next = Downcast(CanonicalizeBindings(next));
+next = Downcast(EliminateCommonSubexpr(next));
+next = Downcast(RemoveAllUnused(next));
+if (struct_equal(start_of_loop, next)) {
+ break;
+}
+ }
+
+ if (struct_equal(prev, next)) {
+return std::move(next);
+ }
+
+ // Reset all knowledge of bindings that were collected from
+ // this SeqExpr. The collected bindings are only after
+ // the point where they were collected, and we are repeating
+ // the mutation of this SeqExpr.
+ bindings_ = cache;
+ prev = next;
+}
+ }
+
+ void VisitBinding_(const VarBindingNode* binding) override {
+auto expr = VisitExpr(binding->value);
+bindings_.Set(binding->var, expr);
+ReEmitBinding(binding, expr);
+ }
+
+ Expr VisitExpr(const Expr& expr) override {
+auto node = ExprMutator::VisitExpr(expr);
+
+std::vector matches_top_level;
+if (auto rewritten = TryRewrite(node, pattern_, &matches_top_level)) {
+ return builder_->Normalize(rewritten.value());
+}
+
+return node;
+ }
+
+ private:
+ Optional TryRewrite(const Expr& expr, const DFPattern& pattern,
+std::vector* matches_top_level) {
+ICHECK(matches_top_level);
+
+// Special handling if the user-supplied pattern is a `OrPattern`.
+// While the `ExtractMatchedExpr` can handle match the
Review Comment:
Looks like a typo. I assume it's supposed to be "handle matching," correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [Cutlass] Add check for group gemm param shapes [tvm]
vinx13 opened a new pull request, #16788: URL: https://github.com/apache/tvm/pull/16788 cc @tqchen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [Codegen] Add check to disable invalid reinterpret [tvm]
vinx13 opened a new pull request, #16786: URL: https://github.com/apache/tvm/pull/16786 cc @tqchen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Target] Use LLVM target parser for determining Arm(R) A-Profile Architecture features [tvm]
cbalint13 commented on PR #16425: URL: https://github.com/apache/tvm/pull/16425#issuecomment-2019043694 > Thanks for the quick response @cbalint13! I didn't try with llvm18 yet, only llvm17. Calling GetAllLLVMCpuFeatures() and GetAllLLVMTargetArches() should reproduce it, but I'll be able to come up with a more concrete example tomorrow I'l test it for llvm18 too, just ping me if you have a concrete sample, until than I try invoking as you said (hope to catch it). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Target] Use LLVM target parser for determining Arm(R) A-Profile Architecture features [tvm]
lhutton1 commented on PR #16425: URL: https://github.com/apache/tvm/pull/16425#issuecomment-2019038835 Thanks for the quick response @cbalint13! I didn't try with llvm18 yet, only llvm17. Calling GetAllLLVMCpuFeatures() and GetAllLLVMTargetArches() should reproduce it, but I'll be able to come up with a more concrete example tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Target] Use LLVM target parser for determining Arm(R) A-Profile Architecture features [tvm]
cbalint13 commented on PR #16425:
URL: https://github.com/apache/tvm/pull/16425#issuecomment-2019031916
> It seems CI was failing due to a memory leak observed when calling
`GetAllLLVMCpuFeatures()` and `GetAllLLVMTargetArches()`. The following is a
valgrind report for an executable that creates a new target `tvm::Target("llvm
-mtriple=aarch64-linux-gnu")` with this PR's changes:
>
> ```
> ...
> ==356347== 8,040 (1,560 direct, 6,480 indirect) bytes in 1 blocks are
definitely lost in loss record 42,621 of 42,667
> ==356347==at 0x4849013: operator new(unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==356347==by 0x1136B1B9: ??? (in
/usr/lib/x86_64-linux-gnu/libLLVM-17.so.1)
> ==356347==by 0xBC44347:
llvm::Target::createTargetMachine(llvm::StringRef, llvm::StringRef,
llvm::StringRef, llvm::TargetOptions const&, std::optional,
std::optional, llvm::CodeGenOpt::Level, bool) const
(TargetRegistry.h:488)
> ==356347==by 0xBC3EF2D:
tvm::codegen::CreateLLVMTargetMachine(llvm::Target const*,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, llvm::TargetOptions
const&, llvm::Reloc::Model const&, llvm::CodeModel::Model const&,
llvm::CodeGenOpt::Level const&) (llvm_instance.cc:398)
> ==356347==by 0xBC3F0B2:
tvm::codegen::GetLLVMSubtargetInfo(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&) (llvm_instance.cc:413)
> ==356347==by 0xBC41E09:
tvm::codegen::LLVMTargetInfo::GetAllLLVMTargetArches() (llvm_instance.cc:843)
> ==356347==by 0xBC3D5CA:
tvm::codegen::LLVMTargetInfo::LLVMTargetInfo(tvm::codegen::LLVMInstance&,
tvm::runtime::Map
const&) (llvm_instance.cc:220)
> ==356347==by 0xAD6FCE8:
tvm::target::parsers::aprofile::GetFeatures(tvm::runtime::Map) (aprofile.cc:101)
> ==356347==by 0xAD70A5E:
tvm::target::parsers::aprofile::ParseTarget(tvm::runtime::Map) (aprofile.cc:137)
> ==356347==by 0xAD71C25:
tvm::target::parsers::cpu::ParseTarget(tvm::runtime::Map) (cpu.cc:55)
> ==356347==by 0xAEAB127:
tvm::runtime::TypedPackedFunc (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*) const (packed_func.h:1826)
> ==356347==by 0xAEB3382:
tvm::runtime::PackedFuncObj::Extractor (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*,
tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) (packed_func.h:1252)
> ==356347==
> ==356347== 8,040 (1,560 direct, 6,480 indirect) bytes in 1 blocks are
definitely lost in loss record 42,622 of 42,667
> ==356347==at 0x4849013: operator new(unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==356347==by 0x1136B1B9: ??? (in
/usr/lib/x86_64-linux-gnu/libLLVM-17.so.1)
> ==356347==by 0xBC44347:
llvm::Target::createTargetMachine(llvm::StringRef, llvm::StringRef,
llvm::StringRef, llvm::TargetOptions const&, std::optional,
std::optional, llvm::CodeGenOpt::Level, bool) const
(TargetRegistry.h:488)
> ==356347==by 0xBC3EF2D:
tvm::codegen::CreateLLVMTargetMachine(llvm::Target const*,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, llvm::TargetOptions
const&, llvm::Reloc::Model const&, llvm::CodeModel::Model const&,
llvm::CodeGenOpt::Level const&) (llvm_instance.cc:398)
> ==356347==by 0xBC3F0B2:
tvm::codegen::GetLLVMSubtargetInfo(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&) (llvm_instance.cc:413)
> ==356347==by 0xBC4217B:
tvm::codegen::LLVMTargetInfo::GetAllLLVMCpuFeatures() (llvm_instance.cc:868)
> ==356347==by 0xAD6FD01:
tvm::target::parsers::aprofile::GetFeatures(tvm::runtime::Map) (aprofile.cc:102)
> ==356347==by 0xAD70A5E:
tvm::target::parsers::aprofile::ParseTarget(tvm::runtime::Map) (aprofile.cc:137)
> ==356347==by 0xAD71C25:
tvm::target::parsers::cpu::ParseTarget(tvm::runtime::Map) (cpu.cc:55)
> ==356347==by 0xAEAB127:
tvm::runtime::TypedPackedFunc (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*) const (packed_func.h:1826)
> ==356347==
Re: [PR] [Target] Use LLVM target parser for determining Arm(R) A-Profile Architecture features [tvm]
lhutton1 commented on PR #16425:
URL: https://github.com/apache/tvm/pull/16425#issuecomment-2019015265
It seems CI was failing due to a memory leak observed when calling
`GetAllLLVMCpuFeatures()` and `GetAllLLVMTargetArches()`. The following is a
valgrind report for an executable that creates a new target `tvm::Target("llvm
-mtriple=aarch64-linux-gnu")` with this PR's changes:
```
...
==356347== 8,040 (1,560 direct, 6,480 indirect) bytes in 1 blocks are
definitely lost in loss record 42,621 of 42,667
==356347==at 0x4849013: operator new(unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==356347==by 0x1136B1B9: ??? (in
/usr/lib/x86_64-linux-gnu/libLLVM-17.so.1)
==356347==by 0xBC44347:
llvm::Target::createTargetMachine(llvm::StringRef, llvm::StringRef,
llvm::StringRef, llvm::TargetOptions const&, std::optional,
std::optional, llvm::CodeGenOpt::Level, bool) const
(TargetRegistry.h:488)
==356347==by 0xBC3EF2D:
tvm::codegen::CreateLLVMTargetMachine(llvm::Target const*,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, llvm::TargetOptions
const&, llvm::Reloc::Model const&, llvm::CodeModel::Model const&,
llvm::CodeGenOpt::Level const&) (llvm_instance.cc:398)
==356347==by 0xBC3F0B2:
tvm::codegen::GetLLVMSubtargetInfo(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&) (llvm_instance.cc:413)
==356347==by 0xBC41E09:
tvm::codegen::LLVMTargetInfo::GetAllLLVMTargetArches() (llvm_instance.cc:843)
==356347==by 0xBC3D5CA:
tvm::codegen::LLVMTargetInfo::LLVMTargetInfo(tvm::codegen::LLVMInstance&,
tvm::runtime::Map
const&) (llvm_instance.cc:220)
==356347==by 0xAD6FCE8:
tvm::target::parsers::aprofile::GetFeatures(tvm::runtime::Map) (aprofile.cc:101)
==356347==by 0xAD70A5E:
tvm::target::parsers::aprofile::ParseTarget(tvm::runtime::Map) (aprofile.cc:137)
==356347==by 0xAD71C25:
tvm::target::parsers::cpu::ParseTarget(tvm::runtime::Map) (cpu.cc:55)
==356347==by 0xAEAB127:
tvm::runtime::TypedPackedFunc (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*) const (packed_func.h:1826)
==356347==by 0xAEB3382:
tvm::runtime::PackedFuncObj::Extractor (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*,
tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) (packed_func.h:1252)
==356347==
==356347== 8,040 (1,560 direct, 6,480 indirect) bytes in 1 blocks are
definitely lost in loss record 42,622 of 42,667
==356347==at 0x4849013: operator new(unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==356347==by 0x1136B1B9: ??? (in
/usr/lib/x86_64-linux-gnu/libLLVM-17.so.1)
==356347==by 0xBC44347:
llvm::Target::createTargetMachine(llvm::StringRef, llvm::StringRef,
llvm::StringRef, llvm::TargetOptions const&, std::optional,
std::optional, llvm::CodeGenOpt::Level, bool) const
(TargetRegistry.h:488)
==356347==by 0xBC3EF2D:
tvm::codegen::CreateLLVMTargetMachine(llvm::Target const*,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, llvm::TargetOptions
const&, llvm::Reloc::Model const&, llvm::CodeModel::Model const&,
llvm::CodeGenOpt::Level const&) (llvm_instance.cc:398)
==356347==by 0xBC3F0B2:
tvm::codegen::GetLLVMSubtargetInfo(std::__cxx11::basic_string, std::allocator > const&,
std::__cxx11::basic_string, std::allocator >
const&, std::__cxx11::basic_string,
std::allocator > const&) (llvm_instance.cc:413)
==356347==by 0xBC4217B:
tvm::codegen::LLVMTargetInfo::GetAllLLVMCpuFeatures() (llvm_instance.cc:868)
==356347==by 0xAD6FD01:
tvm::target::parsers::aprofile::GetFeatures(tvm::runtime::Map) (aprofile.cc:102)
==356347==by 0xAD70A5E:
tvm::target::parsers::aprofile::ParseTarget(tvm::runtime::Map) (aprofile.cc:137)
==356347==by 0xAD71C25:
tvm::target::parsers::cpu::ParseTarget(tvm::runtime::Map) (cpu.cc:55)
==356347==by 0xAEAB127:
tvm::runtime::TypedPackedFunc (tvm::runtime::Map)>::AssignTypedLambda
(*)(tvm::runtime::Map)>(tvm::runtime::Map (*)(tvm::runtime::Map))::{lambda(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&,
tvm::runtime::TVMRetValue*) const (packed_func.h:1826)
==356347==by 0xAEB3382:
tvm::runtime::PackedFuncObj::Extractor (tvm::run
Re: [PR] [Relax] Improve CanonicalizeBindings in DataflowVar edge case [tvm]
slyubomirsky commented on code in PR #16783:
URL: https://github.com/apache/tvm/pull/16783#discussion_r1538290737
##
src/relax/transform/canonicalize_bindings.cc:
##
@@ -91,18 +91,20 @@ class CanonicalizePlanner : public ExprVisitor {
bound_to = opt.value();
}
- if (bound_var.as() || !bound_to.as()) {
+ if (bound_var.as() || !bound_to.as() ||
+ !visitor.used_outside_home_dataflow_.count(bound_var)) {
// Case 1: Var = Var
// Case 2: DataflowVar = Var
// Case 3: DataflowVar = DataflowVar
+// Case 4a: Var = DataflowVar, but used outside this DataflowBlock
//
// For these three cases, the trivial binding can be
Review Comment:
```suggestion
// For these four cases, the trivial binding can be
```
:wink:
##
src/relax/transform/canonicalize_bindings.cc:
##
@@ -91,18 +91,20 @@ class CanonicalizePlanner : public ExprVisitor {
bound_to = opt.value();
}
- if (bound_var.as() || !bound_to.as()) {
+ if (bound_var.as() || !bound_to.as() ||
+ !visitor.used_outside_home_dataflow_.count(bound_var)) {
// Case 1: Var = Var
// Case 2: DataflowVar = Var
// Case 3: DataflowVar = DataflowVar
+// Case 4a: Var = DataflowVar, but used outside this DataflowBlock
Review Comment:
```suggestion
// Case 4a: Var = DataflowVar, but not used outside this
DataflowBlock
```
It appears the text was copied from case 4b. I also think it would be
clearer to write out that it is the var that is not used outside the DF block
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Relax] Improve CanonicalizeBindings in DataflowVar edge case [tvm]
slyubomirsky commented on code in PR #16783:
URL: https://github.com/apache/tvm/pull/16783#discussion_r1538290478
##
src/relax/transform/canonicalize_bindings.cc:
##
@@ -91,18 +91,20 @@ class CanonicalizePlanner : public ExprVisitor {
bound_to = opt.value();
}
- if (bound_var.as() || !bound_to.as()) {
+ if (bound_var.as() || !bound_to.as() ||
+ !visitor.used_outside_home_dataflow_.count(bound_var)) {
// Case 1: Var = Var
// Case 2: DataflowVar = Var
// Case 3: DataflowVar = DataflowVar
+// Case 4a: Var = DataflowVar, but used outside this DataflowBlock
//
// For these three cases, the trivial binding can be
Review Comment:
```suggestion
// For these four cases, the trivial binding can be
```
:wink:
##
src/relax/transform/canonicalize_bindings.cc:
##
@@ -91,18 +91,20 @@ class CanonicalizePlanner : public ExprVisitor {
bound_to = opt.value();
}
- if (bound_var.as() || !bound_to.as()) {
+ if (bound_var.as() || !bound_to.as() ||
+ !visitor.used_outside_home_dataflow_.count(bound_var)) {
// Case 1: Var = Var
// Case 2: DataflowVar = Var
// Case 3: DataflowVar = DataflowVar
+// Case 4a: Var = DataflowVar, but used outside this DataflowBlock
//
// For these three cases, the trivial binding can be
Review Comment:
```suggestion
// For these four cases, the trivial binding can be
```
:wink:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Unity] Check for transpose and dynamic shape in AdjustMatmulOrder [tvm]
Lunderberg commented on PR #16589: URL: https://github.com/apache/tvm/pull/16589#issuecomment-2018762304 Rebased on top of https://github.com/apache/tvm/pull/16735, all new unit tests passing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Restore "pytest.mark.gpu" for RELAX tests [tvm]
Lunderberg commented on code in PR #16741:
URL: https://github.com/apache/tvm/pull/16741#discussion_r1538074865
##
tests/python/relax/test_codegen_cudnn.py:
##
@@ -36,12 +36,8 @@ def reset_seed():
has_cudnn = tvm.get_global_func("relax.ext.cudnn", True)
-cudnn_enabled = pytest.mark.skipif(
-not has_cudnn,
-reason="cuDNN not enabled.",
-)
-pytestmark = [cudnn_enabled]
+pytestmark = [*tvm.testing.requires_cudnn.marks()]
Review Comment:
Nitpick: The `tvm.testing.requires_*` marks can be used as marks directly,
rather than needing to be explicitly expanded with `foo.marks()`. In addition,
the `pytestmark` can be a single marker, rather than a list. Between these
two, this can cleaned up as:
```python
pytestmark = tvm.testing.requires_cudnn
```
##
tests/python/relax/test_codegen_cudnn.py:
##
@@ -36,12 +36,8 @@ def reset_seed():
has_cudnn = tvm.get_global_func("relax.ext.cudnn", True)
Review Comment:
Do we still need `has_cudnn` now that it isn't used in `pytest.mark.skipif`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(tvm) branch main updated (ef46f4e8d3 -> b2204ae698)
This is an automated email from the ASF dual-hosted git repository. lunderberg pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/tvm.git from ef46f4e8d3 Revert "[SLM] Allow modules to define pre-processing of weights" (#16777) add b2204ae698 [IR] Default to empty attributes, instead of NULL (#16745) No new revisions were added by this update. Summary of changes: include/tvm/ir/attrs.h | 7 ++ include/tvm/ir/module.h| 3 ++- include/tvm/relax/expr.h | 5 ++--- include/tvm/relay/function.h | 2 +- include/tvm/runtime/object.h | 14 ++-- include/tvm/script/ir_builder/tir/frame.h | 2 +- include/tvm/tir/function.h | 2 +- python/tvm/contrib/cutlass/build.py| 2 +- python/tvm/contrib/relay_viz/interface.py | 23 +-- python/tvm/dlight/base/transform.py| 2 -- python/tvm/dlight/gpu/matmul.py| 4 ++-- python/tvm/driver/build_module.py | 2 +- python/tvm/ir/attrs.py | 4 python/tvm/meta_schedule/relax_integration.py | 2 +- python/tvm/relax/backend/contrib/cutlass.py| 4 ++-- python/tvm/relax/frontend/common.py| 2 +- python/tvm/relax/training/setup_trainer.py | 12 -- .../tvm/relax/transform/lazy_transform_params.py | 4 ++-- python/tvm/relay/backend/contrib/ethosu/util.py| 2 +- python/tvm/relay/function.py | 3 +++ .../tvm/relay/quantize/_partition_conversions.py | 4 ++-- python/tvm/relay/testing/py_converter.py | 6 - python/tvm/testing/aot.py | 6 ++--- python/tvm/tir/function.py | 3 +++ src/relay/analysis/type_solver.cc | 2 +- src/relay/backend/vm/lambda_lift.cc| 2 +- src/relay/ir/dataflow_matcher.cc | 2 +- src/relay/ir/function.cc | 4 +++- src/relay/transforms/dynamic_to_static.cc | 3 +-- src/relay/transforms/to_cps.cc | 4 ++-- src/script/ir_builder/ir/frame.cc | 2 +- src/script/ir_builder/relax/ir.cc | 21 +++-- src/script/ir_builder/tir/frame.cc | 16 +++-- src/script/ir_builder/tir/ir.cc| 26 ++ tests/python/contrib/test_coreml_codegen.py| 2 +- .../test_meta_schedule_cpu_dot_product.py | 2 +- tests/python/relax/test_codegen_cutlass.py | 6 ++--- tests/python/tir-base/test_tir_nodes.py| 2 +- .../tir-transform/test_tir_transform_helpers.py| 20 - 39 files changed, 127 insertions(+), 107 deletions(-)
Re: [PR] [IR] Default to empty attributes, instead of NULL [tvm]
Lunderberg merged PR #16745: URL: https://github.com/apache/tvm/pull/16745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [SLM] Allow modules to define pre-processing of weights [tvm]
Lunderberg commented on code in PR #16757:
URL: https://github.com/apache/tvm/pull/16757#discussion_r1538023044
##
python/tvm/relax/frontend/nn/exporter.py:
##
@@ -176,35 +183,26 @@ def _unwrap_ret(expr: typing.Any) -> typing.Any:
def _convert_input(arg):
if isinstance(arg, tir.Var):
return rx.Var(arg.name, struct_info=ShapeStructInfo(values=[arg]))
-if isinstance(arg, (core.Tensor, core.Object)):
+elif isinstance(arg, (core.Tensor, core.Object)):
return arg._expr # pylint: disable=protected-access
-if isinstance(arg, _spec.Tuple):
+elif isinstance(arg, _spec.Tuple):
return rx.Var(
arg.name,
struct_info=TupleStructInfo(
[_convert_input(arg_i).struct_info for arg_i in
arg.elements]
),
)
-raise TypeError(f"Unsupported input type: {type(arg)}")
+elif isinstance(arg, rx.Expr):
+return arg
+else:
+raise TypeError(f"Unsupported input type: {type(arg)}")
def _params(mode: str) -> typing.List[rx.Var]:
inputs: typing.List[rx.Var] = []
-def _get_var(shape_var: tir.Var) -> tir.Var:
-name = shape_var.name
-if name in str2var_params:
-return str2var_params[name]
Review Comment:
@MasterJH5574 Can you test against https://github.com/apache/tvm/pull/16785?
It re-implements the same functionality that I require on my side, while
maintaining the handling of dynamic shapes specified by strings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [SLM] Provide consistent dynamic shape when specified as python string [tvm]
Lunderberg closed pull request #16781: [SLM] Provide consistent dynamic shape when specified as python string URL: https://github.com/apache/tvm/pull/16781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [SLM] Provide consistent dynamic shape when specified as python string [tvm]
Lunderberg commented on PR #16781: URL: https://github.com/apache/tvm/pull/16781#issuecomment-2018588838 This PR is superseded by https://github.com/apache/tvm/pull/16785. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [SLM] Allow modules to define pre-processing of weights [tvm]
Lunderberg commented on PR #16785: URL: https://github.com/apache/tvm/pull/16785#issuecomment-2018587753 This PR is currently marked as a draft, as it depends on a bugfix applied in https://github.com/apache/tvm/pull/16783, and the unit tests added in https://github.com/apache/tvm/pull/16784. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [SLM] Allow modules to define pre-processing of weights [tvm]
Lunderberg opened a new pull request, #16785: URL: https://github.com/apache/tvm/pull/16785 Prior to this commit, the weights used by `nn.Module` instances were required to be `nn.Parameter` instances. This commit allows the weights to instead be `nn.Tensor` instances, defined in terms of other `nn.Parameter` weights. This allows a model to define both the original weights that would be present in an external checkpoint (e.g. a Pytorch or Safetensors file), and the pre-processing that should be performed on those weights. This is a re-implementation of https://github.com/apache/tvm/pull/16757, which was reverted in https://github.com/apache/tvm/pull/16777. The re-implementation preserves the handling of dynamic shapes specified as python strings, enabling the test cases that were added in https://github.com/apache/tvm/pull/16784. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Fix] Fix numpy dtype map [tvm]
MasterJH5574 commented on PR #16780: URL: https://github.com/apache/tvm/pull/16780#issuecomment-2018526099 @mshr-h Thank you for the great suggestion! I agree that the way I was doing might not be the best. Feel free to send the change you propose. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Web] Support web indexDB cache for larger model storage [tvm]
CharlieFRuan commented on code in PR #16733:
URL: https://github.com/apache/tvm/pull/16733#discussion_r1537927064
##
web/src/runtime.ts:
##
@@ -1052,6 +1062,191 @@ export class ArtifactCache implements
ArtifactCacheTemplate {
}
}
+/**
+ * Cache by IndexDB to support caching model data
+ */
+export class ArtifactIndexDBCache implements ArtifactCacheTemplate {
+ private dbName?: string;
+ private dbVersion = 1;
+ private db: IDBDatabase | undefined;
+
+ private async initDB() {
+if (this.db != null){
+ return; // the db is already inialized
+}
+return new Promise((resolve, reject) => {
+ const request = indexedDB.open(this.dbName, this.dbVersion);
+ request.onupgradeneeded = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+if (!this.db.objectStoreNames.contains('urls')) {
+ this.db.createObjectStore('urls', { keyPath: 'url' });
+}
+ };
+ request.onsuccess = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+resolve();
+ };
+ request.onerror = (event) => {
+console.error("Database error: ", (event.target as
IDBOpenDBRequest).error);
+reject((event.target as IDBOpenDBRequest).error);
+ };
+});
+ }
+
+ /* Check if the URL is in DB or not */
+ private async isUrlInDB(url: string): Promise {
+return new Promise((resolve, reject) => {
+ const transaction = this.db?.transaction(['urls'], 'readonly');
+ if (transaction === undefined){
+return false;
+ }
+ const store = transaction.objectStore('urls');
+ const request = store.get(url);
+ request.onsuccess = () => {
+resolve(request.result !== undefined);
+ };
+ request.onerror = (event) => {
+reject((event.target as IDBRequest).error);
+ };
+});
+ }
+
+ constructor(dbName: string){
+this.dbName = dbName;
+ }
+
+ async asyncGetHelper(url: string){
+return new Promise((resolve, reject) => {
+ let result: any;
+ const transaction = this.db?.transaction(['urls'], 'readonly');
+ if (transaction === undefined){
+return false;
+ }
+ transaction.oncomplete = () => resolve(result);
+ transaction.onerror = () => reject(transaction.error);
+ const objectStore = transaction.objectStore('urls');
+ const getRequest = objectStore.get(url);
+ getRequest.onsuccess = () => {
+result = getRequest.result;
+ }
+})
+ }
+
+ async fetchWithCache(url: string, storetype?: string) {
+await this.initDB(); // await the initDB process
+const isInDB = await this.isUrlInDB(url);
+if (!isInDB) {
+ const response = await this.addToCache(url, storetype);
+ return response;
+} else {
+ // URL found in DB, just fetch without storing
+ const result = await this.asyncGetHelper(url);
+ if (result != null && typeof result === "object" && "data" in result){
+return result.data;
+ } else if (result === null){
+// previously null data in cache!
+await this.deleteInCache(url);
+const response = await this.addToCache(url, storetype);
+return response;
+ }
+ return null;
+}
+ }
+
+ async addToIndexDB(url: string, response: any, storetype?: string){
+await this.initDB();
+let data: any;
+if (storetype != undefined){
+ if (storetype.toLowerCase() === "json"){
+data = await response.json();
+ } else if (storetype.toLocaleLowerCase() === "arraybuffer"){
+data = await response.arrayBuffer();
+ } else {
+console.error("Unsupported Type in IndexDB");
+ }
+}
+return new Promise((resolve, reject) => {
+ const transaction = this.db?.transaction(['urls'], 'readwrite');
+ if (transaction === undefined){
+return;
+ }
+ const store = transaction.objectStore('urls');
+ const request = store.add({data, url}); // Index DB follows a {value,
key} format, instead of {key, value} format!
+ request.onsuccess = () => resolve();
+ request.onerror = (event) => reject((event.target as IDBRequest).error);
+});
+ }
+
+ async addToCache(url: string, storetype?: string) :Promise{
+let response: Response;
+try {
+ response = await fetch(url);
+ if (!response.ok) {
+throw new Error('Network response was not ok');
+ }
+ const response_copy = response.clone();
+ await this.addToIndexDB(url, response_copy, storetype);
+ if (storetype.toLowerCase() === "arraybuffer"){
+return await response.arrayBuffer();
+ } else if (storetype.toLowerCase() == "json"){
+return await response.json();
+ } else {
+return response;
+ }
+} catch (error) {
+ console.error("There was a problem fetching the data:", error);
+}
+ }
+
+ async hasAllKeys(keys: string[]) :Promise {
Review Comment:
When testing end-to
Re: [PR] [Web] Support web indexDB cache for larger model storage [tvm]
CharlieFRuan commented on code in PR #16733:
URL: https://github.com/apache/tvm/pull/16733#discussion_r1537852720
##
web/src/index.ts:
##
@@ -22,7 +22,7 @@ export {
PackedFunc, Module, NDArray,
TVMArray, TVMObject, VirtualMachine,
InitProgressCallback, InitProgressReport,
- ArtifactCache, Instance, instantiate, hasNDArrayInCache, deleteNDArrayCache
+ ArtifactCache, ArtifactIndexDBCache, Instance, instantiate,
hasNDArrayInCache, deleteNDArrayCache
Review Comment:
I think the cache is called `indexedDB` instead of `indexDB`, let's change
all the namings.
##
web/src/runtime.ts:
##
@@ -1052,6 +1062,191 @@ export class ArtifactCache implements
ArtifactCacheTemplate {
}
}
+/**
+ * Cache by IndexDB to support caching model data
+ */
+export class ArtifactIndexDBCache implements ArtifactCacheTemplate {
+ private dbName?: string;
+ private dbVersion = 1;
+ private db: IDBDatabase | undefined;
+
+ private async initDB() {
+if (this.db != null){
+ return; // the db is already inialized
+}
+return new Promise((resolve, reject) => {
+ const request = indexedDB.open(this.dbName, this.dbVersion);
+ request.onupgradeneeded = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+if (!this.db.objectStoreNames.contains('urls')) {
+ this.db.createObjectStore('urls', { keyPath: 'url' });
+}
+ };
+ request.onsuccess = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+resolve();
+ };
+ request.onerror = (event) => {
+console.error("Database error: ", (event.target as
IDBOpenDBRequest).error);
+reject((event.target as IDBOpenDBRequest).error);
+ };
+});
+ }
+
+ /* Check if the URL is in DB or not */
+ private async isUrlInDB(url: string): Promise {
+return new Promise((resolve, reject) => {
+ const transaction = this.db?.transaction(['urls'], 'readonly');
+ if (transaction === undefined){
+return false;
+ }
+ const store = transaction.objectStore('urls');
+ const request = store.get(url);
+ request.onsuccess = () => {
+resolve(request.result !== undefined);
+ };
+ request.onerror = (event) => {
+reject((event.target as IDBRequest).error);
+ };
+});
+ }
+
+ constructor(dbName: string){
+this.dbName = dbName;
+ }
Review Comment:
Let's move the constructor to the top
##
web/src/runtime.ts:
##
@@ -1052,6 +1062,191 @@ export class ArtifactCache implements
ArtifactCacheTemplate {
}
}
+/**
+ * Cache by IndexDB to support caching model data
+ */
+export class ArtifactIndexDBCache implements ArtifactCacheTemplate {
+ private dbName?: string;
+ private dbVersion = 1;
+ private db: IDBDatabase | undefined;
+
+ private async initDB() {
+if (this.db != null){
+ return; // the db is already inialized
+}
+return new Promise((resolve, reject) => {
+ const request = indexedDB.open(this.dbName, this.dbVersion);
+ request.onupgradeneeded = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+if (!this.db.objectStoreNames.contains('urls')) {
+ this.db.createObjectStore('urls', { keyPath: 'url' });
+}
+ };
+ request.onsuccess = (event) => {
+this.db = (event.target as IDBOpenDBRequest).result;
+resolve();
+ };
+ request.onerror = (event) => {
+console.error("Database error: ", (event.target as
IDBOpenDBRequest).error);
+reject((event.target as IDBOpenDBRequest).error);
+ };
+});
+ }
+
+ /* Check if the URL is in DB or not */
+ private async isUrlInDB(url: string): Promise {
+return new Promise((resolve, reject) => {
+ const transaction = this.db?.transaction(['urls'], 'readonly');
+ if (transaction === undefined){
+return false;
+ }
+ const store = transaction.objectStore('urls');
+ const request = store.get(url);
+ request.onsuccess = () => {
+resolve(request.result !== undefined);
+ };
+ request.onerror = (event) => {
+reject((event.target as IDBRequest).error);
+ };
+});
+ }
+
+ constructor(dbName: string){
+this.dbName = dbName;
+ }
+
+ async asyncGetHelper(url: string){
+return new Promise((resolve, reject) => {
+ let result: any;
+ const transaction = this.db?.transaction(['urls'], 'readonly');
+ if (transaction === undefined){
+return false;
+ }
+ transaction.oncomplete = () => resolve(result);
+ transaction.onerror = () => reject(transaction.error);
+ const objectStore = transaction.objectStore('urls');
+ const getRequest = objectStore.get(url);
+ getRequest.onsuccess = () => {
+result = getRequest.result;
+ }
+})
+ }
+
+ async fetchWithCache(url: string, storetype?: string) {
+await this.initDB(); // await
Re: [PR] [SLM] Add unit tests for SLM to Relax exporter [tvm]
Lunderberg commented on PR #16784: URL: https://github.com/apache/tvm/pull/16784#issuecomment-2018450951 @tqchen @MasterJH5574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [Relax] Improve CanonicalizeBindings in DataflowVar edge case [tvm]
Lunderberg opened a new pull request, #16783: URL: https://github.com/apache/tvm/pull/16783 If there is a trivial binding of `Var = DataflowVar`, but the non-dataflow variable is never used outside the dataflow block in which is is declared, then we should keep the name of the upstream `DataflowVar`, as it is more likely to be the human-readable name (e.g. a function parameter). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Revert "[SLM] Allow modules to define pre-processing of weights" [tvm]
Lunderberg commented on PR #16777: URL: https://github.com/apache/tvm/pull/16777#issuecomment-2018422177 I have a parametrized version of the test case, along with testing for cases beyond `nn.Linear`, that I'll open as a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [SVE] Support scalable vectors in LoopVectorizer [tvm]
ekalda commented on PR #16782: URL: https://github.com/apache/tvm/pull/16782#issuecomment-2018282951 cc the usual suspects @lhutton1 @leandron @Anndrey24 @tqchen @Lunderberg -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [SVE] Support scalable vectors in LoopVectorizer [tvm]
ekalda opened a new pull request, #16782: URL: https://github.com/apache/tvm/pull/16782 This patch add support for turning loops marked for vectorizing into scalable vectors if the extent of the loop is a vscale dependent expression in a correct form. The testing for both scalable and fixed length vectors in test_tir_transform.py has been extended and most of the tests have been converted to TVMScript based testing against expected output. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Revert "[SLM] Allow modules to define pre-processing of weights" [tvm]
MasterJH5574 commented on PR #16777: URL: https://github.com/apache/tvm/pull/16777#issuecomment-2018232941 Merged without the unit test to keep the reversion standalone. Let's add the test case separately. Thank you folks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated: Revert "[SLM] Allow modules to define pre-processing of weights" (#16777)
This is an automated email from the ASF dual-hosted git repository.
ruihangl pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new ef46f4e8d3 Revert "[SLM] Allow modules to define pre-processing of
weights" (#16777)
ef46f4e8d3 is described below
commit ef46f4e8d33f1946dca9cd61f6db5eec79c7deab
Author: Tianqi Chen
AuthorDate: Mon Mar 25 11:10:19 2024 -0400
Revert "[SLM] Allow modules to define pre-processing of weights" (#16777)
Revert "[SLM] Allow modules to define pre-processing of weights (#16757)"
This reverts commit 13b5d65cae743a2becb7e256c05897af29ca.
---
python/tvm/relax/frontend/nn/core.py | 17 +-
python/tvm/relax/frontend/nn/exporter.py | 40 +-
tests/python/relax/test_frontend_nn_exporter.py| 443 -
.../python/relax/test_frontend_nn_extern_module.py | 10 +-
tests/python/relax/test_frontend_nn_modules.py | 3 +-
tests/python/relax/test_frontend_nn_op.py | 27 +-
tests/python/relax/test_frontend_nn_packing.py | 3 +-
tests/python/relax/test_frontend_nn_subroutines.py | 13 +-
8 files changed, 58 insertions(+), 498 deletions(-)
diff --git a/python/tvm/relax/frontend/nn/core.py
b/python/tvm/relax/frontend/nn/core.py
index 820acd235d..b7b3f411ed 100644
--- a/python/tvm/relax/frontend/nn/core.py
+++ b/python/tvm/relax/frontend/nn/core.py
@@ -591,22 +591,7 @@ def wrap_nested(expr: rx.Expr, name: str) -> Union[Tensor,
Sequence[Tensor]]:
The computed result.
"""
if not isinstance(expr, rx.DataflowVar):
-block_builder = BlockBuilder.current()
-if block_builder is None:
-# Normalize to make sure we have valid StructInfo, but
-# wait until we are actually building the function to
-# flatten nested expressions.
-#
-# TODO(Lunderberg): Make this easier to call. Infering
-# struct info for a nested expression should be doable in
-# a free function, without requiring an active
-# BlockBuilder and an active FunctionFrame.
-builder = BlockBuilder()
-with builder.function("dummy_scope", params=[]):
-expr = builder.normalize(expr)
-builder.emit_func_output([])
-else:
-expr = BlockBuilder.current().emit(expr, name)
+expr = BlockBuilder.current().emit(expr, name)
if isinstance(expr.struct_info_, TensorStructInfo):
return Tensor(_expr=expr)
if isinstance(expr.struct_info_, TupleStructInfo):
diff --git a/python/tvm/relax/frontend/nn/exporter.py
b/python/tvm/relax/frontend/nn/exporter.py
index 525d689f49..1a7dcd6a64 100644
--- a/python/tvm/relax/frontend/nn/exporter.py
+++ b/python/tvm/relax/frontend/nn/exporter.py
@@ -111,8 +111,7 @@ class Exporter:
return result
# pylint: enable=protected-access
-
-params = _params()
+params = None
effects = _effects()
ext_mods = self.extern_mods
with self:
@@ -122,6 +121,7 @@ class Exporter:
outputs = _emit_effect_init(self.builder, effects)
self.builder.emit_func_output(outputs, params=[])
for method_name, method_spec in zip(spec.method_names,
spec.method_specs):
+params = _params() # Re-initialize so symbolic shapes not
shared across methods
len_args = len(method_spec.arg_specs)
len_effects = {
"packed": 1,
@@ -135,18 +135,9 @@ class Exporter:
with self.builder.dataflow():
outputs, inputs = _emit_method(self.builder,
method_spec, params, effects)
self.builder.emit_func_output(outputs, inputs)
-
-# TODO(Lunderberg): Make a `ir.transform.ConvertSSA`,
-# similar to the existing `tir.transform.ConvertSSA`,
-# that converts an entire module to SSA, including TIR
-# variable definitions used in either TIR or Relax.
-mod = self.builder.get()
-mod[method_name] =
rx.utils.copy_with_new_vars(mod[method_name])
-
mod = self.builder.finalize()
assert rx.analysis.well_formed(mod)
-mod = rx.transform.CanonicalizeBindings()(mod)
return mod, params, ext_mods
@@ -170,6 +161,8 @@ def _emit_method( # pylint:
disable=too-many-locals,too-many-branches,too-many-
effects: typing.Optional[typing.List[typing.Tuple[str, core.Effect]]],
):
# pylint: disable=protected-access
+# symbolic shape's name mapping to its tir.Var for reuse
+str2var_params: typing.Dict[str, tir.Var] = {}
def _unwrap_ret(expr: typing.Any) -> typing.Any:
if isinstance(expr, (core.Tensor, core.Object)):
@@ -183,26 +176,35 @@ def _emit_method( # pylint:
Re: [PR] Revert "[SLM] Allow modules to define pre-processing of weights" [tvm]
MasterJH5574 merged PR #16777: URL: https://github.com/apache/tvm/pull/16777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Relax] Improve malform error msg [tvm]
tqchen merged PR #16779: URL: https://github.com/apache/tvm/pull/16779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated: [Relax] Improve malform error msg (#16779)
This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new 5a8d928b84 [Relax] Improve malform error msg (#16779)
5a8d928b84 is described below
commit 5a8d928b84c09b19cea8a17c8c85911fcae1e61d
Author: Ruihang Lai
AuthorDate: Mon Mar 25 09:17:32 2024 -0400
[Relax] Improve malform error msg (#16779)
There are a few places in the malform check that prints pointers.
This PR updates them to their references.
---
src/relax/analysis/well_formed.cc | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/relax/analysis/well_formed.cc
b/src/relax/analysis/well_formed.cc
index 499a988a9f..9f840afe33 100644
--- a/src/relax/analysis/well_formed.cc
+++ b/src/relax/analysis/well_formed.cc
@@ -149,13 +149,13 @@ class WellFormedChecker : public relax::ExprVisitor,
GlobalVar var = GetRef(op);
if (!(mod_->ContainGlobalVar(var->name_hint) &&
mod_->GetGlobalVar(var->name_hint).same_as(var))) {
- Malformed(Diagnostic::Error(var) << "GlobalVar " << op << " is not
defined.");
+ Malformed(Diagnostic::Error(var) << "GlobalVar " << GetRef(op) <<
" is not defined.");
}
if (op->checked_type_.defined()) {
if ((!op->checked_type_->IsInstance()) &&
(!op->checked_type_->IsInstance())) {
-Malformed(Diagnostic::Error(var) << "The checked_type_ of GlobalVar "
<< op
+Malformed(Diagnostic::Error(var) << "The checked_type_ of GlobalVar "
<< GetRef(op)
<< " must be either FuncType or
PackedFuncType.");
}
}
@@ -190,7 +190,7 @@ class WellFormedChecker : public relax::ExprVisitor,
void VisitExpr_(const VarNode* op) final {
Var var = GetRef(op);
if (var_set_.count(var) == 0 && recur_vars_.count(var) == 0) {
- Malformed(Diagnostic::Error(var) << "Var " << op << " is not defined.");
+ Malformed(Diagnostic::Error(var) << "Var " << GetRef(op) << " is
not defined.");
}
CheckStructInfo(op);
}
@@ -199,10 +199,10 @@ class WellFormedChecker : public relax::ExprVisitor,
DataflowVar var = GetRef(op);
if (!is_dataflow_) {
Malformed(Diagnostic::Error(var)
-<< "DataflowVar " << op << " is used outside DataflowBlock.");
+<< "DataflowVar " << GetRef(op) << " is used outside
DataflowBlock.");
}
if (dataflow_var_set_.count(var) == 0) {
- Malformed(Diagnostic::Error(var) << "DataflowVar " << op << " is not
defined.");
+ Malformed(Diagnostic::Error(var) << "DataflowVar " << GetRef(op)
<< " is not defined.");
}
CheckStructInfo(op);
}
@@ -234,7 +234,7 @@ class WellFormedChecker : public relax::ExprVisitor,
// ensure the purity attributes are valid
if
(op->GetAttr(relax::attr::kForcePure).value_or(Bool(false))->value &&
!op->is_pure) {
Malformed(Diagnostic::Error(op->span)
-<< "Function " << op << " has true for " <<
relax::attr::kForcePure
+<< "Function " << GetRef(op) << " has true for " <<
relax::attr::kForcePure
<< " but false for is_pure; " << relax::attr::kForcePure
<< " should be true only if is_pure is also true.");
}
(tvm) branch main updated (9899f9cd28 -> b3981d2f77)
This is an automated email from the ASF dual-hosted git repository. tqchen pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/tvm.git from 9899f9cd28 [AOT][Testing] Improve output mismatch information on test failure (#16765) add b3981d2f77 [Fix] Fix numpy dtype map (#16780) No new revisions were added by this update. Summary of changes: python/tvm/_ffi/runtime_ctypes.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Re: [PR] [Fix] Fix numpy dtype map [tvm]
tqchen merged PR #16780: URL: https://github.com/apache/tvm/pull/16780 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Fix] Fix numpy dtype map [tvm]
mshr-h commented on PR #16780:
URL: https://github.com/apache/tvm/pull/16780#issuecomment-2017591582
This is more Pythonic?
```diff
-if np.__version__.startswith("1."):
-NUMPY2STR[np.dtype(np.float_)] = "float64"
+ try:
+ NUMPY2STR[np.dtype(np.float_)] = "float64"
+ except AttributeError:
+ pass
```
[LBYL](https://docs.python.org/3/glossary.html#term-LBYL)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Fix] Fix numpy dtype map [tvm]
mshr-h commented on PR #16780:
URL: https://github.com/apache/tvm/pull/16780#issuecomment-2017574238
I think this is better.
```diff
-if np.__version__.startswith("1."):
-NUMPY2STR[np.dtype(np.float_)] = "float64"
+if hasattr(np, "float_"):
+NUMPY2STR[np.dtype(np.float_)] = "float64"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [AOT][Testing] Improve output mismatch information on test failure [tvm]
lhutton1 commented on PR #16765: URL: https://github.com/apache/tvm/pull/16765#issuecomment-2017518847 Thanks @Anndrey24 @ekalda! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated: [AOT][Testing] Improve output mismatch information on test failure (#16765)
This is an automated email from the ASF dual-hosted git repository.
lukhut pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new 9899f9cd28 [AOT][Testing] Improve output mismatch information on test
failure (#16765)
9899f9cd28 is described below
commit 9899f9cd2801b3234437df5cd8ab10504b9608bc
Author: Andrei Hutu
AuthorDate: Mon Mar 25 09:06:49 2024 +
[AOT][Testing] Improve output mismatch information on test failure (#16765)
Enhanced AOT test harness to include overall mismatch percentage and the
individual mismatch positions from the output tensor for debugging test
failures. Both of these are still gated behind `print_output_on_mismatch ==
True`.
I also added tests to check for the presence and correctness of this new
debug information.
Sample output:
```
Element [Position]: Actual, Reference
-
Element [0, 8, 8, 7]: 521.846313, 521.847412
Element [0, 9, 8, 51]: 478.874359, 478.875549
Element [0, 9, 9, 6]: 462.901001, 462.899658
Mismatched elements: 3 / 16384 (0.02%)
...
```
---
python/tvm/testing/aot.py | 48 ---
tests/python/relay/aot/test_aot_test_harness.py | 52 -
2 files changed, 85 insertions(+), 15 deletions(-)
diff --git a/python/tvm/testing/aot.py b/python/tvm/testing/aot.py
index 3a117624df..959d1cf58e 100644
--- a/python/tvm/testing/aot.py
+++ b/python/tvm/testing/aot.py
@@ -476,20 +476,40 @@ def _emit_main_compare(
if print_output_on_mismatch:
main_file.write(
-f"int mismatch = 0;"
-f'printf("Actual, Reference\\n");\n'
-f"for (int i = 0; i<{data_length_var_name}; i++) {{\n"
-f"\tif ({comparison_function}({actual_data_name}[i]-"
-f"{expected_data_name}[i]) > {tolerance}) {{\n"
-f'\t\tprintf("{value_format_specifier},
{value_format_specifier}\\n"'
-f", {actual_data_name}[i], {expected_data_name}[i]);\n"
-f"\t\tmismatch = 1;\n"
-f"\t}}\n"
-f"}}"
-f"if (mismatch == 1) {{\n"
-f'\tprintf("{AOT_FAILURE_TOKEN}\\n");\n'
-f"\treturn -1;\n"
-f"}}"
+f"""
+{{
+int mismatch = 0;
+int out_ndim = {outputs[key].ndim};
+int out_shape[] = {{{','.join(map(str, outputs[key].shape))}}};
+int out_indices[out_ndim];
+printf("Element [Position]: Actual, Reference\\n");
+printf("-\\n");
+for (int i = 0; i<{data_length_var_name}; i++) {{
+ if ({comparison_function}({actual_data_name}[i] -
+ {expected_data_name}[i]) > {tolerance}) {{
+int flat_index = i;
+for (int j = out_ndim - 1; j >= 0; j--){{
+ out_indices[j] = flat_index % out_shape[j];
+ flat_index /= out_shape[j];
+}}
+printf("Element [%d", out_indices[0]);
+for (int j = 1; j < out_ndim; j++)
+ printf(", %d", out_indices[j]);
+printf("]: {value_format_specifier},
{value_format_specifier}\\n",
+ {actual_data_name}[i], {expected_data_name}[i]);
+mismatch += 1;
+ }}
+}}
+if (mismatch >= 1) {{
+ float percent_mismatched =
+ ((float) mismatch) / ((float) {data_length_var_name}) *
100;
+ printf("\\nMismatched elements: %d / %zu (%.2f%%)\\n",
+ mismatch, {data_length_var_name}, percent_mismatched);
+ printf("{AOT_FAILURE_TOKEN}\\n");
+ return -1;
+}}
+}}
+"""
)
else:
main_file.write(
diff --git a/tests/python/relay/aot/test_aot_test_harness.py
b/tests/python/relay/aot/test_aot_test_harness.py
index 8ec9506f9f..3d10f15d4a 100644
--- a/tests/python/relay/aot/test_aot_test_harness.py
+++ b/tests/python/relay/aot/test_aot_test_harness.py
@@ -46,7 +46,57 @@ def test_output_on_mismatch_option():
).astype(dtype)
}
-msg = ".*Actual, Reference\n2.00, 0.00\nAOT_TEST_FAILURE.*"
+msg = ".*Actual, Reference(\n|.)*2.00,
0.00(\n|.)*AOT_TEST_FAILURE.*"
+with pytest.raises(RuntimeError, match=msg):
+compile_and_run(
+AOTTestModel(module=tvm.IRModule.from_expr(func), inputs={},
outputs=outputs),
+test_runner,
+interface_api,
+use_unpacked_api,
+print_output_
Re: [PR] [AOT][Testing] Improve output mismatch information on test failure [tvm]
lhutton1 merged PR #16765: URL: https://github.com/apache/tvm/pull/16765 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
