[I] [RESULT][VOTE] Release Apache TVM v0.15.0 [tvm]
ysh329 opened a new issue, #16475: URL: https://github.com/apache/tvm/issues/16475 Dear TVM community, I'm glad to announce the results of the vote for the release of the `v0.14.0.rc0` package. This vote passes with 3 "+1 votes" (3 binding), no "0 votes", and no "-1 vote". ## +1 votes - Star Yuan (ysh329) - Siyuan Feng (Hzfengsy) (binding) - Ruihang Lai (MasterJH5574) (binding) - Tianqi Chen (tqchen) (binding) ## 0 votes - None ## -1 votes - None The list of members can be found [here](https://github.com/apache/tvm/blob/main/CONTRIBUTORS.md). According to release procedure, [for the dev@ vote, there must be at least 3 binding +1 votes and more +1 votes than -1 votes](https://tvm.apache.org/docs/contribute/release_process.html?highlight=release#call-a-vote-on-the-release-candidate). Also see: - the vote thread on GitHub in https://github.com/apache/tvm/issues/16428 - the release schedule in https://github.com/apache/tvm/issues/16277 - the release notes in https://github.com/apache/tvm/issues/16391 - the pre-release page in https://github.com/apache/tvm/releases/tag/v0.15.0.rc0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [VOTE] Release Apache TVM v0.15.0.rc0 [tvm]
Hzfengsy commented on issue #16428: URL: https://github.com/apache/tvm/issues/16428#issuecomment-1911507621 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [VOTE] Release Apache TVM v0.15.0.rc0 [tvm]
ysh329 commented on issue #16428: URL: https://github.com/apache/tvm/issues/16428#issuecomment-1911507376 cc @Hzfengsy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [BugTIR] fix thread_sync occurs in letstmt [tvm]
JackWeiw commented on code in PR #16454:
URL: https://github.com/apache/tvm/pull/16454#discussion_r1467227960
##
src/tir/transforms/storage_access.cc:
##
@@ -94,6 +98,20 @@ void StorageAccessVisitor::VisitStmt_(const EvaluateNode*
op) {
allow_append_ = false;
}
+void StorageAccessVisitor::VisitStmt_(const LetStmtNode* op) {
+ allow_append_ = true;
+ curr_stmt_.access.clear();
Review Comment:
can you check my new solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [VOTE] Release Apache TVM v0.15.0.rc0 [tvm]
ysh329 commented on issue #16428: URL: https://github.com/apache/tvm/issues/16428#issuecomment-1911491329 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch nightly updated (593a4bd9d7 -> 8ade9c30ed)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly in repository https://gitbox.apache.org/repos/asf/tvm.git from 593a4bd9d7 [Relax] NDArray Cache Update with DLTensor Support (#16464) add b98bc66584 [Relax] Memory planning for "partially dynamic" shapes (#16466) add bbbc8956a9 [Contrib] Workspace for cuBLAS backend (#16413) add 5012208e4b [LINT] Fix pylint issues in test_dma_builtin.py (#16468) add 47424b374b [Unity] Improve error message in tensor_to_shape struct inference (#16473) add 8ade9c30ed [Unity] Improved error message in tvm::relax::UpdateStructInfo (#16472) No new revisions were added by this update. Summary of changes: src/relax/ir/struct_info.cc| 6 +- src/relax/op/op.cc | 4 +- src/relax/transform/static_plan_block_memory.cc| 12 ++-- src/runtime/contrib/cublas/cublas.cc | 20 ++- src/runtime/contrib/cublas/cublas_json_runtime.cc | 5 +- src/runtime/contrib/cublas/cublas_utils.cc | 14 - src/runtime/contrib/cublas/cublas_utils.h | 11 +++- .../contrib/test_hexagon/test_dma_builtin.py | 4 +- .../test_transform_static_plan_block_memory.py | 64 ++ 9 files changed, 125 insertions(+), 15 deletions(-)
Re: [PR] [BugTIR] fix thread_sync occurs in letstmt [tvm]
JackWeiw commented on code in PR #16454:
URL: https://github.com/apache/tvm/pull/16454#discussion_r1467213250
##
tests/python/tir-transform/test_tir_transform_thread_sync.py:
##
@@ -160,8 +160,49 @@ def expected(A: T.Buffer((4, 4), "float32"), E:
T.Buffer((4, 4), "float32")):
tvm.ir.assert_structural_equal(mod["main"], expected)
+@tvm.testing.requires_cuda
+def test_sync_let_stmt():
+@T.prim_func(private=True)
+def func(A: T.Buffer((16 * 512), "float32")):
+blockIdx_x = T.launch_thread("blockIdx.x", 16)
+A_shared = T.allocate([512], "float32", "shared")
+in_thread_A_temp = T.allocate([1], "float32", "local")
+cross_thread_A_temp = T.allocate([1], "float32", "local")
+threadIdx_x = T.launch_thread("threadIdx.x", 128)
+A_shared_1 = T.Buffer((512,), data=A_shared, scope="shared")
+for ax0 in range(512):
+A_shared_1[ax0] = A[blockIdx_x * 512 + ax0]
+in_thread_A_temp_1 = T.Buffer((1,), data=in_thread_A_temp,
scope="local")
+in_thread_A_temp_1[0] = T.float32(0)
+with T.LetStmt(in_thread_A_temp_1[0] + A_shared_1[threadIdx_x]) as
A_temp:
Review Comment:
in case here, before take change, `StorageAccessVisitor` will call
`VisitExpr_(const BufferLoadNode* op)` directly (not called by father call like
`VisitStmt_(const BufferStoreNode* op)` traverse child), so
`ICHECK(allow_append_) << op << " " << scope.to_string();` will return error.
if we take `VisitStmt_(const LetStmtNode* op)` and traverse child of
`LetStmtNode`
(case here it will traverse `BufferLoadNode` A_shared which will increase
curr_stmt_.access by 1, and then visit `BufferStoreNode` `in_thread_A_temp` ,
`ICHECK_EQ(curr_stmt_.access.size(), 0U);` will return error)
Do you have any insights on how to solve this problem? @vinx13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] Update flash attention to integrate flash decoding with paged KV cache [tvm]
masahi opened a new pull request, #16474: URL: https://github.com/apache/tvm/pull/16474 Flash attention recently added support for loading from paged KV cache in https://github.com/Dao-AILab/flash-attention/commit/54e80a3829c6d2337570d01e78ebd9529c02d342. The support was added to Flash-Decoding kernel, which we haven't used so far. This PR let's us use Flash Decoding with paged KV cache support from TVM. We already use other kernels from Flash attention via BYOC, but due to the specialized nature of this kernel, it is supported as a contrib kernel (similar to vllm). @vinx13 @sunggg -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Unity][Cutlass] Fix C source generation of dense operation [tvm]
creaiter closed pull request #16463: [Unity][Cutlass] Fix C source generation of dense operation URL: https://github.com/apache/tvm/pull/16463 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Redo][Unity] Split DecomposeOpsForTraining into two steps [tvm]
slyubomirsky commented on PR #16465: URL: https://github.com/apache/tvm/pull/16465#issuecomment-1911284435 Ah, that seems like a good reason then. :+1: Some bigger simplifications in the works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Transform] Preserve all reachable functions in FuseTIR [tvm]
slyubomirsky commented on code in PR #16471:
URL: https://github.com/apache/tvm/pull/16471#discussion_r1467148384
##
src/relax/transform/fuse_tir.cc:
##
@@ -950,63 +940,66 @@ class TIRFuseMutator : public ExprMutator {
if (call->op->IsInstance()) {
// Case 1. It is a relax cross function call
- GlobalVar old_gv = Downcast(call->op);
- auto relax_func = Downcast(mod_->Lookup(old_gv));
- auto it = fused_tir_funcs_.find(old_gv);
- if (it != fused_tir_funcs_.end()) {
-const tir::PrimFunc& fused_tir = (*it).second;
-// Case 1.1. It calls a primitive relax function, update the call into
a call_tir
-GlobalVar fused_tir_gv = this->builder_->AddFunction(fused_tir,
old_gv->name_hint);
-// Step a. Flatten all args since call_tir does not support Tuple
value.
-Array arg_list;
-Array tir_vars;
-for (size_t i = 0; i < call->args.size(); ++i) {
- auto arg = call->args[i];
- auto sinfo = GetStructInfo(arg);
-
-
ICHECK(!relax_func->params[i]->struct_info_->IsInstance()
&&
- !sinfo.as())
- << "InternalError: "
- << "All tuple parameters should be expanded before this point in
FuseTIR. "
- << "However, argument " << arg << " with struct info " <<
arg->struct_info_
- << " is passed as argument " << i << " to Primitive Relax
function " << old_gv
- << ", which expects parameter " << relax_func->params[i] << " to
have struct info "
- << relax_func->params[i]->struct_info_;
-
- if (const auto* shape = sinfo.as()) {
-CHECK(shape->values.defined())
-<< "FuseTIR requires all shape input has struct_info value.";
-for (const PrimExpr& prim_value : shape->values.value()) {
- CHECK(prim_value->IsInstance())
- << "All shape inputs are expected to be single tir var.";
- tir_vars.push_back(prim_value);
-}
- } else if (const auto* prim_value = sinfo.as()) {
-CHECK(prim_value->value.defined())
-<< "FuseTIR requires all R.Prim arguments to have a known
value.";
-PrimExpr expr = prim_value->value.value();
-CHECK(expr->IsInstance())
-<< "FuseTIR currently requires all R.Prim arguments to provide
a single tir::Var.";
-tir_vars.push_back(expr);
-
- } else {
-arg_list.push_back(arg);
+ GlobalVar old_gvar = Downcast(call->op);
+ auto relax_func = Downcast(orig_mod_->Lookup(old_gvar));
+
+ if (!relax_func->HasNonzeroAttr(attr::kPrimitive)) {
+// The callee is not a primitive function, no need to fuse.
+return call;
+ }
+
+ tir::PrimFunc fused_tir = FusedTIRConstructor::GetFusedTIR(orig_mod_,
old_gvar);
Review Comment:
Are you sure we aren't likely to call this more than once per global var? As
long as it's not likely or costly, this approach is probably fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Transform] Preserve all reachable functions in FuseTIR [tvm]
slyubomirsky commented on PR #16471: URL: https://github.com/apache/tvm/pull/16471#issuecomment-1911262047 Could you add the test case from that PR? It probably will solve that problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated: [Unity] Improved error message in tvm::relax::UpdateStructInfo (#16472)
This is an automated email from the ASF dual-hosted git repository.
sslyu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git
The following commit(s) were added to refs/heads/main by this push:
new 8ade9c30ed [Unity] Improved error message in
tvm::relax::UpdateStructInfo (#16472)
8ade9c30ed is described below
commit 8ade9c30ed756df307b00c42b94a2abcf639d662
Author: Eric Lunderberg
AuthorDate: Thu Jan 25 19:22:54 2024 -0600
[Unity] Improved error message in tvm::relax::UpdateStructInfo (#16472)
If the struct info cannot be updated, provide the expression, the
previous struct info, and the new struct info in the error message.
---
src/relax/ir/struct_info.cc | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/relax/ir/struct_info.cc b/src/relax/ir/struct_info.cc
index 302534b414..ac71454f8c 100644
--- a/src/relax/ir/struct_info.cc
+++ b/src/relax/ir/struct_info.cc
@@ -209,7 +209,11 @@ TVM_REGISTER_GLOBAL("relax.FuncStructInfoOpaqueFunc")
// Helper functions
void UpdateStructInfo(Expr expr, StructInfo struct_info) {
ICHECK(!expr->struct_info_.defined())
- << "the struct_info_ of the Expr to be updated must be nullptr for
idempotency";
+ << "To ensure idempotency, "
+ << "the expression passed to UpdateStructInfo "
+ << "must not have any prior StructInfo. "
+ << "However, expression " << expr << " has struct info " <<
expr->struct_info_
+ << ", which cannot be overwritten with " << struct_info;
expr->struct_info_ = struct_info;
// also set checked type
expr->checked_type_ = GetStaticType(struct_info);
Re: [PR] [Unity] Improved error message in tvm::relax::UpdateStructInfo [tvm]
slyubomirsky merged PR #16472: URL: https://github.com/apache/tvm/pull/16472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated: [Unity] Improve error message in tensor_to_shape struct inference (#16473)
This is an automated email from the ASF dual-hosted git repository. sslyu pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tvm.git The following commit(s) were added to refs/heads/main by this push: new 47424b374b [Unity] Improve error message in tensor_to_shape struct inference (#16473) 47424b374b is described below commit 47424b374b00b751d5575818374b9bc1712033c3 Author: Eric Lunderberg AuthorDate: Thu Jan 25 19:22:04 2024 -0600 [Unity] Improve error message in tensor_to_shape struct inference (#16473) If the tensor argument does not have a valid shape, then the error message will include the expression, the argument, and the struct info of the argument. --- src/relax/op/op.cc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/relax/op/op.cc b/src/relax/op/op.cc index f51d2cc74f..489886e50f 100644 --- a/src/relax/op/op.cc +++ b/src/relax/op/op.cc @@ -780,7 +780,9 @@ StructInfo ReturnTensorToShapeStructInfo(const Call& call, const BlockBuilder& c const auto* tsinfo = GetStructInfoAs(call->args[0]); ICHECK(tsinfo && tsinfo->shape.defined()); ShapeExpr shape_expr = Downcast(tsinfo->shape.value()); - ICHECK(shape_expr->values.size() == 1); + ICHECK(shape_expr->values.size() == 1) << "relax.tensor_to_shape expected argument to be 1-d, " + << "but " << call << " has argument " << call->args[0] + << " with struct info " << call->args[0]->struct_info_; const IntImmNode* ndim = shape_expr->values[0].as(); ICHECK(ndim); return ShapeStructInfo(ndim->value);
Re: [PR] [Unity] Improve error message in tensor_to_shape struct inference [tvm]
slyubomirsky merged PR #16473: URL: https://github.com/apache/tvm/pull/16473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [DRAFT] test the temporary aws credentials [tvm]
yongwww closed pull request #16459: [DRAFT] test the temporary aws credentials URL: https://github.com/apache/tvm/pull/16459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [BugTIR] fix thread_sync occurs in letstmt [tvm]
vinx13 commented on code in PR #16454:
URL: https://github.com/apache/tvm/pull/16454#discussion_r1466957964
##
tests/python/tir-transform/test_tir_transform_thread_sync.py:
##
@@ -160,8 +160,49 @@ def expected(A: T.Buffer((4, 4), "float32"), E:
T.Buffer((4, 4), "float32")):
tvm.ir.assert_structural_equal(mod["main"], expected)
+@tvm.testing.requires_cuda
+def test_sync_let_stmt():
+@T.prim_func(private=True)
+def func(A: T.Buffer((16 * 512), "float32")):
+blockIdx_x = T.launch_thread("blockIdx.x", 16)
+A_shared = T.allocate([512], "float32", "shared")
+in_thread_A_temp = T.allocate([1], "float32", "local")
+cross_thread_A_temp = T.allocate([1], "float32", "local")
+threadIdx_x = T.launch_thread("threadIdx.x", 128)
+A_shared_1 = T.Buffer((512,), data=A_shared, scope="shared")
+for ax0 in range(512):
+A_shared_1[ax0] = A[blockIdx_x * 512 + ax0]
+in_thread_A_temp_1 = T.Buffer((1,), data=in_thread_A_temp,
scope="local")
+in_thread_A_temp_1[0] = T.float32(0)
+with T.LetStmt(in_thread_A_temp_1[0] + A_shared_1[threadIdx_x]) as
A_temp:
+in_thread_A_temp_1[0] = A_temp
+with T.LetStmt(in_thread_A_temp_1[0] + A_shared_1[threadIdx_x + 128])
as A_temp:
+in_thread_A_temp_1[0] = A_temp
+with T.LetStmt(in_thread_A_temp_1[0] + A_shared_1[threadIdx_x + 256])
as A_temp:
+in_thread_A_temp_1[0] = A_temp
+with T.LetStmt(in_thread_A_temp_1[0] + A_shared_1[threadIdx_x + 384])
as A_temp:
+in_thread_A_temp_1[0] = A_temp
+cross_thread_A_temp_1 = T.Buffer((1,), data=cross_thread_A_temp,
scope="local")
+with T.attr(
+T.comm_reducer(lambda x0, y0: x0 + y0, [T.float32(0)]),
+"reduce_scope",
+T.reinterpret("handle", T.uint64(0)),
+):
+T.tvm_thread_allreduce(
+T.uint32(1),
+in_thread_A_temp_1[0],
+T.bool(True),
+cross_thread_A_temp_1[0],
+threadIdx_x,
+)
+
+mod = run_passes(func)
+assert "T.tvm_storage_sync" in str(mod)
Review Comment:
it's better to use `tvm.ir.assert_structural_equal` if possible since it's
not obvious where `T.tvm_storage_sync` should be inserted
##
src/tir/transforms/storage_access.cc:
##
@@ -94,6 +98,20 @@ void StorageAccessVisitor::VisitStmt_(const EvaluateNode*
op) {
allow_append_ = false;
}
+void StorageAccessVisitor::VisitStmt_(const LetStmtNode* op) {
+ allow_append_ = true;
+ curr_stmt_.access.clear();
Review Comment:
shouldn't call clear here, it can only be used after finishing handling a
statement
##
src/tir/transforms/storage_access.cc:
##
@@ -51,6 +51,10 @@ void StorageAccessVisitor::VisitExpr_(const BufferLoadNode*
op) {
}
// traverse child
StmtExprVisitor::VisitExpr_(op);
+ // push to the scope
+ scope_.back().push_back(curr_stmt_);
+ // clear access entry.
+ curr_stmt_.access.clear();
Review Comment:
`BufferLoad` is not a statement, it is an expression, we shouldn't push and
clear here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(tvm) branch main updated: [LINT] Fix pylint issues in test_dma_builtin.py (#16468)
This is an automated email from the ASF dual-hosted git repository. lukhut pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tvm.git The following commit(s) were added to refs/heads/main by this push: new 5012208e4b [LINT] Fix pylint issues in test_dma_builtin.py (#16468) 5012208e4b is described below commit 5012208e4b54026a21eed7a3308e0c0fae1010a7 Author: Elen Kalda AuthorDate: Thu Jan 25 17:39:52 2024 + [LINT] Fix pylint issues in test_dma_builtin.py (#16468) We see `no-self-argument` and `wrong-import-order` pylint errors in downstream CI. Eyeballing the file, both of these errors are legit, it's not clear why the upstream CI didn't catch them, potentially because of the inherent instability of pylint running on multiple threads. --- tests/python/contrib/test_hexagon/test_dma_builtin.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/python/contrib/test_hexagon/test_dma_builtin.py b/tests/python/contrib/test_hexagon/test_dma_builtin.py index 74f25acaba..11f4d2d540 100644 --- a/tests/python/contrib/test_hexagon/test_dma_builtin.py +++ b/tests/python/contrib/test_hexagon/test_dma_builtin.py @@ -19,6 +19,7 @@ Test relax vm builtin to enable DMA copy and wait operations. """ +import numpy as np import tvm import tvm.script from tvm import relax @@ -27,9 +28,8 @@ from tvm.script.parser import relax as R from tvm.script.parser import tir as T import tvm.contrib.hexagon import tvm.testing -import numpy as np -# pylint: disable=invalid-name, missing-class-docstring, missing-function-docstring +# pylint: disable=invalid-name, missing-class-docstring, missing-function-docstring, no-self-argument @I.ir_module
Re: [PR] [LINT] Fix pylint issues in test_dma_builtin.py [tvm]
lhutton1 merged PR #16468: URL: https://github.com/apache/tvm/pull/16468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [Unity] Improved error message in tvm::relax::UpdateStructInfo [tvm]
Lunderberg opened a new pull request, #16472: URL: https://github.com/apache/tvm/pull/16472 If the struct info cannot be updated, provide the expression, the previous struct info, and the new struct info in the error message. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Unity][Relax] Avoid removing `PrimFunc`s that are used for in-place calls in `FuseTIR` [tvm]
Lunderberg commented on PR #16462: URL: https://github.com/apache/tvm/pull/16462#issuecomment-1910354803 > These PrimFuncs will not be dead code either, since they will actually be called in the final program. Hmm. My impression for the `funcs_to_keep` was that it forwarded all `PrimFunc`s that were called by the user-facing functions, and discarded all the rest. Any functions that were discarded could instead have been discarded by `DeadCodeElimination`. I'll test out the idea and make a separate PR if it works. For now, I think this PR is good to go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Redo][Unity] Split DecomposeOpsForTraining into two steps [tvm]
Lunderberg commented on PR #16465: URL: https://github.com/apache/tvm/pull/16465#issuecomment-1910312936 It isn't strictly necessary, but I'd like to move in that direction as a first step in removing the `LegalizeOpsForInference` step altogether. This came up in a conversation [here](https://github.com/apache/tvm/pull/15842#discussion_r1358449742) regarding potential use cases of allowing `FLegalize` to be implemented in terms of other relax operations. Currently, there are two distinct transforms, `DecomposeOpsForTraining` and `DecomposeOpsForInference`. If a model uses either `R.nn.batch_norm` or `R.nn.layer_norm`, a user must apply one of these two transforms prior to calling `tvm.relax.build`. This long-term goal of this change is to remove that requirement. By splitting the transform into two steps, the training flow has an optional pass `MutateOpsForTraining` and a mandatory pass `DecomposeOps`, while the inference flow has a single mandatory pass `DecomposeOps`. Since the `DecomposeOps` pass is required for both use cases, and the change it makes is a special case of `FLegalize`, a follow-up PR can then define `FLegalize` for `R.nn.batch_norm` and `R.nn.layer_norm`, and remove `DecomposeOps` altogether. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [BugTIR] fix thread_sync occurs in letstmt [tvm]
JackWeiw commented on PR #16454: URL: https://github.com/apache/tvm/pull/16454#issuecomment-1909980005 cc @vinx13 @spectrometerHBH please spend a little time do a review if you see this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [TVMC] Enable VM from tvmc command line and cross compile [tvm]
krishnaraj36 opened a new pull request, #16469: URL: https://github.com/apache/tvm/pull/16469 1. Enable VM executor in tvmc command line script (--use-vm) 2. Enable cross-compiler option for VM executor compile, it fixed opencl compilation for adreno platform. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [LINT] Fix pylint issues in test_dma_builtin.py [tvm]
ekalda opened a new pull request, #16468: URL: https://github.com/apache/tvm/pull/16468 We see `no-self-argument` and `wrong-import-order` pylint errors in downstream CI. Eyeballing the file, both of these errors are legit, it's not clear why the upstream CI didn't catch them, potentially because of the inherent instability of pylint running on multiple threads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(tvm) branch main updated (b98bc66584 -> bbbc8956a9)
This is an automated email from the ASF dual-hosted git repository. masahi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/tvm.git from b98bc66584 [Relax] Memory planning for "partially dynamic" shapes (#16466) add bbbc8956a9 [Contrib] Workspace for cuBLAS backend (#16413) No new revisions were added by this update. Summary of changes: src/runtime/contrib/cublas/cublas.cc | 20 +--- src/runtime/contrib/cublas/cublas_json_runtime.cc | 5 +++-- src/runtime/contrib/cublas/cublas_utils.cc| 14 +- src/runtime/contrib/cublas/cublas_utils.h | 11 ++- 4 files changed, 43 insertions(+), 7 deletions(-)
Re: [PR] [Contrib] Workspace for cuBLAS backend [tvm]
masahi merged PR #16413: URL: https://github.com/apache/tvm/pull/16413 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Contrib] Workspace for cuBLAS backend [tvm]
masahi commented on code in PR #16413:
URL: https://github.com/apache/tvm/pull/16413#discussion_r1466127313
##
src/runtime/contrib/cublas/cublas_utils.h:
##
@@ -80,7 +80,14 @@ struct CuBlasThreadEntry {
struct CuBlasLtThreadEntry {
CuBlasLtThreadEntry();
~CuBlasLtThreadEntry();
+
cublasLtHandle_t handle{nullptr};
+ cublasLtMatmulPreference_t matmul_pref_desc{nullptr};
+ void* workspace_ptr{nullptr};
+ // 32MB workspace as suggested by NVIDIA
+ // https://docs.nvidia.com/cuda/cublas/index.html#cublassetworkspace.
+ static constexpr const size_t workspace_size = 33554432;
Review Comment:
I'm assuming that 32MB is also good for pre-Hopper since it is bigger than
the recommended size, 4MB. @vinx13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
