[GitHub] [tvm] luyaor opened a new issue #7245: [Bug] Wrong result when model contains LpPool

[GitBox]

luyaor opened a new issue #7245:
URL: https://github.com/apache/tvm/issues/7245


   ## Description
   
   When compiling following model with TVM, it will output wrong result.
   
   The model(with ONNX as frontend) with error is as follows, check bug.onnx in 
[bug6.zip](https://github.com/apache/tvm/files/5794682/bug6.zip).
   
   
![image](https://user-images.githubusercontent.com/7541296/104161312-a1381200-542d-11eb-9108-6842e5e0151c.png)
   
   I think it is because TVM relay's wrong implementation for LpPool. In this 
case, p = 1, the result should be 
   L1 norm of [ 0.5186008, -0.2654016] = 
[0.5387418866157532-0.02014109119772911, 
0.6362205147743225-0.9016221165657043], which is 0.7840024, while tvm output 
sum of its element which is 0.25319922.
   
   ## How to reproduce
   
   ### Environment
   
   Python3, with tvm, onnx, onnxruntime
   
   tvm version: 
[`c31e338`](https://github.com/apache/tvm/commit/c31e338d5f98a8e8c97286c5b93b20caee8be602)
 Wed Dec 9 14:52:58 2020 +0900
   
   1. Download [bug6.zip](https://github.com/apache/tvm/files/5794682/bug6.zip)
   2. Run `python check.py`.
   
   
   
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] luyaor closed issue #7245: [Bug] Wrong result when model contains LpPool (when p=1)

[GitBox]

luyaor closed issue #7245:
URL: https://github.com/apache/tvm/issues/7245


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] luyaor commented on issue #7245: [Bug] Wrong result when model contains LpPool (when p=1)

[GitBox]

luyaor commented on issue #7245:
URL: https://github.com/apache/tvm/issues/7245#issuecomment-757792787


   Sorry, maybe I need to check again for definition of LpPool.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] giuseros commented on pull request #7070: Add autoscheduler support to tvmc

[GitBox]

giuseros commented on pull request #7070:
URL: https://github.com/apache/tvm/pull/7070#issuecomment-757848183


   Hi @merrymercy ,
   Thanks for the comment! Sure, I will open a new PR to add this in



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] zhuwenxi opened a new issue #7246: [BUG][Tensorize] race condition when using "tvm.tir.call_packed()" in a parallel schedule.

[GitBox]

zhuwenxi opened a new issue #7246:
URL: https://github.com/apache/tvm/issues/7246


   ### Problem Statement
   This bug was encountered when I was trying to use external as "micro-kernel" 
to optimize the Matmul op. Basically I was just following the [TVM Tensorize 
Tutorial](https://tvm.apache.org/docs/tutorials/language/tensorize.html), and 
did a little modification by replacing the `tvm.tir.call_extern('int32', 
'gemv_update'` with `tvm.tir.call_packed("tvm.contrib.cblas.matmul"...`, which 
of course because I'm trying to leverage existing blas library to do tensorize. 
The code works pretty well, until I add a `s[C].parallel(xo)`, It crashed:
   
   
![image](https://user-images.githubusercontent.com/4969797/104176940-f4688f80-5442-11eb-8625-0d18a6f209b9.png)
   
   ### Environment
   * CPU: CacadeLake-X
   * OS: CentOS 7.0
   * TVM: 0.7
   * LLVM: 9.0
   
   ### Code to reproduce this bug
   import tvm
   from tvm import te
   import numpy as np
   import sys
   from tvm import testing
   
   # Fail case:
   M, K, N = 4, 4, 2
   
   A = te.placeholder((M, K), name='A')
   B = te.placeholder((K, N), name='B')
   k = te.reduce_axis((0, K), name='k')
   C = te.compute((M, N), lambda i, j: te.sum(A[i, k] * B[k, j], axis=k), 
name='C')
   s = te.create_schedule(C.op)
   
   bn = 2
   xo, yo, xi, yi = s[C].tile(C.op.axis[0], C.op.axis[1], bn, bn)
   s[C].reorder(xo, yo, xi, yi, k)
   s[C].parallel(xo)
   
   def intrin_libxsmm(m, k, n):
 a = te.placeholder((m, k), name='a')
 b = te.placeholder((k, n), name='b')
 k = te.reduce_axis((0, k), name='k')
 c = te.compute((m, n), lambda i, j: te.sum(a[i, k] * b[k, j], axis=k), 
name='c')
 a_buffer = tvm.tir.decl_buffer(a.shape, a.dtype, name='a_buffer', 
offset_factor=1, strides=[te.var('s1'), 1])
 b_buffer = tvm.tir.decl_buffer(b.shape, b.dtype, name='b_buffer', 
offset_factor=1, strides=[te.var('s2'), 1])
 c_buffer = tvm.tir.decl_buffer(c.shape, c.dtype, name='c_buffer', 
offset_factor=1, strides=[te.var('s3'), 1])
   
 def intrin_func(ins, outs):
   ib = tvm.tir.ir_builder.create()
   ib.emit(
 tvm.tir.call_packed(
   "tvm.contrib.cblas.matmul", ins[0], ins[1], outs[0], False, False, 
1.0, 0.0
 )
   )
   return ib.get()
   
 return te.decl_tensor_intrin(c.op, intrin_func, binds={a: a_buffer, b: 
b_buffer, c: c_buffer})
   
   micro_kernel = intrin_libxsmm(bn, K, bn)
   s[C].tensorize(xi, micro_kernel)
   ctx = tvm.cpu(0)
   func = tvm.build(s, [A, B, C], target='llvm')
   a = tvm.nd.array(np.random.uniform(size=(M, K)).astype(A.dtype), ctx)
   b = tvm.nd.array(np.random.uniform(size=(K, N)).astype(B.dtype), ctx)
   c = tvm.nd.array(np.zeros((M, N), dtype=C.dtype), ctx)
   func(a, b, c)
   tvm.testing.assert_allclose(c.asnumpy(), np.dot(a.asnumpy(), b.asnumpy()), 
rtol=1e-5)



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] zhuwenxi commented on issue #7246: [BUG][Tensorize] race condition when using "tvm.tir.call_packed()" in a parallel schedule.

[GitBox]

zhuwenxi commented on issue #7246:
URL: https://github.com/apache/tvm/issues/7246#issuecomment-757894841


   And by digging into TVM source code, I believe I've already root cause this 
bug.
   
   Short answer: the TIR generated for "@tvm.tir.call_packed()" is not 
thread-safe, thus causes a race condition in a multi-threads environment.
   
   For detail explanations, please see my analysis here: 
https://discuss.tvm.apache.org/t/tensorize-tensorize-couldnt-work-properly-with-parallel-schedule/8752



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] insop commented on a change in pull request #7230: [FRONTEND][Mxnet][nuymp] Adding _npi_advanced_indexing_multiple

[GitBox]

insop commented on a change in pull request #7230:
URL: https://github.com/apache/tvm/pull/7230#discussion_r555005379



##
File path: tests/python/frontend/mxnet/test_forward.py
##
@@ -1935,6 +1935,29 @@ def verify(data_shape, axis, use_length, length):
 verify((2, 3, 4), 2, True, np.array([[3, 4, 2], [1, 2, 
1]]).astype("int32"))
 
 
+@pytest.mark.parametrize(
+"data_shape, row_sel, col",
+[
+((5, 7), (0, 1, 2, 3, 4,), 2),
+],
+)
+@pytest.mark.parametrize("dtype", ["float64", "float32"])
+@tvm.testing.parametrize_targets
+@pytest.mark.parametrize("kind", ["graph", "vm", "debug"])
+def test_forward_npi_advanced_indexing_multiple(data_shape, row_sel, col, 
dtype, target, ctx, kind):
+data_np = np.random.uniform(size=data_shape).astype(dtype)
+data = mx.sym.var("data")
+ref_res = mx.np.array(data_np)[row_sel, col]
+
+# TODO need to add the proper symbol operator
+mx_sym = mx.sym.np.(data.as_np_ndarray()[row_sel, col])

Review comment:
   Hi @sxjscience
   
   Thank you for the suggestion; however, I still not clear how to create 
`mx_sym` so that I can use it for `relay.frontend.from_mxnet`.
   
   Any suggestion would appreciated.
   Thank you.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] wejoncy opened a new issue #7247: [BUG] Generate float type of index for vectorize schedule

[GitBox]

wejoncy opened a new issue #7247:
URL: https://github.com/apache/tvm/issues/7247


   ## Description
   
   TVM will generate float type of  index expr, like 
`B_local[float)get_local_id(1)) + ((float)(((int)get_group_id(2)) * 
8];` , which is illegal.
   
   ## Error information
   ```
   :41:227: error: array subscript is not an integer
   B[float)((int)get_group_id(0)) * 131072) + (yy_inner * 16384)) + 
(((int)get_group_id(1)) * 4096)) + (((int)get_local_id(0)) * 512))) + 
(((float)get_local_id(1)) + ((float)(((int)get_group_id(2)) * 8)] = 
B_local[float)get_local_id(1)) + ((float)(((int)get_group_id(2)) * 8];
   ```
   
   ## Env
   TVM::main
   
   ## Minimal code to reproduce.
   ```
   
   import numpy as np
   import tvm
   from tvm import te
   
   # The sizes of inputs and filters
   batch = 1
   in_channel = 256
   out_channel = 512
   in_size = 32
   kernel = 1
   pad = 0
   stride = 1
   
   # Algorithm
   A = te.placeholder((in_size, in_size, in_channel, batch), name="A")
   W = te.placeholder((kernel, kernel, in_channel, out_channel), name="W")
   out_size = (in_size - kernel + 2 * pad) // stride + 1
   # Pad input
   Apad = te.compute(
   (in_size + 2 * pad, in_size + 2 * pad, in_channel, batch),
   lambda yy, xx, cc, nn: tvm.tir.if_then_else(
   tvm.tir.all(yy >= pad, yy - pad < in_size, xx >= pad, xx - pad < 
in_size),
   A[yy - pad, xx - pad, cc, nn],
   tvm.tir.const(0.0, "float32"),
   ),
   name="Apad",
   )
   # Create reduction variables
   rc = te.reduce_axis((0, in_channel), name="rc")
   ry = te.reduce_axis((0, kernel), name="ry")
   rx = te.reduce_axis((0, kernel), name="rx")
   # Compute the convolution
   B = te.compute(
   (out_size, out_size, out_channel, batch),
   lambda yy, xx, ff, nn: te.sum(
   Apad[yy * stride + ry, xx * stride + rx, rc, nn] * W[ry, rx, rc, 
ff], axis=[ry, rx, rc]
   ),
   name="B",
   )
   
   # Designate the memory hierarchy
   s = te.create_schedule(B.op)
   s[Apad].compute_inline()  # compute Apad inline
   AA = s.cache_read(Apad, "shared", [B])
   WW = s.cache_read(W, "shared", [B])
   AL = s.cache_read(AA, "local", [B])
   WL = s.cache_read(WW, "local", [B])
   BL = s.cache_write(B, "local")
   
   # tile consts
   tile = 1
   num_thread = 8
   block_factor = tile * num_thread
   step = 8
   vthread = 2
   
   # Get the GPU thread indices
   block_x = te.thread_axis("blockIdx.x")
   block_y = te.thread_axis("blockIdx.y")
   block_z = te.thread_axis("blockIdx.z")
   thread_x = te.thread_axis((0, num_thread), "threadIdx.x")
   thread_y = te.thread_axis((0, num_thread/4), "threadIdx.y")
   thread_xz = te.thread_axis((0, vthread), "vthread", name="vx")
   thread_yz = te.thread_axis((0, vthread), "vthread", name="vy")
   
   # Split the workloads
   hi, wi, fi, ni = s[B].op.axis
   bz, fi = s[B].split(fi, factor=block_factor)
   bx, hi = s[B].split(hi, factor=block_factor)
   by, wi = s[B].split(wi, factor=block_factor)
   # Bind the iteration variables to GPU thread indices
   s[B].bind(bz, block_z)
   s[B].bind(by, block_y)
   s[B].bind(bx, block_x)
   
   
   ty, fi = s[B].split(fi, nparts=num_thread)
   tx, wi = s[B].split(wi, nparts=num_thread)
   
   s[B].reorder(bz,by,bx,ty,wi, fi)
   
   
   s[B].bind(ty, thread_y)
   s[B].bind(tx, thread_x)
   
   
   # Schedule BL local write
   s[BL].compute_at(s[B],ni)
   fi, noo = s[B].split(fi, factor=4)
   s[B].vectorize(noo)  # vectorize memory load
   yi, xi, fi, ni = s[BL].op.axis
   ry, rx, rc = s[BL].op.reduce_axis
   rco, rci = s[BL].split(rc, factor=step)
   s[BL].reorder(rco, ry, rx, rci, fi, ni)
   
   # Attach computation to iteration variables
   s[AA].compute_at(s[BL], rx)
   s[WW].compute_at(s[BL], rx)
   s[AL].compute_at(s[BL], rci)
   s[WL].compute_at(s[BL], rci)
   
   yi, xi, ci, ni = s[AA].op.axis
   tx, ni = s[AA].split(ni, nparts=num_thread)
   _, ci = s[AA].split(ci, factor=4)
   
   s[AA].bind(tx, thread_x)
   s[AA].vectorize(ci)  # vectorize memory load
   
   # Schedule for W's shared memory load
   yi,xi,ci,fi=s[WW].op.axis
   ty, ci = s[WW].split(ci, nparts=num_thread)
   fi, fv = s[WW].split(fi, factor=4)
   s[WW].bind(ty, thread_y)
   s[WW].vectorize(fv)  # vectorize memory load
   
   target="opencl"
   func = tvm.build(s, [A, W, B], target)
   print("--opencl code--")
   print(func.imported_modules[0].get_source()) if len(func.imported_modules) > 
0 else print("source not imported")
   ctx = tvm.context(target, 0)
   np.random.seed(5)
   a_np = np.random.uniform(size=(in_size, in_size, in_channel, 
batch)).astype("float32")
   w_np = np.random.uniform(size=(kernel, kernel, in_channel, 
out_channel)).astype("float32")
   
   a = tvm.nd.array(a_np, ctx)
   w = tvm.nd.array(w_np, ctx)
   b = tvm.nd.array(np.zeros((out_size, out_size, out_channel, batch), 
dtype="float32"), ctx)
   func(a, w, b)
   np.savetxt("filename.txt",b.asnumpy()[:,:,0,0])
   evaluator = func.time_eva

[tvm] branch main updated: [TFLite] Quantized version of unit test for Dense (#7113)

[mbaret]

This is an automated email from the ASF dual-hosted git repository.

mbaret pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
 new 7f14769  [TFLite] Quantized version of unit test for Dense (#7113)
7f14769 is described below

commit 7f1476957cefcaeb61d17781c4e663d876d9fe0f
Author: Dmitriy Smirnov 
AuthorDate: Mon Jan 11 14:24:42 2021 +

[TFLite] Quantized version of unit test for Dense (#7113)

Added quantized version of unit test for FullyConnected/Dense
Added check for -1 in case if bias not supplied
---
 python/tvm/relay/frontend/tflite.py  | 19 ---
 tests/python/frontend/tflite/test_forward.py | 83 +---
 2 files changed, 73 insertions(+), 29 deletions(-)

diff --git a/python/tvm/relay/frontend/tflite.py 
b/python/tvm/relay/frontend/tflite.py
index 94e9e0c..7a2aada 100644
--- a/python/tvm/relay/frontend/tflite.py
+++ b/python/tvm/relay/frontend/tflite.py
@@ -982,7 +982,7 @@ class OperatorConverter(object):
 
 input_tensors = self.get_input_tensors(op)
 assert len(input_tensors) >= 1, "input tensors should greater than 1"
-in_exprs = [self.get_expr(input_tensor.tensor_idx) for input_tensor in 
input_tensors]
+in_exprs = [self.get_tensor_expr(_) for _ in input_tensors]
 
 output_tensors = self.get_output_tensors(op)
 assert len(output_tensors) == 1, "output tensors length should be 1"
@@ -1830,14 +1830,15 @@ class OperatorConverter(object):
 # if we have bias
 if len(input_tensors) == 3:
 bias_tensor = input_tensors[2]
-bias_tensor_type = bias_tensor.tensor.Type()
-# bias tensor type should be INT32 (quantization) or FLOAT32
-assert bias_tensor_type in (TensorType.INT32, TensorType.FLOAT32)
-bias_tensor_type_str = self.get_tensor_type_str(bias_tensor_type)
-bias_expr = self.exp_tab.new_const(
-self.get_tensor_value(bias_tensor), dtype=bias_tensor_type_str
-)
-out = _op.nn.bias_add(out, bias_expr)
+if bias_tensor.tensor_idx != -1:
+bias_tensor_type = bias_tensor.tensor.Type()
+# bias tensor type should be INT32 (quantization) or FLOAT32
+assert bias_tensor_type in (TensorType.INT32, 
TensorType.FLOAT32)
+bias_tensor_type_str = 
self.get_tensor_type_str(bias_tensor_type)
+bias_expr = self.exp_tab.new_const(
+self.get_tensor_value(bias_tensor), 
dtype=bias_tensor_type_str
+)
+out = _op.nn.bias_add(out, bias_expr)
 
 # Finally if the dense is quantized. Add a requantize at the end.
 if output_tensor.qnn_params:
diff --git a/tests/python/frontend/tflite/test_forward.py 
b/tests/python/frontend/tflite/test_forward.py
index 52dde38..c8bd094 100644
--- a/tests/python/frontend/tflite/test_forward.py
+++ b/tests/python/frontend/tflite/test_forward.py
@@ -3342,9 +3342,9 @@ def test_forward_sparse_to_dense():
 ###
 # Fully Connected
 # ---
-
-
-def _test_fully_connected(tensor_in_sizes, const_input, filter_in_sizes, 
bias_in_size=None):
+def _test_fully_connected(
+tensor_in_sizes, const_input, filter_in_sizes, bias_in_size=None, 
quantized=False
+):
 """ One iteration of fully connected """
 
 total_size_1 = np.prod(tensor_in_sizes)
@@ -3356,11 +3356,11 @@ def _test_fully_connected(tensor_in_sizes, const_input, 
filter_in_sizes, bias_in
 
 # Initializes the input tensor with array containing incrementing
 # numbers from 1.
-data_array = np.arange(1, total_size_1 + 1, dtype=np.float32)
-filter_array = np.arange(1, total_size_2 + 1, dtype=np.float32)
+data_array = np.arange(1, total_size_1 + 1, dtype=np.uint8 if quantized 
else np.float32)
+filter_array = np.arange(1, total_size_2 + 1, dtype=np.uint8 if quantized 
else np.float32)
+in_name = "input"
 
 with tf.Graph().as_default():
-in_name = "input"
 in_data = (
 constant_op.constant(data_array, shape=tensor_in_sizes, 
dtype=np.float32, name=in_name)
 if const_input
@@ -3368,30 +3368,73 @@ def _test_fully_connected(tensor_in_sizes, const_input, 
filter_in_sizes, bias_in
 )
 
 in_filter = constant_op.constant(filter_array, shape=filter_in_sizes, 
dtype=np.float32)
-
-# reshape N H W C into N H*W*C
-in_data_reshape = array_ops.reshape(in_data, [tensor_in_sizes[0], -1])
-
-out = math_ops.mat_mul(in_data_reshape, in_filter)
+data_array = np.reshape(data_array, tensor_in_sizes)
 
 # if we have bias
 if bias_in_size:
 assert bias_in_size[0] == filter_in_sizes[1], "bias and filter 
size are mismatched"
-bias_array = np.arange(1, bi

[GitHub] [tvm] mbaret commented on pull request #7113: [TFLite] Quantized version of unit test for Dense

[GitBox]

mbaret commented on pull request #7113:
URL: https://github.com/apache/tvm/pull/7113#issuecomment-757984123


   Thanks @d-smirnov @siju-samuel 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] mbaret merged pull request #7113: [TFLite] Quantized version of unit test for Dense

[GitBox]

mbaret merged pull request #7113:
URL: https://github.com/apache/tvm/pull/7113


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] mbaret merged pull request #7206: [BYOC][ACL] Depthwise convolution support

[GitBox]

mbaret merged pull request #7206:
URL: https://github.com/apache/tvm/pull/7206


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated: [BYOC][ACL] Depthwise convolution support (#7206)

[mbaret]

This is an automated email from the ASF dual-hosted git repository.

mbaret pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
 new 77e4fd1  [BYOC][ACL] Depthwise convolution support (#7206)
77e4fd1 is described below

commit 77e4fd16bfb8175c83638870af9646d1027f0de7
Author: Dmitriy Smirnov 
AuthorDate: Mon Jan 11 14:26:05 2021 +

[BYOC][ACL] Depthwise convolution support (#7206)

* [BYOC][ACL] Depthwise convolution support

Added support for depthwise convolution. ACL only supports depth-wise 
convolution when kernel size is 3x3 and 5x5 and strides are (1, 1) or (2, 2), 
if this is not the case then fallback to TVM.

Also rework tests to remove non-deterministic trials.

*Compute Library for the Arm Architecture (ACL).
*All credits to Luke Hutton @lhutton1

Change-Id: Ida1f5802a65377b84325edf14a0149242c1af857

* linter

* CHECK -> ICHECK

Co-authored-by: Luke Hutton 
---
 docs/deploy/arm_compute_lib.rst|   8 +-
 python/tvm/relay/op/contrib/arm_compute_lib.py | 109 +++-
 python/tvm/relay/testing/__init__.py   |   6 +-
 .../backend/contrib/arm_compute_lib/codegen.cc |  48 ++--
 src/runtime/contrib/arm_compute_lib/acl_runtime.cc |  69 -
 src/runtime/contrib/arm_compute_lib/acl_utils.cc   |  10 +
 src/runtime/contrib/arm_compute_lib/acl_utils.h|   9 +
 .../contrib/test_arm_compute_lib/infrastructure.py |  42 ---
 .../contrib/test_arm_compute_lib/test_conv2d.py| 283 +
 .../contrib/test_arm_compute_lib/test_dense.py |  83 +++---
 .../contrib/test_arm_compute_lib/test_network.py   |   4 +-
 11 files changed, 450 insertions(+), 221 deletions(-)

diff --git a/docs/deploy/arm_compute_lib.rst b/docs/deploy/arm_compute_lib.rst
index a2eaa5f..5d11241 100644
--- a/docs/deploy/arm_compute_lib.rst
+++ b/docs/deploy/arm_compute_lib.rst
@@ -15,7 +15,7 @@
 specific language governing permissions and limitations
 under the License.
 
-Relay Arm :sup:`®` Compute Library Integration
+Relay Arm:sup:`®` Compute Library Integration
 ==
 **Author**: `Luke Hutton `_
 
@@ -195,12 +195,14 @@ Operator support
 |  |   Simple: nn.conv2d   
  |
 |  |   Composite: nn.pad?, nn.conv2d, nn.bias_add?, 
nn.relu? |
 |  |   
  |
-|  | (only groups = 1 supported)   
  |
+|  | Normal and depth-wise (when kernel is 3x3 or 5x5 and 
strides are 1x1|
+|  | or 2x2) convolution supported. Grouped convolution is 
not supported.|
 
+--+-+
 | qnn.conv2d   | uint8:
  |
 |  |   Composite: nn.pad?, nn.conv2d, nn.bias_add?, 
nn.relu?, qnn.requantize |
 |  |   
  |
-|  | (only groups = 1 supported)   
  |
+|  | Normal and depth-wise (when kernel is 3x3 or 5x5 and 
strides are 1x1|
+|  | or 2x2) convolution supported. Grouped convolution is 
not supported.|
 
+--+-+
 | nn.dense | fp32: 
  |
 |  |   Simple: nn.dense
  |
diff --git a/python/tvm/relay/op/contrib/arm_compute_lib.py 
b/python/tvm/relay/op/contrib/arm_compute_lib.py
index a78ad29..8a03cb1 100644
--- a/python/tvm/relay/op/contrib/arm_compute_lib.py
+++ b/python/tvm/relay/op/contrib/arm_compute_lib.py
@@ -19,12 +19,15 @@
 import numpy as np
 import tvm
 
+from tvm._ffi import register_func
 from tvm.relay.expr import const
 from tvm.relay import transform
 from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.testing.temp_op_attr import TempOpAttr
 
 from ...dataflow_pattern import wildcard, is_op, is_constant, is_expr
 from .register import register_pattern_table
+from ..strategy.generic import is_depthwise_conv2d
 
 
 def is_arm_compute_runtime_enabled():
@@ -71,6 +74,61 @@ def partition_for_arm_compute_lib(mod, params=None):
 return seq(mod)
 
 
+@register_func("relay.ext.arm_compute_lib.optimize")
+def preprocess_module(mod):
+"""
+Pre-process a module containing functions ready for ACL codegen. For now 
we enforce OHWI
+ke

[GitHub] [tvm] mbaret commented on pull request #7206: [BYOC][ACL] Depthwise convolution support

[GitBox]

mbaret commented on pull request #7206:
URL: https://github.com/apache/tvm/pull/7206#issuecomment-757985029


   Thanks @d-smirnov @lhutton1 ! This is now merged.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] sxjscience commented on a change in pull request #7230: [FRONTEND][Mxnet][nuymp] Adding _npi_advanced_indexing_multiple

[GitBox]

sxjscience commented on a change in pull request #7230:
URL: https://github.com/apache/tvm/pull/7230#discussion_r555115290



##
File path: tests/python/frontend/mxnet/test_forward.py
##
@@ -1935,6 +1935,29 @@ def verify(data_shape, axis, use_length, length):
 verify((2, 3, 4), 2, True, np.array([[3, 4, 2], [1, 2, 
1]]).astype("int32"))
 
 
+@pytest.mark.parametrize(
+"data_shape, row_sel, col",
+[
+((5, 7), (0, 1, 2, 3, 4,), 2),
+],
+)
+@pytest.mark.parametrize("dtype", ["float64", "float32"])
+@tvm.testing.parametrize_targets
+@pytest.mark.parametrize("kind", ["graph", "vm", "debug"])
+def test_forward_npi_advanced_indexing_multiple(data_shape, row_sel, col, 
dtype, target, ctx, kind):
+data_np = np.random.uniform(size=data_shape).astype(dtype)
+data = mx.sym.var("data")
+ref_res = mx.np.array(data_np)[row_sel, col]
+
+# TODO need to add the proper symbol operator
+mx_sym = mx.sym.np.(data.as_np_ndarray()[row_sel, col])

Review comment:
   Is it possible to do the following?
   ```python
   row_sel_sym = mx.sym.var('row_sel').as_np_ndarray()
   col_sym = mx.sym.var('col').as_np_ndarray()
   data_sym = mx.sym.var('data').as_np_ndarray()
   mx_sym = data_sym[row_sel_sym, col_sym]
   
   ```
   
   Then, you may pass `mx_sym` to the `relay.frontend.from_mxnet`.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tkonolige commented on pull request #7235: [FIX] Fix make format to work with arbitrary upstream names

[GitBox]

tkonolige commented on pull request #7235:
URL: https://github.com/apache/tvm/pull/7235#issuecomment-758084048


   I've hit problems with not using git-merge-base with git-clang-format in the 
past, but I can't for the life of me remember the specifics (maybe 
git-clang-format fixed these issues?). Without merge-base, git-clang-format is 
doing extra work by trying to format all files changed between merge-base and 
the tip of main. You just don't see any output because the files were already 
correctly formatted. Maybe if clang-format was slow this would an issue?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555209975



##
File path: include/tvm/topi/einsum.h
##
@@ -0,0 +1,930 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file topi/einsum.h
+ * \brief Einstein summation op
+ */
+#ifndef TVM_TOPI_EINSUM_H_
+#define TVM_TOPI_EINSUM_H_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace topi {
+
+using namespace tvm::te;
+using namespace topi::detail;
+
+/*!
+ * \brief Compute the stride of the given shape.
+ *
+ * \param shape for the operation.
+ *
+ * \return the stride of the shape.
+ */
+inline Array GetStride(const Array shape) {
+  size_t ndim = shape.size();
+  int prod = 1;
+  Array stride = Array(ndim, -1);
+  for (int i = ndim - 1; i >= 0; i--) {
+stride.Set(i, if_then_else(shape[i] > 1, prod, 0));
+prod = prod * GetConstInt(shape[i]);
+  }
+  return stride;
+}
+
+/*!
+ * \brief Pad the shape with 1.
+ *
+ * \param shape the input shape to be padded
+ * \param odim the padding size of the objective shape.
+ *
+ * \return the padded shape.
+ */
+inline Array Pad(const Array shape, int odim) {
+  int ndim = shape.size();
+  CHECK_GE(odim, ndim);
+  Array ret(static_cast(odim), 1);
+  for (int idim = 0; idim < ndim; ++idim) {
+ret.Set(idim, shape[idim]);
+  }
+  return ret;
+}
+
+/*!
+ * \brief Parse the subscripts for one operand into an output of 'ndim' labels.
+ *
+ * \param subscripts the subscripts for to be parsed.
+ * \param length subscripts[0: length] represents the current operand.
+ * \param ndim the ndim of current operand.
+ * \param iop the index of the operand.
+ * \param op_labels the parsing result.
+ *For Example:
+ *   subscripts="abbcbc",  ndim=6 -> op_labels=[97, 98, -1, 99, -3, 
-2].
+ *   subscripts="ab...bc", ndim=6 -> op_labels=[97, 98, 0, 0, -3, 99].
+ * \param label_counts Count the number the label appears.
+ * \param min_label Save the minimal label according to ASCII.
+ * \param max_label Save the maximal label according to ASCII.
+ *
+ * \return 0.
+ */
+inline int ParseOperandSubscripts(const char* subscripts, int length, int 
ndim, int iop,
+  char* op_labels, char* label_counts, int* 
min_label,
+  int* max_label) {
+  int i;
+  int idim = 0;
+  int ellipsis = -1;
+
+  /* Process all labels for this operand */
+  for (i = 0; i < length; ++i) {
+int label = subscripts[i];
+
+/* A proper label for an axis. */
+if (label > 0 && isalpha(label)) {
+  /* Check we don't exceed the operator dimensions. */
+  CHECK(idim < ndim) << "einstein sum subscripts string contains "
+ << "too many subscripts for operand " << iop;
+
+  op_labels[idim++] = label;
+  if (label < *min_label) {
+*min_label = label;
+  }
+  if (label > *max_label) {
+*max_label = label;
+  }
+  label_counts[label]++;
+} else if (label == '.') {
+  /* The beginning of the ellipsis. */
+  /* Check it's a proper ellipsis. */
+  CHECK(
+  !(ellipsis != -1 || i + 2 >= length || subscripts[++i] != '.' || 
subscripts[++i] != '.'))
+  << "einstein sum subscripts string contains a "
+  << "'.' that is not part of an ellipsis ('...') "
+  << "in operand " << iop;
+
+  ellipsis = idim;
+} else {
+  CHECK(label == ' ') << "invalid subscript '" << static_cast(label)
+  << "' in einstein sum "
+  << "subscripts string, subscripts must "
+  << "be letters";
+}
+  }
+
+  /* No ellipsis found, labels must match dimensions exactly. */
+  if (ellipsis == -1) {
+CHECK(idim == ndim) << "operand has more dimensions than subscripts "
+<< "given in einstein sum, but no '...' ellipsis "
+<< "provided to broadcast the extra dimensions.";
+  } else if (idim < ndim) {
+/* Ellipsis found, may have to add broadcast dimensions. */
+/* Move labels after

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555210651



##
File path: include/tvm/topi/einsum.h
##
@@ -0,0 +1,930 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file topi/einsum.h
+ * \brief Einstein summation op
+ */
+#ifndef TVM_TOPI_EINSUM_H_
+#define TVM_TOPI_EINSUM_H_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace topi {
+
+using namespace tvm::te;
+using namespace topi::detail;
+
+/*!
+ * \brief Compute the stride of the given shape.
+ *
+ * \param shape for the operation.
+ *
+ * \return the stride of the shape.
+ */
+inline Array GetStride(const Array shape) {
+  size_t ndim = shape.size();
+  int prod = 1;
+  Array stride = Array(ndim, -1);
+  for (int i = ndim - 1; i >= 0; i--) {
+stride.Set(i, if_then_else(shape[i] > 1, prod, 0));
+prod = prod * GetConstInt(shape[i]);
+  }
+  return stride;
+}
+
+/*!
+ * \brief Pad the shape with 1.
+ *
+ * \param shape the input shape to be padded
+ * \param odim the padding size of the objective shape.
+ *
+ * \return the padded shape.
+ */
+inline Array Pad(const Array shape, int odim) {
+  int ndim = shape.size();
+  CHECK_GE(odim, ndim);
+  Array ret(static_cast(odim), 1);
+  for (int idim = 0; idim < ndim; ++idim) {
+ret.Set(idim, shape[idim]);
+  }
+  return ret;
+}
+
+/*!
+ * \brief Parse the subscripts for one operand into an output of 'ndim' labels.
+ *
+ * \param subscripts the subscripts for to be parsed.
+ * \param length subscripts[0: length] represents the current operand.
+ * \param ndim the ndim of current operand.
+ * \param iop the index of the operand.
+ * \param op_labels the parsing result.
+ *For Example:
+ *   subscripts="abbcbc",  ndim=6 -> op_labels=[97, 98, -1, 99, -3, 
-2].
+ *   subscripts="ab...bc", ndim=6 -> op_labels=[97, 98, 0, 0, -3, 99].
+ * \param label_counts Count the number the label appears.
+ * \param min_label Save the minimal label according to ASCII.
+ * \param max_label Save the maximal label according to ASCII.
+ *
+ * \return 0.
+ */
+inline int ParseOperandSubscripts(const char* subscripts, int length, int 
ndim, int iop,
+  char* op_labels, char* label_counts, int* 
min_label,
+  int* max_label) {
+  int i;
+  int idim = 0;
+  int ellipsis = -1;
+
+  /* Process all labels for this operand */
+  for (i = 0; i < length; ++i) {
+int label = subscripts[i];
+
+/* A proper label for an axis. */
+if (label > 0 && isalpha(label)) {
+  /* Check we don't exceed the operator dimensions. */
+  CHECK(idim < ndim) << "einstein sum subscripts string contains "
+ << "too many subscripts for operand " << iop;
+
+  op_labels[idim++] = label;
+  if (label < *min_label) {
+*min_label = label;
+  }
+  if (label > *max_label) {
+*max_label = label;
+  }
+  label_counts[label]++;
+} else if (label == '.') {
+  /* The beginning of the ellipsis. */
+  /* Check it's a proper ellipsis. */
+  CHECK(
+  !(ellipsis != -1 || i + 2 >= length || subscripts[++i] != '.' || 
subscripts[++i] != '.'))
+  << "einstein sum subscripts string contains a "
+  << "'.' that is not part of an ellipsis ('...') "
+  << "in operand " << iop;
+
+  ellipsis = idim;
+} else {
+  CHECK(label == ' ') << "invalid subscript '" << static_cast(label)
+  << "' in einstein sum "
+  << "subscripts string, subscripts must "
+  << "be letters";
+}
+  }
+
+  /* No ellipsis found, labels must match dimensions exactly. */
+  if (ellipsis == -1) {
+CHECK(idim == ndim) << "operand has more dimensions than subscripts "
+<< "given in einstein sum, but no '...' ellipsis "
+<< "provided to broadcast the extra dimensions.";
+  } else if (idim < ndim) {
+/* Ellipsis found, may have to add broadcast dimensions. */
+/* Move labels after

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555211155



##
File path: include/tvm/topi/einsum.h
##
@@ -0,0 +1,930 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file topi/einsum.h
+ * \brief Einstein summation op
+ */
+#ifndef TVM_TOPI_EINSUM_H_
+#define TVM_TOPI_EINSUM_H_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace topi {
+
+using namespace tvm::te;
+using namespace topi::detail;
+
+/*!
+ * \brief Compute the stride of the given shape.
+ *
+ * \param shape for the operation.
+ *
+ * \return the stride of the shape.
+ */
+inline Array GetStride(const Array shape) {
+  size_t ndim = shape.size();
+  int prod = 1;
+  Array stride = Array(ndim, -1);
+  for (int i = ndim - 1; i >= 0; i--) {
+stride.Set(i, if_then_else(shape[i] > 1, prod, 0));
+prod = prod * GetConstInt(shape[i]);
+  }
+  return stride;
+}
+
+/*!
+ * \brief Pad the shape with 1.
+ *
+ * \param shape the input shape to be padded
+ * \param odim the padding size of the objective shape.
+ *
+ * \return the padded shape.
+ */
+inline Array Pad(const Array shape, int odim) {
+  int ndim = shape.size();
+  CHECK_GE(odim, ndim);
+  Array ret(static_cast(odim), 1);
+  for (int idim = 0; idim < ndim; ++idim) {
+ret.Set(idim, shape[idim]);
+  }
+  return ret;
+}
+
+/*!
+ * \brief Parse the subscripts for one operand into an output of 'ndim' labels.
+ *
+ * \param subscripts the subscripts for to be parsed.
+ * \param length subscripts[0: length] represents the current operand.
+ * \param ndim the ndim of current operand.
+ * \param iop the index of the operand.
+ * \param op_labels the parsing result.
+ *For Example:
+ *   subscripts="abbcbc",  ndim=6 -> op_labels=[97, 98, -1, 99, -3, 
-2].
+ *   subscripts="ab...bc", ndim=6 -> op_labels=[97, 98, 0, 0, -3, 99].
+ * \param label_counts Count the number the label appears.
+ * \param min_label Save the minimal label according to ASCII.
+ * \param max_label Save the maximal label according to ASCII.
+ *
+ * \return 0.
+ */
+inline int ParseOperandSubscripts(const char* subscripts, int length, int 
ndim, int iop,
+  char* op_labels, char* label_counts, int* 
min_label,
+  int* max_label) {
+  int i;
+  int idim = 0;
+  int ellipsis = -1;
+
+  /* Process all labels for this operand */
+  for (i = 0; i < length; ++i) {
+int label = subscripts[i];
+
+/* A proper label for an axis. */
+if (label > 0 && isalpha(label)) {
+  /* Check we don't exceed the operator dimensions. */
+  CHECK(idim < ndim) << "einstein sum subscripts string contains "
+ << "too many subscripts for operand " << iop;
+
+  op_labels[idim++] = label;
+  if (label < *min_label) {
+*min_label = label;
+  }
+  if (label > *max_label) {
+*max_label = label;
+  }
+  label_counts[label]++;
+} else if (label == '.') {
+  /* The beginning of the ellipsis. */
+  /* Check it's a proper ellipsis. */
+  CHECK(
+  !(ellipsis != -1 || i + 2 >= length || subscripts[++i] != '.' || 
subscripts[++i] != '.'))
+  << "einstein sum subscripts string contains a "
+  << "'.' that is not part of an ellipsis ('...') "
+  << "in operand " << iop;
+
+  ellipsis = idim;
+} else {
+  CHECK(label == ' ') << "invalid subscript '" << static_cast(label)
+  << "' in einstein sum "
+  << "subscripts string, subscripts must "
+  << "be letters";
+}
+  }
+
+  /* No ellipsis found, labels must match dimensions exactly. */
+  if (ellipsis == -1) {
+CHECK(idim == ndim) << "operand has more dimensions than subscripts "
+<< "given in einstein sum, but no '...' ellipsis "
+<< "provided to broadcast the extra dimensions.";
+  } else if (idim < ndim) {
+/* Ellipsis found, may have to add broadcast dimensions. */
+/* Move labels after

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555211434



##
File path: include/tvm/topi/einsum.h
##
@@ -0,0 +1,930 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file topi/einsum.h
+ * \brief Einstein summation op
+ */
+#ifndef TVM_TOPI_EINSUM_H_
+#define TVM_TOPI_EINSUM_H_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace topi {
+
+using namespace tvm::te;
+using namespace topi::detail;
+
+/*!
+ * \brief Compute the stride of the given shape.
+ *
+ * \param shape for the operation.
+ *
+ * \return the stride of the shape.
+ */
+inline Array GetStride(const Array shape) {
+  size_t ndim = shape.size();
+  int prod = 1;
+  Array stride = Array(ndim, -1);
+  for (int i = ndim - 1; i >= 0; i--) {
+stride.Set(i, if_then_else(shape[i] > 1, prod, 0));
+prod = prod * GetConstInt(shape[i]);
+  }
+  return stride;
+}
+
+/*!
+ * \brief Pad the shape with 1.
+ *
+ * \param shape the input shape to be padded
+ * \param odim the padding size of the objective shape.
+ *
+ * \return the padded shape.
+ */
+inline Array Pad(const Array shape, int odim) {
+  int ndim = shape.size();
+  CHECK_GE(odim, ndim);
+  Array ret(static_cast(odim), 1);
+  for (int idim = 0; idim < ndim; ++idim) {
+ret.Set(idim, shape[idim]);
+  }
+  return ret;
+}
+
+/*!
+ * \brief Parse the subscripts for one operand into an output of 'ndim' labels.
+ *
+ * \param subscripts the subscripts for to be parsed.
+ * \param length subscripts[0: length] represents the current operand.
+ * \param ndim the ndim of current operand.
+ * \param iop the index of the operand.
+ * \param op_labels the parsing result.
+ *For Example:
+ *   subscripts="abbcbc",  ndim=6 -> op_labels=[97, 98, -1, 99, -3, 
-2].
+ *   subscripts="ab...bc", ndim=6 -> op_labels=[97, 98, 0, 0, -3, 99].
+ * \param label_counts Count the number the label appears.
+ * \param min_label Save the minimal label according to ASCII.
+ * \param max_label Save the maximal label according to ASCII.
+ *
+ * \return 0.
+ */
+inline int ParseOperandSubscripts(const char* subscripts, int length, int 
ndim, int iop,
+  char* op_labels, char* label_counts, int* 
min_label,
+  int* max_label) {
+  int i;
+  int idim = 0;
+  int ellipsis = -1;
+
+  /* Process all labels for this operand */
+  for (i = 0; i < length; ++i) {
+int label = subscripts[i];
+
+/* A proper label for an axis. */
+if (label > 0 && isalpha(label)) {
+  /* Check we don't exceed the operator dimensions. */
+  CHECK(idim < ndim) << "einstein sum subscripts string contains "
+ << "too many subscripts for operand " << iop;
+
+  op_labels[idim++] = label;
+  if (label < *min_label) {
+*min_label = label;
+  }
+  if (label > *max_label) {
+*max_label = label;
+  }
+  label_counts[label]++;
+} else if (label == '.') {
+  /* The beginning of the ellipsis. */
+  /* Check it's a proper ellipsis. */
+  CHECK(
+  !(ellipsis != -1 || i + 2 >= length || subscripts[++i] != '.' || 
subscripts[++i] != '.'))
+  << "einstein sum subscripts string contains a "
+  << "'.' that is not part of an ellipsis ('...') "
+  << "in operand " << iop;
+
+  ellipsis = idim;
+} else {
+  CHECK(label == ' ') << "invalid subscript '" << static_cast(label)
+  << "' in einstein sum "
+  << "subscripts string, subscripts must "
+  << "be letters";
+}
+  }
+
+  /* No ellipsis found, labels must match dimensions exactly. */
+  if (ellipsis == -1) {
+CHECK(idim == ndim) << "operand has more dimensions than subscripts "
+<< "given in einstein sum, but no '...' ellipsis "
+<< "provided to broadcast the extra dimensions.";
+  } else if (idim < ndim) {
+/* Ellipsis found, may have to add broadcast dimensions. */
+/* Move labels after

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555211652



##
File path: include/tvm/topi/einsum.h
##
@@ -0,0 +1,930 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file topi/einsum.h
+ * \brief Einstein summation op
+ */
+#ifndef TVM_TOPI_EINSUM_H_
+#define TVM_TOPI_EINSUM_H_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+namespace tvm {
+namespace topi {
+
+using namespace tvm::te;
+using namespace topi::detail;
+
+/*!
+ * \brief Compute the stride of the given shape.
+ *
+ * \param shape for the operation.
+ *
+ * \return the stride of the shape.
+ */
+inline Array GetStride(const Array shape) {
+  size_t ndim = shape.size();
+  int prod = 1;
+  Array stride = Array(ndim, -1);
+  for (int i = ndim - 1; i >= 0; i--) {
+stride.Set(i, if_then_else(shape[i] > 1, prod, 0));
+prod = prod * GetConstInt(shape[i]);
+  }
+  return stride;
+}
+
+/*!
+ * \brief Pad the shape with 1.
+ *
+ * \param shape the input shape to be padded
+ * \param odim the padding size of the objective shape.
+ *
+ * \return the padded shape.
+ */
+inline Array Pad(const Array shape, int odim) {
+  int ndim = shape.size();
+  CHECK_GE(odim, ndim);
+  Array ret(static_cast(odim), 1);
+  for (int idim = 0; idim < ndim; ++idim) {
+ret.Set(idim, shape[idim]);
+  }
+  return ret;
+}
+
+/*!
+ * \brief Parse the subscripts for one operand into an output of 'ndim' labels.
+ *
+ * \param subscripts the subscripts for to be parsed.
+ * \param length subscripts[0: length] represents the current operand.
+ * \param ndim the ndim of current operand.
+ * \param iop the index of the operand.
+ * \param op_labels the parsing result.
+ *For Example:
+ *   subscripts="abbcbc",  ndim=6 -> op_labels=[97, 98, -1, 99, -3, 
-2].
+ *   subscripts="ab...bc", ndim=6 -> op_labels=[97, 98, 0, 0, -3, 99].
+ * \param label_counts Count the number the label appears.
+ * \param min_label Save the minimal label according to ASCII.
+ * \param max_label Save the maximal label according to ASCII.
+ *
+ * \return 0.
+ */
+inline int ParseOperandSubscripts(const char* subscripts, int length, int 
ndim, int iop,
+  char* op_labels, char* label_counts, int* 
min_label,
+  int* max_label) {
+  int i;
+  int idim = 0;
+  int ellipsis = -1;
+
+  /* Process all labels for this operand */
+  for (i = 0; i < length; ++i) {
+int label = subscripts[i];
+
+/* A proper label for an axis. */
+if (label > 0 && isalpha(label)) {
+  /* Check we don't exceed the operator dimensions. */
+  CHECK(idim < ndim) << "einstein sum subscripts string contains "
+ << "too many subscripts for operand " << iop;
+
+  op_labels[idim++] = label;
+  if (label < *min_label) {
+*min_label = label;
+  }
+  if (label > *max_label) {
+*max_label = label;
+  }
+  label_counts[label]++;
+} else if (label == '.') {
+  /* The beginning of the ellipsis. */
+  /* Check it's a proper ellipsis. */
+  CHECK(
+  !(ellipsis != -1 || i + 2 >= length || subscripts[++i] != '.' || 
subscripts[++i] != '.'))
+  << "einstein sum subscripts string contains a "
+  << "'.' that is not part of an ellipsis ('...') "
+  << "in operand " << iop;
+
+  ellipsis = idim;
+} else {
+  CHECK(label == ' ') << "invalid subscript '" << static_cast(label)
+  << "' in einstein sum "
+  << "subscripts string, subscripts must "
+  << "be letters";
+}
+  }
+
+  /* No ellipsis found, labels must match dimensions exactly. */
+  if (ellipsis == -1) {
+CHECK(idim == ndim) << "operand has more dimensions than subscripts "
+<< "given in einstein sum, but no '...' ellipsis "
+<< "provided to broadcast the extra dimensions.";
+  } else if (idim < ndim) {
+/* Ellipsis found, may have to add broadcast dimensions. */
+/* Move labels after

[GitHub] [tvm] hanke580 commented on a change in pull request #6370: [TOPI] Add einsum operator

[GitBox]

hanke580 commented on a change in pull request #6370:
URL: https://github.com/apache/tvm/pull/6370#discussion_r555211968



##
File path: python/tvm/topi/transform.py
##
@@ -502,6 +502,14 @@ def tensordot(a, b, axes):
 return cpp.tensordot(a, b, axes[0], axes[1])
 
 
+def einsum(subscripts, a_tuple):

Review comment:
   Created einsum.py in topi for this op. Thx!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tmoreau89 commented on pull request #6126: [VTA][OpenCL] intelfocl

[GitBox]

tmoreau89 commented on pull request #6126:
URL: https://github.com/apache/tvm/pull/6126#issuecomment-758111954


   @zhanghaohit can you update this PR to update the VTA submodule to the 
latest? Currently other PRs that update the VTA submodule will break unless we 
merge this PR. Thanks!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tmoreau89 closed pull request #7219: [VTA] update vta-hw to the latest

[GitBox]

tmoreau89 closed pull request #7219:
URL: https://github.com/apache/tvm/pull/7219


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tmoreau89 commented on pull request #7219: [VTA] update vta-hw to the latest

[GitBox]

tmoreau89 commented on pull request #7219:
URL: https://github.com/apache/tvm/pull/7219#issuecomment-758112682


   Updating the vta submodule is blocked on 
https://github.com/apache/tvm/pull/6126. I suggest we push for this PR (6126) 
to land and close this one.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tkonolige opened a new pull request #7248: [FIX,TUTORIALS] Import tvm.testing in tutorials that use it

[GitBox]

tkonolige opened a new pull request #7248:
URL: https://github.com/apache/tvm/pull/7248


   Fixes the issue encountered here: 
https://discuss.tvm.apache.org/t/attributeerror-module-tvm-has-no-attribute-testing/8834



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] codeislife99 opened a new pull request #7249: Add SampleOpCorrect

[GitBox]

codeislife99 opened a new pull request #7249:
URL: https://github.com/apache/tvm/pull/7249


   Thanks for contributing to TVM!   Please refer to guideline 
https://tvm.apache.org/docs/contribute/ for useful information and tips. After 
the pull request is submitted, please request code reviews from 
[Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers)
 by @ them in the pull request thread.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch opened a new pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

areusch opened a new pull request #7250:
URL: https://github.com/apache/tvm/pull/7250


   In porting up the µTVM AutoTVM code, seems that `check_correctness` assumes 
that each AutoTVM task is instantiated on the `llvm` builder with the same 
input shape as is used on the target. Even in tuning a conv2d for x86, I see 
conv2d_NCHWc used with AutoTVM trying different combinations of `C` and `c`. I 
don't think this approach is viable and from chatting with a few people it 
seems like it was a debug tool that might just be deprecated now. I propose we 
remove until we have something more dependable via AutoScheduling or AutoTIR.
   
   @merrymercy @tqchen @jwfromm @junrushao1994 @tmoreau89 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] merrymercy commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

merrymercy commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758212157


   Although it is buggy for x86 cpu.
   This feature is still useful for some targets like metal according to 
@antinucleon .
   CUDA runtime always throws runtime errors if the GPU kernels are invalid but 
metal runtime may just return wrong results without any runtime error.  So we 
need this check_correctness for metal.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] merrymercy edited a comment on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

merrymercy edited a comment on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758212157


   Although it is buggy for x86 cpu, this feature is still useful for some 
targets like metal according to @antinucleon .
   CUDA runtime always throws runtime errors if the GPU kernels are invalid but 
metal runtime may just return wrong results without any runtime error.  So we 
need this check_correctness for metal.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] d-smirnov opened a new pull request #7251: [BYOC][ACL] removed ACL 20.05 limitations

[GitBox]

d-smirnov opened a new pull request #7251:
URL: https://github.com/apache/tvm/pull/7251


   This PR removes checks for padding in according with changes in ACL 20.11 
which currently does not require padded data.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] merrymercy edited a comment on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

merrymercy edited a comment on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758212157


   Although it is buggy for x86 cpu, this feature is still useful for some 
targets like metal according to @antinucleon .
   CUDA runtime always throws runtime errors if the GPU kernels are invalid, 
but metal runtime may just return wrong results without any runtime error.  So 
we need this check_correctness for metal.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] d-smirnov commented on pull request #7251: [BYOC][ACL] removed ACL 20.05 limitations

[GitBox]

d-smirnov commented on pull request #7251:
URL: https://github.com/apache/tvm/pull/7251#issuecomment-758212449


   @mbaret 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

areusch commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758217721


   @merrymercy @antinucleon should this maybe be done instead by overriding 
some part of Runner? I don't think we should keep it as a generic mechanism in 
the API if it's only useful for 1 backend. at minimum, we need to provide a 
warning or add more checks to verify that we are testing the right thing when 
we do `check_correctness`. I spent several hours trying to debug this because i 
thought i'd broken it.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] codeislife99 opened a new issue #7252: [BUG][LLVM-Codegen] TVMError: Do not have a default for tir.ProducerLoad

[GitBox]

codeislife99 opened a new issue #7252:
URL: https://github.com/apache/tvm/issues/7252


   `ProducerLoadNode` seems to be included in 
`https://github.com/apache/tvm/blob/main/include/tvm/tir/expr_functor.h#L122` 
yet when I run [this PR](https://github.com/apache/tvm/pull/7249/files) using 
`python3 test_dynamic_op_level3.py` I get the following error: 
   
   ```
 File "test_dynamic_op_level3.py", line 34, in verify_func
   intrp = relay.create_executor(kind, mod=mod, ctx=ctx, target=target)
 File "/home/ubuntu/tvm/python/tvm/relay/build_module.py", line 459, in 
create_executor
   return VMExecutor(mod, ctx, target)
 File "/home/ubuntu/tvm/python/tvm/relay/backend/vm.py", line 258, in 
__init__
   self.executable = compile(mod, target)
 File "/home/ubuntu/tvm/python/tvm/relay/backend/vm.py", line 70, in compile
   compiler.codegen()
 File "/home/ubuntu/tvm/python/tvm/relay/backend/vm.py", line 140, in 
codegen
   self._codegen()
 File "/home/ubuntu/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, 
in __call__
   raise get_last_ffi_error()
   tvm._ffi.base.TVMError: Traceback (most recent call last):
 [bt] (8) 
/home/ubuntu/tvm/build/libtvm.so(tvm::codegen::CodeGenCPU::VisitStmt_(tvm::tir::AssertStmtNode
 const*)+0x374) [0x7f310e6fd254]
 [bt] (7) 
/home/ubuntu/tvm/build/libtvm.so(tvm::codegen::CodeGenLLVM::VisitStmt_(tvm::tir::AssertStmtNode
 const*)+0x76) [0x7f310e710596]
 [bt] (6) /home/ubuntu/tvm/build/libtvm.so(tvm::tir::StmtFunctor::VisitStmt(tvm::tir::Stmt const&)+0x7f) 
[0x7f310da8a93f]
 [bt] (5) 
/home/ubuntu/tvm/build/libtvm.so(tvm::codegen::CodeGenCPU::VisitStmt_(tvm::tir::AssertStmtNode
 const*)+0x3d) [0x7f310e6fcf1d]
 [bt] (4) 
/home/ubuntu/tvm/build/libtvm.so(tvm::tir::ExprFunctor::VisitExpr(tvm::PrimExpr const&)+0x75) [0x7f310e6f4565]
 [bt] (3) 
/home/ubuntu/tvm/build/libtvm.so(tvm::codegen::CodeGenLLVM::VisitExpr_(tvm::tir::EQNode
 const*)+0x28) [0x7f310e712978]
 [bt] (2) 
/home/ubuntu/tvm/build/libtvm.so(tvm::tir::ExprFunctor::VisitExpr(tvm::PrimExpr const&)+0x75) [0x7f310e6f4565]
 [bt] (1) 
/home/ubuntu/tvm/build/libtvm.so(tvm::tir::ExprFunctor::VisitExprDefault_(tvm::runtime::Object const*)+0x87) 
[0x7f310e6ed367]
 [bt] (0) /home/ubuntu/tvm/build/libtvm.so(+0x11d8be2) [0x7f310e6ecbe2]
 File "/home/ubuntu/tvm/include/tvm/tir/expr_functor.h", line 155
   TVMError: Do not have a default for tir.ProducerLoad 
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] trevor-m opened a new pull request #7253: [BYOC][TRT] Fix weight conversion when first dim of weight shape is 1

[GitBox]

trevor-m opened a new pull request #7253:
URL: https://github.com/apache/tvm/pull/7253


   Fixes conversion of weight shape to work with TensorRT's implicit batch 
dimension. 
   Added test cases which failed before this change.
   Before this PR, a conv2d or dense op which has a kernel weight beginning 
with 1 (`[1, 32, 3, 3]` or `[1, 128]` for example) would cause an error.
   
   Also, there is an unrelated change to add conv2d_transpose to ConvertLayout 
list in partition_for_tensorrt.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated (77e4fd1 -> 72c9a51)

[comaniac]

This is an automated email from the ASF dual-hosted git repository.

comaniac pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git.


from 77e4fd1  [BYOC][ACL] Depthwise convolution support (#7206)
 add 72c9a51  [FIX,TUTORIALS] Import tvm.testing in tutorials that use it 
(#7248)

No new revisions were added by this update.

Summary of changes:
 tutorials/autotvm/tune_conv2d_cuda.py| 1 +
 tutorials/autotvm/tune_simple_template.py| 3 ++-
 tutorials/frontend/using_external_lib.py | 1 +
 tutorials/get_started/relay_quick_start.py   | 1 +
 tutorials/language/extern_op.py  | 1 +
 tutorials/language/tensorize.py  | 1 +
 tutorials/optimize/opt_matmul_auto_tensorcore.py | 1 +
 7 files changed, 8 insertions(+), 1 deletion(-)

[GitHub] [tvm] comaniac commented on pull request #7248: [FIX,TUTORIALS] Import tvm.testing in tutorials that use it

[GitBox]

comaniac commented on pull request #7248:
URL: https://github.com/apache/tvm/pull/7248#issuecomment-758251983


   Thanks @tkonolige @tqchen 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] comaniac merged pull request #7248: [FIX,TUTORIALS] Import tvm.testing in tutorials that use it

[GitBox]

comaniac merged pull request #7248:
URL: https://github.com/apache/tvm/pull/7248


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] gromero opened a new pull request #7254: [µTVM] Add ST STM32F746 disco board to tflite tutorial script

[GitBox]

gromero opened a new pull request #7254:
URL: https://github.com/apache/tvm/pull/7254


   Currently tutorial script 'micro_tflite.py' assumes all boards with target
   STM32F746 are the same. As a consequence once that target is selected the
   script automatically defaults to the Nucleo board. However, the STM32F746
   is also used on Discovery Kit boards (aka disco) which are quite similar
   but have some differences, so Nucleo config and final image don't work on
   the disco boards.
   
   That commit adds a way to select a different dev board and adds comments
   accordingly, informing how to use the script with the disco boards.
   
   Signed-off-by: Gustavo Romero 
   
   Thanks for contributing to TVM!   Please refer to guideline 
https://tvm.apache.org/docs/contribute/ for useful information and tips. After 
the pull request is submitted, please request code reviews from 
[Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers)
 by @ them in the pull request thread.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] gromero commented on pull request #7254: [µTVM] Add ST STM32F746 disco board to tflite tutorial script

[GitBox]

gromero commented on pull request #7254:
URL: https://github.com/apache/tvm/pull/7254#issuecomment-758277449


   @areusch Would you mind to review it? It's about our discussion last week 
about adding disco boards to the tflite tutorial as well.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on issue #7252: [BUG][LLVM-Codegen] TVMError: Do not have a default for tir.ProducerLoad

[GitBox]

tqchen commented on issue #7252:
URL: https://github.com/apache/tvm/issues/7252#issuecomment-758279309


   ProducerLoad is expected to be lowered to BufferLoad, during TE lowering and 
we do not expect to see it in a valid TIR.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] gromero edited a comment on pull request #7254: [µTVM] Add ST STM32F746 disco board to tflite tutorial script

[GitBox]

gromero edited a comment on pull request #7254:
URL: https://github.com/apache/tvm/pull/7254#issuecomment-758277449


   @areusch Would you mind to review it? It's about our discussion last week 
about adding disco boards to the tflite tutorial as well. It depends on #7225



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

tqchen commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758288934


   How about we consider add an warning when hitting this option for now 
stating that it does not work for cases with layout transform.
   
   In some sense, tuning the layout transform together with the autoTVM routine 
is stretching the original AutoTVM's capability (since the kernel is no longer 
equivalent). So that was an unconventional use of autotvm.  In other cases, 
setting a higher `tol` usually works.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

areusch commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758292075


   I would be okay with that except that this was the network I tried to tune:
   
   ```
   model = keras.models.Sequential()
   model.add(keras.layers.Conv2D(2,  3, input_shape=(64, 64, 3)))
   model.build()
   ```
   
   I could be wrong but I think this is a pretty common, basic use case. with 
conv2d_NCHWc schedule, it doesn't seem like the warning case is an edge case 
(other than that perhaps we consider `check_correctness` to be an edge 
case--but...I would push back pretty strongly that a new developer is going to 
think: "why would I not want this?" and turn it on).
   
   is there an easy way we can detect which operator a layout transformation 
belongs to in task extraction maybe?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch edited a comment on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

areusch edited a comment on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758292075


   I would be okay with that except that this was the network I tried to tune:
   
   ```
   model = keras.models.Sequential()
   model.add(keras.layers.Conv2D(2,  3, input_shape=(64, 64, 3)))
   model.build()
   ```
   
   I could be wrong but I think this is a pretty common, basic use case. with 
conv2d_NCHWc schedule on by default, it doesn't seem like the warning case is 
an edge case (other than that perhaps we consider `check_correctness` to be an 
edge case--but...I would push back pretty strongly that a new developer is 
going to think: "why would I not want this?" and turn it on).
   
   is there an easy way we can detect which operator a layout transformation 
belongs to in task extraction maybe?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

tqchen commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758300529


   OK, given such verification is not yet done in autoscheduling and the 
limited usecase. I agree that we can remove it for now. For specific backend, 
we could add optional callbacks that have those correctness checks later. Let 
us keep this thread open this thread for another day



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7250: Remove check_correctness in AutoTVM, which is busted

[GitBox]

areusch commented on pull request #7250:
URL: https://github.com/apache/tvm/pull/7250#issuecomment-758308337


   yes also to be clear I'm very open to other suggestions...it just seems like 
if we are keeping this top-level API option checked-in when it only works on 
metal and maybe vta, we should either add detection for layout transforms or 
only enable it for those targets or maybe move it into e.g. a Runner subclass. 
I do think we should avoid putting this as a kwarg on the generic Runner since 
it doesn't work for many cases.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] ptrendx opened a new pull request #7255: Do not use ICHECK in nnvm

[GitBox]

ptrendx opened a new pull request #7255:
URL: https://github.com/apache/tvm/pull/7255


   This PR reverts the changes from #6758 for NNVM subdirectory, since NNVM is 
separate project and does not know anything about the rest of TVM (including 
the ICHECK macro).
   
   Thanks for contributing to TVM!   Please refer to guideline 
https://tvm.apache.org/docs/contribute/ for useful information and tips. After 
the pull request is submitted, please request code reviews from 
[Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers)
 by @ them in the pull request thread.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] ptrendx commented on pull request #6758: More CHECK to ICHECK

[GitBox]

ptrendx commented on pull request #6758:
URL: https://github.com/apache/tvm/pull/6758#issuecomment-758317229


   @tqchen Thanks, I just did (#7255).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] trevor-m commented on pull request #7154: [Torch] Restore class-aware NMS for detection models by graph rewrite

[GitBox]

trevor-m commented on pull request #7154:
URL: https://github.com/apache/tvm/pull/7154#issuecomment-758329448


   I highly agree with you guys.
   
   For class-aware NMS, the [batch, num_anchors, 6] format seems very 
inefficient. It means all anchors need to be checked just to see if the classes 
match. A [batch, num_classes, num_anchors, 5] format would give us a nicely 
defined slice of memory where the same-class anchors are located.
   
   > TF takes a 1D tensor of scores and concats it to the boxes before 
performing get_valid_counts and nms. I'm not sure if the rest of the TF graph 
is handling the loop over batch size and classes.
   
   That's correct, TF's NMS is only for single class and single batch, so the 
TF graph loops over batches and classes. To do that, they use 
[tf.map_fn](https://www.tensorflow.org/api_docs/python/tf/map_fn) so the 
execution of each NMS can actually still run in parallel. However, this turns 
into a mess of control flow operators and TensorArrays, so Relay isn't able to 
do the same parallelization. This PR's graph rewrite could actually benefit TF 
OD models as well, but the pattern is a lot more complicated for TF.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] trevor-m commented on pull request #7253: [BYOC][TRT] Fix weight conversion when first dim of weight shape is 1

[GitBox]

trevor-m commented on pull request #7253:
URL: https://github.com/apache/tvm/pull/7253#issuecomment-758338084


   > LGTM
   
   @comaniac Thanks for the review!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] jwfromm commented on pull request #7243: Add op_name in error message for Pool

[GitBox]

jwfromm commented on pull request #7243:
URL: https://github.com/apache/tvm/pull/7243#issuecomment-758338609


   More info in error messages is always good. LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] jwfromm commented on pull request #7242: [Relay][Frontend][Onnx] Set default value of p in LpPool as 2

[GitBox]

jwfromm commented on pull request #7242:
URL: https://github.com/apache/tvm/pull/7242#issuecomment-758340015


   Can you add a test that would trigger the error? Once that's included in 
this PR it should be good to go. Thanks for identifying and fixing this.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] zhanghaohit commented on pull request #6126: [VTA][OpenCL] intelfocl

[GitBox]

zhanghaohit commented on pull request #6126:
URL: https://github.com/apache/tvm/pull/6126#issuecomment-758342566


   > @zhanghaohit can you update this PR to update the VTA submodule to the 
latest? Currently other PRs that update the VTA submodule will break unless we 
merge this PR. Thanks!
   
   Sure. Apology for the delay. I think there are some unit tests failed for 
this PR. I'll try to fix it.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #7195: [THRUST] Faster multi dimensional argsort by segmented sort

[GitBox]

masahi commented on pull request #7195:
URL: https://github.com/apache/tvm/pull/7195#issuecomment-758343079


   @Laurawly @kazum This is ready to go, please have a look



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on pull request #7146: [CUDA]batch_matmul tensorcore schedule

[GitBox]

tqchen commented on pull request #7146:
URL: https://github.com/apache/tvm/pull/7146#issuecomment-758346419


   @jcf94 @Meteorix @jwfromm because our TOPI test stage does not gaurantee 
uses the tensorcore GPU(we had two pascal GPUs), it would be useful to 
optionally skip it, to avoid flaky CI error on the main. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #7237: [ONNX] Fix issues for Clip and RoiAlign

[GitBox]

masahi commented on pull request #7237:
URL: https://github.com/apache/tvm/pull/7237#issuecomment-758357315


   Maybe you can tell us what the issue was.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] huanleo opened a new issue #7256: [bug] memory leak, whether on cpu or gpu，after loaded and then release the model

[GitBox]

huanleo opened a new issue #7256:
URL: https://github.com/apache/tvm/issues/7256


   Thanks for participating in the TVM community! We use https://discuss.tvm.ai 
for any general usage questions and discussions. The issue tracker is used for 
actionable items such as feature proposals discussion, roadmaps, and bug 
tracking.  You are always welcomed to post on the forum first :)
   
   Issues that are inactive for a period of time may get closed. We adopt this 
policy so that we won't lose track of actionable issues that may fall at the 
bottom of the pile. Feel free to reopen a new one if you feel there is an 
additional problem that needs attention when an old one gets closed.
   
   For bug reports, to help the developer act on the issues, please include a 
description of your environment, preferably a minimum script to reproduce the 
problem.
   
   For feature proposals, list clear, small actionable items so we can track 
the progress of the change.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] masahi opened a new pull request #7257: Unpack NMS inputs into bbox, scores and class ids

[GitBox]

masahi opened a new pull request #7257:
URL: https://github.com/apache/tvm/pull/7257


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] aaltonenzhang opened a new issue #7258: tvm doesn't support mix-precision inputs for qnn conv2d

[GitBox]

aaltonenzhang opened a new issue #7258:
URL: https://github.com/apache/tvm/issues/7258


   I have a tflite model with float32 feature map and int8 weights, when I look 
into convert_conv() in tflite frontend, I found that _qnn.op.conv2d is 
constructed only if feature map is int8(uint8) typed. So, I tried to modify 
code like:
   
   > 
   if (input_tensor.qnn_params or weight_tensor.qnn_params):
   qnn_conv2d_params = dict(params)
   qnn_conv2d_params["input_zero_point"] = 
input_tensor.qnn_params["zero_point"] if input_tensor.qnn_params else 
relay.const(0, "int32")
   qnn_conv2d_params["kernel_zero_point"] = 
weight_tensor.qnn_params["zero_point"] if weight_tensor.qnn_params else 
relay.const(0, "int32")
   qnn_conv2d_params["out_dtype"] = "int32" if 
(input_tensor.qnn_params and weight_tensor.qnn_params) else "float32"
   qnn_conv2d_params["input_scale"] = 
input_tensor.qnn_params["scale"] if input_tensor.qnn_params else relay.const(1, 
"float32")
   qnn_conv2d_params["kernel_scale"] = 
weight_tensor.qnn_params["scale"] if weight_tensor.qnn_params else 
relay.const(1, "float32")
   out = _qnn.op.conv2d(in_expr, weight_expr, **qnn_conv2d_params)
   else:
   out = _op.nn.conv2d(in_expr, weight_expr, **params)
   
   And I found that another check in QnnConv2DRel() has failed: "Expected qnn 
conv2d type(int8, uint8) for input but was float32".
   Is it ok to simply modify code like I mentioned above and is it ok to ignore 
the ICHECK()? I don't know are there any other inner constraints for the 
mix-precision issue, and I hope somebody could support this feature officially. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] xutianming opened a new pull request #7259: [ONNX Frontend] add default value for leaky relu alpha

[GitBox]

xutianming opened a new pull request #7259:
URL: https://github.com/apache/tvm/pull/7259


   Fix Issue #7244 
   
   cc @luyaor @masahi @mbrookhart 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] huanleo closed issue #7256: [bug] memory leak, whether on cpu or gpu，after loaded and then release the model

[GitBox]

huanleo closed issue #7256:
URL: https://github.com/apache/tvm/issues/7256


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] huanleo commented on issue #7256: [bug] memory leak, whether on cpu or gpu，after loaded and then release the model

[GitBox]

huanleo commented on issue #7256:
URL: https://github.com/apache/tvm/issues/7256#issuecomment-758423973


   my faulty，sorry



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] luyaor commented on pull request #7259: [ONNX Frontend] add default value for leaky relu alpha

[GitBox]

luyaor commented on pull request #7259:
URL: https://github.com/apache/tvm/pull/7259#issuecomment-758438231


   Thanks for @xutianming, LGTM.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] lixiaoquan commented on pull request #7237: [ONNX] Fix issues for Clip and RoiAlign

[GitBox]

lixiaoquan commented on pull request #7237:
URL: https://github.com/apache/tvm/pull/7237#issuecomment-758478199


   > Maybe you can tell us what the issue was.
   
   For RoiAlign, 1) attr.get(“mode", "avg”) will return a string "avg" as 
default value, which always fails the following test `if avg == b"avg"` 2) 
infer_type(rois).type_annotation only works when rois is variable, but it could 
be CallNode
   
   For Clip  1) It is possible "min"/"max" don't exist in attrs 
   




This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org