[GitHub] [tvm] merrymercy commented on a change in pull request #6987: [AutoScheduler] Support layout rewrite for whole networks

2020-12-01 Thread GitBox 


merrymercy commented on a change in pull request #6987:
URL: https://github.com/apache/tvm/pull/6987#discussion_r533939309



##
File path: include/tvm/ir/transform.h
##
@@ -197,6 +197,13 @@ class PassContext : public ObjectRef {
*/
   TVM_DLL void Trace(const IRModule& module, const PassInfo& info, bool 
is_before) const;
 
+  /*!
+   * \brief Check if a pass is enabled.

Review comment:
   ```suggestion
  * \brief Check whether a pass is enabled.
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] anijain2305 commented on pull request #7011: [BYOC][TRT] Use channels attribute in Conv2D op converter

2020-12-01 Thread GitBox 


anijain2305 commented on pull request #7011:
URL: https://github.com/apache/tvm/pull/7011#issuecomment-737013165


   Thanks @trevor-m This is merged.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated: Use channels from attrs if possible (#7011)

2020-12-01 Thread anijain2305 
This is an automated email from the ASF dual-hosted git repository.

anijain2305 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
 new 0778afd  Use channels from attrs if possible (#7011)
0778afd is described below

commit 0778afd6d0fb0283fba5d4839f27e2ac548a3284
Author: Trevor Morris 
AuthorDate: Tue Dec 1 22:04:43 2020 -0800

Use channels from attrs if possible (#7011)
---
 src/runtime/contrib/tensorrt/tensorrt_ops.cc | 4 
 tests/python/contrib/test_tensorrt.py| 5 +
 2 files changed, 9 insertions(+)

diff --git a/src/runtime/contrib/tensorrt/tensorrt_ops.cc 
b/src/runtime/contrib/tensorrt/tensorrt_ops.cc
index 057743c..c3ff1c4 100644
--- a/src/runtime/contrib/tensorrt/tensorrt_ops.cc
+++ b/src/runtime/contrib/tensorrt/tensorrt_ops.cc
@@ -243,6 +243,10 @@ class Conv2DOpConverter : public TensorRTOpConverter {
 auto str_padding = 
params->node.GetAttr>("padding");
 int groups = 
std::stoi(params->node.GetAttr>("groups")[0]);
 int channels = weight_shape[0];
+if (params->node.HasAttr("channels") &&
+
!params->node.GetAttr>("channels")[0].empty()) {
+  channels = 
std::stoi(params->node.GetAttr>("channels")[0]);
+}
 // TRT conv2d op doesn't support asymmetric padding before 5.1, so we
 // workaround by adding a padding layer before the pooling op.
 nvinfer1::DimsHW prepadding, postpadding;
diff --git a/tests/python/contrib/test_tensorrt.py 
b/tests/python/contrib/test_tensorrt.py
index 10c311a..de98222 100644
--- a/tests/python/contrib/test_tensorrt.py
+++ b/tests/python/contrib/test_tensorrt.py
@@ -352,6 +352,7 @@ def test_conv2d():
 padding=(0, 0),
 strides=(1, 1),
 dilation=(1, 1),
+channels=None,
 ):
 x = relay.var("x", shape=(x_shape), dtype="float32")
 kernel = relay.var("kernel", shape=(k_shape), dtype="float32")
@@ -363,6 +364,7 @@ def test_conv2d():
 padding=padding,
 strides=strides,
 dilation=dilation,
+channels=channels,
 )
 f = relay.Function([x, kernel], out)
 return f, {"x": x_shape, "kernel": k_shape}, ["kernel"]
@@ -380,6 +382,9 @@ def test_conv2d():
 dilation=dilation,
 )
 )
+run_and_verify_func(
+get_graph((1, 3, 16, 16), (3, 8, 7, 7), 3, [2, 2, 3, 3], [2, 2], [1, 
1], 24)
+)
 
 
 def test_conv2d_nhwc():

[GitHub] [tvm] anijain2305 merged pull request #7011: [BYOC][TRT] Use channels attribute in Conv2D op converter

2020-12-01 Thread GitBox 


anijain2305 merged pull request #7011:
URL: https://github.com/apache/tvm/pull/7011


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] merrymercy edited a comment on pull request #6987: [AutoScheduler] Support layout rewrite for whole networks

2020-12-01 Thread GitBox 


merrymercy edited a comment on pull request #6987:
URL: https://github.com/apache/tvm/pull/6987#issuecomment-736981545


   @jcf94 @comaniac @FrozenGene @junrushao1994  Let us merge this as soon as 
possible, so we can get good performance on CPU and publish the CPU tutorial.
   I will do more clean up in the follow-up PRs when I add layout rewrite 
support for more ops (conv3d, dense, batch matmul).
   It is easier to figure out the best way to modify TOPI compute when I add 
the support for more ops.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] FrozenGene commented on a change in pull request #6987: [AutoScheduler] Support layout rewrite for whole networks

2020-12-01 Thread GitBox 


FrozenGene commented on a change in pull request #6987:
URL: https://github.com/apache/tvm/pull/6987#discussion_r533889606



##
File path: python/tvm/topi/nn/conv2d.py
##
@@ -371,8 +379,29 @@ def conv2d_nhwc(Input, Filter, stride, padding, dilation, 
out_dtype="float32"):
 else:
 dilation_h, dilation_w = dilation
 
+if auto_scheduler_rewritten_layout:
+# Infer shape for the rewritten layout

Review comment:
   As discussed, we need extract it. If conveniently, we could mark this 
one todo.

##
File path: tests/python/relay/test_auto_scheduler_layout_rewrite.py
##
@@ -0,0 +1,122 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Test layout rewrite support for whole neural networks"""
+import tempfile
+
+import numpy as np
+
+import tvm
+from tvm import relay, auto_scheduler
+from tvm.contrib import graph_runtime
+import tvm.testing
+
+
+def get_np_array(var, dtype):
+return np.random.randn(*[int(x) for x in 
var.type_annotation.shape]).astype(dtype)
+
+
+def get_relay_conv2d(
+outc=128,
+inc=64,
+height=14,
+width=14,
+kh=3,
+kw=3,
+batch=1,
+pad=0,
+stride=1,
+dilation=1,
+layout="NHWC",
+):
+dtype = "float32"
+if layout == "NHWC":
+kernel_layout = "HWIO"
+d = relay.var("data", shape=(batch, height, width, inc), dtype=dtype)
+w = relay.var("weight", shape=(kh, kw, inc, outc), dtype=dtype)
+elif layout == "NCHW":
+kernel_layout = "OIHW"
+d = relay.var("data", shape=(batch, inc, height, width), dtype=dtype)
+w = relay.var("weight", shape=(outc, inc, kh, kw), dtype=dtype)
+
+y = relay.nn.conv2d(
+d,
+w,
+padding=pad,
+kernel_size=(kh, kw),
+strides=(stride, stride),
+dilation=(dilation, dilation),
+channels=outc,
+groups=1,
+data_layout=layout,
+kernel_layout=kernel_layout,
+)
+mod = tvm.IRModule()
+mod["main"] = relay.Function([d, w], y)
+data, weight = get_np_array(d, dtype), get_np_array(w, dtype)
+return mod, data, weight
+
+
+def tune_and_check(mod, data, weight):
+# Extract tasks from a relay program
+target = tvm.target.Target("llvm")
+tasks, task_weights = auto_scheduler.extract_tasks(mod, target=target, 
params={})
+
+with tempfile.NamedTemporaryFile() as fp:
+log_file = fp.name
+# log_file = "test_layout_rewrite.json"

Review comment:
   remove it. If we want to merge it soon, could leave it in the clean up 
pr.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] merrymercy commented on pull request #6987: [AutoScheduler] Support layout rewrite for whole networks

2020-12-01 Thread GitBox 


merrymercy commented on pull request #6987:
URL: https://github.com/apache/tvm/pull/6987#issuecomment-736981545


   @jcf94 @comaniac @FrozenGene @junrushao1994  Let us merge this as soon as 
possible, so we can get good performance on CPU and publish the CPU tutorial.
   I will do more clean up in the follow-up PRs when I add layout rewrite 
support for more ops (conv3d, dense, batch matmul).
   I will figure out the best way to modify TOPI compute when I add the support 
for more ops.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch opened a new pull request #7012: [µTVM] Minor fixes to the Reference VM tutorial

2020-12-01 Thread GitBox 


areusch opened a new pull request #7012:
URL: https://github.com/apache/tvm/pull/7012


   A few minor updates to the reference vm tutorials. @tqchen



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] slyubomirsky commented on a change in pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


slyubomirsky commented on a change in pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#discussion_r533867812



##
File path: python/tvm/relay/frontend/keras.py
##
@@ -927,6 +1031,66 @@ def _convert_repeat_vector(inexpr, keras_layer, _):
 return out
 
 
+def _convert_time_distributed(inexpr, keras_layer, etab, input_shape=None, 
data_layout=None):
+# TimeDistributed: split input tensor along the second dimension (assumed 
to be time),
+# apply inner layer to each split individually,
+# and then combine the results
+if input_shape is None:
+input_shape = keras_layer.input_shape
+if data_layout is None:
+data_layout = etab.data_layout
+
+assert len(input_shape) >= 2, "Input to TimeDistributed must have at least 
two dimensions"
+
+inner_layer = keras_layer.layer
+inner_input_shape = [d for (i, d) in enumerate(input_shape) if i != 1]
+
+# for NDHWC, inner data layout will drop the D
+inner_data_layout = None
+if data_layout == "NDHWC":
+inner_data_layout = "NHWC"
+
+# some code duplication from keras_op_to_relay
+# but it's useful to avoid cluttering the etab
+inner_layer_op_name = type(keras_layer.layer).__name__
+if inner_layer_op_name not in _convert_map:
+raise tvm.error.OpNotImplemented(
+"The inner layer for TimeDistributed {} is not supported for 
frontend Keras.".format(
+inner_layer_op_name
+)
+)
+
+conversion_func = lambda expr: _convert_map[inner_layer_op_name](
+expr, inner_layer, etab, input_shape=inner_input_shape, 
data_layout=inner_data_layout

Review comment:
   After reading over the `ExprTable` implementation, I do not think it was 
a good idea to attach the data layout (a property that has nothing to do with 
the `ExprTable` specifically) to the `etab` in the first place -- it seems like 
unnecessary coupling. Maybe there should be a different way of handling the 
shared state across these conversions.
   
   Fwiw, I could just overwrite the `data_layout` field in the `ExprTable` when 
handling the `TimeDistributed` case and avoid having to modify the other 
conversion functions. I am not sure whether that would be a good idea from the 
standpoint of maintainability, though.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] liangfu commented on a change in pull request #6948: [µTVM] Allow for platform-specific malloc in runtime

2020-12-01 Thread GitBox 


liangfu commented on a change in pull request #6948:
URL: https://github.com/apache/tvm/pull/6948#discussion_r533859211



##
File path: src/runtime/crt/common/crt_runtime_api.c
##
@@ -315,21 +315,30 @@ int TVMFuncFree(TVMFunctionHandle func) {
   return 0;
 }
 
-tvm_crt_error_t TVMInitializeRuntime(uint8_t* memory_pool, size_t 
memory_pool_size_bytes,
- size_t page_size_bytes_log2) {
+tvm_crt_error_t TVMInitializeRuntime() {
   int idx;
   tvm_crt_error_t error;
+  void* func_registry_memory;
 
-  error =
-  TVMInitializeGlobalMemoryManager(memory_pool, memory_pool_size_bytes, 
page_size_bytes_log2);
+  system_lib_handle = kTVMModuleHandleUninitialized;
+
+  DLContext ctx = {kDLCPU, 0};
+  error = TVMPlatformMemoryAllocate(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES, 
ctx,
+_registry_memory);
   if (error != kTvmErrorNoError) {
 return error;
   }
 
   system_lib_handle = kTVMModuleHandleUninitialized;
 
-  TVMMutableFuncRegistry_Create(_func_registry,
-
vmalloc(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES),
+  void* registry_backing_memory;
+  error = TVMPlatformMemoryAllocate(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES, 
ctx,
+_backing_memory);
+  if (error != kTvmErrorNoError) {
+return error;

Review comment:
   When we reach here, we might need to release func_registry_memory before 
reporting an error?
   
   I would suggest 1) initialize func_registry_memory to nullptr on creation of 
the pointer, 2) have a single return statement at the end, 3) we could check 
all failed malloc and release all allocated memory before reporting an error. 
Facing a mixture of successful and failed mallocs inside a function before 
returning an error status, I think we might have created a memory leak.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] liangfu commented on a change in pull request #6948: [µTVM] Allow for platform-specific malloc in runtime

2020-12-01 Thread GitBox 


liangfu commented on a change in pull request #6948:
URL: https://github.com/apache/tvm/pull/6948#discussion_r533859211



##
File path: src/runtime/crt/common/crt_runtime_api.c
##
@@ -315,21 +315,30 @@ int TVMFuncFree(TVMFunctionHandle func) {
   return 0;
 }
 
-tvm_crt_error_t TVMInitializeRuntime(uint8_t* memory_pool, size_t 
memory_pool_size_bytes,
- size_t page_size_bytes_log2) {
+tvm_crt_error_t TVMInitializeRuntime() {
   int idx;
   tvm_crt_error_t error;
+  void* func_registry_memory;
 
-  error =
-  TVMInitializeGlobalMemoryManager(memory_pool, memory_pool_size_bytes, 
page_size_bytes_log2);
+  system_lib_handle = kTVMModuleHandleUninitialized;
+
+  DLContext ctx = {kDLCPU, 0};
+  error = TVMPlatformMemoryAllocate(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES, 
ctx,
+_registry_memory);
   if (error != kTvmErrorNoError) {
 return error;
   }
 
   system_lib_handle = kTVMModuleHandleUninitialized;
 
-  TVMMutableFuncRegistry_Create(_func_registry,
-
vmalloc(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES),
+  void* registry_backing_memory;
+  error = TVMPlatformMemoryAllocate(TVM_CRT_GLOBAL_FUNC_REGISTRY_SIZE_BYTES, 
ctx,
+_backing_memory);
+  if (error != kTvmErrorNoError) {
+return error;

Review comment:
   When we reach here, we might need to release func_registry_memory before 
reporting an error?
   
   I would suggest 1) initialize func_registry_memory to nullptr on creation of 
the pointer, 2) have a single return statement at the end, 3) we could check 
all failed malloc and release all allocated memory before reporting an error. 
Facing a mixture of successful and failed mallocs inside a function before 
returning an error status, I think we might have create a memory leak.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7001: [µTVM] Modify reference VMs to support new µTVM demo

2020-12-01 Thread GitBox 


areusch commented on pull request #7001:
URL: https://github.com/apache/tvm/pull/7001#issuecomment-736942300


   @tqchen 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] slyubomirsky commented on a change in pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


slyubomirsky commented on a change in pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#discussion_r533841208



##
File path: python/tvm/relay/frontend/keras.py
##
@@ -927,6 +1031,66 @@ def _convert_repeat_vector(inexpr, keras_layer, _):
 return out
 
 
+def _convert_time_distributed(inexpr, keras_layer, etab, input_shape=None, 
data_layout=None):
+# TimeDistributed: split input tensor along the second dimension (assumed 
to be time),
+# apply inner layer to each split individually,
+# and then combine the results
+if input_shape is None:
+input_shape = keras_layer.input_shape
+if data_layout is None:
+data_layout = etab.data_layout
+
+assert len(input_shape) >= 2, "Input to TimeDistributed must have at least 
two dimensions"
+
+inner_layer = keras_layer.layer
+inner_input_shape = [d for (i, d) in enumerate(input_shape) if i != 1]
+
+# for NDHWC, inner data layout will drop the D
+inner_data_layout = None
+if data_layout == "NDHWC":
+inner_data_layout = "NHWC"
+
+# some code duplication from keras_op_to_relay
+# but it's useful to avoid cluttering the etab
+inner_layer_op_name = type(keras_layer.layer).__name__
+if inner_layer_op_name not in _convert_map:
+raise tvm.error.OpNotImplemented(
+"The inner layer for TimeDistributed {} is not supported for 
frontend Keras.".format(
+inner_layer_op_name
+)
+)
+
+conversion_func = lambda expr: _convert_map[inner_layer_op_name](
+expr, inner_layer, etab, input_shape=inner_input_shape, 
data_layout=inner_data_layout

Review comment:
   https://www.tensorflow.org/api_docs/python/tf/keras/models/clone_model I 
might be able to copy the inner layer to `TimeDistributed` and set its 
`input_shape` but I'm not sure whether it's possible. I will check





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] kevinthesun commented on pull request #7005: [Relay][Topi] Fix GPU NMS when return_indices is True

2020-12-01 Thread GitBox 


kevinthesun commented on pull request #7005:
URL: https://github.com/apache/tvm/pull/7005#issuecomment-736927234


   @zhiics That's a good idea since we will need it for arghwere and topk as 
well.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] trevor-m opened a new pull request #7011: [BYOC][TRT] Use channels attribute in Conv2D op converter

2020-12-01 Thread GitBox 


trevor-m opened a new pull request #7011:
URL: https://github.com/apache/tvm/pull/7011


   Previous code was getting the number of output channels from the kernel 
shape, however this didn't work for some grouped convs. Now it will get it from 
the channels attr of the Relay Conv2D op if available.
   
   Added test case for one such conv2d which failed before.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] slyubomirsky commented on a change in pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


slyubomirsky commented on a change in pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#discussion_r533816124



##
File path: python/tvm/relay/frontend/keras.py
##
@@ -927,6 +1031,66 @@ def _convert_repeat_vector(inexpr, keras_layer, _):
 return out
 
 
+def _convert_time_distributed(inexpr, keras_layer, etab, input_shape=None, 
data_layout=None):
+# TimeDistributed: split input tensor along the second dimension (assumed 
to be time),
+# apply inner layer to each split individually,
+# and then combine the results
+if input_shape is None:
+input_shape = keras_layer.input_shape
+if data_layout is None:
+data_layout = etab.data_layout
+
+assert len(input_shape) >= 2, "Input to TimeDistributed must have at least 
two dimensions"
+
+inner_layer = keras_layer.layer
+inner_input_shape = [d for (i, d) in enumerate(input_shape) if i != 1]
+
+# for NDHWC, inner data layout will drop the D
+inner_data_layout = None
+if data_layout == "NDHWC":
+inner_data_layout = "NHWC"
+
+# some code duplication from keras_op_to_relay
+# but it's useful to avoid cluttering the etab
+inner_layer_op_name = type(keras_layer.layer).__name__
+if inner_layer_op_name not in _convert_map:
+raise tvm.error.OpNotImplemented(
+"The inner layer for TimeDistributed {} is not supported for 
frontend Keras.".format(
+inner_layer_op_name
+)
+)
+
+conversion_func = lambda expr: _convert_map[inner_layer_op_name](
+expr, inner_layer, etab, input_shape=inner_input_shape, 
data_layout=inner_data_layout

Review comment:
   Another problem is that the `etab` is used in the end to produce the new 
Relay module. Copying the `etab` would be okay if I move all the definitions 
over to the original one and make sure there's no chance of a name conflict 
(that is probably doable).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] slyubomirsky commented on a change in pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


slyubomirsky commented on a change in pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#discussion_r533814428



##
File path: python/tvm/relay/frontend/keras.py
##
@@ -927,6 +1031,66 @@ def _convert_repeat_vector(inexpr, keras_layer, _):
 return out
 
 
+def _convert_time_distributed(inexpr, keras_layer, etab, input_shape=None, 
data_layout=None):
+# TimeDistributed: split input tensor along the second dimension (assumed 
to be time),
+# apply inner layer to each split individually,
+# and then combine the results
+if input_shape is None:
+input_shape = keras_layer.input_shape
+if data_layout is None:
+data_layout = etab.data_layout
+
+assert len(input_shape) >= 2, "Input to TimeDistributed must have at least 
two dimensions"
+
+inner_layer = keras_layer.layer
+inner_input_shape = [d for (i, d) in enumerate(input_shape) if i != 1]
+
+# for NDHWC, inner data layout will drop the D
+inner_data_layout = None
+if data_layout == "NDHWC":
+inner_data_layout = "NHWC"
+
+# some code duplication from keras_op_to_relay
+# but it's useful to avoid cluttering the etab
+inner_layer_op_name = type(keras_layer.layer).__name__
+if inner_layer_op_name not in _convert_map:
+raise tvm.error.OpNotImplemented(
+"The inner layer for TimeDistributed {} is not supported for 
frontend Keras.".format(
+inner_layer_op_name
+)
+)
+
+conversion_func = lambda expr: _convert_map[inner_layer_op_name](
+expr, inner_layer, etab, input_shape=inner_input_shape, 
data_layout=inner_data_layout

Review comment:
   That would work for `data_layout`. `input_shape` is a field in the Keras 
layer and is not mutable, unfortunately





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] jwfromm commented on a change in pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


jwfromm commented on a change in pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#discussion_r533796286



##
File path: python/tvm/relay/frontend/keras.py
##
@@ -927,6 +1031,66 @@ def _convert_repeat_vector(inexpr, keras_layer, _):
 return out
 
 
+def _convert_time_distributed(inexpr, keras_layer, etab, input_shape=None, 
data_layout=None):
+# TimeDistributed: split input tensor along the second dimension (assumed 
to be time),
+# apply inner layer to each split individually,
+# and then combine the results
+if input_shape is None:
+input_shape = keras_layer.input_shape
+if data_layout is None:
+data_layout = etab.data_layout
+
+assert len(input_shape) >= 2, "Input to TimeDistributed must have at least 
two dimensions"
+
+inner_layer = keras_layer.layer
+inner_input_shape = [d for (i, d) in enumerate(input_shape) if i != 1]
+
+# for NDHWC, inner data layout will drop the D
+inner_data_layout = None
+if data_layout == "NDHWC":
+inner_data_layout = "NHWC"
+
+# some code duplication from keras_op_to_relay
+# but it's useful to avoid cluttering the etab
+inner_layer_op_name = type(keras_layer.layer).__name__
+if inner_layer_op_name not in _convert_map:
+raise tvm.error.OpNotImplemented(
+"The inner layer for TimeDistributed {} is not supported for 
frontend Keras.".format(
+inner_layer_op_name
+)
+)
+
+conversion_func = lambda expr: _convert_map[inner_layer_op_name](
+expr, inner_layer, etab, input_shape=inner_input_shape, 
data_layout=inner_data_layout

Review comment:
   it seems like you could have just make a copy of etab but set a new 
'input_shape' and 'data_layout' here. This would remove the need to change 
every other convert function. Am I missing a reason that wouldnt work? It's 
unclear to me what other assumptions are being made about etab.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] slyubomirsky commented on pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


slyubomirsky commented on pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#issuecomment-736882577


   Thanks, Jared!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] jroesch commented on pull request #7006: [frontend][keras] Add support for TimeDistributed

2020-12-01 Thread GitBox 


jroesch commented on pull request #7006:
URL: https://github.com/apache/tvm/pull/7006#issuecomment-736880166


   cc @mbrookhart @tkonolige @jwfromm 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] zhiics commented on pull request #7005: [Relay][Topi] Fix GPU NMS when return_indices is True

2020-12-01 Thread GitBox 


zhiics commented on pull request #7005:
URL: https://github.com/apache/tvm/pull/7005#issuecomment-736877850


   Can we just enable thrust in the gpu CI script?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] altanh commented on issue #7010: [TEST][FLAKY] test_op_grad_level2.py::test_conv2d_grad.py

2020-12-01 Thread GitBox 


altanh commented on issue #7010:
URL: https://github.com/apache/tvm/issues/7010#issuecomment-736863288


   I suspect some recent PR might have broke something, this is the error: 
```tests/python/relay/test_op_grad_level2.py::test_conv2d_grad Fatal Python 
error: Aborted.```
   
   Doesn't seem to me like a numerical issue with the gradient



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on issue #7010: [TEST][FLAKY] test_op_grad_level2.py::test_conv2d_grad.py

2020-12-01 Thread GitBox 


tqchen commented on issue #7010:
URL: https://github.com/apache/tvm/issues/7010#issuecomment-736860281


   cc @altanh @jroesch @antinucleon would be great if you can take a look



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen opened a new issue #7010: [TEST][FLAKY] test_op_grad_level2.py::test_conv2d_grad.py

2020-12-01 Thread GitBox 


tqchen opened a new issue #7010:
URL: https://github.com/apache/tvm/issues/7010


   
   https://ci.tlcpack.ai/job/tvm/job/main/245/execution/node/218/log/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] kevinthesun commented on pull request #7005: [Relay][Topi] Fix GPU NMS when return_indices is True

2020-12-01 Thread GitBox 


kevinthesun commented on pull request #7005:
URL: https://github.com/apache/tvm/pull/7005#issuecomment-736851932


   @zhiics @anijain2305 @tqchen We need Thrust library for gpu dynamic nms to 
pass ci.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] kevinthesun commented on a change in pull request #7009: [TFLite] Bugfix - ensure pad calcalution to be in int32

2020-12-01 Thread GitBox 


kevinthesun commented on a change in pull request #7009:
URL: https://github.com/apache/tvm/pull/7009#discussion_r533703042



##
File path: python/tvm/relay/frontend/tflite.py
##
@@ -3210,7 +3210,7 @@ def get_pad_value(data, kernel, stride):
 """
 
 out = int(math.ceil(float(data) / float(stride)))
-pad = max(0, (out - 1) * stride + kernel - data)
+pad = max(0, (out - 1) * int(stride) + int(kernel) - int(data))

Review comment:
   Do we want to somehow test the compilation for skylake/cascadelake 
target?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] anijain2305 opened a new pull request #7009: [TFLite] Bugfix - ensure pad calcalution to be in int32

2020-12-01 Thread GitBox 


anijain2305 opened a new pull request #7009:
URL: https://github.com/apache/tvm/pull/7009


   @kevinthesun @zhiics @giuseros 
   
   TFLite prequantized Resent model was failing to compile. I found that the 
pad calculation introduce int64 datatype in the shape of max pool2d, that later 
causes issues with tensorization. Int64 was introduced because of the 
calculation between python int and np.int32, which results in np.int64. This PR 
ensures that all the padding calculations happen in python int.
   
   Additionally, I have added a small check to print i64 in AsText. This helped 
in identifying which op was introducing the int64 datatype.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] comaniac commented on pull request #7007: [Contrib] Fix call mkl gemm in mkldnn.py

2020-12-01 Thread GitBox 


comaniac commented on pull request #7007:
URL: https://github.com/apache/tvm/pull/7007#issuecomment-736785713


   Thanks @CaramelFc @cchung100m 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] comaniac merged pull request #7007: [Contrib] Fix call mkl gemm in mkldnn.py

2020-12-01 Thread GitBox 


comaniac merged pull request #7007:
URL: https://github.com/apache/tvm/pull/7007


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated (73a1a9a -> fe4c66b)

2020-12-01 Thread comaniac 
This is an automated email from the ASF dual-hosted git repository.

comaniac pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git.


from 73a1a9a  [TOPI] deformable_conv2d in NHWC (#6999)
 add fe4c66b  Fix call mkl gemm in mkldnn.py (#7007)

No new revisions were added by this update.

Summary of changes:
 python/tvm/contrib/mkldnn.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[GitHub] [tvm] comaniac commented on pull request #6999: [TOPI] deformable_conv2d in NHWC

2020-12-01 Thread GitBox 


comaniac commented on pull request #6999:
URL: https://github.com/apache/tvm/pull/6999#issuecomment-736784024


   Thanks @vinx13 @merrymercy 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated: [TOPI] deformable_conv2d in NHWC (#6999)

2020-12-01 Thread comaniac 
This is an automated email from the ASF dual-hosted git repository.

comaniac pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
 new 73a1a9a  [TOPI] deformable_conv2d in NHWC (#6999)
73a1a9a is described below

commit 73a1a9a69f62281d61148280a023e58e6dcd08f0
Author: Wuwei Lin 
AuthorDate: Tue Dec 1 14:59:09 2020 -0500

[TOPI] deformable_conv2d in NHWC (#6999)

* [TOPI] deformable_conv2d in NHWC

* Update python/tvm/topi/generic/nn.py

Co-authored-by: Cody Yu 

* Update python/tvm/topi/testing/deformable_conv2d_python.py

Co-authored-by: Cody Yu 

* style

* fix

* style

Co-authored-by: Cody Yu 
---
 include/tvm/topi/detail/tensor_utils.h |  37 +++
 python/tvm/topi/generic/nn.py  |  18 
 python/tvm/topi/nn/deformable_conv2d.py| 110 -
 python/tvm/topi/testing/__init__.py|   2 +-
 ..._nchw_python.py => deformable_conv2d_python.py} |  49 +
 src/topi/schedule.cc   |   4 +
 .../topi/python/test_topi_deformable_conv2d.py |  95 +-
 7 files changed, 311 insertions(+), 4 deletions(-)

diff --git a/include/tvm/topi/detail/tensor_utils.h 
b/include/tvm/topi/detail/tensor_utils.h
index 7004c35..65a760b 100644
--- a/include/tvm/topi/detail/tensor_utils.h
+++ b/include/tvm/topi/detail/tensor_utils.h
@@ -89,6 +89,43 @@ inline PrimExpr bilinear_sample_nchw(const Tensor& input, 
const Array&
  D * x_lerp * y_lerp;
 }
 
+/*!
+ * \brief Sample a point in a tensor using bilinear interpolation.
+ *
+ * \param input The input tensor.
+ * \param indices The index of the target point, which can be fractional
+ * \param max_y The maximum of y dimension
+ * \param max_x The maximum of x dimension
+ *
+ * \return The interpolated value in the given index.
+ */
+inline PrimExpr bilinear_sample_nhwc(const Tensor& input, const 
Array& indices,
+ const PrimExpr max_y, const PrimExpr 
max_x) {
+  auto in_y = indices[1];
+  auto yf = tvm::floor(in_y);
+  auto yc = tvm::cast(DataType::Int(32), tvm::ceil(in_y));
+
+  auto y0 = tvm::cast(DataType::Int(32), tvm::floor(in_y));
+  auto y1 = tvm::if_then_else((yc > max_y), max_y, yc);
+  auto y_lerp = in_y - yf;
+
+  auto in_x = indices[2];
+  auto xf = tvm::floor(in_x);
+  auto xc = tvm::cast(DataType::Int(32), tvm::ceil(in_x));
+
+  auto x0 = tvm::cast(DataType::Int(32), tvm::floor(in_x));
+  auto x1 = tvm::if_then_else((xc > max_x), max_x, xc);
+  auto x_lerp = in_x - xf;
+
+  auto A = input(indices[0], y0, x0, indices[3]);
+  auto B = input(indices[0], y0, x1, indices[3]);
+  auto C = input(indices[0], y1, x0, indices[3]);
+  auto D = input(indices[0], y1, x1, indices[3]);
+
+  return A * (1 - x_lerp) * (1 - y_lerp) + B * x_lerp * (1 - y_lerp) + C * (1 
- x_lerp) * y_lerp +
+ D * x_lerp * y_lerp;
+}
+
 }  // namespace detail
 }  // namespace topi
 }  // namespace tvm
diff --git a/python/tvm/topi/generic/nn.py b/python/tvm/topi/generic/nn.py
index 4bc3f97..60ccd0d 100644
--- a/python/tvm/topi/generic/nn.py
+++ b/python/tvm/topi/generic/nn.py
@@ -462,6 +462,24 @@ def schedule_deformable_conv2d_nchw(outs):
 return _default_schedule(outs, False)
 
 
+def schedule_deformable_conv2d_nhwc(outs):
+"""Schedule for deformable_conv2d_nhwc.
+We only use the default schedule here and rely on auto_scheduler.
+
+Parameters
+--
+outs: Array of Tensor
+  The computation graph description of deformable_conv2d_nhwc
+  in the format of an array of tensors.
+
+Returns
+---
+sch: Schedule
+The computation schedule for the op.
+"""
+return _default_schedule(outs, False)
+
+
 def schedule_bitserial_conv2d_nchw(outs):
 """Schedule for bitserial_conv2d_nchw
 
diff --git a/python/tvm/topi/nn/deformable_conv2d.py 
b/python/tvm/topi/nn/deformable_conv2d.py
index a8c2745..780530c 100644
--- a/python/tvm/topi/nn/deformable_conv2d.py
+++ b/python/tvm/topi/nn/deformable_conv2d.py
@@ -21,7 +21,7 @@ from tvm import te
 
 from .utils import get_pad_tuple
 from ..utils import get_const_tuple
-from ..cpp.utils import bilinear_sample_nchw
+from ..cpp.utils import bilinear_sample_nchw, bilinear_sample_nhwc
 
 
 def deformable_conv2d_nchw(
@@ -130,3 +130,111 @@ def deformable_conv2d_nchw(
 ),
 tag="deformable_conv2d_nchw",
 )
+
+
+def deformable_conv2d_nhwc(
+data, offset, kernel, strides, padding, dilation, deformable_groups, 
groups, out_dtype
+):
+"""Deformable conv2D operator in NHWC layout.
+
+The deformable convolution operation is described in 
https://arxiv.org/abs/1703.06211
+
+Parameters
+--
+data : tvm.te.Tensor
+4-D with shape [batch, in_height, in_width, in_channel]
+
+offset : tvm.te.Tensor
+

[GitHub] [tvm] comaniac merged pull request #6999: [TOPI] deformable_conv2d in NHWC

2020-12-01 Thread GitBox 


comaniac merged pull request #6999:
URL: https://github.com/apache/tvm/pull/6999


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated: [RELAY,TOPI] Add scatter_nd op (#6854)

2020-12-01 Thread jroesch 
This is an automated email from the ASF dual-hosted git repository.

jroesch pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git


The following commit(s) were added to refs/heads/main by this push:
 new 0421efb  [RELAY,TOPI] Add scatter_nd op (#6854)
0421efb is described below

commit 0421efba4c3a42c6cf8d692734c24fe8e08e3884
Author: Tristan Konolige 
AuthorDate: Tue Dec 1 11:20:09 2020 -0800

[RELAY,TOPI] Add scatter_nd op (#6854)

* [RELAY,TOPI] Add scatter_nd op

Scatter_nd is the inverse of gather_nd and also happens to be its
gradient. The implementation here is not optimized. There are no cpu or
gpu specific implementations.

* formatting

* Fix tests

* formatting

* specify types on test

* Fix grad test

* scatter_nd cuda impl

* cuda impl

* x86 impl

* formatting

* fix shape rel

* fix tests

* formatting
---
 include/tvm/relay/attrs/transform.h   |   8 ++
 python/tvm/relay/backend/compile_engine.py|   5 +-
 python/tvm/relay/op/_tensor_grad.py   |   7 ++
 python/tvm/relay/op/_transform.py |   9 ++
 python/tvm/relay/op/strategy/cuda.py  |  13 +++
 python/tvm/relay/op/strategy/generic.py   |  22 +
 python/tvm/relay/op/strategy/x86.py   |  13 +++
 python/tvm/relay/op/transform.py  |  24 ++
 python/tvm/relay/testing/__init__.py  |   2 +
 python/tvm/te/operation.py|   6 +-
 python/tvm/topi/cuda/scatter.py   | 106 +++
 python/tvm/topi/scatter.py| 120 +-
 python/tvm/topi/testing/__init__.py   |   1 +
 python/tvm/topi/testing/common.py |  31 +++
 python/tvm/topi/x86/__init__.py   |   1 +
 python/tvm/topi/x86/scatter.py| 109 +++
 src/relay/analysis/type_solver.cc |   9 +-
 src/relay/op/tensor/transform.cc  |  68 +++
 tests/python/relay/test_any.py|   5 +-
 tests/python/relay/test_op_grad_level3.py |   9 ++
 tests/python/topi/python/test_topi_scatter.py |  67 ++
 21 files changed, 627 insertions(+), 8 deletions(-)

diff --git a/include/tvm/relay/attrs/transform.h 
b/include/tvm/relay/attrs/transform.h
index a7830cf..3ed6b83 100644
--- a/include/tvm/relay/attrs/transform.h
+++ b/include/tvm/relay/attrs/transform.h
@@ -129,6 +129,14 @@ struct ScatterAddAttrs : public 
tvm::AttrsNode {
   }
 };
 
+struct ScatterNDAttrs : public tvm::AttrsNode {
+  Array out_shape;
+
+  TVM_DECLARE_ATTRS(ScatterNDAttrs, "relay.attrs.ScatterNDAttrs") {
+TVM_ATTR_FIELD(out_shape).describe("Output shape of the scatter.");
+  }
+};
+
 struct GatherAttrs : public tvm::AttrsNode {
   Integer axis;
 
diff --git a/python/tvm/relay/backend/compile_engine.py 
b/python/tvm/relay/backend/compile_engine.py
index 32affe7..a39f72e 100644
--- a/python/tvm/relay/backend/compile_engine.py
+++ b/python/tvm/relay/backend/compile_engine.py
@@ -122,7 +122,10 @@ def get_valid_implementations(op, attrs, inputs, out_type, 
target):
 The list of all valid op implementations.
 """
 fstrategy = op.get_attr("FTVMStrategy")
-assert fstrategy is not None, "%s doesn't have FTVMStrategy registered" % 
op.name
+assert fstrategy is not None, (
+"%s doesn't have an FTVMStrategy registered. You can register "
+"one in python with `tvm.relay.op.register_strategy`." % op.name
+)
 with target:
 strategy = fstrategy(attrs, inputs, out_type, target)
 analyzer = tvm.arith.Analyzer()
diff --git a/python/tvm/relay/op/_tensor_grad.py 
b/python/tvm/relay/op/_tensor_grad.py
index b070d9f..9c84411 100644
--- a/python/tvm/relay/op/_tensor_grad.py
+++ b/python/tvm/relay/op/_tensor_grad.py
@@ -62,6 +62,7 @@ from .transform import (
 squeeze,
 strided_set,
 arange,
+scatter_nd,
 )
 
 
@@ -803,3 +804,9 @@ def arange_grad(orig, grad):
 grad_step = cast_like(_sum(grad_step), step)
 
 return [grad_start, grad_stop, grad_step]
+
+
+@register_gradient("gather_nd")
+def gather_nd_grad(orig, grad):
+data, indices = orig.args
+return [scatter_nd(grad, indices, data.checked_type.concrete_shape), 
zeros_like(indices)]
diff --git a/python/tvm/relay/op/_transform.py 
b/python/tvm/relay/op/_transform.py
index 439d44b..e1cb9e9 100644
--- a/python/tvm/relay/op/_transform.py
+++ b/python/tvm/relay/op/_transform.py
@@ -115,6 +115,15 @@ def compute_scatter_add(attrs, inputs, output_type):
 
 _reg.register_strategy("scatter_add", strategy.scatter_add_strategy)
 
+# scatter
+@_reg.register_compute("scatter_nd")
+def compute_scatter_nd(attrs, inputs, output_type):
+"""Compute definition of scatter_nd"""
+return [topi.scatter_nd(inputs[0], inputs[1], attrs.out_shape)]
+
+
+_reg.register_strategy("scatter_nd",

[GitHub] [tvm] jroesch merged pull request #6854: [RELAY,TOPI] Add scatter_nd op

2020-12-01 Thread GitBox 


jroesch merged pull request #6854:
URL: https://github.com/apache/tvm/pull/6854


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] kevinthesun commented on a change in pull request #6978: [Relay][Topi][Dynamic] Add a Sort op and use the legalization pass to perform dynamic topk on GPU

2020-12-01 Thread GitBox 


kevinthesun commented on a change in pull request #6978:
URL: https://github.com/apache/tvm/pull/6978#discussion_r533639174



##
File path: python/tvm/topi/cuda/sort.py
##
@@ -593,3 +694,82 @@ def schedule_topk(outs):
   The computation schedule for the op.
 """
 return _schedule_sort(outs)
+
+
+def _dyn_topk_legalize(attrs, inputs, arg_types):

Review comment:
   I also get segfault using current topi dynamic strided slice to cut topk 
result. I'll spend some time on it.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


areusch commented on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736741647


   @rkimball weird. it's not that I don't believe you, just I am fairly sure 
that's what i saw before and it is listed in the [man 
page](https://man7.org/linux/man-pages/man2/accept.2.html) for accept(). i'll 
see if I can try it out later this afternoon.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #6948: [µTVM] Allow for platform-specific malloc in runtime

2020-12-01 Thread GitBox 


areusch commented on pull request #6948:
URL: https://github.com/apache/tvm/pull/6948#issuecomment-736726055


   @tqchen @liangfu please take a look when you have a minute



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] rkimball commented on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


rkimball commented on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736715585


   @areusch I tried it and updated my test repo if you want to have a look. My 
signal handler does nothing until the third time it gets SIGINT when it aborts, 
just to allow me to kill the program.
   EINTR is never triggered. The blocked network calls remain blocked until we 
exit.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] mbrookhart commented on a change in pull request #6978: [Relay][Topi][Dynamic] Add a Sort op and use the legalization pass to perform dynamic topk on GPU

2020-12-01 Thread GitBox 


mbrookhart commented on a change in pull request #6978:
URL: https://github.com/apache/tvm/pull/6978#discussion_r533605262



##
File path: python/tvm/topi/cuda/sort.py
##
@@ -593,3 +694,82 @@ def schedule_topk(outs):
   The computation schedule for the op.
 """
 return _schedule_sort(outs)
+
+
+def _dyn_topk_legalize(attrs, inputs, arg_types):

Review comment:
   Any tips for debugging generated code? I have a branch where I've done 
this a a topi composition, it passes if I compile with AddressSanitizer, but 
segfaults with a normal build in the generated code. @zhiics attempted to do 
argwhere, but gets compilation errors with topi dropping variables.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] mbrookhart commented on pull request #6978: [Relay][Topi][Dynamic] Add a Sort op and use the legalization pass to perform dynamic topk on GPU

2020-12-01 Thread GitBox 


mbrookhart commented on pull request #6978:
URL: https://github.com/apache/tvm/pull/6978#issuecomment-736714836


   @anijain2305 I'll add a shape func for sort and a regression test
   
   @kevinthesun I added a thrust implementation of sort, and tested dynamic 
topk with thrust enabled, it worked just fine.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] mbrookhart commented on a change in pull request #6978: [Relay][Topi][Dynamic] Add a Sort op and use the legalization pass to perform dynamic topk on GPU

2020-12-01 Thread GitBox 


mbrookhart commented on a change in pull request #6978:
URL: https://github.com/apache/tvm/pull/6978#discussion_r533605262



##
File path: python/tvm/topi/cuda/sort.py
##
@@ -593,3 +694,82 @@ def schedule_topk(outs):
   The computation schedule for the op.
 """
 return _schedule_sort(outs)
+
+
+def _dyn_topk_legalize(attrs, inputs, arg_types):

Review comment:
   Any tips for debugging generated code? I have a branch where I've done 
this a a topi composition, it passes if I compile with AddressSanitizer, but 
segfaults with a normal build. @zhiics attempted to do argwhere, but gets 
compilation errors with topi dropping variables.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


areusch commented on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736707997


   @rkimball thanks for testing this. can you modify your test as follows, 
which would replicate the way python does Ctrl+C handling:
   
   1. override the SIGINT handle using signal.
   2. return from the signal handler without exiting the program (you can set 
some global flag if you want to, which is what the python signal handler does)
   3. now see if you get EINTR from the network call.
   
   if so, you need to check the global flag between retries. unfortunately, in 
the typical TVM use case, that global flag is in python land, and merely says 
"please jump the python interpreter to the python-land signal handler at the 
next instruction"



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] rkimball commented on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


rkimball commented on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736692350


   I have confirmed that nothing bad or unexpected happens if ctrl-c is entered 
when we are waiting on a long-running network call. I made a test repo here 
https://github.com/rkimball/network_test.
   The test waits on accept and when a connection is accepted it spawns an echo 
client, so there are two blocking calls, one in the main thread waiting on 
accept and one in the echo client waiting on recv. While this is running I type 
ctrl-c and the program exits and does not print out EINTR so neither call 
receives an EINTR from ctrl-c.
   This test was performed on Ubuntu 20.04
   Here is the output of the test run
   ```
   (venv3) rkimball@Callisto:~/dev/network_test/build$ ./test
   main thread accepting connection now
   waiting on call accept
   call done
   main thread got a connection
   main thread accepting connection now
   waiting on call accept
   waiting on call recv
   call done
   got abc
   
   waiting on call recv
   ^C
   (venv3) rkimball@Callisto:~/dev/network_test/build$
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] anijain2305 commented on pull request #6955: Dynamic Batch Support for TRT

2020-12-01 Thread GitBox 


anijain2305 commented on pull request #6955:
URL: https://github.com/apache/tvm/pull/6955#issuecomment-736685415


   Thanks @codeislife99 @trevor-m This is merged!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[tvm] branch main updated (93758ca -> 7950ea1)

2020-12-01 Thread anijain2305 
This is an automated email from the ASF dual-hosted git repository.

anijain2305 pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/tvm.git.


from 93758ca  [TVMC] use target_host when it is set (#6855)
 add 7950ea1  Dynamic Batch Support for TRT  (#6955)

No new revisions were added by this update.

Summary of changes:
 python/tvm/relay/op/contrib/tensorrt.py  | 117 --
 src/relay/backend/utils.h|   3 +-
 src/runtime/contrib/tensorrt/tensorrt_runtime.cc |  40 +++--
 tests/python/contrib/test_tensorrt.py| 185 +++
 4 files changed, 318 insertions(+), 27 deletions(-)

[GitHub] [tvm] anijain2305 merged pull request #6955: Dynamic Batch Support for TRT

2020-12-01 Thread GitBox 


anijain2305 merged pull request #6955:
URL: https://github.com/apache/tvm/pull/6955


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen edited a comment on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


tqchen edited a comment on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736571499







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] tqchen commented on pull request #6953: Add retry to sockets on EINTR error

2020-12-01 Thread GitBox 


tqchen commented on pull request #6953:
URL: https://github.com/apache/tvm/pull/6953#issuecomment-736571499


   It could be hard to reproduce what @areusch said.
   
   Because such interruption need to happen during a socket call, nevertheless, 
it would be useful to confirm the behavior(what will happen when ctril c get 
pressed during a socket call) before check in the change.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] Meteorix opened a new issue #7008: [RELAY][BUG]type inference is slow

2020-12-01 Thread GitBox 


Meteorix opened a new issue #7008:
URL: https://github.com/apache/tvm/issues/7008


   For a large model, tvm compilation is really slow. I perf it and find that 
type inference costs most of the time.
   ```
   # Children  Self  Command  Shared Object Symbol
   #     ...    
.
   #
   93.18% 0.00%  python   libtvm.so [.] 
tvm::relay::PatternRewriter::Rewrite
   |
   ---tvm::relay::PatternRewriter::Rewrite
  |
   --93.17%--tvm::relay::InferTypeWithModule
 |
  --93.05%--tvm::transform::Pass::operator()
tvm::transform::PassNode::operator()

tvm::transform::ModulePassNode::operator()

tvm::runtime::PackedFunc::operator()
std::function::operator()
std::_Function_handler::AssignTypedLambda::AssignTypedLambdahttps://github.com/apache/tvm/blob/main/src/relay/transforms/type_infer.cc#L805
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [tvm] CaramelFc opened a new pull request #7007: [Contrib] Fix call mkl gemm in mkldnn.py

2020-12-01 Thread GitBox 


CaramelFc opened a new pull request #7007:
URL: https://github.com/apache/tvm/pull/7007


   Thanks for contributing to TVM!   Please refer to guideline 
https://tvm.apache.org/docs/contribute/ for useful information and tips. After 
the pull request is submitted, please request code reviews from 
[Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers)
 by @ them in the pull request thread.
   fix call mkl gemm in mkldnn.py
   
   @cchung100m please kindly review



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org