kevinthesun commented on a change in pull request #6449:
URL: https://github.com/apache/incubator-tvm/pull/6449#discussion_r487256873
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "32"]. What I can do is to enhance
```_pytorch_promote_types``` so that we do _infer_type for every input and get
actual input dtype, rather than solely relying on pytorch input dtype. Sounds
like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
This comes from weird behavior of ```prim::NumToTensor```. It converts
int32 to int64 silently:
```
%11 : int = aten::size(%img.1, %10), scope: __module.model #
/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/generalized_rcnn.py:62:0
%im_h : Long() = prim::NumToTensor(%11), scope: __module.model
```
Right now py frontend just follow use the same dtype for this op output. For
an elemwise op, pytorch input dtype is ["int64", "int64"] which is fine.
However, the actual input dtype is ["int64", "int32"]. What I can do is to
enhance ```_pytorch_promote_types``` so that we do _infer_type for every input
and get actual input dtype, rather than solely relying on pytorch input dtype.
Sounds like a plan?
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
The try except block is mainly for _infer_value. Currently there is no
very secure way to try _infer_value with explicit error types. That's why a
general Exception is used here.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -127,8 +128,22 @@ def _is_quantized_tensor(data, prelude):
# operator implementation
def _elemwise(name):
def _impl(inputs, input_types):
- data0, data1 = _pytorch_promote_types(inputs[:2], input_types[:2])
- return get_relay_op(name)(data0, data1)
+ dtype0, dtype1 = input_types[:2]
+ if isinstance(inputs[0], _expr.Expr):
+ dtype0 = _infer_type(inputs[0]).checked_type.dtype
+ if isinstance(inputs[1], _expr.Expr):
+ dtype1 = _infer_type(inputs[1]).checked_type.dtype
+
Review comment:
```%11 : int = aten::size(%img.1, %10)``` generates int32 but ```%im_h :
Long() = prim::NumToTensor(%11)``` automatically converts it to int64, without
any hint. When we converting ```prim::NumToTenso```, we can just follow the
input type which is int32 here since there is no any other information. So this
is about the weird behavior of ```prim::NumToTenso``` rather than indexing. I'm
not sure how many other ops in pytorch has such behavior, but it looks like
inferring actual input type in ```_pytorch_promote_types``` would fix these
kind of issues.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -364,7 +438,11 @@ def _impl(inputs, input_types):
def _topk():
def _impl(inputs, input_types):
data = inputs[0]
- k = int(inputs[1])
+ try:
+ k = int(_infer_value(inputs[1], {}).asnumpy().tolist())
+ k = _expr.const(k)
+ except Exception:
+ k = inputs[1]
Review comment:
Sure. I can do what I did for arange. It's checking whether input is
type _expr.Expr.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
+
if isinstance(inputs[3], str) and inputs[3].isdigit():
- end[dim] = min(end[dim], int(inputs[3]))
+ target_end = int(inputs[3])
else:
- if isinstance(inputs[3], _expr.Call):
- target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ if isinstance(inputs[3], _expr.Expr):
+ try:
+ target_end = np.asscalar(_infer_value(inputs[3],
{}).asnumpy().astype(np.int))
+ except Exception:
Review comment:
```if isinstance(inputs[3], _expr.Expr):```
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -429,25 +507,56 @@ def _impl(inputs, input_types):
return _impl
+def _full_impl(data, fill_value, dtype):
+ size = []
+ need_reshape = False
+ new_shape = []
+ for dim in data:
+ if isinstance(dim, _expr.Expr):
+ if isinstance(dim, _expr.Constant):
+ dim = int(dim.data.asnumpy())
+ if isinstance(size, list):
+ size.append(dim)
+ new_shape.append(dim)
+ else:
+ try:
+ dim = int(_infer_value(dim, {}).asnumpy())
Review comment:
Same. These try except blocks are necessary to handle dynamic operators.
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -274,38 +295,91 @@ def _impl(inputs, input_types):
def _slice():
def _impl(inputs, input_types):
+ index_size_limit = 2**63 - 1
data = inputs[0]
- strides = []
+ dshape = _infer_shape(data)
+ ndim = len(dshape)
+ end = []
+ for dim in dshape:
+ if isinstance(dim, tvm.tir.Any):
+ end = _op.shape_of(data)
+ break
+ end.append(int(dim))
- if isinstance(data, _expr.Expr):
- inferred_shape = _infer_shape(data)
- end = []
- for infer in inferred_shape:
- end.append(int(infer))
- if isinstance(data, _expr.Var):
- end = inferred_shape
- end = list(end)
- else:
- end = data.shape
-
- begin = [0] * len(end)
+ begin = [0] * ndim
dim = int(inputs[1])
+ stride = int(inputs[4])
if isinstance(inputs[2], _expr.Call):
- begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ try:
+ begin[dim] = np.asscalar(_infer_value(inputs[2],
{}).asnumpy().astype(np.int))
+ except Exception:
+ begin[dim] = inputs[2]
else:
begin[dim] = int(inputs[2])
+ # Process begin
+ if not isinstance(begin[dim], int):
+ tmp = []
+ for b in begin:
+ if isinstance(b, int):
+ tmp.append(_op.expand_dims(_expr.const(b, "int64"),
axis=0))
+ else:
+ tmp.append(_op.cast(_op.expand_dims(b, axis=0), "int64"))
+ begin = _op.concatenate(tmp, axis=0)
+ btype = _infer_type(begin).checked_type.dtype
+ if str(btype) != "int32":
+ begin = _op.cast(begin, "int32")
Review comment:
Use int64 now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
