[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r372389879 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -747,6 +749,158 @@ def convert_squared_difference(self, op): out = _op.power(difference, relay.const(2, exp_type)) return out +def convert_gather(self, op): +"""Method to Convert TFLite Gather operator""" +# Check if the input tensor is quantized, call QNN op +if self.is_quantized(op): +raise tvm.error.OpNotImplemented( +'TFlite quantized gather operator is not supported yet.') Review comment: Since this is a tensor manipulation operation rather than something that requires any specific qnn dialect, I would prefer we handle the qnn ops here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r372390838 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -747,6 +749,158 @@ def convert_squared_difference(self, op): out = _op.power(difference, relay.const(2, exp_type)) return out +def convert_gather(self, op): +"""Method to Convert TFLite Gather operator""" +# Check if the input tensor is quantized, call QNN op +if self.is_quantized(op): +raise tvm.error.OpNotImplemented( +'TFlite quantized gather operator is not supported yet.') +input_tensors = self.get_input_tensors(op) + +try: +from tflite.BuiltinOptions import BuiltinOptions +from tflite.GatherOptions import GatherOptions +from tflite.TensorType import TensorType +except ImportError: +raise ImportError("The tflite package must be installed") + +assert op.BuiltinOptionsType() == BuiltinOptions.GatherOptions +op_options = op.BuiltinOptions() +gather_options = GatherOptions() +gather_options.Init(op_options.Bytes, op_options.Pos) +axis = gather_options.Axis() + +data = self.get_expr(input_tensors[0].tensor_idx) + +indices = input_tensors[1] +indices_type = indices.tensor.Type() + +assert indices_type in (TensorType.INT32, TensorType.INT64) +indices_type_str = self.get_tensor_type_str(indices_type) +indices = self.exp_tab.new_const(self.get_tensor_value(indices), + dtype=indices_type_str) +out = _op.take(data, indices, axis=axis) +return out + +def convert_strided_slice(self, op): +"""Method to Convert TFLite Strided Slice operator""" +# Check if the input tensor is quantized, call QNN op +if self.is_quantized(op): +raise tvm.error.OpNotImplemented( +'TFlite quantized strided slice operator is not supported yet.') Review comment: Same comment as above - this is a tensor manipulation op rather than anything that intrinsically requires quantization knowledge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r374500086 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -792,6 +794,147 @@ def convert_not_equal(self, op): 'TFlite quantized NOT_EQUAL operator is not supported yet.') return self._convert_elemwise(_op.not_equal, op) +def convert_gather(self, op): +"""Method to Convert TFLite GATHER operator""" +try: +from tflite.BuiltinOptions import BuiltinOptions +from tflite.GatherOptions import GatherOptions +from tflite.TensorType import TensorType +except ImportError: +raise ImportError("The tflite package must be installed") + +input_tensors = self.get_input_tensors(op) +data = self.get_expr(input_tensors[0].tensor_idx) + +indices = input_tensors[1] +indices_type = indices.tensor.Type() +assert indices_type in (TensorType.INT32, TensorType.INT64) +indices_type_str = self.get_tensor_type_str(indices_type) +indices = self.exp_tab.new_const(self.get_tensor_value(indices), + dtype=indices_type_str) + +assert op.BuiltinOptionsType() == BuiltinOptions.GatherOptions +op_options = op.BuiltinOptions() +gather_options = GatherOptions() +gather_options.Init(op_options.Bytes, op_options.Pos) +axis = gather_options.Axis() + +out = _op.take(data, indices, axis=axis) +return out + Review comment: A comment here explaining the lowering from the tflite operator to the relay IR seems appropriate as the code is non-obvious especially as it appears tflite.strided_slice is not identical to relay.op.strided_slice and a final reshape is needed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r374497747 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -244,6 +244,74 @@ def test_forward_slice(): _test_slice(np.arange(8, dtype=np.int32).reshape((2, 4)), begin=[0, 1], size=[-1, -1]) _test_slice(np.arange(5, dtype=np.int32).reshape((5, )), begin=[4], size=[-1]) +### +# Gather +# -- + +def _test_gather(dshape, indices, axis, dtype): +""" One iteration of Gather """ +data = np.random.uniform(1, 10, size=dshape).astype(dtype) +indices = np.asarray(indices).astype('int32') + +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype) +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm(data, 'Placeholder:0', [in_data], [out]) + +#Test quantized input +data = np.random.uniform(1, 10, size=dshape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype, name="in_data") +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) + +def test_forward_gather(): +""" GATHER """ +_test_gather((4,), [1], 0, 'float32') +_test_gather((1, 4), [0], 0, 'int32') +_test_gather((4,), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 1, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((3, 3, 3), [[[1, 0]]], 0, 'int32') +_test_gather((3, 3, 3), [[[1, 0]]], 2, 'int32') +_test_gather((4, 3, 5, 6), [[2, 1, 0, 0]], 0, 'float32') Review comment: I think a testcase is missing to test that out of bounds indices behave as tflite expects it to behave. See the documentation for relay.op.transform.take with the last optional parameter for "mode" See here for more . https://docs.tvm.ai/api/python/relay/op.html Further, axis is an optional input to gather in tflite and _op.take . Do the 2 semantics match up as I cannot see this obviously in the tflite documentation ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r374498740 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -244,6 +244,74 @@ def test_forward_slice(): _test_slice(np.arange(8, dtype=np.int32).reshape((2, 4)), begin=[0, 1], size=[-1, -1]) _test_slice(np.arange(5, dtype=np.int32).reshape((5, )), begin=[4], size=[-1]) +### +# Gather +# -- + +def _test_gather(dshape, indices, axis, dtype): +""" One iteration of Gather """ +data = np.random.uniform(1, 10, size=dshape).astype(dtype) +indices = np.asarray(indices).astype('int32') + +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype) +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm(data, 'Placeholder:0', [in_data], [out]) + +#Test quantized input +data = np.random.uniform(1, 10, size=dshape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype, name="in_data") +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) + +def test_forward_gather(): +""" GATHER """ +_test_gather((4,), [1], 0, 'float32') +_test_gather((1, 4), [0], 0, 'int32') +_test_gather((4,), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 1, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((3, 3, 3), [[[1, 0]]], 0, 'int32') +_test_gather((3, 3, 3), [[[1, 0]]], 2, 'int32') +_test_gather((4, 3, 5, 6), [[2, 1, 0, 0]], 0, 'float32') + +### +# StridedSlice +# + +def _test_stridedslice(ip_shape, begin, end, stride, dtype, + begin_mask=0, end_mask=0, new_axis_mask=0, + shrink_axis_mask=0, ellipsis_mask=0): +""" One iteration of a Stridedslice """ +data = np.random.uniform(size=ip_shape).astype(dtype) +with tf.Graph().as_default(): +in_data = tf.placeholder(dtype, ip_shape, name="in_data") +out = array_ops.strided_slice(in_data, begin, end, stride, + begin_mask=begin_mask, + end_mask=end_mask, new_axis_mask=new_axis_mask, + shrink_axis_mask=shrink_axis_mask, + ellipsis_mask=ellipsis_mask) +compare_tflite_with_tvm(data, 'in_data:0', [in_data], [out]) + +#Test with quantized inputs +data = np.random.uniform(size=ip_shape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = tf.placeholder(dtype, ip_shape, name="in_data") +out = array_ops.strided_slice(in_data, begin, end, stride, + begin_mask=begin_mask, + end_mask=end_mask, new_axis_mask=new_axis_mask, + shrink_axis_mask=shrink_axis_mask, + ellipsis_mask=ellipsis_mask) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) Review comment: Can we instead change the caller to call _test_stridedslice with a uint8 data type and avoid duplication of code ? If the data type can't be passed, can the data be created in the caller with an appropriate type and passed in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r374499522 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -244,6 +244,74 @@ def test_forward_slice(): _test_slice(np.arange(8, dtype=np.int32).reshape((2, 4)), begin=[0, 1], size=[-1, -1]) _test_slice(np.arange(5, dtype=np.int32).reshape((5, )), begin=[4], size=[-1]) +### +# Gather +# -- + +def _test_gather(dshape, indices, axis, dtype): +""" One iteration of Gather """ +data = np.random.uniform(1, 10, size=dshape).astype(dtype) +indices = np.asarray(indices).astype('int32') + +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype) +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm(data, 'Placeholder:0', [in_data], [out]) + +#Test quantized input +data = np.random.uniform(1, 10, size=dshape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype, name="in_data") +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) Review comment: Can we remove this by calling _test_gather with np.uint8 instead of duplicating logic ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r376673796 ## File path: tests/python/frontend/tflite/test_forward.py ## @@ -244,6 +244,74 @@ def test_forward_slice(): _test_slice(np.arange(8, dtype=np.int32).reshape((2, 4)), begin=[0, 1], size=[-1, -1]) _test_slice(np.arange(5, dtype=np.int32).reshape((5, )), begin=[4], size=[-1]) +### +# Gather +# -- + +def _test_gather(dshape, indices, axis, dtype): +""" One iteration of Gather """ +data = np.random.uniform(1, 10, size=dshape).astype(dtype) +indices = np.asarray(indices).astype('int32') + +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype) +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm(data, 'Placeholder:0', [in_data], [out]) + +#Test quantized input +data = np.random.uniform(1, 10, size=dshape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = array_ops.placeholder(shape=data.shape, dtype=data.dtype, name="in_data") +out = array_ops.gather(in_data, indices, axis=axis) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) + +def test_forward_gather(): +""" GATHER """ +_test_gather((4,), [1], 0, 'float32') +_test_gather((1, 4), [0], 0, 'int32') +_test_gather((4,), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 1, 'int32') +_test_gather((2, 2), [[[1, 0], [0, 1]]], 0, 'float32') +_test_gather((3, 3, 3), [[[1, 0]]], 0, 'int32') +_test_gather((3, 3, 3), [[[1, 0]]], 2, 'int32') +_test_gather((4, 3, 5, 6), [[2, 1, 0, 0]], 0, 'float32') + +### +# StridedSlice +# + +def _test_stridedslice(ip_shape, begin, end, stride, dtype, + begin_mask=0, end_mask=0, new_axis_mask=0, + shrink_axis_mask=0, ellipsis_mask=0): +""" One iteration of a Stridedslice """ +data = np.random.uniform(size=ip_shape).astype(dtype) +with tf.Graph().as_default(): +in_data = tf.placeholder(dtype, ip_shape, name="in_data") +out = array_ops.strided_slice(in_data, begin, end, stride, + begin_mask=begin_mask, + end_mask=end_mask, new_axis_mask=new_axis_mask, + shrink_axis_mask=shrink_axis_mask, + ellipsis_mask=ellipsis_mask) +compare_tflite_with_tvm(data, 'in_data:0', [in_data], [out]) + +#Test with quantized inputs +data = np.random.uniform(size=ip_shape).astype(np.uint8) +with tf.Graph().as_default(): +in_data = tf.placeholder(dtype, ip_shape, name="in_data") +out = array_ops.strided_slice(in_data, begin, end, stride, + begin_mask=begin_mask, + end_mask=end_mask, new_axis_mask=new_axis_mask, + shrink_axis_mask=shrink_axis_mask, + ellipsis_mask=ellipsis_mask) +compare_tflite_with_tvm([data], ['in_data:0'], [in_data], [out], quantized=True) + +def test_forward_stridedslice(): +'''test StridedSlice''' Review comment: @wyc-ruiker - might help to specify what situations you see missing to help with actionable feedback ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r381990653 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -852,6 +855,193 @@ def convert_logical_or(self, op): """Convert tflite LOGICAL_OR""" return self._convert_logical_binary(_op.logical_or, op) +def convert_gather(self, op): +"""Method to Convert TFLite GATHER operator""" +try: +from tflite.BuiltinOptions import BuiltinOptions +from tflite.GatherOptions import GatherOptions +from tflite.TensorType import TensorType +except ImportError: +raise ImportError("The tflite package must be installed") + +input_tensors = self.get_input_tensors(op) +assert len(input_tensors) == 2, "input tensors length should be 2" + +data = self.get_expr(input_tensors[0].tensor_idx) + +indices = input_tensors[1] +indices_type = indices.tensor.Type() +assert indices_type in (TensorType.INT32, TensorType.INT64) +indices_type_str = self.get_tensor_type_str(indices_type) +indices = self.exp_tab.new_const(self.get_tensor_value(indices), + dtype=indices_type_str) + +assert op.BuiltinOptionsType() == BuiltinOptions.GatherOptions +op_options = op.BuiltinOptions() +gather_options = GatherOptions() +gather_options.Init(op_options.Bytes, op_options.Pos) +axis = gather_options.Axis() + +# Check the indices are oob, tflite is unpredictable in case of oob. Review comment: Maybe improve comment to : Check that the indices are within bounds ? Is the tflite specification clear that out of bounds indices results in unpredictable behaviour ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added
u99127 commented on a change in pull request #4788: [FRONTEND][TFLITE]Gather, StridedSlice op support added URL: https://github.com/apache/incubator-tvm/pull/4788#discussion_r381991135 ## File path: python/tvm/relay/frontend/tflite.py ## @@ -852,6 +855,193 @@ def convert_logical_or(self, op): """Convert tflite LOGICAL_OR""" return self._convert_logical_binary(_op.logical_or, op) +def convert_gather(self, op): +"""Method to Convert TFLite GATHER operator""" +try: +from tflite.BuiltinOptions import BuiltinOptions +from tflite.GatherOptions import GatherOptions +from tflite.TensorType import TensorType +except ImportError: +raise ImportError("The tflite package must be installed") + +input_tensors = self.get_input_tensors(op) +assert len(input_tensors) == 2, "input tensors length should be 2" + +data = self.get_expr(input_tensors[0].tensor_idx) + +indices = input_tensors[1] +indices_type = indices.tensor.Type() +assert indices_type in (TensorType.INT32, TensorType.INT64) +indices_type_str = self.get_tensor_type_str(indices_type) +indices = self.exp_tab.new_const(self.get_tensor_value(indices), + dtype=indices_type_str) + +assert op.BuiltinOptionsType() == BuiltinOptions.GatherOptions +op_options = op.BuiltinOptions() +gather_options = GatherOptions() +gather_options.Init(op_options.Bytes, op_options.Pos) +axis = gather_options.Axis() + +# Check the indices are oob, tflite is unpredictable in case of oob. +data_shape = list(input_tensors[0].tensor.ShapeAsNumpy()) +data_dim = len(data_shape) + +axis_n = axis +if axis_n < 0: +axis_n += axis_n + data_dim +assert axis_n >= 0, "Axis out of bounds" +assert axis_n < data_dim, "Axis out of bounds" + +indices_val = self.get_tensor_value(input_tensors[1]) +indices_shape = list(indices_val.shape) +indices_len = len(indices_shape) + +out_shape = [] +for i in range(data_dim): +if axis_n == i: +for j in range(indices_len): +out_shape.append(indices_shape[j]) +else: +out_shape.append(data_shape[i]) + +loopover = [range(s) for s in out_shape] +for idx in list(itertools.product(*loopover)): +indices_position = [idx[j] for j in range(axis_n, axis_n+indices_len)] + +real_indices = [idx[j] for j in range(axis_n)] +real_indices.append(indices_val[tuple(indices_position)]) +real_indices.extend([idx[j] for j in range(axis_n + indices_len, len(idx))]) +for r, d in zip(real_indices, data_shape): +if r >= d: +raise ValueError("TFLite out of bound indices are not supported.") + +# Use mode as fast since already checked for oob. +out = _op.take(data, indices, axis=axis, mode="fast") +return out + +def convert_strided_slice(self, op): +"""Method to Convert TFLite STRIDED_SLICE operator. + Note: Eventhough tf2.0 supports begin_mask, end_mask, ellipsis_mask, new_axis_mask + and shrink_axis_mask, tflite doesn't support these and expect these values to be zero. + But in future, they may open up the mask implementation, so kept the implementation + same as tensorflow. + TVM Relay implementation of doesn't support mask, so the mask values are processed here Review comment: relay.op.strided_slice ? Instead of "similar to tensorflow parsing" a bit more context would be better here about what is actually being done because it is not clear what exactly you refer to as there are no comments next to op.strided_slice in tensorflow.py ;) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services