echuraev opened a new pull request, #14996: URL: https://github.com/apache/tvm/pull/14996
When the number of output channels is less than 4, then we cannot pack such convolution to textures, although we can repack and extend tensors from 4d to 5d in runtime. It is happened because function `PropBoundToInputs` is invoked for all stages in when InferBound pass or LowerSchedule function is called. `PropBoundToInputs` has a logic in it that helps to developer to avoid out of bound access. And based on the output shape, it propagates it to inputs. Imagine that we want to transform a 4d tensor with 3 channels to 5d, extend its number of channels to 4 and then transform it back to 4d tensor with number of channels equal to 3. Example below: ``` [1, 3, 6, 6] -> [1, 1, 6, 6, 4] -> [1, 3, 6, 6] ``` In this case, we might write a boundary check in the repacking and extending compute function to handle the case when the iterator by the channels is out of bounds for the intermediate tensor. To avoid such problem, `PropBoundToInputs` has a logic which propagates a bounds from output tensor to input. In case when it is a possible situation that the compute has out of bounds access, then the range is decreased to value when such situation cannot be achieved. That means that for the example above the loop by channels which should filling intermediate tensor will iterate in range [0, 2] instead of [0, 3]. As it was mentioned, to avoid such problem we use buffers and cuda schedules instead of textures for cases when number of output channels is less than 4 and we cannot pack such tensor to texture. I evaluated performance of such approach and it doesn't introduce any performance degradation. For such small convolutions performance with buffers even a bit better than with textures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
