| Issue |
243218
|
| Summary |
[mlir][tosa] `TransposeOp::reifyResultShapes` crashes on unranked tensor input
|
| Labels |
mlir
|
| Assignees |
|
| Reporter |
quiet-maple-47
|
## Description
`--resolve-shaped-type-result-dims` crashes (or exhausts memory / fails `SmallVector` allocation) when `tosa.transpose` takes an unranked tensor (`tensor<*xi32>`) as input.
`tosa::TransposeOp::reifyResultShapes` calls `inputType.getRank()` without checking `inputType.hasRank()`, allocating a `SmallVector` with `kDynamic` (`INT64_MIN`) elements on non-assertions builds.
## Reproducer
`input.mlir`:
```mlir
func.func @f(%arg0: tensor<*xi32>) -> index {
%c0 = arith.constant 0 : index
%0 = tosa.transpose %arg0 perms([0, 2, 1, 3]) : (tensor<*xi32>) -> tensor<?x3x2x2xi32>
%d = tensor.dim %0, %c0 : tensor<?x3x2x2xi32>
return %d : index
}
```
```bash
mlir-opt --resolve-shaped-type-result-dims input.mlir
```
[Reproduce on Compiler Explorer (LLVM trunk)](https://godbolt.org/z/5zdPs7Md8)
## Actual Behavior
On LLVM trunk (Compiler Explorer), `mlir-opt` is terminated with `SIGKILL` (exit code 137) while trying to allocate `INT64_MIN` elements for `SmallVector<OpFoldResult>`.
On Homebrew LLVM 23.1.5 (with `{perms = ...}` syntax), `mlir-opt` aborts with:
```text
LLVM ERROR: SmallVector unable to grow. Requested capacity (505012822184) is larger than maximum value for size type (4294967295)
```
On assertions-enabled builds (LLVM 23 `b3df0665`), `mlir-opt` aborts in `ShapedTypeTrait<TensorType>::getRank()`:
```text
mlir-opt: mlir/include/mlir/IR/BuiltinTypeInterfaces.h.inc:758: int64_t mlir::detail::ShapedTypeTrait<mlir::TensorType>::getRank() const [ConcreteType = mlir::TensorType]: Assertion `(*static_cast<const ConcreteType *>(this)).hasRank() && "cannot query rank of unranked shaped type"' failed.
```
## Expected Behavior
`TransposeOp::reifyResultShapes` should return `failure()` when `inputType` is unranked (or derive rank/dimensions safely when unranked), rather than querying `getRank()` on an `UnrankedTensorType`.
## Root Cause
[`tosa::TransposeOp::reifyResultShapes`](https://github.com/llvm/llvm-project/blob/e0c9350a3f88ca5a37aaac4c7cc9caebf5c309e0/mlir/lib/Dialect/Tosa/IR/TosaOps.cpp#L3267-L3289) casts `input.getType()` to `TensorType` and immediately constructs `SmallVector<OpFoldResult> returnedDims(inputType.getRank())` at line 3275 without checking `inputType.hasRank()`.
Checking `if (!inputType.hasRank()) return failure();` (or using `RankedTensorType` / `transposePerms.size()`) prevents the crash.
## Version
- LLVM trunk on Compiler Explorer (`mliropttrunk`, LLVM 24.0.0git)
- Homebrew LLVM 23.1.5 (macOS arm64; using `{perms = ...}` syntax)
- `llvm-project` `b3df0665` (Linux x86_64, assertions enabled; using `{perms = ...}` syntax)
- Relevant code verified on `main` at `e0c9350a3f88` (2026-11-16)
## Claim
This investigation and write-up were AI-assisted. I've reviewed and verified everything above, and I'm happy to answer questions.
**Assisted-by:** Gemini / Antigravity
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs