baibaichen opened a new pull request, #6745:
URL: https://github.com/apache/incubator-gluten/pull/6745
## What changes were proposed in this pull request?
This PR avoids such transition by returning `Convention` which ouputs both
`VanillaRow` and `BatchType`.
```scala
case _: ColumnarWriteFilesExec.ExecuteWriteCompatible =>
// ColumnarWriteFilesExec neither output Row nor columnar data.
// We output both row and columnar to avoid c2r and r2c transitions.
Convention.of(
Convention.RowType.VanillaRow,
BackendsApiManager.getSparkPlanExecApiInstance.batchType)
```
### why need
`ColumnarWriteFilesExec` neither outputs Columnar batch data nor
`InternalRow`, and hence we can't add c2r or r2c on top of it, otherwise it
would be am internal error.
1. In the normal code path, `DataWritingCommandExec` is on top of
`ColumnarWriteFilesExec` and `DataWritingCommandExec`'s Convention is
`VanillaRow` which avoid c2r transition.
2. In ClickHouse Backend, we are preparing to support native delta write in
Spark 3.5 and delta 3.2, `WriteFile` is directly added on top of delta logical
plan and hence the root `SparkPlan` is `ColumnarWriteFilesExec` which casue
internal error without this change.
(Fixes: \#6705)
## How was this patch tested?
Existed UTs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
