zhztheplayer commented on code in PR #6745:
URL: https://github.com/apache/incubator-gluten/pull/6745#discussion_r1708181313
##########
gluten-core/src/main/scala/org/apache/gluten/extension/columnar/transition/ConventionFunc.scala:
##########
@@ -92,6 +92,12 @@ object ConventionFunc {
}
val conv = Convention.of(rowType, batchType)
conv
+ case _: ColumnarWriteFilesExec.ExecuteWriteCompatible =>
+ // ColumnarWriteFilesExec neither output Row nor columnar data.
+ // We output both row and columnar to avoid c2r and r2c transitions.
+ Convention.of(
+ Convention.RowType.VanillaRow,
+ BackendsApiManager.getSparkPlanExecApiInstance.batchType)
Review Comment:
Basically this list is for overriding conventions of vanilla Spark's
operators. Would you use interface `KnowBatchType` / `KnownRowType` instead?
##########
gluten-core/src/main/scala/org/apache/gluten/extension/columnar/transition/ConventionFunc.scala:
##########
@@ -156,6 +158,10 @@ object ConventionFunc {
case write: DataWritingCommandExec if
SparkShimLoader.getSparkShims.isPlannedV1Write(write) =>
// To align with
ApplyColumnarRulesAndInsertTransitions#insertTransitions
ConventionReq.any
+ case _: ColumnarWriteFilesExec.ExecuteWriteCompatible =>
+ ConventionReq.of(
+ ConventionReq.RowType.Any,
+
ConventionReq.BatchType.Is(BackendsApiManager.getSparkPlanExecApiInstance.batchType))
Review Comment:
The same. Use interface `KnownChildrenConventions` instead.
##########
gluten-core/src/main/scala/org/apache/gluten/extension/columnar/transition/Transitions.scala:
##########
@@ -107,8 +100,13 @@ object Transitions {
}
private def toBatchPlan(plan: SparkPlan, toBatchType: Convention.BatchType):
SparkPlan = {
+ convert(
+ plan,
+ ConventionReq.of(ConventionReq.RowType.Any,
ConventionReq.BatchType.Is(toBatchType)))
+ }
+
+ private def convert(plan: SparkPlan, req: ConventionReq): SparkPlan = {
Review Comment:
I'd suggest rename `convert` which sounds too general. How about
``enforceReq`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
