kevinwilfong commented on code in PR #10697:
URL:
https://github.com/apache/incubator-gluten/pull/10697#discussion_r2377196072
##########
cpp/velox/substrait/SubstraitToVeloxPlan.cc:
##########
@@ -1272,26 +1272,30 @@ core::PlanNodePtr
SubstraitToVeloxPlanConverter::toVeloxPlan(const ::substrait::
SubstraitParser::parseColumnTypes(baseSchema, columnTypes);
}
- // Velox requires Filter Pushdown must being enabled.
- bool filterPushdownEnabled = true;
auto names = colNameList;
auto types = veloxTypeList;
- auto dataColumns = ROW(std::move(names), std::move(types));
+ // The columns we project from the file.
+ auto baseSchema = ROW(std::move(names), std::move(types));
+ // The columns present in the table, if not available default to the
baseSchema.
+ auto tableSchema = splitInfo->tableSchema ? splitInfo->tableSchema :
baseSchema;
Review Comment:
Just for clarity, we could always set the full table schema in the
HiveTableHandle and Velox would work (whether or not we're mapping by index or
reading from a text file). The reason I only set it when it's absolutely
necessary is because if the table schema contains types that aren't supported
in the Substrait, sending the table schema would cause the query to fail (see
my reasoning for not putting it in the ReadRel in the description).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
