yuchuanchen commented on code in PR #21563:
URL: https://github.com/apache/flink/pull/21563#discussion_r1067625428
##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableSource.java:
##########
@@ -282,12 +296,37 @@ public void applyDynamicFiltering(List<String>
candidateFilterFields) {
@Override
public boolean supportsNestedProjection() {
- return false;
+ if (!flinkConf.get(SOURCE_NESTED_PROJECTION_PUSHDOWN_ENABLED)
+ || flinkConf.get(TABLE_EXEC_HIVE_FALLBACK_MAPRED_READER)) {
+ return false;
+ }
+ List<String> supportedFormats =
+
flinkConf.get(SOURCE_NESTED_PROJECTION_PUSHDOWN_SUPPORTED_FORMATS);
+ return supportedFormats != null
+ &&
supportedFormats.stream().anyMatch(getSerializationLib()::contains);
+ }
+
+ private String getSerializationLib() {
Review Comment:
getSerializationLib() accesses hive metastore to find out whether it is
parquet format(I didn't find another way, pls correct me if I am wrong). And
currently, only parquet format supports nested projection pushdown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
