>From Peeyush Gupta <[email protected]>: Peeyush Gupta has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20664?usp=email )
Change subject: [ASTERIXDB-3679][EXT] Fix parquet filter pushdown ...................................................................... [ASTERIXDB-3679][EXT] Fix parquet filter pushdown - user model changes: no - storage format changes: no - interface changes: no Ext-ref: MB-69838 Change-Id: I7c5455be80e60290fb819b2228018ad9f07603b1 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20664 Reviewed-by: Ali Alsuliman <[email protected]> Reviewed-by: Peeyush Gupta <[email protected]> Tested-by: Peeyush Gupta <[email protected]> Integration-Tests: Jenkins <[email protected]> --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/ParquetFilterPushdownProcessor.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ParquetFilterBuilder.java 2 files changed, 2 insertions(+), 7 deletions(-) Approvals: Ali Alsuliman: Looks good to me, approved Anon. E. Moose #1000171: Peeyush Gupta: Looks good to me, but someone else must approve; Verified Jenkins: Verified diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/ParquetFilterPushdownProcessor.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/ParquetFilterPushdownProcessor.java index f6c94c6..50d5ca6 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/ParquetFilterPushdownProcessor.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/pushdown/processor/ParquetFilterPushdownProcessor.java @@ -79,5 +79,6 @@ } else { scanDefDesc.setRowGroupFilterExpression(inlinedExpr); } + scanDefDesc.getFilterPaths().putAll(paths); } } diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ParquetFilterBuilder.java b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ParquetFilterBuilder.java index 38ad119..6cdef59 100644 --- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ParquetFilterBuilder.java +++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/filter/ParquetFilterBuilder.java @@ -52,9 +52,6 @@ import org.apache.parquet.filter2.predicate.Operators; import org.apache.parquet.io.api.Binary; -import io.delta.kernel.expressions.Column; -import io.delta.kernel.expressions.Predicate; - public class ParquetFilterBuilder extends AbstractFilterBuilder { private static final org.apache.logging.log4j.Logger LOGGER = LogManager.getLogger(); @@ -71,12 +68,9 @@ try { parquetFilterPredicate = createFilterExpression(filterExpression); } catch (Exception e) { - LOGGER.error("Error creating Parquet row-group filter expression ", e); + LOGGER.error("Error creating Parquet row-group filter expression ", e.getMessage()); } } - if (parquetFilterPredicate != null && !(parquetFilterPredicate instanceof Predicate)) { - parquetFilterPredicate = null; - } return parquetFilterPredicate; } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20664?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I7c5455be80e60290fb819b2228018ad9f07603b1 Gerrit-Change-Number: 20664 Gerrit-PatchSet: 4 Gerrit-Owner: Peeyush Gupta <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Peeyush Gupta <[email protected]>
