[ https://issues.apache.org/jira/browse/DRILL-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anton Gozhiy closed DRILL-6259. ------------------------------- Verified with Drill version 1.14.0-SNAPSHOT, commit id: 24193b1b038a6315681a65c76a67034b64f71fc5 > Support parquet filter push down for complex types > -------------------------------------------------- > > Key: DRILL-6259 > URL: https://issues.apache.org/jira/browse/DRILL-6259 > Project: Apache Drill > Issue Type: Improvement > Affects Versions: 1.13.0 > Reporter: Arina Ielchiieva > Assignee: Arina Ielchiieva > Priority: Major > Labels: ready-to-commit > Fix For: 1.14.0 > > > Currently parquet filter push down is not working for complex types > (including arrays). > This Jira aims to implement filter push down for complex types which > underneath type is among supported simple types for filter push down. For > instance, currently Drill does not support filter push down for varchars, > decimals etc. Though once Drill will start support, this support will be > applied for complex type automatically. > Complex fields will be pushed down the same way regular fields are, except > for one case with arrays. > Query with predicate {{where users.hobbies_ids[2] is null}} won't be able to > push down because we are not able to determine exact number of nulls in > arrays fields. > {{Consider [1, 2, 3]}} vs {{[1, 2]}} if these arrays are in different files. > Statistics for the second case won't show any nulls but when querying from > two files, in terms of data the third value in array is null. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)