Arina Ielchiieva created DRILL-6259:
---------------------------------------
Summary: Implement parquet filter push down for complex types
Key: DRILL-6259
URL: https://issues.apache.org/jira/browse/DRILL-6259
Project: Apache Drill
Issue Type: Improvement
Affects Versions: 1.13.0
Reporter: Arina Ielchiieva
Assignee: Arina Ielchiieva
Fix For: 1.14.0
Currently parquet filter push down is not working for complex types (including
arrays).
This Jira aims to implement filter push down for complex types which underneath
type is among supported simple types for filter push down. For instance,
currently Drill does not support filter push down for varchars, decimals etc.
Though once Drill will start support, this support will be applied for complex
type automatically.
Complex fields will be pushed down the same way regular fields are, except for
one case with arrays.
Query with predicate {{where users.hobbies_ids[2] is null}} won't be able to
push down because we are not able to determine exact number of nulls in arrays
fields.
{{Consider [1, 2, 3]}} vs {{[1, 2]. If}} these arrays are in different files.
Statistics for the second case won't show any nulls but when querying from two
files, in terms of data the third value in array is null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
