[
https://issues.apache.org/jira/browse/HUDI-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kudinkin updated HUDI-3739:
----------------------------------
Status: In Progress (was: Open)
> Fix translation of isNotNull predicates in Data Skipping
> --------------------------------------------------------
>
> Key: HUDI-3739
> URL: https://issues.apache.org/jira/browse/HUDI-3739
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Alexey Kudinkin
> Assignee: Alexey Kudinkin
> Priority: Blocker
> Fix For: 0.11.0
>
>
> Right now isNotNull predicates are translated incorrectly
> {code:java}
> // Filter "colA is not null"
> // Translates to "colA_num_nulls = 0" for index lookup
> case IsNotNull(attribute: AttributeReference) =>
> getTargetIndexedColumnName(attribute, indexSchema)
> .map(colName => EqualTo(genColNumNullsExpr(colName), Literal(0))) {code}
> Instead of "colA_num_nulls = 0" the condition should be "colA_num_nulls !=
> colA_numRecords"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
