cmathiesen commented on a change in pull request #1326:
URL: https://github.com/apache/iceberg/pull/1326#discussion_r472896565
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java
##########
@@ -51,6 +58,17 @@
forwardConfigSettings(job);
+ //Convert Hive filter to Iceberg filter
+ String hiveFilter = job.get(TableScanDesc.FILTER_EXPR_CONF_STR);
Review comment:
Yeah I think you're right here, and I was initially attempting to do it
that way too but I was having some trouble finding the correct method of doing
that. I attempted to update the conf in `decomposePredicate` but the way I was
doing it caused some bugs where the value I was setting wasn't getting reset
between queries if one query had a filter (and decomposePredicate would get
called) and the next didn't (so decomposePredicate doesn't get called). And in
the other methods like `configureJobConf`, the filter information wasn't
available so it couldn't be set.
I would really appreciate some input on this, I may have missed something
with how the StorageHandler works with Hive!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
