simhadri-g commented on code in PR #4761:
URL: https://github.com/apache/hive/pull/4761#discussion_r1508014085
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputCommitter.java:
##########
@@ -416,12 +417,13 @@ private void commitTable(FileIO io, ExecutorService
executor, OutputTable output
Table table = null;
String branchName = null;
-
+ Expression filterExpr = Expressions.alwaysTrue();
for (JobContext jobContext : outputTable.jobContexts) {
JobConf conf = jobContext.getJobConf();
table = Optional.ofNullable(table).orElse(Catalogs.loadTable(conf,
catalogProperties));
branchName = conf.get(InputFormatConfig.OUTPUT_TABLE_SNAPSHOT_REF);
-
+ filterExpr = Expressions.and(filterExpr, (Expression)
SessionStateUtil.getResource(conf,
Review Comment:
The following example creates 2 job contexts per update query:
Thread 1: UPDATE customers SET first_name='Changed' WHERE
last_name='Taylor'
Thread 2: UPDATE customers SET first_name='Changed' WHERE
last_name='Donnel'
In the previous commit it would have multiple duplicate filters with AND.
But i this commit, i have added a if condition to update the the filter only if
they are different. This should help minimise the duplicate filters as seen in
image below.
<img width="1708" alt="image"
src="https://github.com/apache/hive/assets/26441677/11f02125-5bb4-44e0-a6c0-9d9813279c80";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
