difin commented on code in PR #5621:
URL: https://github.com/apache/hive/pull/5621#discussion_r2072244617
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java:
##########
@@ -317,16 +317,22 @@ public List<String> findColumnsWithStats(CompactionInfo
ci) throws MetaException
@Override
public void updateCompactorState(CompactionInfo ci, long compactionTxnId)
throws MetaException {
+ boolean runInTxn = compactionTxnId > 0;
jdbcResource.execute(
"UPDATE \"COMPACTION_QUEUE\" SET \"CQ_HIGHEST_WRITE_ID\" =
:highestWriteId, " +
- "\"CQ_RUN_AS\" = :runAs, \"CQ_TXN_ID\" = :txnId WHERE \"CQ_ID\" =
:id",
+ "\"CQ_RUN_AS\" = :runAs, \"CQ_TXN_ID\" = :txnId, \"CQ_TYPE\" =
:type WHERE \"CQ_ID\" = :id",
new MapSqlParameterSource()
.addValue("highestWriteId", ci.highestWriteId)
Review Comment:
Correct, it is set. I was confused thinking it was not set because in the
output of `SHOW COMPACTIONS` command after running Iceberg compaction `Highest
WriteId` is null.
```
CompactionId Database Table Partition Type State Worker
host Worker Enqueue Time Start Time Duration(ms) HadoopJobId
Error message Initiator host Initiator Pool name TxnId Next
TxnId Commit Time Highest WriteId
#Masked# default ice_orc dept_id=2 MAJOR succeeded
#Masked# manual default 0 0 0 ---
#Masked# default ice_orc dept_id=3 SMART_OPTIMIZE refused
#Masked# manual default 0 0 0 ---
#Masked# default ice_orc --- SMART_OPTIMIZE refused #Masked#
manual default 0 0 0 ---
```
But it appears to be an existing bug in Hive in
`ShowCompactHandler#extractData`:
it sets `highestTxnId` instead of `highestWriteId`
https://github.com/apache/hive/blob/208488089df71e8a0dafd95b2812e5e4864baed9/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/jdbc/queries/ShowCompactHandler.java#L174
`highestWriteId` remains unset in `ShowCompactResponse` and is shown as
"---" in the `SHOW COMPACTIONS` output.
I can open a JIRA for this defect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
