veghlaci05 commented on code in PR #4096:
URL: https://github.com/apache/hive/pull/4096#discussion_r1126069879
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##########
@@ -3647,9 +3647,12 @@ long generateCompactionQueueId(Statement stmt) throws
SQLException, MetaExceptio
+ "no record found in next_compaction_queue_id");
}
long id = rs.getLong(1);
- s = "UPDATE \"NEXT_COMPACTION_QUEUE_ID\" SET \"NCQ_NEXT\" = " + (id + 1);
+ s = "UPDATE \"NEXT_COMPACTION_QUEUE_ID\" SET \"NCQ_NEXT\" = " + (id + 1)
+ " WHERE \"NCQ_NEXT\" = " + id;
LOG.debug("Going to execute update <{}>", s);
- stmt.executeUpdate(s);
+ if (stmt.executeUpdate(s) != 1) {
Review Comment:
This is a short term quick and dirty fix, as the id generation for
compaction requests is broken. The old approach did not guarantee a unique id
at all, and since Initiator processes the candidates in a parallel way, this
could be a production issue too. Actually I have seen this duplicate key error
in production logs before. The long term solution is to modify the id to auto
increment field. It is already done this way for TXNS table. However, this
change will be bigger, therefore I created a separate task for it:
https://issues.apache.org/jira/browse/HIVE-27121
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
