pvary commented on a change in pull request #1085:
URL: https://github.com/apache/hive/pull/1085#discussion_r442034137
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -5363,6 +5363,121 @@ private void acquireTxnLock(Statement stmt, boolean
shared) throws SQLException,
LOG.debug("TXN lock locked by {} in mode {}",
quoteString(TxnHandler.hostname), shared);
}
+
+ @Override
+ @RetrySemantics.Idempotent
+ public void requestCleanup(CompactionInfo ci) throws MetaException {
+ try {
+ Connection dbConn = null;
+ Statement stmt = null;
+ PreparedStatement pst = null;
+ TxnStore.MutexAPI.LockHandle handle = null;
+ try {
+ lockInternal();
+ /**
+ * MUTEX_KEY.CompactionScheduler lock ensures that there is only 1
entry in
+ * Initiated/Working state for any resource. This ensures that we
don't run concurrent
+ * compactions for any resource.
+ */
+ handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
+ dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
+ stmt = dbConn.createStatement();
+
+ long id = generateCompactionQueueId(stmt);
+
+ List<String> params = new ArrayList<>();
+ StringBuilder sb = new StringBuilder("select cq_id, cq_state from
COMPACTION_QUEUE where").
+ append(" cq_state IN(").append(quoteChar(INITIATED_STATE)).
+ append(",").append(quoteChar(WORKING_STATE)).
+ append(") AND cq_database=?").
+ append(" AND cq_table=?").append(" AND ");
Review comment:
Further clarified thoughts:
- It would be good to have a single method inserting to the COMPACTION_QUEUE
table
- We should use a preparedstatement for this so JDBC execution could be
faster
- TxnHandler.compact() should check for only INITIATED/WORKING status - it
might still worth tho start a new compaction, even if the cleanup not finished
yet
- TxnHandler.requestCleanup() should check for
INITIATED/WORKING/READY_TO_CLEAN status (the first 2 should not be there
anyway), so we do not queue multiple compactions for the same table/partition
Thanks,
Peter
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
