pvary commented on PR #5036: URL: https://github.com/apache/iceberg/pull/5036#issuecomment-1155994292
@SinghAsDev: Quick question: Did you face a specific issue around this, or we are just fixing a long outstanding `TODO` here? I am asking, because I have a feeling that the need for heartbeat is more like a theoretical issue than a concrete one. The code is like this: ``` lock loadTable if needed createTable some object value settings persistTable ``` The `acquireLock` part should heartbeat the lock until the lock is acquired, and the other parts should be fast. I was trying to find situations where we need heartbeat, and I was able to come up with these: - Wrong configuration: Too low heartbeat interval configured on the HMS - JVM pause on the client, or the HMS side For the JVM pause situations the problem is still unlikely to happen (even without the heartbeat), and the heartbeating thread is also suspended in this case, so there is no guarantees that it will help. Since Hive 4 is around the release, we might try to add a `checkAndSet` method to the HMS API for setting the `metadata_location` atomically. This could be the ultimate solution for the issue, if we find this fix important. All that said: using the default configuration on the HMS, the timeout is set to 300s. If we can not finish the commit in 5 mins, then we already are in serious trouble. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
