jackye1995 commented on pull request #1823: URL: https://github.com/apache/iceberg/pull/1823#issuecomment-752298764
@rdblue sorry for the delay, I went back and forth with myself for this and took more time than expected to solidify different parts. Major changes: 1. `LockManager` interface now has all config values removed out of the method signatures. In both the current use case and the future use case for multi-table transaction, the config values are always fixed and passed from catalog properties, so it can be stored as the lock manager's internal property instead of being a dynamic parameter in a method. A `LockManagerBase` abstract class is used to initialize those common fields. 2. all lock properties are put in `CatalogProperties` instead of `AwsProperties` to be shared across different implementations in the future, and avoid the need to deprecate duplicated config keys in the future. 3. now lock acquisition throws an exception for failure instead of returning false, so it is easier to be used in `Tasks`. 4. `LockId` object is added with an additional `ownerId` field to avoid releasing a lock from a wrong owner. The new metadata location is used as the owner ID. IllegalArgumentException is thrown if owner ID is wrong. 5. `DynamoLockManager` clock skew issue is fixed, now when a lock is acquired, it starts to heartbeat in the background to extend the heartbeat timeout. Lock acquisition will wait for the heartbeat timeout and then try to overwrite an expired lock. With this feature, there is no need to set a very large time for the total commit completion time. The default values can be viewed in `CatalogProperties`, please let me know if the default values look reasonable. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
