jackye1995 commented on a change in pull request #4216: URL: https://github.com/apache/iceberg/pull/4216#discussion_r816090930
########## File path: docs/versioned/integrations/aws.md ########## @@ -188,21 +180,22 @@ However, if you are streaming data to Iceberg, this will easily create a lot of Therefore, it is recommended to turn off the archive feature in Glue by setting `glue.skip-archive` to `true`. For more details, please read [Glue Quotas](https://docs.aws.amazon.com/general/latest/gr/glue.html) and the [UpdateTable API](https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateTable.html). -#### DynamoDB for Commit Locking +#### Optimistic Locking -Glue does not have a strong guarantee over concurrent updates to a table. -Although it throws `ConcurrentModificationException` when detecting two processes updating a table at the same time, -there is no guarantee that one update would not clobber the other update. -Therefore, [DynamoDB](https://aws.amazon.com/dynamodb) can be used for Glue, so that for every commit, -`GlueCatalog` first obtains a lock using a helper DynamoDB table and then try to safely modify the Glue table. +By default, Iceberg uses Glue's optimistic locking for concurrent updates to a table. +With optimistic locking, each table has a version id. +If you retrieve the table metadata, Iceberg records the version id of that table. +You can update the table, but only if the version id on the server side has not changed. +If there is a version mismatch, it means that someone else has modified the table before you did. +The update attempt fails, because you have a stale version of the table. +If this happens, Iceberg simply tries again by retrieving the table metadata and then tries to update it. Review comment: > Iceberg simply tries again Should not say simply, Iceberg refreshes metadata and checks if there might be potential conflict. If there is no commit conflict, the operation will be retried. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
