CTTY commented on code in PR #1484:
URL: https://github.com/apache/iceberg-rust/pull/1484#discussion_r2178995973
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -152,13 +159,59 @@ impl Transaction {
}
/// Commit transaction.
- pub async fn commit(mut self, catalog: &dyn Catalog) -> Result<Table> {
+ pub async fn commit(self, catalog: &dyn Catalog) -> Result<Table> {
if self.actions.is_empty() {
// nothing to commit
return Ok(self.table);
}
- self.do_commit(catalog).await
+ let backoff = Self::build_backoff(self.table.metadata().properties());
+ let tx = self;
+
+ (|mut tx: Transaction| async {
+ let result = tx.do_commit(catalog).await;
+ (tx, result)
+ })
+ .retry(backoff)
+ .sleep(tokio::time::sleep)
+ .context(tx)
+ .when(|e| e.retryable())
+ .await
+ .1
+ }
+
+ fn build_backoff(props: &HashMap<String, String>) -> ExponentialBackoff {
+ ExponentialBuilder::new()
+ .with_min_delay(Duration::from_millis(
+ props
+ .get(COMMIT_MIN_RETRY_WAIT_MS)
+ .map(|s| s.parse())
+ .unwrap_or_else(|| Ok(COMMIT_MIN_RETRY_WAIT_MS_DEFAULT))
+ .expect("Invalid value for commit.retry.min-wait-ms"),
+ ))
+ .with_max_delay(Duration::from_millis(
+ props
+ .get(COMMIT_MAX_RETRY_WAIT_MS)
+ .map(|s| s.parse())
+ .unwrap_or_else(|| Ok(COMMIT_MAX_RETRY_WAIT_MS_DEFAULT))
+ .expect("Invalid value for commit.retry.max-wait-ms"),
+ ))
+ .with_total_delay(Some(Duration::from_millis(
+ props
+ .get(COMMIT_TOTAL_RETRY_TIME_MS)
+ .map(|s| s.parse())
+ .unwrap_or_else(|| Ok(COMMIT_TOTAL_RETRY_TIME_MS_DEFAULT))
+ .expect("Invalid value for commit.retry.total-timeout-ms"),
+ )))
+ .with_max_times(
+ props
+ .get(COMMIT_NUM_RETRIES)
+ .map(|s| s.parse())
+ .unwrap_or_else(|| Ok(COMMIT_NUM_RETRIES_DEFAULT))
+ .expect("Invalid value for commit.retry.num-retries"),
+ )
+ .with_factor(2.0)
+ .build()
Review Comment:
Not really related to this change, but something I noticed while writing the
PR:
1. There is not a centralized properties class, which makes properties hard
to locate in the code base: Some properties are defined in table_metadata.rs,
and some are defined in the classes that use them:
[example](https://github.com/apache/iceberg-rust/blob/e93dd04e1ba661c783b05b5dced424c624be8af5/crates/iceberg/src/writer/file_writer/location_generator.rs#L34)
2. No orthogonal way to fetch and parse property value, which makes error
handling inconsistent. For example, this
[LOC](https://github.com/apache/iceberg-rust/blob/772ae1ab194aa039b653648d9d8310d074609619/crates/iceberg/src/transaction/snapshot.rs#L300)
will silently fallback to the default value if property parsing failed
For 1, we can have a dedicated file like `table_metadata.rs` to hold all
hardcoded property keys, or we can have a property struct to define properties
we support like [what datafusion
does](https://github.com/apache/datafusion/blob/9d06baf45298bc736966d0239fa78ec7a434ca54/datafusion/common/src/config.rs#L197)
For 2, something like iceberg-java's
[PropertyUtil](https://github.com/apache/iceberg/blob/fe23584fc3af9f0ea1371989030b0a99affb233f/core/src/main/java/org/apache/iceberg/util/PropertyUtil.java#L35)
can be very useful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
