yihua commented on code in PR #666:
URL: https://github.com/apache/hudi-rs/pull/666#discussion_r4409410367
##########
crates/core/src/config/table.rs:
##########
@@ -175,6 +181,7 @@ impl AsRef<str> for HudiTableConfig {
Self::TimelineHistoryPath => "hoodie.timeline.history.path",
Self::MetadataTableEnabled => "hoodie.metadata.enable",
Self::MetadataTablePartitions =>
"hoodie.table.metadata.partitions",
+ Self::RecordIndexEnabled => "hoodie.metadata.record.index.enable",
Review Comment:
Addressed in #851: the old key went away with `RecordIndexEnabled`
(2c10dffa), and `with_option` now reads
`hoodie.metadata.global.record.level.index.enable`, or its alternative
`hoodie.metadata.record.index.enable`, as `with_record_index`, with the primary
key taking precedence like the Java write config (7d9382b5).
##########
crates/core/src/storage/mod.rs:
##########
@@ -398,6 +400,78 @@ impl Storage {
}
Ok(file_metadata)
}
+
+ /// Resolve a relative path to an [`ObjPath`] under this storage's base
URL.
+ fn relative_obj_path(&self, relative_path: &str) -> Result<ObjPath> {
+ let obj_url = join_url_segments(&self.base_url, &[relative_path])?;
+ Ok(ObjPath::from_url_path(obj_url.path())?)
+ }
+
+ /// Write bytes to a path relative to the table base path.
+ pub async fn put_file(&self, relative_path: &str, bytes: impl Into<Bytes>)
-> Result<()> {
+ let obj_path = self.relative_obj_path(relative_path)?;
+ self.object_store
+ .put(&obj_path, PutPayload::from(bytes.into()))
+ .await?;
+ Ok(())
+ }
+
+ /// Write a file only if nothing exists at the path (`PutMode::Create`).
+ ///
+ /// Timeline files must never silently replace one another: two writers
+ /// minting the same instant otherwise clobber each other's commit with no
+ /// error. An existing object surfaces as
+ /// [`StorageError::AlreadyExists`](error::StorageError) so callers can
+ /// report the conflict.
+ pub async fn put_file_if_absent(
Review Comment:
It does already: `put_file_if_absent` uses `PutMode::Create`, and
object_store 0.13 sends that to S3 as a conditional PUT (`If-None-Match: *`,
its default `ETagMatch` mode). The create is not retried on a timeout, so a
retry cannot turn a PUT that landed into `AlreadyExists`. Setting
`aws_conditional_put=disabled` makes these writes fail with `NotImplemented`
rather than overwrite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
