yihua commented on code in PR #666:
URL: https://github.com/apache/hudi-rs/pull/666#discussion_r4407455317
##########
crates/core/src/metadata/table/records.rs:
##########
@@ -282,7 +282,11 @@ pub fn decode_files_partition_record_with_schema(
});
}
- let avro_value = decode_avro_value(value, schema)?;
+ // Prefer the record's own writer schema (from the HFile or log block it
+ // was read from) — `schema` is only a fallback for records with no
+ // container schema. Mixed-writer tables (rs + Spark) have differing
+ // HoodieMetadataRecord schemas per container.
Review Comment:
The MDT writer schema mismatch between hudi-rs and Spark Java writers should
already be fixed; let's revisit the logic and update the comment.
##########
crates/core/src/storage/mod.rs:
##########
@@ -398,6 +400,78 @@ impl Storage {
}
Ok(file_metadata)
}
+
+ /// Resolve a relative path to an [`ObjPath`] under this storage's base
URL.
+ fn relative_obj_path(&self, relative_path: &str) -> Result<ObjPath> {
+ let obj_url = join_url_segments(&self.base_url, &[relative_path])?;
+ Ok(ObjPath::from_url_path(obj_url.path())?)
+ }
+
+ /// Write bytes to a path relative to the table base path.
+ pub async fn put_file(&self, relative_path: &str, bytes: impl Into<Bytes>)
-> Result<()> {
+ let obj_path = self.relative_obj_path(relative_path)?;
+ self.object_store
+ .put(&obj_path, PutPayload::from(bytes.into()))
+ .await?;
+ Ok(())
+ }
+
+ /// Write a file only if nothing exists at the path (`PutMode::Create`).
+ ///
+ /// Timeline files must never silently replace one another: two writers
+ /// minting the same instant otherwise clobber each other's commit with no
+ /// error. An existing object surfaces as
+ /// [`StorageError::AlreadyExists`](error::StorageError) so callers can
+ /// report the conflict.
+ pub async fn put_file_if_absent(
Review Comment:
Should this use S3's conditional PUT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
