yihua commented on code in PR #666:
URL: https://github.com/apache/hudi-rs/pull/666#discussion_r4409410071
##########
crates/core/src/config/table.rs:
##########
@@ -145,6 +145,12 @@ pub enum HudiTableConfig {
/// When creating a metadata table instance, this value should be passed
as the
/// PartitionFields option.
MetadataTablePartitions,
+
+ /// Enable the metadata-table record-level index (RLI) used to tag
upsert/delete keys.
+ ///
+ /// When false, writers fall back to [`crate::index::SimpleIndex`] (full
table scan).
+ /// Requires [`Self::MetadataTableEnabled`].
+ RecordIndexEnabled,
Review Comment:
Addressed in #851 (2c10dffa): the independent `RecordIndexEnabled` config is
gone, and whether the record index is on is read from
`hoodie.table.metadata.partitions`.
##########
crates/core/src/config/table.rs:
##########
@@ -175,6 +181,7 @@ impl AsRef<str> for HudiTableConfig {
Self::TimelineHistoryPath => "hoodie.timeline.history.path",
Self::MetadataTableEnabled => "hoodie.metadata.enable",
Self::MetadataTablePartitions =>
"hoodie.table.metadata.partitions",
+ Self::RecordIndexEnabled => "hoodie.metadata.record.index.enable",
Review Comment:
Addressed in #851 (2c10dffa): the old key went away with
`RecordIndexEnabled`, and the table builder rejects
`hoodie.metadata.global.record.level.index.enable` through `with_option`, since
`with_record_index` sets it (8420bab1).
##########
crates/core/src/metadata/commit.rs:
##########
@@ -193,21 +208,6 @@ impl HoodieCommitMetadata {
.unwrap_or_default()
}
- /// Get the file IDs to be replaced for a specific partition
- pub fn get_partition_replace_file_ids(&self, partition: &str) ->
Option<&Vec<String>> {
Review Comment:
Done in #851 (48736922): `partition_to_replace_file_ids` and its accessors
are back on `HoodieCommitMetadata`, with their tests.
##########
crates/core/src/metadata/table/records.rs:
##########
@@ -282,7 +282,11 @@ pub fn decode_files_partition_record_with_schema(
});
}
- let avro_value = decode_avro_value(value, schema)?;
+ // Prefer the record's own writer schema (from the HFile or log block it
+ // was read from) — `schema` is only a fallback for records with no
+ // container schema. Mixed-writer tables (rs + Spark) have differing
+ // HoodieMetadataRecord schemas per container.
Review Comment:
Addressed in #851 (bdacbd6a): `HFileRecord` is a plain key/value pair again
and the comment is gone. Each HFile or log block now hands its writer schema to
the decoder with its records.
##########
crates/core/src/lib.rs:
##########
@@ -46,12 +46,17 @@
//! }
//! ```
+// The write-path async fns (e.g. `Table::delete_keys`) nest deeply enough
that computing their
+// layout exceeds the default limit of 128 on Linux targets.
+#![recursion_limit = "256"]
+
Review Comment:
Tracked in #860, keeping the limit for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
