This is an automated email from the ASF dual-hosted git repository.
xuanwo pushed a change to branch merge-catalog-memory
in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git
from b61ad8de Address comments
add 69686ba8 add parsing of positional delete files to the
`DeleteFileLoader`
add 8edd7087 fix: fix batch transform modify schema (#1483)
add 36cc1208 feat(datafusion): support metadata tables for Datafusion
(#879)
add a3bf8294 refine: add new_manifest_writer in SnapshotProducer (#1481)
add d9fbc5c9 refine: remove data_file_content in FileScanTask (#1485)
add 2cb3fb51 chore(deps): Bump crate-ci/typos from 1.33.1 to 1.34.0 (#1491)
add 27dc7202 chore(deps): Bump serde_with from 3.13.0 to 3.14.0 (#1494)
add 11908149 chore(deps): Bump tokio from 1.45.0 to 1.46.1 (#1493)
add 96ec4d5b chore(deps): Bump the arrow-parquet group with 4 updates
(#1492)
add 3fa90278 typo: FILED_ID_PROP -> FIELD_ID_PROP (#1497)
add c1aa7fdb Merge branch 'main' into merge-catalog-memory
No new revisions were added by this update.
Summary of changes:
.github/workflows/ci_typos.yml | 2 +-
Cargo.lock | 94 ++++++++++----
Cargo.toml | 1 +
crates/iceberg/Cargo.toml | 2 +
.../src/arrow/caching_delete_file_loader.rs | 66 ++++++++--
crates/iceberg/src/arrow/delete_file_loader.rs | 133 +-------------------
crates/iceberg/src/arrow/delete_filter.rs | 60 ++++++---
crates/iceberg/src/arrow/reader.rs | 5 +-
.../iceberg/src/arrow/record_batch_transformer.rs | 9 +-
crates/iceberg/src/avro/schema.rs | 14 +--
crates/iceberg/src/delete_vector.rs | 9 ++
crates/iceberg/src/inspect/manifests.rs | 7 +-
crates/iceberg/src/inspect/metadata_table.rs | 103 ++++++---------
crates/iceberg/src/inspect/snapshots.rs | 7 +-
crates/iceberg/src/lib.rs | 1 +
crates/iceberg/src/scan/context.rs | 1 -
crates/iceberg/src/scan/mod.rs | 2 -
crates/iceberg/src/scan/task.rs | 3 -
crates/iceberg/src/test_utils.rs | 78 ++++++++++++
crates/iceberg/src/transaction/snapshot.rs | 48 +++----
.../tests/shared_tests/read_positional_deletes.rs | 17 +--
.../datafusion/src/physical_plan/metadata_scan.rs | 92 ++++++++++++++
.../datafusion/src/physical_plan/mod.rs | 1 +
crates/integrations/datafusion/src/schema.rs | 47 +++++--
.../datafusion/src/table/metadata_table.rs | 87 +++++++++++++
crates/integrations/datafusion/src/table/mod.rs | 10 ++
.../tests/integration_datafusion_test.rs | 138 ++++++++++++++++++++-
27 files changed, 716 insertions(+), 321 deletions(-)
create mode 100644 crates/iceberg/src/test_utils.rs
create mode 100644
crates/integrations/datafusion/src/physical_plan/metadata_scan.rs
create mode 100644 crates/integrations/datafusion/src/table/metadata_table.rs
