JingsongLi opened a new pull request, #10505:
URL: https://github.com/apache/paimon/pull/10505
## Summary
PyPaimon already supports caller `snapshot_properties` in batch, stream and
overwrite commits. The presence of an explicit property map previously forced
the Python publication path, even for `{}` and REST tables which otherwise
qualified for Native.
- Route eligible commits with properties through Native and pass metadata to
the existing batch/stream commit operation.
- Keep callback and non-REST routing on the Python path.
- Preserve the preparation/publication exception boundary: errors once
Native publication begins must not trigger another publication or abort
prepared files.
- Add REST tests for Append, PK and Data Evolution, stream checkpoint
isolation, empty commits, static/dynamic empty overwrite and lost publication
responses.
Properties describe the current commit, are not inherited by subsequent
checkpoints, and empty maps are absent from snapshot metadata. These rules
follow Java `ManifestCommittable.properties` and `FileStoreCommitImpl` snapshot
construction. No new Python capability or API is introduced.
## Dependency
Depends on https://github.com/apache/paimon-rust/pull/1104 for Rust core
snapshot-property publication and the optional binding argument. This PR is
draft until that change reaches Rust main. Native CI continues to build
upstream `apache/paimon-rust@main`.
## Verification
Built the companion Rust extension and enabled Native plan/read/write/commit
for:
```
pypaimon/tests/native_snapshot_properties_test.py
pypaimon/tests/native_commit_test.py
pypaimon/tests/table_commit_test.py
```
- 117 tests passed, with 68 Native plans, 56 reads, 54 writes and 73 commits
exercised.
- Existing Python `table_commit_test.py` and `table/simple_table_test.py`:
41 passed.
- Rust commit regression: 121 passed; Python binding commit suite: 58 passed.
- Clippy 1.98.0, Rust formatting, Python Flake8, Python 3.6 syntax parsing
and diff checks passed.
- Independent core/API and Python adapter reviews have no remaining findings.
New cases assert that property maps cannot leak between stream checkpoints,
empty-map metadata is omitted, static empty overwrite publishes properties
while dynamic empty overwrite creates no snapshot, and publication response
loss preserves both metadata and referenced data files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
