JingsongLi opened a new pull request, #10645: URL: https://github.com/apache/paimon/pull/10645
## Dependency Depends on https://github.com/apache/paimon-rust/pull/1168. Keep this draft until it merges. Native CI continues building Rust main; no compatibility probes, branch overrides or missing-method fallbacks are added. ## Summary Complete remaining routing for existing PyPaimon write APIs: - Enable standard query-authorized REST tables for native writes and commits. The server authorizes publication; SELECT policies still govern explicit reads. Preserve the existing guarded selection for read-based updates. - Correct Python writer sequence-number and HASH-bucket restoration to use internal live manifest metadata, including L0, instead of requesting SELECT authorization through public `plan_all_files`. Preserve partition predicates and explicit snapshot/empty restore baselines. - Share `write_ray` coordination with the ordinary writer without constructing a Python local writer. Workers continue selecting their own native data writers through existing builders. - Respect call-level overwrite/static-partition settings when planning postpone Ray writes in both Python and Native routes. New overwrite input can infer fresh bucket counts; explicitly supplied bucket plans remain authoritative. - Keep complete PK `with_write_type` calls on the native writer without adding PK subset support. Preserve the existing non-video group-boundary validation/no-op behavior without triggering fallback after native data has been written. - Document supported native write/commit behavior and the remaining deliberate boundaries. ## Tests Added regression coverage for native routing before/after staging, canonical PK input order, group validation, Ray arguments and partition precedence, ordinary/native postpone planning with old bucket metadata, query-authorized publication without SELECT, continued explicit-read authorization, and snapshot 0/1 dynamic HASH/sequence restoration. Native integration cases also cover overwrite and truncate without Python mutation fallback. ## Verification - Related Python write/commit/dynamic-bucket/sequence suites: **302 passed, 159 skipped, 17 subtests passed**. - Changed Python files: repository flake8 passed; `git diff --check` passed. - Companion Rust core: **4266 passed, 6 ignored**; core and Python-binding Clippy passed. - Independent review found and fixed call-level postpone overwrite planning and checked all shared table-construction callers. - Real Native extension cases remain for CI; no extension was rebuilt or installed locally. Ray coordinator forwarding is tested without requiring Ray locally; distributed execution remains for CI. ## Scope Only existing Python Parquet/BLOB write and commit behavior is completed. No metadata features, compaction, video, other formats, NaN/signed-zero handling, PK subset writes or new public native APIs are introduced. Non-REST commit and arbitrary custom Python FileIO remain on their existing paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
