JingsongLi opened a new pull request, #1147: URL: https://github.com/apache/paimon-rust/pull/1147
## Purpose Let native Arrow upserts handle conversions already supported by PyPaimon's row-ID updates. Upsert currently normalizes every payload column as an ordinary append before matching. This rejects valid matched-only updates such as STRING to INT and converts values that are later discarded by last-write-wins or excluded by the update column list. ## Changes - Normalize only match and partition keys before deduplication and matching. - Normalize surviving append rows with the ordinary write contract, preserving Java append write-type and file metadata semantics. - Send surviving matched payloads through the existing core row-ID update conversion rules; ignore unwritten payload columns. - Keep binary-containing values on the existing write layout normalization, without introducing Binary/LargeBinary conversions, including through nested and cross-container inputs. - Reuse existing nested field projection for update column selection. No new public API, dependency, or Python-side matching logic. ## Validation - `cargo test -p paimon --lib upsert`: 9 passed. - `cargo +1.98.0 clippy --locked -p paimon --lib --tests -- -D warnings`: passed. - `cargo +1.98.0 fmt --all -- --check`: passed. - Built the Python extension from this branch and ran the selected-value, existing Arrow layout, row-shape, Blob row/update and nested update suites: 495 passed, 2 skipped. - Regression coverage includes last-write-wins across batches, target fan-out, ignored payloads, mixed matched/append input types, invalid append rejection, pinned snapshots, and binary layout rejection. The companion PyPaimon change only adjusts native eligibility and adds end-to-end parity coverage. Its Native CI will require this change on Rust main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
