JingsongLi opened a new pull request, #10635:
URL: https://github.com/apache/paimon/pull/10635
### Purpose
Enable Native for additional existing PyPaimon upsert key layouts and make
DECIMAL predicate results consistent across Python and Native.
### Changes
- Allow the Native upsert bridge to pass floating-width and
Binary/LargeBinary key families to core. Core validates exact identity;
selected payloads and unmatched-row append validation retain the original
arrays.
- Fix Python bitmap global-index filtering for finer-scale DECIMAL literals.
For example, `DECIMAL(10,2) != Decimal('1.235')` previously rounded the lookup
literal to `1.24` and discarded that matching row. Decline index evaluation
when the serializer cannot represent the literal exactly, leaving the original
predicate for data filtering, consistent with Native.
- Add regressions covering dictionary and composite/partition keys,
duplicate-key last-write-wins across chunks, NULLs, batch/stream writes,
rejected lossy conversions, strict unmatched appends, and a two-byte
LargeBinary key after a 2 GiB mmap prefix.
- Add real BTree/bitmap global-index tests for compact/wide decimals,
comparisons, membership/NULL/range/compound predicates, independent Native
plan/read settings, and predicate updates/callables/deletes.
### Dependency
Depends on https://github.com/apache/paimon-rust/pull/1150. Keep this draft
until it is merged. Native CI continues to build Rust main; no extension pin or
fallback is added.
### Verification
1953 tests passed against the companion extension with Native
plan/read/write execution counters checked. This includes the new
decimal/upsert regressions and existing encoded-input, Dictionary MERGE, MERGE
INTO, writer-capability, decimal-expression, and index-evaluator suites. Flake8
passed for all changed Python files. Two independent review rounds completed
with confirmed findings repaired.
Only current read/write/update paths are changed; no metadata features or
legacy Java compatibility is added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
