JingsongLi opened a new pull request, #10661: URL: https://github.com/apache/paimon/pull/10661
### Problem and change Python resolved a data-evolution file's `write_cols` against the current schema. Renaming a column after a column-only update could make the newer provider disappear from the merge plan, silently returning the old value. This affects Parquet reads and ROW sidecar reads alike. Resolve these historical column names against the schema identified by the file's `schema_id`, then use stable field IDs for provider selection. Add real global bitmap/btree regressions with Parquet primary files and aligned ROW sidecars, covering nonzero row IDs, ordinary and nested Variant values, SQL NULL versus Variant null, reordered projections, column-only updates and historical renames. Delete all participating Parquet files and require indexed reads to keep returning the exact updated values. Disabling the global index must fail once those primary files are gone. Java regression coverage also checks file bitmap, global bitmap and global btree queries across single-file and column-merged layouts, including Variant extraction projections. Java production code is unchanged. ### Validation - `mvn -pl paimon-core -am -DwildcardSuites=none '-Dtest=DataEvolutionFileIndexTest#testVariantIndexQueryReadsRowSidecar' -DfailIfNoTests=false test` — 6 passed with normal Maven validation checks. - `PYTHONPATH=paimon-python python3 -m pytest paimon-python/pypaimon/tests/variant_row_sidecar_index_test.py paimon-python/pypaimon/tests/schema_evolution_read_test.py paimon-python/pypaimon/tests/schema_evolution_nested_read_test.py paimon-python/pypaimon/tests/test_format_row_table.py paimon-python/pypaimon/tests/global_index_family_parity_test.py -q` — 82 passed, 33 subtests passed. - `PYTHONPATH=paimon-python python3 -m pytest paimon-python/pypaimon/tests/data_evolution_formats_test.py -k row_sidecar -q` — 7 passed, 2 subtests passed. - `uvx --from 'flake8>=7,<8' flake8 --config paimon-python/dev/cfg.ini paimon-python/pypaimon/read/split_read.py paimon-python/pypaimon/tests/variant_row_sidecar_index_test.py` — passed. - The four Python-written warehouses were also read successfully by standalone Java and Rust probes after all primary Parquet files were removed. Both readers verified real index selections, updated and renamed Variant fields, nested values, NULL distinctions and row IDs 25 through 29. The Python regressions explicitly use Python planning, reading, writing and commit implementations. The regression failed in both column-merged cases before the fix and passes in all four cases afterward. Related Rust sidecar selection fix and regression coverage: https://github.com/apache/paimon-rust/pull/1173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
