zhuanshenbsj1 opened a new pull request, #10537:
URL: https://github.com/apache/paimon/pull/10537
### Purpose
Global index files are bound to bucket 0 and keyed by the table-wide
`_ROW_ID`
space, so every global index type requires an unaware-bucket table (`bucket
= -1`).
In pypaimon this requirement was only enforced for *generic* global indexes
(vector / full-text), because the bucket check lived inside
`_validate_generic_index_table()`, which is only called for
`_GENERIC_INDEX_IDENTIFIERS`. Sorted index types (btree / bitmap) skipped the
check entirely, so a global index could be built on a `bucket != -1` table,
producing a semantically wrong index.
How Java handles this, for reference:
- For **generic** global index, Java validates explicitly in
`GenericGlobalIndexScanner#scan()`:
`checkArgument(table.coreOptions().bucket() == -1, ...)`.
- For **sorted / data-evolution** global index, Java has no explicit bucket
check (`DataEvolutionGlobalIndexScanner` does not check bucket); it relies
on
schema-level validation guaranteeing `bucket = -1` at table creation
(`data-evolution` requires `row-tracking`, which requires an unaware-bucket
table with no primary key).
pypaimon tables created by older versions may still carry row tracking with a
fixed bucket (they skipped that schema validation), so relying on the
schema-level guarantee is not safe here. This PR adds an explicit bucket
check
for **all** global index types at index creation.
### Tests
- `tests/global_index_build_test.py`: new
`test_create_sorted_global_index_rejects_legacy_fixed_bucket_table`.
It builds a legacy fixed-bucket table (a legal table whose committed schema
file is then overwritten to set `bucket = 4`, simulating a table left by an
older pypaimon version), and asserts that creating a btree or bitmap global
index on it is rejected — including when the caller passes
`options={'bucket':'-1'}`,
proving the check reads the table options and cannot be bypassed. It also
asserts no index snapshot is committed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
