jiangxt2 opened a new issue, #12915:
URL: https://github.com/apache/gravitino/issues/12915
### Version
main branch
### Describe what's wrong
ClickHouse data-skipping indexes may use expressions such as `lower(name)`.
When Gravitino loads such an index, the current parser recursively removes
function wrappers and exposes it as a normal `Index` over the column `name`.
Recreating a table from that metadata emits an index over `name`, silently
changing the indexed expression.
Gravitino should not fabricate a representable column index from an
expression it cannot preserve. Plain identifiers and tuples of identifiers
should remain loadable, while unsupported expression indexes should follow the
existing warning-and-skip behavior so the table itself remains loadable.
### Error message and/or stacktrace
No exception is raised. The problem is a silent metadata change:
`lower(name)` is returned through `Index.fieldNames()` as `name`.
### How to reproduce
1. Using the Gravitino main branch and ClickHouse 25.3.2.39, create a native
table with a function-based index expression:
```sql
CREATE TABLE expression_index (
id UInt64,
name String,
INDEX idx_lower lower(name) TYPE minmax GRANULARITY 1
) ENGINE = MergeTree ORDER BY id;
```
2. Load the table through the Gravitino ClickHouse catalog and inspect
`Table.index()`.
3. Observe that `idx_lower` is reported with `fieldNames()` equal to
`[["name"]]`.
4. Recreate a table using the loaded metadata and inspect `SHOW CREATE
TABLE`; the recreated index targets `name` rather than `lower(name)`.
### Additional context
`Index.fieldNames()` represents column references and has no first-class
expression field. This issue proposes skipping unsupported expression indexes
during readback; it does not propose a new expression AST, storing raw SQL in
`Index.properties()`, or accepting arbitrary SQL expressions on CREATE or ALTER.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
