jiangxt2 opened a new issue, #12748:
URL: https://github.com/apache/gravitino/issues/12748
### Version
main branch
### Describe what's wrong
The ClickHouse catalog reads each column of a composite primary key as a
separate Gravitino `PRIMARY_KEY` index. For a native ClickHouse table with
`PRIMARY KEY (id, ts)`, `loadTable` returns two single-column indexes named
`PRIMARY` instead of one composite index containing both fields in key order.
This loses the identity and ordering of the composite primary key. Metadata
consumers may therefore treat one ClickHouse primary key as multiple
independent constraints, and a metadata round-trip cannot preserve the original
index shape.
Expected index metadata:
```text
type=PRIMARY_KEY, name=PRIMARY, fieldNames=[[id], [ts]]
```
Actual index metadata:
```text
type=PRIMARY_KEY, name=PRIMARY, fieldNames=[[id]]
type=PRIMARY_KEY, name=PRIMARY, fieldNames=[[ts]]
```
### Error message and/or stacktrace
No exception is thrown. The connector silently returns the incorrect index
metadata shown above.
### How to reproduce
Use Gravitino from the main branch and ClickHouse 25.3.2.39.
Create a native ClickHouse table with a composite primary key:
```sql
CREATE TABLE composite_primary_key_test
(
id UInt64,
ts DateTime,
value String,
PRIMARY KEY (id, ts)
)
ENGINE = MergeTree
ORDER BY (id, ts);
```
Configure a Gravitino ClickHouse catalog for the database containing this
table, call `loadTable` for `composite_primary_key_test`, and inspect
`Table.index()`.
Observe that the returned metadata contains two separate single-column
`PRIMARY_KEY` indexes instead of one index whose `fieldNames` are `[[id],
[ts]]`.
### Additional context
This was reproduced against Gravitino main commit
`f08dc93cad344fa37c52cc3a7d897dd101bfef17`. The affected ClickHouse source is
unchanged in current main commit `416ef9bece606eecf843b9c112803c04f1d88893`.
The ClickHouse-specific primary-key query returns one row per key column,
including `PK_NAME=PRIMARY` and the numeric `KEY_SEQ`. For the example above it
returns `id` with sequence 1 and `ts` with sequence 2.
`ClickHouseTableOperations.getIndexes()` currently constructs an `Index` for
each row and does not use `KEY_SEQ`, so the composite key is split during table
loading.
The ClickHouse JDBC driver does not currently provide the required
primary-key metadata through `DatabaseMetaData.getPrimaryKeys`, so the
connector uses this ClickHouse-specific system-table query. The rows need to be
aggregated into one primary-key index in `KEY_SEQ` order while leaving
data-skipping indexes unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
