jiangxt2 opened a new issue, #13376: URL: https://github.com/apache/gravitino/issues/13376
### Describe the feature Preserve metadata for legacy `annoy` and `usearch` data-skipping indexes when the ClickHouse catalog loads existing tables. Keep each legacy index distinguishable from the current `vector_similarity` HNSW index. ### Motivation The catalog currently warns and skips unrecognized index types, which can lose legacy index metadata during table loading. ClickHouse replaced the experimental `annoy` and `usearch` types with `vector_similarity` in v24.8. Its [upstream migration PR](https://github.com/ClickHouse/ClickHouse/pull/63675) states that old table definitions may still load, but inserts and vector searches through those legacy indexes fail and migration is recommended. Treating either old name as HNSW would misrepresent the original index. ### Describe the solution Provide a Gravitino representation that keeps `annoy` and `usearch` distinct and retains the legacy metadata exposed by ClickHouse. The catalog must not silently recreate either legacy type as `vector_similarity`. If the target ClickHouse version cannot create a legacy type, the catalog should return a clear error with table and index context. Migration to `vector_similarity` should remain explicit. ### Additional context Related to [#11935](https://github.com/apache/gravitino/issues/11935), which covers the current `vector_similarity` index after its scope is narrowed. This issue covers legacy metadata compatibility; it does not restore ClickHouse insert or vector-search support for legacy indexes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
