nevzheng opened a new pull request, #13333: URL: https://github.com/apache/gravitino/pull/13333
Stacked on #13331 and #13332: please review only the last commit. ### What changes were proposed in this pull request? Add a `PropertiesMetadata.validateProperties(Map)` hook for rules that relate several properties, which a single property entry's decoder cannot check. The default does nothing. `PropertiesMetadataHelpers` calls it at catalog create with the requested properties, and at catalog alter with the stored properties plus the change. `CatalogManager` passes the stored properties to the alter check. The Iceberg catalog implements the hook for the table-format-version properties from #13331. A `table-format-version.default` above `table-format-version.max`, or a conflicting `table-default.format-version`, now fails the create or alter request, and the catalog keeps its properties. The load-time check from #13331 stays for catalogs stored earlier: they load, but every schema and table operation fails with HTTP 400 until an alter fixes them. ### Why are the changes needed? Without this, an invalid combination is accepted, stored, and only fails on the catalog's first operation, which leaves a broken catalog behind. Checking at create and alter makes the misconfiguration visible to the operator who made it. Fix: #13330 ### Does this PR introduce _any_ user-facing change? - A catalog create or alter with an invalid table-format-version combination now fails with HTTP 400 instead of storing a catalog whose operations fail. `docs/lakehouse-iceberg-catalog.md` is updated. - A new default method, `PropertiesMetadata.validateProperties`, for connector authors. Existing implementations are unaffected. ### How was this patch tested? Unit tests only. This PR covers the last row of the format-version coverage (the others are in #13331 and #13332): | Case | Gravitino API | Iceberg REST service | |---|---|---| | Default above max, or conflicting `table-default.format-version`, at catalog create and alter | refused, catalog unchanged | n/a: catalogs are managed through the Gravitino API | New unit tests in `TestPropertiesMetadataHelpers` and `TestCatalogManager` cover the hook at create and at alter with the change applied. `TestIcebergCatalogPropertiesMetadata` covers the Iceberg rules. `./gradlew :core:test :catalogs:catalog-common:test :catalogs:catalog-lakehouse-iceberg:test :iceberg:iceberg-common:test :iceberg:iceberg-rest-server:test -PskipITs`: 2960 tests, 0 failures, 0 errors, 2 skipped (core 2249 with 2 skipped, catalog-common 40, catalog-lakehouse-iceberg 125, iceberg-common 83, iceberg-rest-server 463). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
