markhoerth commented on issue #11128: URL: https://github.com/apache/gravitino/issues/11128#issuecomment-4500933705
Thanks both, and thanks @lasdf1234 for #11153. The PR closes the silent-drop, which was the operator-facing bug as filed. Server-level `gravitino.iceberg-rest.table-metadata-cache-*` keys in `gravitino.conf` will now actually take effect on the dynamic path. That is the right fix for the immediate problem. One product gap remains in the new behavior. The test `testNamedDynamicCatalogMergesNamedStaticConfigOnly` makes clear that root server keys only merge into the default dynamic catalog. Named catalogs get only `catalog.<name>.*` entries, not the root settings. For a deployment with many catalogs provisioned through self-service portals, programmatic APIs, and Terraform, an operator still cannot set cache configuration once and have it apply to every dynamic catalog. The per-catalog tedium just moves from the catalog API to `gravitino.conf`: 50 catalogs needs 50 `catalog.<name>.table-metadata-cache-impl` entries, or 50 setProperty calls at creation time. A way to define these once and have them apply consistently across all dynamic catalogs would be beneficial. Per-catalog configuration leaves room for omissions when new catalogs are created, especially when provisioning is distributed across multiple tools or teams. Happy to test the PR against our environment before merge. @roryqi the yaml proposal is worth its own issue, since it sits on the static side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
