masfworld opened a new issue, #532: URL: https://github.com/apache/iceberg-go/issues/532
## Summary The iceberg-go REST catalog cannot connect to Google Cloud BigLake because it lacks support for the required `x-goog-user-project` header in the initial `/v1/config` request. ## Problem Description BigLake requires the `x-goog-user-project` header for quota/billing attribution, but the current `rest.NewCatalog()` implementation fails during the initial configuration call. ### Current Behavior ```go // This fails with BigLake cat, err := rest.NewCatalog(ctx, "catalog", "https://biglake.googleapis.com/iceberg/v1beta/restcatalog";, rest.WithOAuthToken(token), rest.WithAdditionalProps(iceberg.Properties{ "header.x-goog-user-project": "my-project", // Doesn't work for initial config call }), ) // Error: "biglake.googleapis.com API requires a quota project" ``` ### Expected Behavior The catalog should successfully connect to BigLake when the proper headers are provided. ## Root Cause 1. `rest.NewCatalog()` makes an initial call to `/v1/config` to fetch catalog configuration 2. This initial call uses the base HTTP client without custom headers from `WithAdditionalProps` 3. BigLake rejects the request due to missing `x-goog-user-project` header 4. The `header.` prefix in additional properties only applies to subsequent calls, not the config call ## Evidence Direct HTTP calls with proper headers work perfectly: ```bash curl -H "Authorization: Bearer $TOKEN" \ -H "x-goog-user-project: my-project" \ "https://biglake.googleapis.com/iceberg/v1beta/restcatalog/v1/config?warehouse=gs://my-bucket"; # Returns 200 OK with catalog configuration ``` ## Proposed Solutions ### Option 1: Add WithHeaders function ```go func WithHeaders(headers map[string]string) Option { // Apply headers to ALL requests including initial config } ``` ### Option 2: Add WithHTTPClient function ```go func WithHTTPClient(client *http.Client) Option { // Allow custom HTTP client injection } ``` ### Option 3: Fix WithAdditionalProps header handling Ensure `header.` prefixed properties in `WithAdditionalProps` are applied to the initial config request. ## Impact This issue prevents iceberg-go from working with: - Google Cloud BigLake - Any REST catalog requiring custom headers for authentication/billing - Enterprise environments with header-based routing/auth ## Workaround Currently requires bypassing `rest.NewCatalog()` and implementing direct HTTP calls to BigLake REST API endpoints. ## Environment - iceberg-go version: v0.3.0 - Go version: 1.21+ - Platform: Google Cloud BigLake REST Catalog ## Related This is similar to how Apache Spark's Iceberg connector handles BigLake via the `spark.sql.catalog.catalog_name.header.x-goog-user-project` configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
