aboueleyes opened a new issue, #17759: URL: https://github.com/apache/iceberg/issues/17759
### Apache Iceberg version 1.11.0 (latest release) ### Query engine None ### Please describe the bug 🐞 The Java REST client uses form encoding for namespace levels, table names, view names, and scan plan IDs placed in URL paths. A space therefore becomes `+` instead of `%20`. Project Nessie treats the plus sign as a literal character in these paths, making names containing spaces unreachable through the Java client. This was reproduced for namespace and table names. View names and scan plan IDs are affected by the same path construction but were not exercised against a live server. The problem remains on Iceberg `main` at commit [`900e88215797a84adcadd3e0d53f7c8e0d17455f`](https://github.com/apache/iceberg/commit/900e88215797a84adcadd3e0d53f7c8e0d17455f). ### Root cause [`RESTUtil.encodeString`](https://github.com/apache/iceberg/blob/900e88215797a84adcadd3e0d53f7c8e0d17455f/core/src/main/java/org/apache/iceberg/rest/RESTUtil.java#L140-L143) uses `java.net.URLEncoder`: ```java public static String encodeString(String toEncode) { Preconditions.checkArgument(toEncode != null, "Invalid string to encode: null"); return URLEncoder.encode(toEncode, StandardCharsets.UTF_8); } ``` `URLEncoder` implements `application/x-www-form-urlencoded`, where a space becomes `+`. That is correct when [`encodeFormData`](https://github.com/apache/iceberg/blob/900e88215797a84adcadd3e0d53f7c8e0d17455f/core/src/main/java/org/apache/iceberg/rest/RESTUtil.java#L113-L123) constructs an OAuth form body. It is not correct when [`ResourcePaths`](https://github.com/apache/iceberg/blob/900e88215797a84adcadd3e0d53f7c8e0d17455f/core/src/main/java/org/apache/iceberg/rest/ResourcePaths.java#L80-L199) places the result in a URL path. The current path construction uses the form encoder for: - namespace levels, through `RESTUtil.encodeNamespace` - table names in table, metrics, signing, planning, and task paths - view names - scan plan IDs For example: ```java public String table(TableIdentifier ident) { return SLASH.join( "v1", prefix, "namespaces", pathEncode(ident.namespace()), "tables", RESTUtil.encodeString(ident.name())); } ``` The current[`TestResourcePaths.cancelPlanEndpointPath`](https://github.com/apache/iceberg/blob/900e88215797a84adcadd3e0d53f7c8e0d17455f/core/src/test/java/org/apache/iceberg/rest/TestResourcePaths.java#L316-L337) also expects `plan with spaces` to become `plan+with+spaces` inside a path. ### Reproduction Environment: - Project Nessie REST catalog at `http://localhost:19120` - catalog prefix `main|warehouse` - namespace `sales report` - table `q1 results` - both objects were created through the catalog API before these checks - `demo`, containing two tables, was used as a control namespace First verify that the namespace exists: ```shell curl -s "http://localhost:19120/iceberg/v1/main%7Cwarehouse/namespaces"; ``` The response includes `sales report`. Request the namespace and table with a percent-encoded space and then with a plus sign. The path suffixes below differ only in that encoding: ```text namespaces/sales%20report -> 200 namespaces/sales+report -> 404 namespaces/sales%20report/tables/q1%20results -> 200 namespaces/sales%20report/tables/q1+results -> 404 namespaces/sales+report/tables/q1%20results -> 404 namespaces/sales+report/tables/q1+results -> 404 ``` `ResourcePaths` constructs: ```text namespace path : v1/main|warehouse/namespaces/sales+report tables path : v1/main|warehouse/namespaces/sales+report/tables table path : v1/main|warehouse/namespaces/sales+report/tables/q1+results ``` End-to-end through `RESTCatalog`: ```text listTables(demo) -> 2 tables listTables(sales report) -> 0 tables loadTable(q1 results) -> NoSuchTableException: Table does not exist: sales+report.q1+results ``` The empty `listTables` result is server-dependent. Nessie returns `200` and an empty identifiers array for the unknown `sales+report` tables path. Another server may return `404`. In both cases, the Java client sends a path for a different name than the one requested. ### Expected behavior Namespace levels, table names, view names, and scan plan IDs should use URL-path encoding. A space should be sent as `%20`, while a literal plus sign should be sent as `%2B`. OAuth form bodies must retain form encoding, where a space is represented by `+`. ### Proposed fix Add a path-specific encoder: ```java public static String encodePathSegment(String toEncode) { Preconditions.checkArgument(toEncode != null, "Invalid string to encode: null"); return URLEncoder.encode(toEncode, StandardCharsets.UTF_8).replace("+", "%20"); } ``` Then: 1. Use `encodePathSegment` for each namespace level in `encodeNamespace`. 2. Replace direct `encodeString` calls for table names, view names, and scan plan IDs in `ResourcePaths`. 3. Keep form encoding in `encodeFormData`. 4. The change preserves existing Iceberg namespace decoding, which accepts both legacy + and new %20 representations. Correct URL-path decoders accept the new representation. Compatibility with catalogs that depend specifically on + in path segments requires validation. The exact method name and visibility are open for maintainer guidance. This direction follows the review recommendation on #15948. ### Suggested regression coverage - namespace levels containing spaces - table and view names containing spaces - scan plan IDs containing spaces - a name containing both a literal plus and a space, such as `my+name with+spaces` - multipart namespaces using the configured namespace separator - legacy namespace decoding from `my+namespace` - `encodeFormData` continuing to encode spaces as `+` - all affected paths constructed by `ResourcePaths` For example, `my+name with+spaces` should be encoded as `my%2Bname%20with%2Bspaces` when used as one path segment. ### Compatibility considerations - The outgoing representation of a space in the affected path segments changes from `+` to `%20`. - A literal plus sign remains `%2B`. - OAuth form-body encoding remains unchanged. - `URLDecoder`, which is currently used by the namespace helper, decodes both `%20` and `+` as a space. - Correct URL-path decoders accept `%20`. Catalog-specific behavior outside the reproduced Nessie case has not been tested here. ### Related work - [#12308](https://github.com/apache/iceberg/issues/12308) reported form encoding in path parameters and was closed automatically after becoming stale. - [#14263](https://github.com/apache/iceberg/issues/14263) reproduced the namespace symptom with Polaris and Spark. Maintainers requested an Iceberg reproduction and discussed the client/server boundary. It was later closed automatically after becoming stale. - [#15948](https://github.com/apache/iceberg/pull/15948) attempted a namespace-only fix. Review requested a separate path encoder, literal-plus and legacy-decoding tests, an OAuth form-data test, and coverage for all affected `ResourcePaths`. The PR was closed automatically for inactivity before those comments were addressed. References: - [Java `URLEncoder`](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/net/URLEncoder.html) - [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986.html) ### Willingness to contribute - [x] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
