ningsh7 opened a new pull request, #11270:
URL: https://github.com/apache/gravitino/pull/11270
### What changes were proposed in this pull request?
This PR adds configurable HTTP timeout settings for the Iceberg REST catalog
backend used by the Gravitino Iceberg REST service.
The main changes include:
1. Add REST catalog backend HTTP timeout configs in `IcebergConfig`:
- `rest-client-connection-timeout-ms`, default `10000`
- `rest-client-socket-timeout-ms`, default `60000`
2. Apply the timeout configs before initializing the backend `RESTCatalog`
in `IcebergCatalogUtil.loadRestCatalog`.
The properties written to Iceberg are the ones actually recognized by
Iceberg 1.10.1 `HTTPClient`:
- `rest.client.connection-timeout-ms`
- `rest.client.socket-timeout-ms`
3. Add Gravitino-to-Iceberg property mappings for these timeout configs.
4. Add Docker environment variable mappings:
- `GRAVITINO_REST_CLIENT_CONNECTION_TIMEOUT_MS`
- `GRAVITINO_REST_CLIENT_SOCKET_TIMEOUT_MS`
5. Update the Iceberg REST service documentation.
6. Add tests to verify:
- default timeout values;
- configured timeout values;
- Iceberg property aliases;
- rejection of non-positive timeout values;
- mapping from Gravitino config keys to Iceberg property keys;
- timeout properties are applied to the backend REST catalog properties;
- backend timeout properties are not exposed to REST clients through
`/v1/config`.
### Why are the changes needed?
When the catalog backend is a REST catalog, Gravitino initializes Iceberg
`RESTCatalog` without explicit HTTP timeout settings. In this case, Iceberg
falls back to its HTTP client defaults.
If the remote REST catalog is slow or unresponsive, requests from IRC1 to
the backend REST catalog may block server request threads for too long. Under
concurrent load, this can exhaust the server thread pool and make the Iceberg
REST service unresponsive even though the root cause is the downstream REST
catalog.
This PR adds explicit and tunable backend HTTP timeout configs so operators
can choose appropriate timeout values for their network and remote catalog
characteristics.
Note that the issue description mentions `rest.client.connect-timeout-ms`
and `rest.client.read-timeout-ms`. I checked the Iceberg version currently used
by Gravitino, which is Iceberg 1.10.1. In that version, `HTTPClient` actually
reads `rest.client.connection-timeout-ms` and `rest.client.socket-timeout-ms`,
so this PR writes the Iceberg properties that are actually effective. The
issue-mentioned names are also supported as compatible aliases in
`IcebergConfig`.
Fix: #11136
### Does this PR introduce _any_ user-facing change?
Yes.
This PR adds the following user-facing Iceberg REST service configs:
- `gravitino.iceberg-rest.rest-client-connection-timeout-ms`
- HTTP connection timeout in milliseconds for IRC1 requests to the REST
catalog backend.
- Default: `10000`
- `gravitino.iceberg-rest.rest-client-socket-timeout-ms`
- HTTP socket timeout in milliseconds for IRC1 requests to the REST
catalog backend.
- Default: `60000`
It also adds the following Docker environment variable mappings:
- `GRAVITINO_REST_CLIENT_CONNECTION_TIMEOUT_MS`
- `GRAVITINO_REST_CLIENT_SOCKET_TIMEOUT_MS`
These configs are backend-only settings for IRC1 to access the backend REST
catalog, and they are not exposed to REST clients through `/v1/config`.
### How was this patch tested?
Tested locally with JDK 17.
Commands run:
```bash
./gradlew spotlessApply
./gradlew :catalogs:catalog-common:test \
--tests org.apache.gravitino.lakehouse.iceberg.TestIcebergPropertiesUtils
./gradlew :iceberg:iceberg-common:test \
--tests org.apache.gravitino.iceberg.common.TestIcebergConfig \
--tests org.apache.gravitino.iceberg.common.utils.TestIcebergCatalogUtil
./gradlew :iceberg:iceberg-rest-server:test \
--tests org.apache.gravitino.iceberg.service.TestCatalogWrapperForREST
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
