jadewang-db opened a new pull request, #2664:
URL: https://github.com/apache/arrow-adbc/pull/2664
## Description
This PR implements retry-after behavior for the Spark ADBC driver when
receiving 503 responses with Retry-After headers. This is particularly useful
for Databricks clusters that may return 503 responses when a cluster is
starting up or experiencing temporary unavailability.
## Changes
- Added new configuration parameters:
- `adbc.spark.temporarily_unavailable_retry` (default: 1 - enabled)
- `adbc.spark.temporarily_unavailable_retry_timeout` (default: 900 seconds)
- Created a `RetryHttpHandler` class that wraps the existing
`HttpClientHandler` to handle 503 responses
- Modified `SparkHttpConnection` to use the new retry handler
- Added comprehensive unit tests for the retry behavior
## Implementation Details
When a 503 response with a Retry-After header is received:
1. The handler will wait for the number of seconds specified in the header
2. It will then retry the request
3. If another 503 response is received, it will continue retrying
4. If the total retry time exceeds the configured timeout, it will fail with
an appropriate error message
## Testing
Added unit tests to verify:
- Retry behavior for 503 responses with Retry-After headers
- Timeout behavior when retry time exceeds the configured limit
- Handling of invalid or missing Retry-After headers
- Disabling retry behavior via configuration
- Parameter validationv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
