jrmccluskey commented on code in PR #39004:
URL: https://github.com/apache/beam/pull/39004#discussion_r3507283460
##########
sdks/java/ml/inference/remote/src/main/java/org/apache/beam/sdk/ml/inference/remote/RetryHandler.java:
##########
@@ -34,24 +34,42 @@ public class RetryHandler implements Serializable {
private final Duration initialBackoff;
private final Duration maxBackoff;
private final Duration maxCumulativeBackoff;
+ private final RetryFilter retryFilter;
+
+ @FunctionalInterface
+ public interface RetryFilter extends Serializable {
+ boolean shouldRetry(Exception e);
+ }
private RetryHandler(
- int maxRetries, Duration initialBackoff, Duration maxBackoff, Duration
maxCumulativeBackoff) {
+ int maxRetries,
+ Duration initialBackoff,
+ Duration maxBackoff,
+ Duration maxCumulativeBackoff,
+ RetryFilter retryFilter) {
this.maxRetries = maxRetries;
this.initialBackoff = initialBackoff;
this.maxBackoff = maxBackoff;
this.maxCumulativeBackoff = maxCumulativeBackoff;
+ this.retryFilter =
+ java.util.Objects.requireNonNull(retryFilter, "retryFilter must not be
null");
}
public static RetryHandler withDefaults() {
return new RetryHandler(
3, // maxRetries
Duration.standardSeconds(1), // initialBackoff
Duration.standardSeconds(10), // maxBackoff per retry
- Duration.standardMinutes(1) // maxCumulativeBackoff
+ Duration.standardMinutes(1), // maxCumulativeBackoff
+ e -> true // retryFilter default to retrying all exceptions
);
}
+ public RetryHandler withRetryFilter(RetryFilter retryFilter) {
Review Comment:
My game plan here is to get all of the missing features added (retry filters
here, batching, client-side throttling) and then 1) add them to the existing
Open AI implementation and 2) add a handler using the Java Gemini SDK
(https://github.com/googleapis/java-genai)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
