dongjoon-hyun opened a new pull request, #860: URL: https://github.com/apache/spark-kubernetes-operator/pull/860
### What changes were proposed in this pull request? This PR makes `ReconcilerUtils.isTransientError` classify the failures fabric8 reports without a response status, and makes `shouldBackoffBeforeRetry` delay before retrying them. `KubernetesClientException.getCode()` returns `-1`, not `0`, when no response status was received, so the existing `case 0` never matched anything the client produces. A new `NO_RESPONSE_CODE` constant carries that value, and a new private `brokeOnTheWayToTheApiServer` decides whether such a failure left the request unanswered. It is written as the complement of the failures that repeating cannot change — an unparsable answer, a handshake rejected over a certificate, and a rejection raised before sending, which carries no cause at all — so it does not depend on the HTTP client in use. That matters because the pinned client reports a connection the peer closes mid-response with a type of its own that is not even an `IOException`. `shouldBackoffBeforeRetry` gains the same code: an unanswered request carries no `Status` and so can never supply the `Retry-After` hint that is the only other way to earn a delay. ### Why are the changes needed? No failure ever reported code `0`, so every failure that did not reach a healthy API server was treated as a decision by that API server. Verified against the pinned client by pointing a real `KubernetesClient` at local sockets: | Failure | `getCode()` | Innermost cause | Classified | | --- | --- | --- | --- | | Connection refused | `-1` | `java.net.ConnectException` | transient | | Connect timeout | `-1` | `io.netty.channel.ConnectTimeoutException` | transient | | Read timeout | `-1` | `io.vertx.core.impl.NoStackTraceTimeoutException` | transient | | Connection reset | `-1` | `java.net.SocketException` | transient | | Peer closed mid-response | `-1` | `io.vertx.core.http.HttpClosedException` | transient | | DNS failure | `-1` | `java.net.UnknownHostException` | transient | | Unparsable response body | `-1` | `com.fasterxml.jackson.databind.exc.MismatchedInputException` | permanent | | Rejected before sending | `-1` | none | permanent | The misclassification reached every caller: | Caller | Before | After | | --- | --- | --- | | `getOrCreateSecondaryResource` | rethrew on the first attempt, skipping the `GET` that avoids a duplicate create, which `AppInitStep` turns into a terminal `SchedulingFailure` under the default `RestartPolicy.Never` | retries with backoff | | `getResourceStrictly` | reported an unanswered read as a refusal | reports it as one the create path settles | | `StatusRecorder.persistStatus` | wrote a `StatusUpdateFailed` event per blip, adding load to the server that caused it | stays quiet | | `KueueWorkloadUtils.retryAfterRequestFailure` | warning event plus the default requeue interval | the short interval, no event | The classification also holds in the other direction: an unparsable response and a rejection raised before sending no longer pass as transient, so they keep their warning event and longer requeue instead of being retried forever in silence. ### Does this PR introduce _any_ user-facing change? Yes. A `SparkApplication` or `SparkCluster` that loses its connection to the API server while requesting driver or master resources is now retried, with a delay, instead of ending in a terminal `SchedulingFailure`, and such failures no longer emit `Warning` events. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
