ashwinpankaj opened a new pull request, #23475:
URL: https://github.com/apache/kafka/pull/23475
KIP-909 introduced `bootstrap.resolve.timeout.ms` to opt into
asynchronous bootstrap DNS resolution, surfacing failures as a
`BootstrapResolutionException` from later API calls on a client that
must then be discarded and rebuilt by the caller. #23364 (KAFKA-20939)
disabled this for Kafka Connect (and Streams) by forcing the config to
`0` for every client Connect creates internally, since neither
framework was prepared to handle that recovery contract.
This PR re-enables it for Connect with framework-appropriate defaults,
rather than leaving it permanently off:
- **Worker/herder group-membership client**: defaults
`bootstrap.resolve.timeout.ms` to 10s (`DistributedConfig
.BOOTSTRAP_RESOLVE_TIMEOUT_MS_DEFAULT`), replacing the previous
forced `0`. A user-supplied value is honored as before.
- **Task-level clients** (source/sink producers, consumers, and admin
clients created by `Worker`): default to 60s via the new
`ConnectUtils.applyDefaultTaskBootstrapResolveTimeout`, which only
fills in the property when absent — it does not override a
user-supplied value at any config level. This replaces
`ConnectUtils.enforceSynchronousBootstrapResolution`, which is
removed.
- Config/offset/status backing store clients no longer force the
value to `0` either, so they pick up the worker-level default like
any other internal client.
- `DistributedHerder` and `AbstractHerder` now recognize
`BootstrapResolutionException` in the failure/cause chain and log
and record a short, human-readable summary ahead of the full stack
trace, since this exception is otherwise unrecoverable for the
affected client.
- `docs/upgrade.md` updated to drop the "Connect forces this to 0"
caveat (Streams still does, and is unaffected by this PR).
The intent is to let a Connect worker/task tolerate a transient DNS
blip during startup or rejoin — retrying resolution internally up to
the timeout — while still failing within a bounded time if
`bootstrap.servers` is persistently unresolvable, rather than either
hanging indefinitely or failing immediately on the first blip.
Requesting review from folks involved in #23364, since this PR
re-enables what that one turned off for Connect specifically.
### Test plan
Updated/added unit tests in `DistributedConfigTest`, `WorkerTest`,
`AbstractHerderTest`, `DistributedHerderTest`, `ConnectUtilsTest`, and
`KafkaBasedLogTest` covering the new defaults and the
`BootstrapResolutionException` surfacing behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
