lianetm commented on code in PR #15961:
URL: https://github.com/apache/kafka/pull/15961#discussion_r1601976218
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java:
##########
@@ -1607,6 +1607,7 @@ private Fetch<K, V> pollForFetches(Timer timer) {
wakeupTrigger.clearTask();
}
+ metadata.maybeThrowAnyException();
Review Comment:
uhm I would say that having this here, in the app thread, is not right,
since the metadata updates happen in the background thread, so we could easily
get a race condition I would expect. The metadata object is updated in the
[NetworkClient](https://github.com/apache/kafka/blob/bb3ff0f84a0a4c6d213fbe50eb2c3d3a8cdecdf0/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java#L1193),
when a response to a metadata request is received. So only at that point in
the background thread is that may accurately know that there is an invalid
topic. I imagine we should then propagate an `ErrorEvent` to the app thread,
that would be raised in the poll loop
[here](https://github.com/apache/kafka/blob/bb3ff0f84a0a4c6d213fbe50eb2c3d3a8cdecdf0/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L213))
Seen from a conceptual POV, we have something that could wrong in the
background (invalid topic received in metadata response), and we want to throw
an error in the foreground, so the pattern would be via events. Similar
probably to what is done to raise errors that may happen while resetting
positions in the background thread, that are propagated to the app thread
[here](https://github.com/apache/kafka/blob/bb3ff0f84a0a4c6d213fbe50eb2c3d3a8cdecdf0/clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManager.java#L202).
Difference in this new case is that the failure is detected deep down the
network client (not in the request managers layer), so if we follow this
approach, we would need some wiring there. Just thinking out loud, this is
definitely a tricky missing bit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
