Jackie-Jiang opened a new pull request, #19296: URL: https://github.com/apache/pinot/pull/19296
## Summary When a consuming segment is stopped ([RealtimeSegmentDataManager#stop]), the consumer thread is interrupted (repeatedly, until it exits). If the thread is blocked in `fetchMessages` at that moment, the interrupt surfaces as a transient stream exception, and `handleTransientStreamErrors` — although it already recognizes the interrupted-to-stop case for logging — still sleeps 1 second uninterruptibly and recreates the stream consumer. That recreation is pure waste: the consume loop exits on the very next `_shouldStop` check, so the freshly created consumer is never used. Worse, closing the old Kafka consumer from the interrupted thread makes the Kafka client log a noisy error with a full stack trace on every such shutdown (from inside `KafkaConsumer#close`, so it cannot be suppressed on the Pinot side): ``` ERROR [ClassicKafkaConsumer] ... Failed to close fetcher with a timeout(ms)=30000 org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException ``` This shows up in bulk in integration test logs, since tests constantly stop tables with live consumers, and the interrupt lands inside `fetchMessages` most of the time. Fix: when `_shouldStop` is set, `handleTransientStreamErrors` returns without the retry sleep and without recreating the consumer. The consume loop exits on the next check, and the current consumer is closed exactly once by the regular shutdown path (`closeStreamConsumer`, typically invoked from a non-interrupted thread). Besides removing the log noise, this also shaves the pointless 1s uninterruptible sleep plus a close/recreate cycle off every stop that races an in-flight fetch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
