[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783485#comment-17783485 ] Philip Nee commented on KAFKA-15686: Hi [~ihavenoem...@163.com] - "Now maybe we can use describeTopics method in admin client to check if every partition has a leader." - That would be a feasible solution. > Consumer should be able to detect network problem > - > > Key: KAFKA-15686 > URL: https://issues.apache.org/jira/browse/KAFKA-15686 > Project: Kafka > Issue Type: New Feature > Components: consumer >Affects Versions: 3.5.0 >Reporter: Jiahongchao >Priority: Minor > > When we call poll method in consumer, it will return normally even if some > partitions do not have a leader. > What should we do to detect such failures? Currently we have to check log to > find out broker connection problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17781635#comment-17781635 ] Jiahongchao commented on KAFKA-15686: - We are using the Kafka cluster managed by another team. A few days ago, they shut down some brokers for maintenance, some partitions lost the leader, but our consumers kept running without exceptions, some message didn't get processed on time. So we wanted to monitor from our side for a higher availability. Now maybe we can use describeTopics method in admin client to check if every partition has a leader. > Consumer should be able to detect network problem > - > > Key: KAFKA-15686 > URL: https://issues.apache.org/jira/browse/KAFKA-15686 > Project: Kafka > Issue Type: New Feature > Components: consumer >Affects Versions: 3.5.0 >Reporter: Jiahongchao >Priority: Minor > > When we call poll method in consumer, it will return normally even if some > partitions do not have a leader. > What should we do to detect such failures? Currently we have to check log to > find out broker connection problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17781206#comment-17781206 ] Philip Nee commented on KAFKA-15686: [~ihavenoem...@163.com] - What is the use case of knowing which broker is down? If telemetry is set up for your kafka cluster, you should be able to tell by looking at the broker side metrics. It would be helpful to articulate a clear case to help community to understand the rationale behind the ask. Thanks. > Consumer should be able to detect network problem > - > > Key: KAFKA-15686 > URL: https://issues.apache.org/jira/browse/KAFKA-15686 > Project: Kafka > Issue Type: New Feature > Components: consumer >Affects Versions: 3.5.0 >Reporter: Jiahongchao >Priority: Minor > > When we call poll method in consumer, it will return normally even if some > partitions do not have a leader. > What should we do to detect such failures? Currently we have to check log to > find out broker connection problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780930#comment-17780930 ] Jiahongchao commented on KAFKA-15686: - It's helpful, but from this metric it's hard to know which broker is down. Many thanks anyway. > Consumer should be able to detect network problem > - > > Key: KAFKA-15686 > URL: https://issues.apache.org/jira/browse/KAFKA-15686 > Project: Kafka > Issue Type: New Feature > Components: consumer >Affects Versions: 3.5.0 >Reporter: Jiahongchao >Priority: Minor > > When we call poll method in consumer, it will return normally even if some > partitions do not have a leader. > What should we do to detect such failures? Currently we have to check log to > find out broker connection problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[
https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780851#comment-17780851
]
Philip Nee commented on KAFKA-15686:
Hi [~ihavenoem...@163.com] - Have you tried using client-side JMX metrics?
Metrics such as `connection-count` should give you a pretty good sense of the
active network connections. Would something like this help in this situation?
{code:java}
consumer.metrics().get(new MetricName("connection-count" ... )){code}
> Consumer should be able to detect network problem
> -
>
> Key: KAFKA-15686
> URL: https://issues.apache.org/jira/browse/KAFKA-15686
> Project: Kafka
> Issue Type: New Feature
> Components: consumer
>Affects Versions: 3.5.0
>Reporter: Jiahongchao
>Priority: Minor
>
> When we call poll method in consumer, it will return normally even if some
> partitions do not have a leader.
> What should we do to detect such failures? Currently we have to check log to
> find out broker connection problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[
https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780843#comment-17780843
]
Jiahongchao commented on KAFKA-15686:
-
Hi, [~pnee] , in the following code, even if the broker is unreachable,
consumer.poll alway return an empty result instead of throwing an exception. So
users will not know if the topic has no message or if the broker is down, we
need to check log looking for something like "WARN clients.NetworkClient:
[Consumer clientId=consumer-3, groupId=JAVA] Connection to node -1 could not be
established. Broker may not be available"
{code:java}
Properties props = new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9094");
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.StringDeserializer");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.StringDeserializer");
props.put(ConsumerConfig.GROUP_ID_CONFIG, "test-consumer-group");
try (KafkaConsumer consumer = new KafkaConsumer<>(props)) {
consumer.subscribe(List.of("bar"));
while (true) {
ConsumerRecords records =
consumer.poll(Duration.ofMillis(100));
System.out.println(records.count());
}
}{code}
> Consumer should be able to detect network problem
> -
>
> Key: KAFKA-15686
> URL: https://issues.apache.org/jira/browse/KAFKA-15686
> Project: Kafka
> Issue Type: New Feature
> Components: consumer
>Affects Versions: 3.5.0
>Reporter: Jiahongchao
>Priority: Minor
>
> When we call poll method in consumer, it will return normally even if some
> partitions do not have a leader.
> What should we do to detect such failures? Currently we have to check log to
> find out broker connection problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-15686) Consumer should be able to detect network problem
[ https://issues.apache.org/jira/browse/KAFKA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780187#comment-17780187 ] Philip Nee commented on KAFKA-15686: Hi [~ihavenoem...@163.com] - If the leader is not presented in the current metadata, it should trigger metadata refresh. However, that shouldn't change the way you poll the consumer, because you need to continue to poll to drive the network IO and various consumer tasks. May I ask what kind of failure scenario are you seeing? The client shouldn't even try to send a request if the leader is not present. Could you provide the error messages you see for us to understand the issue? > Consumer should be able to detect network problem > - > > Key: KAFKA-15686 > URL: https://issues.apache.org/jira/browse/KAFKA-15686 > Project: Kafka > Issue Type: New Feature > Components: consumer >Affects Versions: 3.5.0 >Reporter: Jiahongchao >Priority: Minor > > When we call poll method in consumer, it will return normally even if some > partitions do not have a leader. > What should we do to detect such failures? Currently we have to check log to > find out broker connection problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
