Re: [new to kafka/java] Could I check whether a consumer has been closed?

2023-05-29 Thread Daniel Hinojosa
Warning, though. `closed` is volatile. That shouldn't be too much of a problem, but if something is odd, make a mental bookmark. Consumer and KafkaConsumer should have a permanent isClosed() method. It sure would be handy. I'll put it as an issue. On Mon, May 29, 2023 at 8:47 PM Daniel

Re: [new to kafka/java] Could I check whether a consumer has been closed?

2023-05-29 Thread Daniel Hinojosa
As a stop-gap you can create something like the following. private static boolean isClosed(KafkaConsumer consumer) { try { var consumerClass = consumer.getClass(); Field field = consumerClass.getDeclaredField("closed"); field.setAccessible(true); return (Boolean) field.get(consumer); } catch

Re: Patterns for generating ordered streams

2023-05-29 Thread Brebner, Paul
Oh, the Kafka parallel consumer may help potentially? https://www.instaclustr.com/blog/improving-apache-kafka-performance-and-scalability-with-the-parallel-consumer-part-2/ Paul From: Edvard Fagerholm Date: Tuesday, 30 May 2023 at 6:55 am To: users@kafka.apache.org Subject: Patterns for

Re: Patterns for generating ordered streams

2023-05-29 Thread Brebner, Paul
Hi Edvard, interesting problem – I’ve had similar problems with high fan out use cases, but only for demo applications where I’m more interested in scale than order – e.g. have a look at this list of blogs, examples include Anomalia Machina for Kafka+Cassandra, and Kongo, Kafka+IoT.

Errors related to Kafka API checks

2023-05-29 Thread odashima.tatsuya
Hi, there, I have a problem when I routinely run the following command line. In addition, I am experiencing process hangs on one Broker at irregular intervals, but I do not know the relevance of the error content. Has anyone had the same problem to help me? ○Kafka Broker Server Information

Patterns for generating ordered streams

2023-05-29 Thread Edvard Fagerholm
Hi there, Kafka is new to us, so we don't have operational experience from it. I'm building a system that would be broadcasting events to groups of clients. Kafka consumers handle the broadcasting. In other words, kafka topic --- consumers --- clients We need to maintain the order of the events

[new to kafka/java] Could I check whether a consumer has been closed?

2023-05-29 Thread Eric Li
Hello everyone, my question is as the topic. In particular, when I have a Consumer object, I would like to see if https://github.com/apache/kafka/blob/2e1947d240607d53f071f61c875cfffc3fec47fe/clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java#L269 (kafka-clients-3.4.0) has been