Re: kafka in unrecoverable state

If you run kafka-topics.sh --describe --topic __consumer_offsets, does it show that all replicas are in sync? On 23 August 2017 at 23:11, Murad Mamedov wrote: > Hi David, > > Thanks for reply. However, I don't have problem with number of replicas. I > have 3 brokers. And topics configured accord

how to find out the group coordinator in kafka for a given topic?

Re: Kafka Streams: Pseudo Wallclock Punctuate

Update on this. Modifying the state store on another thread actually doesn’t work. I receive the error java.lang.IllegalStateException: This should not happen as offset() should only be called while a record is processed at org.apache.kafka.streams.processor.internals.AbstractProcessorC

Does Kafka Connect have a built-in, connector-agnostic way to throttle its connection to a Kafka Cluster?

Hi, As it can be seen in this diagram: https://www.planttext.com/plantuml/img/ZP7BRi8m44Nt_eehi-O7ka1KYSI221QXOjFDa4p1meoHFH5yVXqUaNXKgjVddhcwdYQbgDt16wDEFSFEwMTFA888DuenLhiOLBsJhJNcc67Nk2E93gJladAqe8HvKRr5hh8Ridn4IaxgXDG5kG1blJ9c4ywmLMZ8Oy5w2d5lCQhSTgUpo2m6-VsZ2qkBhBhv2X-wzkwfoH_2twAdnY_OkzR3jNmQ

Kafka Streams: Pseudo Wallclock Punctuate

Hi I have a kafka streams application, that uses a couple of stateful (i.e. using key value stores) custom transformer implementations. I'm aware that the scheduling of the punctuate method is based on stream time. However, for my use case, this is not what I want, I need wallclock scheduling.

Re: Kafka Streams - groupByKey and Count, null result on join

Hi Ian, I suspect it has something to do with your specified topology, in which it triggers the join first, then the aggregation updates. For example, take a look at this ticket: https://issues.apache.org/jira/browse/KAFKA-4601 As from its printed topology, due to the repartition topic the join

Is it possible to disable caching for some kafka topics?

Hi, So I have a scenario in kafka where there are 2 topics : A and B. Topic A's consumers are online (and need to consume data fast) and topic B's consumers are offline. Since topic B's consumers are offline, I don't need to cache any message related to topic B right now and want topic A to get th

bind kafka tcp port to localhost instead of 0.0.0.0

Hi, kafka vertion: kafka_2.11-0.10.2.0. I change the config files in kafka and I can bind 9092 and 2181 to localhost. But then I check the tcp port, I find that there are another two tcp port used by kafka. The two ports always change after start kafka. [cid:image003.jpg@01D31CEC.5926A

Batch containing 4 record(s) expired due to timeout while requesting metadata from brokers for

Hi, I'm running a two-node kafka cluster. When I (gracefully) shut down one of the kafka servers, the application that publishes message to the cluster keeps giving this error message: org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while reque

Kafka Streams - groupByKey and Count, null result on join

Hi All, I'm building a streams applications where I wish to take action on the input when a certain frequency of the input has been seen. At the moment the application roughly goes: frequency table = Input Stream -> groupByKey -> Count input stream with counts = leftJoin frequency table and inp

Re: Pinning clients to specific brokers

Hi Stephen, Hans, This can help in keeping the traffic isolated in an availability zone, and avoid inter AZ communication. Thanks, Mohit On Thursday, August 24, 2017, Hans Jespersen wrote: > We (Confluent) run Kafka as a SaaS-based cloud offering and we do not see > any reason for this feature