Re: Flush Kafka topic

in 0.11 and above see the CLI command bin//kafka-delete-records.sh -hans > On Aug 23, 2017, at 7:28 PM, Rahul Singh wrote: > > Hello all, > > I am unable to purge the topic data from Kafka. Is there any class to flush > all topic data. > > Thank you

Flush Kafka topic

Hello all, I am unable to purge the topic data from Kafka. Is there any class to flush all topic data. Thank you

Re: Pinning clients to specific brokers

We (Confluent) run Kafka as a SaaS-based cloud offering and we do not see any reason for this feature so I just don’t understand the motivation for it. Please explain. -hans -- /** * Hans Jespersen, Principal Systems Engineer, Confluent Inc. * h...@confluent.io (650)924-2670 */ > On Aug

Re: kafka in unrecoverable state

Hi David, Thanks for reply. However, I don't have problem with number of replicas. I have 3 brokers. And topics configured accordingly, especially __consumer_offsets Topic:__consumer_offsets PartitionCount:50 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.t

Re: kafka straming progress had be done a few minutes later

Hello Jie, Could you provide some more details in your "transformation" logic? For example did you use any state stores? Note that if caching is enabled (by default it is), then you may not see outputs until the cache is flushed during commit. Guozhang On Wed, Aug 23, 2017 at 12:31 PM, Matthias

Re: kafka straming progress had be done a few minutes later

Not sure what your question is... Maybe you refer to commit interval that is 30 seconds by default. It could be, that you don't see any writes to the output topic before that. But it's a wild guess. You can try to set a shorter commit interval via StreamsConfig. -Matthias On 8/22/17 8:09 PM, 杰

Re: kafka in unrecoverable state

|> NotEnoughReplicasException: Number of insync replicas for partition __consumer_offsets-17 is [1], below required minimum [2] Please refer to https://stackoverflow.com/questions/37960767/how-to-change-the-replicas-of-kafka-topic. Hope it helps! On Aug 23, 2017 5:17 AM, "Murad Mamedov" wrote:

Clients list

Hi Kafka users, I maintain the PyKafka python client library, and the PyKafka listing on the clients page is outdated. I made a Confluence account, but I don't see a way to edit the page

Re: Pinning clients to specific brokers

Mohit, Can you describe your use case around why you want this to happen? Thanks From: Joao Reis Sent: Wednesday, August 23, 2017 11:08:02 AM To: users@kafka.apache.org Subject: Re: Pinning clients to specific brokers Hey Mohit, I agree with Hans, and addition

Independent zookeep for kafka multi node

Hi All Can I have three independent zookeepers tagged to three kafka brokers without any clustering or quorum ? Would it be a good idea ?

Re: Pinning clients to specific brokers

Hey Mohit, I agree with Hans, and additionally you may have trouble when the cluster needs to rebalance its partitions or when partition leadership changes node. Also you may loose high-availability and cluster resilience in case of fail-over. Cheers João Reis On 23 Aug 2017, at 08:42, Mohit C

Re: Avro With Kafka

Classpath Schema Registry and Consul Schema Registry which I wrote can be found here: https://github.com/mykidong/kafka-etl-consumer/tree/master/src/main/java/kafka/etl/deserialize Classpath Schema Registry can be used like this: // topic and avro schema classpath properties. // topic key must be

Re: kafka in unrecoverable state

Hi, Did you manage to find the root cause of this issue? Same thing happened here. Thanks in advance On Tue, Jun 13, 2017 at 7:50 PM, Paul van der Linden wrote: > I managed to solve it by: > - stopping and deleting all data on kafka & zookeeper > - stopping all consumers and producers > - sta

Re: about Kafka behavior with distinct groups of consumers, but with the same group.id ?

Consumers with the same group.id are part of the same consumer group. Topic/partitions are load-balanced over the consumer instances based on their topic subscriptions. In this case G1, G2 consumers are part of the same group, T1 is balanced over G1 and T2 over G2. This is a valid scenario. On Wed

about Kafka behavior with distinct groups of consumers, but with the same group.id ?

Hi, I am wondering about Kafka behavior. So, let me explain the use case I face. I have 2 different groups of consumers: * group of consumers G1 to consume only msgs from topic T1 * group of consumers G2 to consume only msgs from topic T2 The question is... What happens if I set the same group

Re: Kafka streams application (v 0.10.0.1) stuck at close

Hi, If you can then i'd recommend upgrading to a newer version. As you said many bugs have been fixed since 0.10.0.1 On Wed, 23 Aug 2017 at 05:08 Balaprassanna Ilangovan < balaprassanna1...@gmail.com> wrote: > Hi, > > I have the following three question regarding Apache Kafka streams. > > 1. I a

Python Kafka client that has support for SASL_SSL with SCRAM-SHA-256/512

Hello, could someone point me in the direction of Python Kafka client that has support for SASL_SSL with SCRAM-SHA-256/512? The reason for asking is that this lib edenhill/librdkafka seems to have configuration properties sasl.*

Re: Pinning clients to specific brokers

Hey Hans, Thanks for your reply. In a cloud environment this can be useful. Perhaps if partitioning and replicas are selected appropriately, this could be possible ? Thanks, Mohit On Tuesday, August 22, 2017, Hans Jespersen wrote: > Doing that doesn't really make sense in a Kafka cluster beca