Re: Any gotchas upgrading to 0.9?

Rajiv, By default, the quota is unlimited until you decide to configure them explicitly. And yes, we did get rid of "replica.lag.max.messages", so that configuration will no longer apply. Aditya On Tue, Dec 1, 2015 at 10:24 AM, Todd Snyder wrote: > The quota page is

Re: using kafka

I think this is what you are looking for: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example On Thu, Sep 24, 2015 at 11:59 PM, 刘振 wrote: > Dear all, > > > I am trying to use kafka to do some job load balance and not sure if kafka > support this

Re: [IMPORTANT] Questions related to AdminUtils thread safety

topicExists simply reads ZK, so yes. createTopic should also be fine unless you try to create the same topic concurrently. AdminUtils itself does not maintain any state, just some static util functions. On Mon, Aug 31, 2015 at 3:00 PM, Sivananda Reddys Thummala Abbigari <

RE: Kafka bottleneck issue

Do you have a single producer thread? Also, how do you calculate the TPS (is it transactions per second?) ? Aditya From: Hawin Jiang [hawin.ji...@gmail.com] Sent: Thursday, July 02, 2015 5:58 PM To: users@kafka.apache.org Subject: Kafka bottleneck issue

RE: Help Us Nominate Apache Kafka for a 2015 Bossie (Best of OSS) Award - Due June 30th

Sent :) From: Gwen Shapira [gshap...@cloudera.com] Sent: Friday, June 26, 2015 11:53 AM To: users@kafka.apache.org Cc: d...@kafka.apache.org Subject: Re: Help Us Nominate Apache Kafka for a 2015 Bossie (Best of OSS) Award - Due June 30th Sent! Thanks for

RE: Changing replication factor for an existing topic

The replica list that you specify can be used to increment/decrement the replication factor. http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor Aditya From: Robin Yamaguchi [ro...@tune.com] Sent: Wednesday, June 10, 2015

RE: Kafka JMS metrics meaning

Number of underreplicated partitions, total request time are some good bets. Aditya From: Otis Gospodnetic [otis.gospodne...@gmail.com] Sent: Tuesday, June 02, 2015 9:56 AM To: users@kafka.apache.org; Marina Subject: Re: Kafka JMS metrics meaning Hi, On

RE: async

On Mon, Jun 1, 2015 at 11:06 AM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: This should be enough: bin/kafka-console-producer.sh --async -batch-size=10 --broker-list localhost:9092 --topic test Aditya From: ram kumar [ramkumarro...@gmail.com

RE: KafkaMetricsConfig not documented

Yeah, they aren't included in KafkaConfig for some reason but I think they should. Can you file a jira? Aditya From: Stevo Slavić [ssla...@gmail.com] Sent: Sunday, May 31, 2015 3:57 PM To: users@kafka.apache.org Subject: KafkaMetricsConfig not documented

RE: async

This should be enough: bin/kafka-console-producer.sh --async -batch-size=10 --broker-list localhost:9092 --topic test Aditya From: ram kumar [ramkumarro...@gmail.com] Sent: Sunday, May 31, 2015 10:18 PM To: users@kafka.apache.org Subject: async hi, is

RE: Java - High Level Consumer

You should receive only new messages if the auto.offset.reset is largest. How do you determine that the messages you are receiving are older? Are you checking the lag on your consumer by ConsumerOffsetChecker? Thanks, Aditya From: Panda, Samaresh

RE: Is fetching from in-sync replicas possible?

Is that necessarily the case? On a cluster hosting partitions, assuming the leaders are evenly distributed, every node should receive a roughly equal share of the traffic. It does help a lot when the consumer throughput of a single partition exceeds the capacity of a single leader but at that

RE: Can't open console producer

Can you do a git status? I accidentally did something similar the other day while copying commands from the quickstart guide. bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test filename will truncate filename to zero length and all subsequent commands will not return

RE: Kafka 0.8.2.1 - Listing partitions owned by consumers

Perhaps you could try the ConsumerOffsetChecker. The Owner field might be what you want.. Aditya From: Bharath Srinivasan [bharath...@gmail.com] Sent: Tuesday, May 12, 2015 7:29 PM To: users@kafka.apache.org Subject: Kafka 0.8.2.1 - Listing partitions

RE: Kafka listener threads - graceful shutdown

I don't think call allDone will cause hasNext() to exit. The new consumer has a timed poll() function on it's API I think. With the old consumer, interrupting the thread calling hasNext might work. Have you tried that? Aditya From: Gomathivinayagam

RE: Significance of SimpleConsumer id string

A client id is used to logically identify an application. Ideally, multiple consumers belonging to the same application should use the same client id. More concretely, metrics can be gathered per client and quotas in the future will be enforced by clientId. Aditya

RE: Delete topic / Recreate = No leader

I believe there were some issues with delete topic in 0.8.10 (other correct me if I am wrong). Can you try with the most recent release ? Thanks, Aditya From: 马哲超 [mazhechaomaill...@gmail.com] Sent: Sunday, May 03, 2015 7:51 PM To: users@kafka.apache.org

RE: Java Consumer API

It'll be officially ready only in version 0.9. Aditya From: Mohit Gupta [success.mohit.gu...@gmail.com] Sent: Thursday, April 30, 2015 8:58 PM To: users@kafka.apache.org Subject: Java Consumer API Hello, Kafka documentation (

RE: Unclaimed partitions

On 4/28/15, 1:40 PM, Aditya Auradkar aaurad...@linkedin.com.INVALID wrote: Couple of questions: - What version of the consumer API are you using? - Are you seeing any rebalance failures in the consumer logs? - How do you determine that some partitions are unassigned? Just confirming that you have

RE: Kafka - preventing message loss

You can use the min.insync.replicas topic level configuration in this case. It must be used with acks=-1 which is a producer config. http://kafka.apache.org/documentation.html#topic-config Aditya From: Gomathivinayagam Muthuvinayagam

RE: Unclaimed partitions

are in the same consumer group. Thanks, Dave On 4/28/15, 1:15 PM, Aditya Auradkar aaurad...@linkedin.com.INVALID wrote: Hi Dave, The simple consumer doesn't do any state management across consumer instances. So I'm not sure how you are assigning partitions in your application code. Did you mean to say

RE: Kafka to Hadoop HDFS

Try this. https://github.com/linkedin/camus Aditya From: Lin Ma [lin...@gmail.com] Sent: Friday, March 06, 2015 8:19 PM To: users@kafka.apache.org Subject: Kafka to Hadoop HDFS Hi Kafka masters, Wondering if any open source solutions, to transfer

RE: batching causes replica out of sync

Xiaoyu, Just FYI - Here's a discussion on this issue if you are interested. https://issues.apache.org/jira/browse/KAFKA-1546 Aditya From: Mayuresh Gharat [gharatmayures...@gmail.com] Sent: Thursday, March 05, 2015 4:41 PM To: users@kafka.apache.org

RE: Kafka ETL Camus Question

Hi Bhavesh, I just checked with one of the devs on the Camus team. We run the Camus job with speculative execution disabled. Aditya From: Pradeep Gollakota [pradeep...@gmail.com] Sent: Monday, February 02, 2015 11:15 PM To: users@kafka.apache.org

RE: LinkedIn Engineering Blog Post - Current and Future

This should work. http://engineering.linkedin.com/kafka/kafka-linkedin-current-and-future Aditya From: Thunder Stumpges [tstump...@ntent.com] Sent: Thursday, January 29, 2015 2:43 PM To: users@kafka.apache.org Subject: RE: LinkedIn Engineering Blog Post