Re: 0.9.0.1 RC1

Thanks everyone for voting. The results are: +1 binding = 4 votes (Ewen Cheslack-Postava, Neha Narkhede, Joel Koshy and Jun Rao) +1 non-binding = 3 votes -1 = 0 votes 0 = 0 votes The vote passes. I will release artifacts to maven central, update the dist svn and download site. Will send out an

Re: 0.9.0.1 RC1

Since the unit test failures are transient. +1 from myself. Thanks, Jun On Thu, Feb 11, 2016 at 6:55 PM, Jun Rao wrote: > This is the first candidate for release of Apache Kafka 0.9.0.1. This a > bug fix release that fixes 70 issues. > > Release Notes for the 0.9.0.1

Re: [kafka-clients] 0.9.0.1 RC1

+1 (non-binding) On Tue, Feb 16, 2016 at 8:59 PM, Joel Koshy wrote: > +1 > > On Thu, Feb 11, 2016 at 6:55 PM, Jun Rao wrote: > > > This is the first candidate for release of Apache Kafka 0.9.0.1. This a > > bug fix release that fixes 70 issues. > > > >

Re: Offset committing on rebalance

Hi I am still seeing this issue while trying to rebalance. The consumer is committing offsets and clearing the fetcher queues while trying to rebalance. I don’t want this to happen. I am using kafka 0.8.2. Pleas help me here Thanks Gayathri

Re: Questions from new user

Hi can anyone help with this? On Fri, Jan 29, 2016 at 11:50 PM, allen chan wrote: > Use case: We are using kafka as broker in one of our elasticsearch > clusters. Kafka caches the logs if elasticsearch has any performance > issues. I have Kafka set to delete logs

0.9 client AbstractCoordinator - Attempt to join group failed due to obsolete coordinator information

Hi, My local Java client consumer and producer fail with log messages I don’t understand. What does "obsolete coordinator information” mean? 2016-02-16 18:49:01,795 INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.9.0.0 2016-02-16 18:49:01,795 INFO

Re: [kafka-clients] 0.9.0.1 RC1

+1 On Thu, Feb 11, 2016 at 6:55 PM, Jun Rao wrote: > This is the first candidate for release of Apache Kafka 0.9.0.1. This a > bug fix release that fixes 70 issues. > > Release Notes for the 0.9.0.1 release >

Re: Replication Factor and number of brokers

Hi Sean, you'll want equal or more brokers than your replication factor. Meaning, if your replication factor is 3, you'll want 3 or more brokers. I'm not sure what Kafka will do if you have fewer brokers than your replication factor. It will either give you the highest replication factor it can

Optimize the performance of inserting data to Cassandra with Kafka and Spark Streaming

Hello Everybody, I have questions using Spark streaming to consume data from Kafka and insert to Cassandra database but not sure whether should post in the Kafka users mailing list or not. I appreciated it if you do have any suggestions to me. 5 AWS instances (each one does have 8 cores, 30GB

Re: Kafka as master data store

I guess I was just drawn in by the elegance of having everything available in one well-defined Kafka topic should I start up some new code. If instead the Kafka topics were on a retention period of say 7 days, that would involve firing up a topic to load the warehoused data from HDFS (or a more

Re: 答复: NoAuth for /controller

Kafka doesn't have security enabled for 0.8.2.2 so make sure zookeeper root that you're using doesn't have any acls set -Harsha On Sun, Feb 14, 2016, at 06:51 PM, 赵景波 wrote: > Can you help me? > > ___ > JingBo

RE: Rest Proxy Question

Hi Martin, I can see the items in the queue, that is not the problem. The issue is when the consumer (using the REST proxy) is trying to consume. -Original Message- From: Martin Gainty [mailto:mgai...@hotmail.com] Sent: Tuesday, February 16, 2016 10:32 AM To: users@kafka.apache.org

RE: Rest Proxy Question

pathping on the contacted proxy during slack periods | tee slack.lst pathping on the contacted proxy during busy periods | tee busy.lst Please Post Results here What i am suggesting is once a packet is transmitted outside your firewall the packet is beyond reach of locally hosted software such

Re: Compression - MessageSet size

I'm not using Java client, I'm developing my own: https://github.com/oleksiyk/kafka And I'm talking about MessageSet on the wire, not the one passed to library user. Example: Consumer is not started. Producer sends a batch #1 of 5 messages, compressed in a single message. delay 50 ms Producer

Re: Compression - MessageSet size

What makes you think there are 2? would you have data or code to share? When compression is enabled, multiple messages will be packed and compressed in a MessageSet. That MessageSet will then have a single message. The interface however will let you iterate over the unpacked messages. See

Re: Rebalancing during the long-running tasks

Great proposal, indeed. If I understand right, Jason suggests committing messages one-by-one instead of ingesting the whole batch. I have nothing against processing the big bunch of tasks from poll() and committing after every single processed task, but in my case the very single task is too

Rest Proxy Question

I am having a hard time with some bug in my code related to the REST proxy. The issue that I am seeing is when the GET /consumers/HIVIEPurchaseOrderCreatedEvent/instances/b0671fa1-d000-4f9b-b795-aff3003e500a/topics/PurchaseOrderCreatedEvent/. Sometimes this will return immediately with results

Re: Rebalancing during the long-running tasks

Hi, I had the same issue and managed to work around it by simulating a heartbeat to kafka. It works really well, i.e., we have had zero issues since it was implemented I have somthing like this: void process() { records = consumer.poll(timeout) dispatcher.dispatch(records)

Re: Rebalancing during the long-running tasks

I think you’ll find some useful context in this KIP Jason wrote. It’s pretty good. https://cwiki.apache.org/confluence/display/KAFKA/KIP-41%3A+KafkaConsumer+Max+Records > On 16 Feb 2016, at 07:15, Насыров

Re: Replication Factor and number of brokers

Then you'll have under-replicated partitions. However, even if you have 3 brokers with a replication factor of 2 and you lose a single broker you'll still likely have under-replicated partitions. Partitions are assigned to brokers, 1 broker will be the leader and n brokers will be followers. If

Rebalancing during the long-running tasks

Hello! I'm trying to use kafka for long-running tasks processing. The tasks can be very short (less than a second) or very long (about 10 minutes). I've got one consumer group for the single queue, and one or more consumers. Sometimes consumers manage to commit their offsets before

Re: Replication Factor and number of brokers

So if I have a replication factor of 2, but only 2 brokers, then replication works, but what if I lose one broker? Thanks, Sean On 2/16/16, 9:14 AM, "Damian Guy" wrote: >Hi, > >You need to have at least replication factor brokers. >replication factor = 1 is no

Re: Replication Factor and number of brokers

Hi, You need to have at least replication factor brokers. replication factor = 1 is no replication. HTH, Damian On 16 February 2016 at 14:08, Sean Morris (semorris) wrote: > Should your number of brokers be atleast one more then your replication > factor of your topic(s)?

Replication Factor and number of brokers

Should your number of brokers be atleast one more then your replication factor of your topic(s)? So if I have a replication factor of 2, I need atleast 3 brokers? Thanks, Sean

Compression - MessageSet size

Hi! The Kafka 0.9 protocol guide https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Compression explicitly states that there should be only single compressed message in a MessageSet, but I'm definitely receiving two compressed messages in a