Re: Single-replication topic behavior on failures

2017-04-12 Thread Manikumar
I think you will get LeaderNotAvailable exception and producer requests will fail. On Thu, Apr 13, 2017 at 10:23 AM, Timur Fayruzov wrote: > Hello, > > We have a number of topics with replication factor of 1 and a single > partition. I want to update the cluster from

Single-replication topic behavior on failures

2017-04-12 Thread Timur Fayruzov
Hello, We have a number of topics with replication factor of 1 and a single partition. I want to update the cluster from 0.10.0 to 0.10.1 by taking nodes down one by one, but it is not clear to me what is the behavior will be for these topics. Brokers have topic auto-create setting set to true.

Re: Kafka best practice on bare metal hardware

2017-04-12 Thread Marcos Juarez
Ali, I don't know of proper benchmarks out there, but I've done some work in this area, when trying to determine what hardware to get for particular use cases. My answers are in-line: On Mon, Apr 10, 2017 at 7:05 PM, Ali Nazemian wrote: > Hi all, > > I was wondering if

[VOTE] 0.10.2.1 RC1

2017-04-12 Thread Gwen Shapira
Hello Kafka users, developers, client-developers, friends, romans, citizens, etc, This is the second candidate for release of Apache Kafka 0.10.2.1. This is a bug fix release and it includes fixes and improvements from 24 JIRAs (including a few critical bugs). Release notes for the 0.10.2.1

Re: [VOTE] 0.10.2.1 RC0

2017-04-12 Thread Gwen Shapira
Thank you for reporting Eno, and thank you for everyone who tested the release. This is really appreciated. I am rolling out a new RC right now. Expect a new vote thread soon :) On Wed, Apr 12, 2017 at 10:01 AM, Eno Thereska wrote: > Hi Gwen, > > Unfortunately we found

Re: Subscription to this list

2017-04-12 Thread Matthias J. Sax
It's self-service: http://kafka.apache.org/contact -Matthias On 4/11/17 11:17 PM, Vidya Priyadarshni Narayan wrote: signature.asc Description: OpenPGP digital signature

Java stdin producer loosing logs

2017-04-12 Thread Milind Vaidya
Hi Background : I have following set up Apache server >> Apache Kafka Producer >> Apache Kafka Cluster >> Apache Storm As a normal scenario, front end boxes run the apache server and populate the log files. The requirement is to read every log and send it to kafka cluster. The java producer

Corrupted message in master partition, but valid in replica

2017-04-12 Thread Dhyan Muralidharan
Hi, I see issue in kafka 0.9 consumer where messages are corrupted in partitions master node . I have details explained in stack-overflow . http://stackoverflow.com/questions/43379306/kafka-message-corrupted-in-master-but-replica-looks-good . Can someone help ? --Dhyan

Consumer offsets resetting

2017-04-12 Thread Lawrence Weikum
We’re seeing a strange error on our Kafka cluster. Users have reported their consumer offsets seem to periodically reset, which makes downstream processes handle a massive amount of duplicates. Our Kafka cluster is currently running 0.10.1.1. Previously the cluster had been at 0.10.0.1. It was

Kafka Support as Service

2017-04-12 Thread Diego Paes Ramalho Pereira
Hello, I work for a Stock Exchange in Brazil and We are looking for a company that can provide us Kafka support as a service. Do you have companies that provide this kind of service? Regards, Diego Paes Ramalho Pereira Senior Infrastructure Analyst Phone.: + 55 11 2565-5004 (Team) Phone.: + 55

Consumer offsets resetting

2017-04-12 Thread Lawrence Weikum
We’re seeing a strange error on our Kafka cluster. Users have reported their consumer offsets seem to periodically reset, which makes downstream processes handle a massive amount of duplicates. Our Kafka cluster is currently running 0.10.1.1. Previously the cluster had been at 0.10.0.1. It was

Re: Kafka MTLS Support?

2017-04-12 Thread Hans Jespersen
Are you asking about Multiplexed Transport Layer Security (MTLS) - https://en.wikipedia.org/wiki/Multiplexed_Transport_Layer_Security or Mutual TLS authentication (mTLS) https://en.wikipedia.org/wiki/Mutual_authentication

Re: How to get metrics from Kafka?

2017-04-12 Thread Sachin Mittal
check this http://kafka.apache.org/documentation.html#monitoring You can use any jmx client to hook up to the running server jvm. https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters or use jconsole. I use https://cwiki.apache.org/confluence/display/KAFKA/jmxterm+quickstart and works

How to get metrics from Kafka?

2017-04-12 Thread Sriram Srinivasaraghavan (srirsri2)
Team, I am new to Kafka. I would like to know how to get general metrics to monitor a kafka cluster. For ex: 1. CPU, Memory details of individual brokers 2. Number of messages produced by a Producer vs written to the broker. 3. Number of messages consumed by the consumer vs lost

Re: [VOTE] 0.10.2.1 RC0

2017-04-12 Thread Michal Borowiecki
FWIW, I upgraded without issue and noticed the speedup from KAFKA-4851/KAFKA-4876. +1 from me (non-binding) On 12/04/17 02:06, Gwen Shapira wrote: Wrong link :) http://kafka.apache.org/documentation/#upgrade and http://kafka.apache.org/documentation/streams#streams_api_changes_0102 On Tue,

Kafka MTLS Support?

2017-04-12 Thread Sriram Srinivasaraghavan (srirsri2)
Team, Would like to know if MTLS is supported as a security protocol, for inter-broker communications and clients to broker communications? Could someone throw some light? Regards, Sriram S

Subscription

2017-04-12 Thread Jaikishan Khatwani
Please subscribe to user questions news letter. Regards,  -Jaikishan

Apache Kafka SSL Deployment

2017-04-12 Thread IT Consultant
Hi All , How can I avoid using password for keystore creation ? We are currently passing keystore password while accessing TLS enabled Kafka instance . I would like to use either passwordless keystore or avoid password for clients accessing Kafka .

Kafka Streams vs Flink (or any other stream processing framework)

2017-04-12 Thread kant kodali
Hi All, I have read enough blogs from Confluent and others and also books that tried to talk about the differences between the two and while it is great to know those differences I hardly find them any useful when it comes to decision making process of which one to pick since I don't see the

Subscription to this list

2017-04-12 Thread Vidya Priyadarshni Narayan
-- Thanks nVidia