How to use ApiVersionRequest and ApiVersionResponse

Hi All I have a requirement where based on the kafka version or compatible api in the broker , we will be dynamically loading the best suited kafka client jar. I bumped into ApiVersionRequest and ApiVersionResponse and even went through some code to in NetworkClient class , but I was not able

Re: Consumer service that supports retry with exponential backoff

Pretty much the implementation that we're using, exception using partitions within a single topic because we have a low number of failures passing through. Granted the first partition will take the brunt of the load vs the higher order partitions. On Tue, Oct 10, 2017 at 7:30 AM, Steven Schlansker

Re: Consumer service that supports retry with exponential backoff

> On Oct 9, 2017, at 2:41 PM, John Walker wrote: > > I have a pair of services. One dispatches commands to the other for > processing. > > My consumer sometimes fails to execute commands as a result of transient > errors. To deal with this, commands are retried after an exponentially > increa

Fwd: Consumer service that supports retry with exponential backoff

I have a pair of services. One dispatches commands to the other for processing. My consumer sometimes fails to execute commands as a result of transient errors. To deal with this, commands are retried after an exponentially increasing delay up to a maximum of 4 times. (Delays: 1hr, 2hr, 4hr, 8h

Re: Serve interactive queries from standby replicas

Hi, I have created a ticker: https://issues.apache.org/jira/browse/KAFKA-6031 Best regards, Stas. 2017-10-06 23:39 GMT+02:00 Guozhang Wang : > Hi Stas, > > Would you mind creating a JIRA for this functionality request so that we > won't forget about it and drop on the floor? > > > Guozhang > >

Re: java.lang.NoSuchMethodError: scala.collection.GenTraversableOnce.$init

Hi Amy Could you check in your pom if any other dependencies defined in the pom is downloading different version of scala-library jar?. Any chance you are running kafka from eclipse?. If so sometimes if you are using scala from eclipse once u add scala nature it inherently picks up its own version

java.lang.NoSuchMethodError: scala.collection.GenTraversableOnce.$init

I'm trying to use kafka_2.11-0.11.* version but it fails with java.lang.NoSuchMethodError: scala.collection.GenTraversableOnce.$init. I don't have Scala installed on my Mac to cause any conflict. Anyone has seen this error? [2017-10-09 11:21:23,846] INFO binding to port 0.0.0.0/0.0.0.0:2181

Re: subscription

See http://kafka.apache.org/contact On 10/9/17 8:27 AM, Emanuele Ianni wrote: > subscription > signature.asc Description: OpenPGP digital signature

How to use ApiVersionRequest and ApiVersionResponse

Hi All I have a requirement where based on the kafka version or compatible api in the broker , we will be dynamically loading the best suited kafka client jar. I bumped into ApiVersionRequest and ApiVersionResponse and even went through some code to in NetworkClient class , but I was not able

subscription

subscription -- Distinti Saluti Emanuele Ianni Le informazioni e gli allegati contenuti in questa e-mail sono considerati confidenziali e possono essere riservati. Qualora non foste il destinatario, siete pregati di distruggere questo messaggio e notificarmi il problema immediatamente. In ogni c

I meet some problem i wanna use kafka connect oracle and hdfs then use hive。 message:org.apache.hadoop.hive.serde2.avro.AvroSerdeException Invalid precision or scale for decimal type

this is my oracle table CREATE TABLE T_JR_CCQK ( CC_ID INTEGER NOT NULL, CC_USER_FK INTEGER, CC_DEPT_FK INTEGER, CC_YEAR INTEGER, CC_MOUTH INTEGER, CC_DAY INTEGER, CC_STATES INTEGER, CC_ZCD VARCHAR2(80), CC_CCD VARCHAR

Re: kafka broker loosing offsets?

Hi tao, we had unclean leader election enabled at the beginning. But then disabled it and also reduced 'max.poll.records' value. It helped little bit. But after today's testing there is strong correlation between lag spike and what broker we crash. For lowest ID (100) broker : 1. it always at

Re: How to reduce time for consumer to take over?

Hello Christian, Assuming that you are using new Java kafka-clients with Kafka's consumer group management, you'd have to tune consumers and broker. See - Broker config https://kafka.apache.org/documentation/#brokerconfigs -- group.initial.rebalance.delay.ms -- group.max.session.timeout.ms -- gro

How to reduce time for consumer to take over?

I have the following case: 1. I have 1 consumer that starts processing a message, then crashes (Java-VM shuts down) 2. I start a second consumer that should process the message instead. It seems that it takes about 60 seconds for the second consumer to take over the processing. Can this be speede

Re: NPE on ConsumerRecords$ConcatenatedIterable$1.makeNext() while iterating records

Hi, Can you reproduce the error? Is it happening at the same offset every time? Try to reproduce with the console-consumer tool. You can raise JIRA issue here. https://issues.apache.org/jira/projects/KAFKA On Mon, Oct 9, 2017 at 3:00 PM, Michael Keinan wrote: > Thank you for your response. > N

Re: NPE on ConsumerRecords$ConcatenatedIterable$1.makeNext() while iterating records

Thank you for your response. No consumer interceptor involved Michael On Oct 8, 2017, at 7:04 PM, Ted Yu mailto:yuzhih...@gmail.com>> wrote: Was there any consumer interceptor involved ? Cheers On Sun, Oct 8, 2017 at 6:29 AM, Michael Keinan mailto:micha...@capitolis.com>> wrote: Hi Using

Re: The idea of "composite key" to make log compaction more flexible - question / proposal

Hey Jay, Thanks for reply. Yes, this should do the job. We were thinking about something that's abstracting away this logic from user (e.g. the same way Cassandra handles its PK definitions in CQL - "hiding" the row key and optional clustering key behind the concept of "primary key"), but introdu