Re: Offset commit for partitions not owned by consumer

Below are the logs: Consumer 1 logs, where issue can be seen: https://pastebin.com/PuQhud91 Consumer 2 logs: https://pastebin.com/yfJDSGPA server.log: https://pastebin.com/QKpk0zLn controller.log: https://pastebin.com/9T0niwEw state-change.log: https://pastebin.com/nrftHPC9 On Fri, Nov 3, 2017

Re: Kafka Streams : CommitFailedException

Hello Tony, Which version of Kafka did you use for the Streams library? And could you share your settings (num. threads in total, num of tasks etc) and your exception stack trace so that we can help you better identifying the root cause of the exception? Guozhang On Thu, Nov 2, 2017 at 4:09

Re: How do I instantiate a metrics reporter in Kafka Streams, with custom config?

Hi James, Thanks for bringing this up. The original design is to let users apply the prefix ONLY if they want to override for the specific client. For example, for `metric.reporters` or `interceptor.classes` which are defined for both producer and consumer (and moving forward we are adding admin

Re: Offset commit for partitions not owned by consumer

Can you pastebin relevant logs from client and broker ? Thanks On Fri, Nov 3, 2017 at 1:37 PM, Manan G wrote: > Hello, > > I am using 0.11.0.0 version of Kakfa broker and Java client library. My > consumer code tracks offsets for each assigned partition and at some time >

Offset commit for partitions not owned by consumer

Hello, I am using 0.11.0.0 version of Kakfa broker and Java client library. My consumer code tracks offsets for each assigned partition and at some time interval manually commits offsets by specifying partition->offset map. What I noticed is, after the rebalance, even if consumer loses some

Kafka log4j appender

Did the log4j appender get deprecated? I've spent a good amount of time on google and don't find any such announcement but I also don't find any documentation since 0.7. Thanks, Dale

Manual offset control and topic compaction

Hi, I'm working on the Kafka connector for Apache Storm, which pulls messages from Kafka and emits them into a Storm topology. The connector uses manual offset control since message processing happens asynchronously to pulling messages from Kafka, and we hit an issue a while back related to topic

Re: Kafka Security SSL and SASL_PLAINTEXT

If you have just one Kafka node, you probably don't care about the replication / interbroker communication. J. On Fri, Nov 3, 2017 at 6:36 PM, chidigam . wrote: > One more point I have missed out,I am using just one zk and one kafka > instance. > Regards > Bhanu > > On

Re: Kafka Security SSL and SASL_PLAINTEXT

One more point I have missed out,I am using just one zk and one kafka instance. Regards Bhanu On Fri, Nov 3, 2017 at 10:51 PM, chidigam . wrote: > Hi Jakub, > I believe there is some thing wrong with inter broker communication. In > producer my send call just blocks. > As

Re: Kafka Security SSL and SASL_PLAINTEXT

If you use the Java Client the logging should be quite easy. You can add the SLF4J simple logger to your dependencies: org.slf4j slf4j-simple 1.7.22 And set the system property org.slf4j.simpleLogger.defaultLogLevel to the log level you want -

Re: Kafka Security SSL and SASL_PLAINTEXT

Hi Jakub, I believe there is some thing wrong with inter broker communication. In producer my send call just blocks. As told you I am new to the Kafka, is there any quick way to enable the producer logs. Regards Bhanu On Fri, Nov 3, 2017 at 10:37 PM, Jakub Scholz wrote: > Do

Re: Kafka Security SSL and SASL_PLAINTEXT

Do you have some logs from your producer? Is it just the producer what is not working? Or is the producer not working because the Interbroker communication doesn't work? J. On Fri, Nov 3, 2017 at 6:02 PM, chidigam . wrote: > Hi Jakub, > Thanks for responding, I tried

Re: Kafka Security SSL and SASL_PLAINTEXT

Hi Jakub, Thanks for responding, I tried what you have suggested, but producer is not working. If I enable SSL for replication,then every thing works fine. Regards Bhanu On Fri, Nov 3, 2017 at 9:52 PM, Jakub Scholz wrote: > Sure, you can use someting like this: >

Re: Kafka Security SSL and SASL_PLAINTEXT

Sure, you can use someting like this: listeners=SSL://:9092,REPLICATION://:19092 listener.security.protocol.map=SSL:SSL,REPLICATION:SASL_PLAINTEXT inter.broker.listener.name=REPLICATION (plus all the SSL and Kerberos configuration) Jakub On Fri, Nov 3, 2017 at 4:48 PM, chidigam .

Kafka Security SSL and SASL_PLAINTEXT

Hi All, I am new to Kafka and trying to understand possible security combinations. I want to do Client authentication and authorization only with SSL. Inter broker communication should be with Kerberos with out SSL. Is this possible? Can anyone help me with configurations. Regards Bhanu

Kafka latency optimization

Hi, everyone My kafka version is 0.10.2.1. My service have really low qps (1msg/sec). And our requirement for rtt is really strict. ( 99.9% < 30ms) Currently I've encounter a problem, when kafka run for a long time, 15 days or so, performance start to go down. 2017-10-21 was like Time .

Re: Kafka 0.11 broker running out of file descriptors

Us too. Read up on this one, released in 1.0.0, and pinned against an 0.11.0.2. https://issues.apache.org/jira/browse/KAFKA-6042 There are some other tickets linked in that also. On Fri, Nov 3, 2017 at 5:40 PM, Madhukar Bharti wrote: > Hi, > > We also facing the

Re: Kafka 0.11 broker running out of file descriptors

Hi, We also facing the same issue when we upgraded our cluste to 0.11.0.1. Can anyone please help us out. It happened in production cluster. Thanks Madhukar On Fri, 15 Sep 2017 at 9:07 PM, Lukas Lalinsky wrote: > Hello, > > I'm dealing with a strange issue in