Re: How to enable Kafka Authentication as on-demand feature and later make mandatory feature?

2018-09-19 Thread Manikumar
One option is to enable multiple listeners (PLAINTEXT, SASL_PLAINTEXT) on brokers and slowly migrate the clients/services. On Wed, Sep 19, 2018 at 10:38 PM Ashwin Sinha wrote: > Hi, > > We have tried setting up Kafka Authentication by SASL/PLAIN >

Re: Does kafka support the delay queue

2018-09-19 Thread Subash Konar
Approach 1: Yes it can be.You have to create a dedicated topic for the same and I think the limitation will be all the messages to be processed should have same delay time in that particular topic and use decision making along with some loop for the messages. Approach 2:I believe the feature in ve

Does kafka support the delay queue

2018-09-19 Thread lx
Hi,kafka.apache.org Does kafka support delay queues?how can kafka implement delay queues? I hope my doubts can be answered.thanks Sincerely Lx

Re: Understanding default.deserialization.exception.handler

2018-09-19 Thread Matthias J. Sax
Thanks for reporting! It might have been an unknown issue. The community relies on reports like this, so we really appreciate that you reached out! -Matthias On 9/19/18 1:08 AM, Tim Ward wrote: > Thanks. No, it's not that big a deal now that I understand it, but as I'd had > to spend a fair am

Re: Questions about manage offset in external storage and consumer failure detect

2018-09-19 Thread Matthias J. Sax
1. If you don't have a good reason to store offsets externally, I would not recommend it, but use the client's built-in mechanism. It will be more work (ie, code you need to write) if you store offsets externally. For some use-cases, it's beneficial to store the offsets in an external system to ge

How to enable Kafka Authentication as on-demand feature and later make mandatory feature?

2018-09-19 Thread Ashwin Sinha
Hi, We have tried setting up Kafka Authentication by SASL/PLAIN and SASL/SCRAM (both w/o SSL). We found that SASL/SCRAM is more convenient as we do not need to

Did not work yet.//How to config the servers when Kafka cluster is behind a NAT?

2018-09-19 Thread XinYi Long
Hello, Thanks for your reply. And my net topology is as the attach picture. The node1's configuration is as below. The others are similar. listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:9001 listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT

Re: Kafka stream issue : Deleting obsolete state directory

2018-09-19 Thread Bhavesh Patel
Hi Bill, Apologies, I was trying to attached image. Please find below the error log, 2018-09-18 09:26:09.112 INFO 1 --- [5-CleanupThread] o.a.k.s.p.internals.StateDirectory : stream-thread [ApplicationName-1ae22d38-32d3-451a-b039-372c79b2e6a5-CleanupThread] Deleting obsolete state directory

RE: Understanding default.deserialization.exception.handler

2018-09-19 Thread Tim Ward
Thanks. No, it's not that big a deal now that I understand it, but as I'd had to spend a fair amount of time working out what was going on I thought I'd flag it up in case it wasn't known about. Tim Ward -Original Message- From: Matthias J. Sax Sent: 14 September 2018 19:57 To: users@k

Re: Kafka producer huge memory usage (leak?)

2018-09-19 Thread Manikumar
Similar issue reported here:KAFKA-7304, but on broker side. maybe you can create a JIRA and upload the heap dump for analysis. On Wed, Sep 19, 2018 at 11:59 AM Shantanu Deshmukh wrote: > Any thoughts on this matter? Someone, please help. > > On Tue, Sep 18, 2018 at 6:05 PM Shantanu Deshmukh >