Re: ACL with SSL is not working

2017-05-21 Thread Raghav
I tried all possible ways (including the way you suggested Michael), but I still get the same error. Is there a step by step guide to get ACLs working in Kafka with SSL ? Thanks. On Fri, May 19, 2017 at 11:40 AM, Michael Rauter wrote: > Hi, > > with SSL client authentication the user identifie

Re: Messages are repeating in kafka

2017-05-21 Thread Girish Aher
Yup, exactly as Kant said. Also make sure that the retention of the offsets topic is an upper bound across all topics. So in this case, don't create any other topics in the future with retention of more than 30 days or otherwise they may have the same problem too. On May 21, 2017 03:25, "Abhimanyu

Re: Messages are repeating in kafka

2017-05-21 Thread Abhimanyu Nagrath
Hi Kant, Thanks for the suggestion. Regards, Abhimanyu On Sun, May 21, 2017 at 3:44 PM, kant kodali wrote: > @Abhimanyu You can try setting offset.retention = 30 (log.retention). At > most, you will have a storage overhead of 5 million msgs per day * 30 > (days) * 8 bytes (for each offset) =

Re: Messages are repeating in kafka

2017-05-21 Thread kant kodali
@Abhimanyu You can try setting offset.retention = 30 (log.retention). At most, you will have a storage overhead of 5 million msgs per day * 30 (days) * 8 bytes (for each offset) = 1.2GB (not that much since you have a TB of hard disk) On Sun, May 21, 2017 at 3:05 AM, kant kodali wrote: > Looking

Re: Messages are repeating in kafka

2017-05-21 Thread kant kodali
Looking at that ticket and reading the comments it looks like one of the concern is as follows. "offsets.retention.minutes is designed to handle the case that a consumer group goes away forever. In that case, we don't want to store the offsets for that group forever." This can simply be addressed

Re: Messages are repeating in kafka

2017-05-21 Thread Abhimanyu Nagrath
Hi, My average message size is less 30 Kb and I get 1 near about 5 Million message per day On Sun, May 21, 2017 at 2:49 PM, kant kodali wrote: > What is your average message size and network speed? > > On Sun, May 21, 2017 at 2:04 AM, Abhimanyu Nagrath < > abhimanyunagr...@gmail.com> wrote: > >

Re: Messages are repeating in kafka

2017-05-21 Thread kant kodali
What is your average message size and network speed? On Sun, May 21, 2017 at 2:04 AM, Abhimanyu Nagrath < abhimanyunagr...@gmail.com> wrote: > Hi Girish, > > I did not set any value for offsets.retention.minutes so therefore what I > think is picking its default value i.e 1440 minutes so what do

Re: Messages are repeating in kafka

2017-05-21 Thread Abhimanyu Nagrath
Hi Girish, I did not set any value for offsets.retention.minutes so therefore what I think is picking its default value i.e 1440 minutes so what do you think what should I set if I am keeping my data for 30 days? Regards, Abhimanyu