Re: Query about LogConfigs

2015-07-24 Thread Gwen Shapira
Agree. All I found on this was KAFKA-1325, which is more about inconsistency than real use-case. Anyway, I'd argue that if we added it in 0.8.2.0, we can't take it out now :) On Fri, Jul 24, 2015 at 7:04 PM, Aditya Auradkar aaurad...@linkedin.com.invalid wrote: Is there actually a use case

Re: Query about LogConfigs

2015-07-24 Thread Mayuresh Gharat
Hmm. This creates confusion for people who might be new to kafka or have never used those properties. Does deprecating the older ones make sense here? Thanks, Mayuresh On Fri, Jul 24, 2015 at 7:15 PM, Gwen Shapira gshap...@cloudera.com wrote: Agree. All I found on this was KAFKA-1325, which

Re: Query about LogConfigs

2015-07-24 Thread Gwen Shapira
Backward compatibility, I think. At least the ms one is fairly new, and I think we left the others to avoid break configuration during upgrade. On Fri, Jul 24, 2015 at 6:34 PM, Mayuresh Gharat gharatmayures...@gmail.com wrote: I was thinking why we have 3 different configs for the same property

Re: Query about LogConfigs

2015-07-24 Thread Mayuresh Gharat
Oh ok. Is there a plan that we should deprecate the older ones. This is because as I said some users might misconfigure it like by using both the log.retention.ms and log.retention.minutes by mistake. We can probably set a warning that only one of this should be used. Also looking at the

Re: Query about LogConfigs

2015-07-24 Thread Mayuresh Gharat
Cool. Yeah +1 on make it consistent. Thanks, Mayuresh On Fri, Jul 24, 2015 at 7:29 PM, Grant Henke ghe...@cloudera.com wrote: They take precedence in order of granularity (ms minutes hours). Below is the relevant snippet of code. I vote for keeping all time related configs in

Re: Query about LogConfigs

2015-07-24 Thread Ewen Cheslack-Postava
This also came up recently when Geoff and I were discussing KAFKA-2257, which needs to add a timeout option to a tool that KAFKA-2276 is introducing. For that timeout, the question is whether we should strive for consistency (use ms), the unit that is most likely convenient (use s), or finest

Query about LogConfigs

2015-07-24 Thread Mayuresh Gharat
I was thinking why we have 3 different configs for the same property (log retention) : log.retention.ms log.retention.minutes log.retention.hours Why don't we only use the Milliseconds? There are other properties as well like log Jitter, LogRollTime which raise the same question in my mind.

Re: Query about LogConfigs

2015-07-24 Thread Aditya Auradkar
Is there actually a use case where we need log.retention.ms? In most cases, people would want to retain their logs for at least a few minutes I'd think. Aditya On Fri, Jul 24, 2015 at 6:49 PM, Gwen Shapira gshap...@cloudera.com wrote: Backward compatibility, I think. At least the ms one is

Re: Query about LogConfigs

2015-07-24 Thread Grant Henke
They take precedence in order of granularity (ms minutes hours). Below is the relevant snippet of code. I vote for keeping all time related configs in milliseconds and deprecating the others. It gives you the most control in a single config. The minutes and hours ones offer no new functionality