+1 on the KIP. bq. introducing offsets.retention.hours config property
Probably the introduction would cause confusion among users due to the existing minutes config. On Fri, Oct 6, 2017 at 10:53 AM, Manikumar <manikumar.re...@gmail.com> wrote: > looks like VOTE thread is started for this KIP. > > On Wed, Aug 16, 2017 at 5:39 PM, Stevo Slavić <ssla...@gmail.com> wrote: > > > +1 for making consistent default log and offsets retention time. > > I like Stephane's suggestion too, log retention override should override > > offset retention too if not explicitly configured. > > > > Please consider additionally: > > - introducing offsets.retention.hours config property > > - syncing log and offsets retention.check.interval.ms, if there's no > real > > reason for the two to differ > > -- consider making retention check interval by default (if not explicitly > > configured) a fraction of retention time > > - name all "offsets" configs with "offsets" prefix (now it's a mix of > > singular/"offset" and plural/"offsets") > > > > > > On Fri, Aug 11, 2017 at 2:01 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > +1 from me > > > > > > On Wed, Aug 9, 2017 at 9:40 AM, Jason Gustafson <ja...@confluent.io> > > > wrote: > > > > > > > +1 on the bump to 7 days. Wanted to mention one minor point. The > > > > OffsetCommit RPC still provides the ability to set the retention time > > > from > > > > the client, but we do not use it in the consumer. Should we consider > > > adding > > > > a consumer config to set this? Given the problems people had with the > > old > > > > default, such a config would probably have gotten a fair bit of use. > > > Maybe > > > > it's less necessary with the new default, but there may be situations > > > where > > > > you don't want to keep the offsets for too long. For example, the > > console > > > > consumer commits offsets with a generated group id. We might want to > > set > > > a > > > > low retention time to keep it from filling the offset cache with > > garbage > > > > from such groups. > > > > > > > > > > I agree with Jason here, but maybe itself deserves a separate KIP > > > discussion. > > > > > > > > > > > > > > -Jason > > > > > > > > On Wed, Aug 9, 2017 at 5:24 AM, Sönke Liebau < > > > > soenke.lie...@opencore.com.invalid> wrote: > > > > > > > > > Just had this create issues at a customer as well, +1 > > > > > > > > > > On Wed, Aug 9, 2017 at 11:46 AM, Mickael Maison < > > > > mickael.mai...@gmail.com> > > > > > wrote: > > > > > > > > > > > Yes the current default is too short, +1 > > > > > > > > > > > > On Wed, Aug 9, 2017 at 8:56 AM, Ismael Juma <ism...@juma.me.uk> > > > wrote: > > > > > > > Thanks for the KIP, +1 from me. > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > On Wed, Aug 9, 2017 at 1:24 AM, Ewen Cheslack-Postava < > > > > > e...@confluent.io > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Hi all, > > > > > > >> > > > > > > >> I posted a simple new KIP for a problem we see with a lot of > > > users: > > > > > > >> KIP-186: Increase offsets retention default to 7 days > > > > > > >> > > > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > > >> 186%3A+Increase+offsets+retention+default+to+7+days > > > > > > >> > > > > > > >> Note that in addition to the KIP text itself, the linked JIRA > > > > already > > > > > > >> existed and has a bunch of discussion on the subject. > > > > > > >> > > > > > > >> -Ewen > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sönke Liebau > > > > > Partner > > > > > Tel. +49 179 7940878 > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - > Germany > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > >
