Thanks for this proposal, Qingsheng! If you want to be a little conservative with the default, 5 minutes might be better than 30 seconds.
The equivalent config in Kafka seems to be metadata.max.age.ms (https://kafka.apache.org/documentation/#consumerconfigs_metadata.max.age.ms), which has a default value of 5 minutes. Other than that, I’m in favor. I agree, this should be on by default. Thanks again, John On Fri, Jan 13, 2023, at 08:26, Leonard Xu wrote: > Thanks Qingsheng for driving this, enable the dynamic partition > discovery would be very useful for kafka topic scale partitions > scenarios. > > +1 for the change. > > CC: Becket > > > Best, > Leonard > > > >> On Jan 13, 2023, at 3:15 PM, Jark Wu <imj...@gmail.com> wrote: >> >> +1 for the change. I think this is beneficial for users and is compatible. >> >> Best, >> Jark >> >> On Fri, 13 Jan 2023 at 14:22, 何军 <xuehaijux...@gmail.com> wrote: >> >>>> >>>> +1 for this idea, we have enabled kafka dynamic partition discovery in >>> all >>>> jobs. >>>> >>>> >>>
