Also, just adding to the above point - we don't necessarily need to explicitly add a new worker configuration right? Instead, we could potentially just use the new proposed default value internally which can be overridden by users through setting a value for "offset.storage.segment.bytes" (via the existing KIP-605 based mechanism).
On Thu, Jul 6, 2023 at 6:04 PM Yash Mayya <yash.ma...@gmail.com> wrote: > Hi hudeqi, > > Thanks for the KIP! Just to clarify - since KIP-605 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-605%3A+Expand+Connect+Worker+Internal+Topic+Settings) > already allows configuring "segment.bytes" for the Connect cluster's > offsets topic via a worker configuration ("offset.storage.segment.bytes", > same as what is being proposed in this KIP), the primary motivation of this > KIP is essentially to override the default value for that topic > configuration to a smaller value and decouple it from the backing Kafka > cluster's "log.segment.bytes" configuration? Also, I'm curious about how > the new default value of 50 MB was chosen (if there were any experiments > that were run etc.)? > > Thanks, > Yash > > On Mon, Jul 3, 2023 at 6:08 PM hudeqi <16120...@bjtu.edu.cn> wrote: > >> Is anyone following this KIP? Bump this thread. > >
