hi Jiunn +1 to Approach A :)
Best, Chia-Ping On 2026/07/07 11:55:14 黃竣陽 wrote: > Hi Jun, Chia-Ping, > > I'd like to first explain why I believe changing the default is worthwhile. > > Documentation can only help users who already know about the problem. > Because the failure mode is silent, most users discover it only after data > has > already been missed. A safer default such as by_duration:PT5S protects users > automatically, without requiring prior awareness of the issue. > > Given that, the question is not whether the issue exists, but how best to > introduce > a safer default. > > Approach A — Follow a deprecation cycle and land the change in 5.0 > - Keep the current default (`latest`) in upcoming 4.x releases, but emit a > warning that > the default will change to `by_duration:PT5S` in 5.0 and explain how users > can retain > the current behavior explicitly. > - Change the default to `by_duration:PT5S` in 5.0. > - Update shell tools (for example, the console consumer) to explicitly use > `latest`, preserving > the expected "show only new messages" behavior. > > Approach B — Change the default in a 4.4 minor release > - Change the default to `by_duration:PT5S` in a 4.x minor release. > - Leave shell tools unchanged. > - Emit a warning explaining the change and how users can explicitly restore > `latest`. > > I favor Approach B: > > Silent data loss during partition expansion is a correctness issue, not a > usability issue. > Delaying the safer default until 5.0 leaves users relying on the implicit > default exposed unnecessarily. > Users who need latest can simply pin it explicitly. > I don't think shell tools need special handling. by_duration:PT5S only > replays a few seconds of data, > which is negligible in practice and preferable to silently missing records. > Keeping the behavior consistent > is simpler. > > Best Regards, > Jiunn-Yang > > > Jun Rao via dev <[email protected]> 於 2026年7月7日 清晨5:49 寫道： > > > > Hi, Chia-Ping, > > > > I am not sure that we need an out-of-the-box solution. The proposal in the > > KIP is also not out-of-the-box and requires users to opt-in. > > > > Thanks, > > > > Jun > > > > On Mon, Jul 6, 2026 at 11:34 AM Chia-Ping Tsai <[email protected]> wrote: > > > >> hi Jun > >> > >> Yes, updating the docs requires minimal changes to the codebase, but it is > >> not an out-of-the-box solution. From my perspective, this issue can be > >> addressed by setting the console consumer's policy back to latest. > >> > >> Let's see what Jiunn thinks > >> > >> Best, > >> Chia-Ping > >> > >> On 2026/07/06 15:59:37 Jun Rao via dev wrote: > >>> Hi, Chia-Ping, > >>> > >>> One potential downside of changing the default is that some existing > >> tools > >>> (e.g. console consumer) may depend on this behavior. For example, a user > >>> testing the system may expect to see only new messages and could be > >>> surprised by the change. Another option is to leave the default > >> unchanged, > >>> but document clearly how to prevent losing messages on new partitions > >>> through by_duration. The users can then make the corresponding config > >>> changes if needed. > >>> > >>> Thanks, > >>> > >>> Jun > >>> > >>> On Wed, Jul 1, 2026 at 9:39 AM Chia-Ping Tsai <[email protected]> > >> wrote: > >>> > >>>> Hi Jun, > >>>> > >>>> Thanks for the feedback. I agree with your conclusion. It would be > >> better > >>>> to "make haste" with a new KIP to address this. We can focus the > >> discussion > >>>> on the following points: > >>>> > >>>> 1) Should we change the default reset policy to by_duration for all > >>>> consumers? > >>>> 2) What is the optimal duration to prevent data loss while avoiding > >> large > >>>> backlogs? Note that the classic consumer has a 3-second heartbeat > >> interval, > >>>> whereas the async consumer uses 5 seconds. > >>>> 3) What are the potential side effects of replacing latest with > >>>> by_duration as the default behavior? > >>>> > >>>> Best, > >>>> Chia-Ping > >>>> > >>>> On 2026/06/30 20:39:24 Jun Rao via dev wrote: > >>>>> Hi, Chia-Ping, > >>>>> > >>>>> I agree that using by_duration with a value of 5s to 10s addresses > >> most > >>>>> common issues with expanded partitions, given the default > >>>>> heartbeat.interval.ms is 3 seconds. We can consider setting that as > >> the > >>>> new > >>>>> default. The current proposal may address some other rarer cases, > >> but it > >>>>> feels a bit like over engineering. > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Jun > >>>>> > >>>>> On Tue, Jun 30, 2026 at 5:58 AM Chia-Ping Tsai <[email protected]> > >>>> wrote: > >>>>> > >>>>>> - Short durations fail during extended downtimes, leading to the > >> exact > >>>>>> silent data > >>>>>> loss this KIP aims to prevent. > >>>>>> > >>>>>> Losing the records produced to expanded partitions during a > >> downtime is > >>>>>> actually the expected behavior for the latest policy. This is > >> exactly > >>>> why, > >>>>>> in the current KIP proposal, we prefer by_duration over earliest > >> for > >>>> the > >>>>>> expanded partitions—we want to avoid forcing users to consume a > >> massive > >>>>>> accumulated backlog just because a partition is technically "new". > >>>>>> > >>>>>> To quote my previous comment from May 15: > >>>>>> "...if a consumer is offline for a while and a new partition is > >> created > >>>>>> during that downtime, the user might actually want to skip to > >> latest > >>>> when > >>>>>> resuming, rather than reading from earliest just because the > >> partition > >>>> is > >>>>>> technically 'new' to the group." > >>>>>> > >>>>>> On 2026/06/30 12:12:47 黃竣陽 wrote: > >>>>>>> Hello all, > >>>>>>> > >>>>>>> The metadata blindness and consumer downtime windows are > >> inherently > >>>>>>> unpredictable, making a fixed by_duration approach unviable. No > >>>> static > >>>>>>> duration can accommodate both extremes: > >>>>>>> > >>>>>>> - Short durations fail during extended downtimes, leading to the > >>>> exact > >>>>>> silent data > >>>>>>> loss this KIP aims to prevent. > >>>>>>> - Long durations (calibrated for worst-case scenarios) cause > >> routine > >>>>>> restarts to > >>>>>>> trigger unnecessary mass replays, undermining the latest offset > >> reset > >>>>>> policy. > >>>>>>> > >>>>>>> This is why the proposed solution uses creation time. It > >> provides a > >>>>>> discrete, > >>>>>>> deterministic signal—verifying whether a partition is actually > >> newer > >>>>>> than the group, > >>>>>>> rather than relying on an arbitrary time window that has no > >>>> correlation > >>>>>> with the consumer’s > >>>>>>> actual downtime. > >>>>>>> > >>>>>>> Best Regards, > >>>>>>> Jiunn-Yang > >>>>>>> > >>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月30日 下午3:05 寫道： > >>>>>>>> > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> I'd like to revisit Jun previous comment about using > >>>> by_duration=5s. I > >>>>>> realize now that having a small backlog is actually an acceptable > >>>>>> trade-off, especially since the latest policy inherently brings a > >> tiny > >>>>>> backlog to starting consumers anyway due to the time gap between > >>>> fetching > >>>>>> the offset and actually consuming. > >>>>>>>> > >>>>>>>> If we bring the simple by_duration=5s solution back to the > >> KIP, I'd > >>>>>> like to discuss a further step: Should we just change the default > >>>> policy > >>>>>> from latest to by_duration=5s specifically for the new consumer and > >>>> share > >>>>>> consumer? > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> Chia-Ping > >>>>>>>> > >>>>>>>> On 2026/06/15 17:39:19 Chia-Ping Tsai wrote: > >>>>>>>>> hi Jun > >>>>>>>>> > >>>>>>>>> The most important part of this story is how users should > >> expect > >>>> the > >>>>>> data > >>>>>>>>> they can see when using the latest or by_duration policy with > >>>> expanded > >>>>>>>>> partitions. > >>>>>>>>> > >>>>>>>>> Yes, the by_duration policy can minimize data loss, but it is > >>>>>>>>> non-deterministic, which means users will either read too many > >>>>>> historical > >>>>>>>>> records from existing partitions or lose some records from > >>>> expanded > >>>>>>>>> partitions. > >>>>>>>>> > >>>>>>>>> Also, I agree that auto.offset.reset.max.age.ms is a bit > >> hard to > >>>>>>>>> understand, and that is why I preferred having a whole new > >> policy > >>>>>> based > >>>>>>>>> entirely on group creation time (KIP-1282) > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Chia-Ping > >>>>>>>>> > >>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月16日週二 > >> 上午1:08寫道： > >>>>>>>>> > >>>>>>>>>> Hi, Chia-Ping and Jiunn-Yang, > >>>>>>>>>> > >>>>>>>>>> Thanks for the reply. I am still trying to understand the > >> value > >>>> of > >>>>>> the new > >>>>>>>>>> configs with the KIP. > >>>>>>>>>> > >>>>>>>>>> The motivation of the KIP is that a user doesn't want to > >> miss the > >>>>>> data if > >>>>>>>>>> the backlog is small. The backlog of the existing partition > >> is > >>>> easy > >>>>>> to > >>>>>>>>>> understand because it relates to retention time. The backlog > >> for > >>>> the > >>>>>> new > >>>>>>>>>> partition is a bit subtle to understand since it depends on > >> the > >>>>>> metadata > >>>>>>>>>> refresh delay. To set auto.offset.reset.max.age.ms, the user > >>>> needs > >>>>>> to > >>>>>>>>>> understand the metadata refresh delay on the consumer side > >> and > >>>> use > >>>>>> it to > >>>>>>>>>> set the config. > >>>>>>>>>> > >>>>>>>>>> Now, let's consider the alternative: setting the same value > >> for > >>>> the > >>>>>>>>>> existing by_duration policy. The KIP lists three issues with > >> this > >>>>>> approach. > >>>>>>>>>> 1. It computes the seek target client-side as now() - > >> duration, > >>>> which > >>>>>>>>>> introduces clock skew across consumers and forces operators > >> to > >>>> choose > >>>>>>>>>> overly large durations, causing unnecessary reprocessing. > >>>>>>>>>> 2. The target timestamp is recomputed on each retry, so > >> failed > >>>>>>>>>> ListOffsetsRequest retries can shift the target forward and > >>>>>> potentially > >>>>>>>>>> miss records produced between attempts. > >>>>>>>>>> 3. It applies uniformly to all partitions without committed > >>>> offsets, > >>>>>> and > >>>>>>>>>> cannot distinguish newly expanded partitions from > >> long-existing > >>>>>> partitions > >>>>>>>>>> newly assigned to the group, leading to unnecessary replay. > >>>>>>>>>> > >>>>>>>>>> Issues 1 and 2 are uncommon and can be mitigated by adding a > >> bit > >>>>>> buffer to > >>>>>>>>>> the metadata refresh delay. We could also consider improving > >> the > >>>>>>>>>> implementation. For issue 3, the metadata refresh delay is > >>>> typically > >>>>>> low > >>>>>>>>>> (in the order of minutes with the classic consumer and tens > >> of > >>>>>> seconds with > >>>>>>>>>> the new consumer). If a user is ok with reading that much > >> backlog > >>>>>> for new > >>>>>>>>>> partitions, it seems they will be ok doing the same for > >> existing > >>>>>>>>>> partitions. > >>>>>>>>>> > >>>>>>>>>> So, instead of introducing a new config, could we just reuse > >> the > >>>>>> existing > >>>>>>>>>> config with better documentation and/or implementation? > >>>>>>>>>> > >>>>>>>>>> Jun > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Sat, Jun 13, 2026 at 12:19 AM 黃竣陽 <[email protected]> > >> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hello Jun, > >>>>>>>>>>> > >>>>>>>>>>> You're right that group creation time is the more intuitive > >>>> answer > >>>>>> at > >>>>>>>>>>> first glance, > >>>>>>>>>>> the KIP's own motivation talks about partitions that > >> "predate > >>>> the > >>>>>> group" > >>>>>>>>>>> vs partitions > >>>>>>>>>>> "created during group runtime," which directly points to a > >>>>>>>>>> group-lifecycle > >>>>>>>>>>> classifier. > >>>>>>>>>>> I'd like to walk through why we landed on partition age, > >> and the > >>>>>>>>>>> trade-offs we considered. > >>>>>>>>>>> > >>>>>>>>>>> We evaluated three candidate signals: > >>>>>>>>>>> > >>>>>>>>>>> 1. `by_duration:5secs` > >>>>>>>>>>> > >>>>>>>>>>> This covers the metadata blindness window, but has issues > >> the > >>>> KIP > >>>>>>>>>>> currently documents > >>>>>>>>>>> under "Why not use `by_duration`?": > >>>>>>>>>>> > >>>>>>>>>>> - Client-side `now() - duration` introduces clock skew > >> across > >>>>>> consumers. > >>>>>>>>>>> - `ListOffsets` retries shift the target forward, > >> potentially > >>>>>> missing > >>>>>>>>>>> records produced between > >>>>>>>>>>> attempts. > >>>>>>>>>>> - It applies uniformly to all partitions without committed > >>>> offsets, > >>>>>>>>>>> including pre-existing partitions > >>>>>>>>>>> newly assigned to the group, causing unnecessary replay. > >>>>>>>>>>> > >>>>>>>>>>> 2. Group creation time as classifier > >>>>>>>>>>> > >>>>>>>>>>> This works cleanly when the consumer is actively running. > >> Our > >>>>>> concern > >>>>>>>>>>> is the idle / late-rejoin case: > >>>>>>>>>>> > >>>>>>>>>>> T=0: Group created. > >>>>>>>>>>> T=1..T=100: Consumer idle (down, disconnected, etc.). > >>>>>>>>>>> T=50: Partition added during the idle window. > >>>>>>>>>>> T=100: Consumer resumes. > >>>>>>>>>>> > >>>>>>>>>>> Under group creation time, the new partition is classified > >> as > >>>> new > >>>>>>>>>>> (`50 > 0`) and reset to `earliest`, replaying everything > >> from > >>>> T=50. > >>>>>>>>>>> But during `[T=1, T=100]`, base partitions also accumulated > >> data > >>>>>> that > >>>>>>>>>>> the consumer accepts as lost — that is precisely the > >> contract of > >>>>>>>>>>> `auto.offset.reset=latest`. There is no principled reason to > >>>> treat > >>>>>>>>>>> the new partition differently; both contain backlog > >> accumulated > >>>>>> during > >>>>>>>>>>> the same idle window. > >>>>>>>>>>> > >>>>>>>>>>> This aligns with the "backlog is backlog” principle you > >> raised > >>>> in > >>>>>>>>>>> the KIP-1282 thread: a `latest` user has tolerated some > >> backlog > >>>> on > >>>>>>>>>>> every other partition during the same idle period; forcing > >>>> 0-backlog > >>>>>>>>>>> tolerance only on new partitions would be inconsistent with > >> that > >>>>>>>>>>> tolerance. > >>>>>>>>>>> > >>>>>>>>>>> 3. Partition age vs threshold > >>>>>>>>>>> > >>>>>>>>>>> Partition age corresponds to the actual silent data loss > >> window, > >>>>>>>>>>> the gap between partition creation and the consumer’s > >> metadata > >>>>>>>>>>> refresh. Within this window, data loss is genuinely silent: > >> the > >>>>>>>>>>> consumer had no opportunity to know about the partition. > >> Outside > >>>>>> this > >>>>>>>>>>> window, missing data reflects either: > >>>>>>>>>>> > >>>>>>>>>>> - (a) the user’s tolerated cost of running with idle > >> consumers, > >>>> or > >>>>>>>>>>> - (b) an operational issue to surface via monitoring, not > >> via > >>>> reset > >>>>>>>>>> policy. > >>>>>>>>>>> > >>>>>>>>>>> We did not choose partition age because it is more elegant > >> than > >>>>>> group > >>>>>>>>>>> creation time — we chose it because its failure mode > >> (requires a > >>>>>>>>>>> threshold) is > >>>>>>>>>>> less invasive than the failure mode of group creation time > >>>>>> (overrides > >>>>>>>>>>> user-stated > >>>>>>>>>>> `latest` intent during idle periods). > >>>>>>>>>>> > >>>>>>>>>>> Best Regards, > >>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>> > >>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年6月13日 上午11:52 > >> 寫道： > >>>>>>>>>>>> > >>>>>>>>>>>> Hi Jun, > >>>>>>>>>>>> > >>>>>>>>>>>> Relying on both creation times will create an inconsistent > >>>>>> scenario. A > >>>>>>>>>>>> consumer that lost all offsets due to a long sleep will > >> seek > >>>> to the > >>>>>>>>>>>> beginning for the partitions created later than the group. > >>>>>>>>>>>> > >>>>>>>>>>>> That is why we initially proposed KIP-1282 to fix the > >>>> inconsistency > >>>>>>>>>>> using a > >>>>>>>>>>>> whole new policy. Since KIP-1282 couldn't reach a > >> consensus, > >>>>>> KIP-1327 > >>>>>>>>>>> goes > >>>>>>>>>>>> back to using flexible configurations to prevent users from > >>>> falling > >>>>>>>>>> into > >>>>>>>>>>>> that pitfall. > >>>>>>>>>>>> > >>>>>>>>>>>> Best, Chia-Ping > >>>>>>>>>>>> > >>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 2026年6月13日週六 > >>>> 上午6:49寫道： > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks for the reply and sorry for the late reply. > >>>>>>>>>>>>> > >>>>>>>>>>>>> JR1. The design of auto.offset.reset.max.age.ms still > >> feels > >>>>>> weird to > >>>>>>>>>>> me. > >>>>>>>>>>>>> It > >>>>>>>>>>>>> categorizes partitions as new or existing based on the > >>>> partition > >>>>>>>>>>> creation > >>>>>>>>>>>>> time. Intuitively, the categorization should be based on > >> the > >>>> group > >>>>>>>>>>> creation > >>>>>>>>>>>>> time: all partitions existing when the group is created > >> are > >>>>>> existing > >>>>>>>>>> and > >>>>>>>>>>>>> all partitions created after the group creation are new > >>>>>> partitions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Jun > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Jun 9, 2026 at 8:51 AM 黃竣陽 <[email protected]> > >>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Manually bumping this thread. If there is no further > >>>>>>>>>>>>>> discussion, I will close the vote. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年6月1日 晚上7:16 寫道： > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hello Jian, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks for your feedback, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Agreed, partition expansion is a common operational > >> task, > >>>> not an > >>>>>>>>>> edge > >>>>>>>>>>>>>>> case. I've updated the Motivation section accordingly. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> jian fu <[email protected]> 於 2026年6月1日 下午5:49 寫道： > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi Jiunn-Yang: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks for the KIP. I think it would be useful to > >> clarify > >>>> that > >>>>>> this > >>>>>>>>>>>>> is a > >>>>>>>>>>>>>>>> common scenario rather than an edge case, which further > >>>>>>>>>> demonstrates > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>> need for this optimization. For example: > >>>>>>>>>>>>>>>> A partition expansion is a common operational task in > >>>> Kafka: To > >>>>>>>>>>>>> balance > >>>>>>>>>>>>>>>> resource utilization and cost, topics are typically > >> created > >>>>>> with a > >>>>>>>>>>>>>> moderate > >>>>>>>>>>>>>>>> default partition count. However, as traffic grows over > >>>> time, > >>>>>> it is > >>>>>>>>>>>>>> often > >>>>>>>>>>>>>>>> necessary to increase the number of partitions to > >>>> accommodate > >>>>>> the > >>>>>>>>>>>>> higher > >>>>>>>>>>>>>>>> workload. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Regards > >>>>>>>>>>>>>>>> Jian > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 于2026年5月30日周六 22:31写道： > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hello chia, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks for the comments, I have updated the KIP! > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 2026年5月30日 > >> 晚上8:29 > >>>> 寫道： > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi Jiunn-Yang, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Would you mind removing the terms "hot" and "cold" > >> when > >>>>>>>>>> describing > >>>>>>>>>>>>>>>>>> partitions in the KIP? I understand you are using > >> them to > >>>>>>>>>> describe > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> "freshness" or the users' need for the records, but > >>>> applying > >>>>>>>>>> these > >>>>>>>>>>>>>> terms > >>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>> the partition itself feels a bit unnatural. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> After all, in this scenario, users don't really care > >>>> whether > >>>>>> a > >>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>> newly expanded or not. Their only expectation is that > >>>> they > >>>>>> won't > >>>>>>>>>>>>>> silently > >>>>>>>>>>>>>>>>>> lose any live records produced to the topic during > >> their > >>>>>> active > >>>>>>>>>>>>>>>>> consumption. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Best, Chia-Ping > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月30日週六 下午12:30寫道： > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Hello Jun, > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks for the feedback, I have updated the KIP > >>>> motivation > >>>>>>>>>>> section. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Jun Rao via dev <[email protected]> 於 > >> 2026年5月30日 > >>>>>> 凌晨1:12 寫道： > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks for the reply. I think we need a stronger > >>>>>> motivation for > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> KIP. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> The KIP says "The core insight is that not all > >>>> partitions > >>>>>>>>>> without > >>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>> committed offset are the same. A newly expanded > >>>> partition > >>>>>> (hot) > >>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>> fundamentally different from a partition the > >> consumer > >>>> has > >>>>>> never > >>>>>>>>>>>>> seen > >>>>>>>>>>>>>>>>>>>> because it predates the group (cold)." Why is the > >> hot > >>>>>> partition > >>>>>>>>>>>>>>>>>>>> fundamentally different from the cold? > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> The KIP says "The existing by_duration policy is > >> also > >>>>>>>>>>> insufficient > >>>>>>>>>>>>>>>>>>> because: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> - The calculated seek time (now() - duration) > >> varies > >>>> across > >>>>>>>>>> nodes > >>>>>>>>>>>>>> due > >>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>> clock skew. To be safe, users must set an overly > >> large > >>>>>>>>>> duration, > >>>>>>>>>>>>>>>>>>> causing > >>>>>>>>>>>>>>>>>>>> unnecessary reprocessing. > >>>>>>>>>>>>>>>>>>>> - On network errors, the client recalculates the > >> seek > >>>> time > >>>>>> on > >>>>>>>>>>>>> retry, > >>>>>>>>>>>>>>>>>>>> shifting the target timestamp forward and risking > >> data > >>>>>> loss." > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> However, both of these situations are rare. If > >> these > >>>> issues > >>>>>>>>>>>>> persist, > >>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>> severe problems likely exist elsewhere. Rare > >> situations > >>>>>> don't > >>>>>>>>>>>>> need a > >>>>>>>>>>>>>>>>>>> common > >>>>>>>>>>>>>>>>>>>> solution. If users care about those rare > >> situations, > >>>> they > >>>>>> can > >>>>>>>>>>>>>> implement > >>>>>>>>>>>>>>>>>>>> customized logic using > >>>>>>>>>>>>>>>>> ConsumerRebalanceListener.onPartitionsAssigned(). > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Sun, May 17, 2026 at 6:50 AM 黃竣陽 < > >>>> [email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Hello chia, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Thanks for the feedback, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > >>>> should > >>>>>> always > >>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>> greater > >>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > >>>>>>>>>>>>>>>>>>>>> I have explicitly mentioned this in the KIP. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > >>>> any > >>>>>>>>>>>>>> field > >>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> The earliest point at which we can detect the > >> version > >>>>>> mismatch > >>>>>>>>>>> is > >>>>>>>>>>>>>>>>> during > >>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> first metadata fetch after assignment, which > >> occurs > >>>> inside > >>>>>>>>>>>>> poll(). > >>>>>>>>>>>>>>>>>>>>> Therefore, the > >>>>>>>>>>>>>>>>>>>>> user would encounter an > >> UnsupportedVersionException > >>>> from > >>>>>>>>>> poll(). > >>>>>>>>>>>>>> I’ll > >>>>>>>>>>>>>>>>>>>>> clarify this in the KIP. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > >> 2026年5月17日 > >>>>>> 下午4:50 寫道： > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> hi Jiunn > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> PartitionAgeMs (int64, default -1): The age of > >> this > >>>>>>>>>> partition > >>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>> milliseconds, computed server-side by the broker > >> as > >>>>>>>>>>>>>>>>> broker_current_time > >>>>>>>>>>>>>>>>>>> - > >>>>>>>>>>>>>>>>>>>>> partition_creation_time. Returns -1 if the broker > >>>> does not > >>>>>>>>>>>>> support > >>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>> feature or the partition creation time is unknown. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> If the creation time exists, the returned value > >>>> should > >>>>>> always > >>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>> greater > >>>>>>>>>>>>>>>>>>>>> than or equal to zero, right? > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> New Old (MetadataResponse v0–13) positive > >>>> any > >>>>>>>>>>>>>> field > >>>>>>>>>>>>>>>>>>>>> absent UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Will user encounter UnsupportedVersionException > >> when > >>>>>> calling > >>>>>>>>>>>>>>>>> `poll()`? > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On 2026/05/16 04:30:49 黃竣陽 wrote: > >>>>>>>>>>>>>>>>>>>>>>> Hello Jun, chia, > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> I've updated KIP-1327 with a design change > >> based on > >>>> the > >>>>>>>>>>>>>> discussion > >>>>>>>>>>>>>>>>>>>>>>> feedback. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> The updated design decouples the new-partition > >> reset > >>>>>>>>>> behavior > >>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>> the base auto.offset.reset policy: > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> - auto.offset.reset.max.age.ms now applies to > >> all > >>>>>>>>>>>>>> auto.offset.reset > >>>>>>>>>>>>>>>>>>>>> values > >>>>>>>>>>>>>>>>>>>>>>> (latest, earliest, by_duration, none). > >>>>>>>>>>>>>>>>>>>>>>> - For new ("hot") partitions, the consumer > >> resets to > >>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new.partitions > >>>>>>>>>>>>>>>>>>>>>>> config setting > >>>>>>>>>>>>>>>>>>>>>>> - For existing ("cold") partitions, the base > >>>>>>>>>> auto.offset.reset > >>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>> continues > >>>>>>>>>>>>>>>>>>>>>>> to apply unchanged. > >>>>>>>>>>>>>>>>>>>>>>> - The new-partition reset behavior is > >> represented > >>>> by a > >>>>>>>>>>> separate > >>>>>>>>>>>>>>>>>>>>> internal config > >>>>>>>>>>>>>>>>>>>>>>> (auto.offset.reset.new.partitions, currently > >> fixed > >>>> to > >>>>>>>>>>>>> earliest). > >>>>>>>>>>>>>>>>> This > >>>>>>>>>>>>>>>>>>>>> decoupled design makes > >>>>>>>>>>>>>>>>>>>>>>> it straightforward to promote the behavior to a > >>>> public > >>>>>>>>>>>>>> user-facing > >>>>>>>>>>>>>>>>>>>>> configuration in a future KIP. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > >> 2026年5月16日 > >>>>>> 清晨7:46 > >>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> hi Jun > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> I see what you mean now. The proposal from me > >> is > >>>> listed > >>>>>>>>>>> below: > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> 1) Add auto.offset.reset.new.partitions with a > >>>> default > >>>>>>>>>> value > >>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>> earliest. It fixes the data loss from both > >>>> by_duration and > >>>>>>>>>>>>> latest, > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>> does not change the logic of > >>>> auto.offset.reset=earliest. > >>>>>>>>>>>>>>>>>>>>>>>> 2) Mark auto.offset.reset.new.partitions as an > >>>>>> internal > >>>>>>>>>>>>>>>>>>>>> configuration. auto.offset.reset.new > >>>> .partitions=earliest > >>>>>>>>>>> already > >>>>>>>>>>>>>>>>>>>>> addresses the issue, and we can discuss the use > >> cases > >>>> of > >>>>>> other > >>>>>>>>>>>>>> values > >>>>>>>>>>>>>>>>>>> in a > >>>>>>>>>>>>>>>>>>>>> separate KIP. > >>>>>>>>>>>>>>>>>>>>>>>> 3) Both configs, auto.offset.reset.new > >> .partitions > >>>> and > >>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age.ms, will be > >> applied > >>>> to > >>>>>> all > >>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>> consistency. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> WDYT? > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/15 20:53:20 Jun Rao via dev wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> Hi, Chia-Ping, > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the reply. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> 1. In the motivation section, the KIP says > >> "When a > >>>>>> Kafka > >>>>>>>>>>>>> topic > >>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>> expanded > >>>>>>>>>>>>>>>>>>>>>>>>> with new partitions, consumers using the > >> latest > >>>> auto > >>>>>>>>>> offset > >>>>>>>>>>>>>> reset > >>>>>>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>>>>>> will silently miss all records produced to > >> those > >>>>>>>>>> partitions > >>>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>> consumer discovers them.". If a user sets > >>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset=by_duration=1sec, the same > >>>> record > >>>>>> loss > >>>>>>>>>>>>> issue > >>>>>>>>>>>>>>>>> could > >>>>>>>>>>>>>>>>>>>>> also > >>>>>>>>>>>>>>>>>>>>>>>>> happen, right? > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> 2. I was thinking auto.offset.reset.new > >>>> .partitions > >>>>>> will > >>>>>>>>>>> take > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> same > >>>>>>>>>>>>>>>>>>>>>>>>> values as auto.offset.reset. So a user could > >> set > >>>> it > >>>>>>>>>>>>>> by_duration if > >>>>>>>>>>>>>>>>>>>>> needed. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 14, 2026 at 4:06 PM Chia-Ping > >> Tsai < > >>>>>>>>>>>>>>>>> [email protected] > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> hi Jun > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. I might be missing > >>>> something > >>>>>>>>>>>>>> important > >>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>> your > >>>>>>>>>>>>>>>>>>>>>>>>>> suggestion, so please bear with me as I try > >> to > >>>>>> clarify > >>>>>>>>>> with > >>>>>>>>>>>>> a > >>>>>>>>>>>>>> few > >>>>>>>>>>>>>>>>>>>>> questions: > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> 1. Is there a strong use case for extending > >> this > >>>>>> logic to > >>>>>>>>>>>>>> other > >>>>>>>>>>>>>>>>>>> reset > >>>>>>>>>>>>>>>>>>>>>>>>>> policies? Unlike latest, policies like > >> earliest > >>>> or > >>>>>>>>>>>>> by_duration > >>>>>>>>>>>>>>>>>>> don't > >>>>>>>>>>>>>>>>>>>>> seem > >>>>>>>>>>>>>>>>>>>>>>>>>> to suffer from the same silent data loss > >> issue > >>>> when a > >>>>>>>>>>>>>> partition > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>> expanded. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> 2. What values would we expect users to > >>>> configure for > >>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.new.partitions? If they > >> set > >>>> it to > >>>>>>>>>>>>> earliest > >>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>> latest, > >>>>>>>>>>>>>>>>>>>>>>>>>> we might run into the exact same edge cases. > >> For > >>>>>> example, > >>>>>>>>>>>>> if a > >>>>>>>>>>>>>>>>>>>>> consumer is > >>>>>>>>>>>>>>>>>>>>>>>>>> offline for a while and a new partition is > >>>> created > >>>>>> during > >>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>> downtime, > >>>>>>>>>>>>>>>>>>>>>>>>>> the user might actually want to skip to > >> latest > >>>> when > >>>>>>>>>>>>> resuming, > >>>>>>>>>>>>>>>>>>> rather > >>>>>>>>>>>>>>>>>>>>> than > >>>>>>>>>>>>>>>>>>>>>>>>>> reading from earliest just because the > >> partition > >>>> is > >>>>>>>>>>>>>> technically > >>>>>>>>>>>>>>>>>>>>> "new" to > >>>>>>>>>>>>>>>>>>>>>>>>>> the group. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> This is exactly why we opted for introducing > >> a > >>>>>> max.age > >>>>>>>>>>>>>> threshold. > >>>>>>>>>>>>>>>>>>> It > >>>>>>>>>>>>>>>>>>>>> gives > >>>>>>>>>>>>>>>>>>>>>>>>>> users a time-bound way to define what is > >>>> genuinely > >>>>>>>>>>> "hot/new" > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>> what is > >>>>>>>>>>>>>>>>>>>>>>>>>> just an old partition they haven't seen yet. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/05/14 20:48:09 Jun Rao via dev wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, Jiunn-Yang, > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I find auto.offset.reset.latest.max.age a > >> bit > >>>>>> weird. It > >>>>>>>>>>>>> only > >>>>>>>>>>>>>>>>>>>>> applies when > >>>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset is latest. However, it > >> seems > >>>> that > >>>>>> the > >>>>>>>>>>>>>>>>> motivation > >>>>>>>>>>>>>>>>>>>>>>>>>> equally > >>>>>>>>>>>>>>>>>>>>>>>>>>> applies when auto.offset.reset is set to > >> other > >>>>>> values > >>>>>>>>>> like > >>>>>>>>>>>>>>>>>>>>> by_duration. > >>>>>>>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>>> intention is that we want to have a separate > >>>> way to > >>>>>>>>>>> control > >>>>>>>>>>>>>>>>> newly > >>>>>>>>>>>>>>>>>>>>> created > >>>>>>>>>>>>>>>>>>>>>>>>>>> partitions vs existing partitions when the > >> group > >>>>>> starts. > >>>>>>>>>>>>>> Have we > >>>>>>>>>>>>>>>>>>>>>>>>>> considered > >>>>>>>>>>>>>>>>>>>>>>>>>>> adding a new config like > >> auto.offset.reset.new > >>>>>>>>>>> .partitions? > >>>>>>>>>>>>>> If > >>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>> new > >>>>>>>>>>>>>>>>>>>>>>>>>>> config is not set, the offset reset policy > >>>> defaults > >>>>>> to > >>>>>>>>>> the > >>>>>>>>>>>>>>>>> policy > >>>>>>>>>>>>>>>>>>>>> used > >>>>>>>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>>>>>> existing partitions. The user could set it > >>>>>> explicitly to > >>>>>>>>>>>>>>>>> customize > >>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>> behavior for new partitions. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, May 7, 2026 at 5:07 AM 黃竣陽 < > >>>>>> [email protected]> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I’d like to manually bump this thread. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 黃竣陽 <[email protected]> 於 2026年5月1日 > >> 晚上10:37 > >>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01/DJ02: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MetadataResponse bumps from v13 to v14. > >> The > >>>>>>>>>>>>>> PartitionMetadata > >>>>>>>>>>>>>>>>>>>>> struct > >>>>>>>>>>>>>>>>>>>>>>>>>>>> gains a new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> field PartitionAgeMs (int64, default -1), > >>>> computed > >>>>>>>>>>>>>> server-side > >>>>>>>>>>>>>>>>>>> by > >>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>> broker as > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> broker_current_time - > >> partition_creation_time. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also add the consumer heartbeat flow. when > >>>>>>>>>>>>>> MembershipManager > >>>>>>>>>>>>>>>>>>>>> detects > >>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>> newly assigned > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition, it explicitly invalidates the > >>>> metadata > >>>>>> for > >>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> affected > >>>>>>>>>>>>>>>>>>>>>>>>>> topic > >>>>>>>>>>>>>>>>>>>>>>>>>>>> and forces a fresh MetadataRequest > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> before making the offset reset decision, > >> even > >>>> if > >>>>>> the > >>>>>>>>>>>>> topic > >>>>>>>>>>>>>> ID > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>> already > >>>>>>>>>>>>>>>>>>>>>>>>>>>> in the cache. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB0: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The consumer learns the broker's maximum > >>>> supported > >>>>>>>>>>>>>>>>>>>>> MetadataResponse > >>>>>>>>>>>>>>>>>>>>>>>>>>>> version via the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ApiVersions negotiation at connection > >> time. > >>>> If the > >>>>>>>>>>>>>> negotiated > >>>>>>>>>>>>>>>>>>>>>>>>>> version is > >>>>>>>>>>>>>>>>>>>>>>>>>>>> unsupported, the consumer > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> knows the broker does not support > >>>> PartitionAgeMs > >>>>>> at > >>>>>>>>>> all > >>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>> can > >>>>>>>>>>>>>>>>>>>>>>>>>> throw an > >>>>>>>>>>>>>>>>>>>>>>>>>>>> UnsupportedVersionException > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> immediately, rather than silently falling > >>>> back to > >>>>>>>>>> latest > >>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>> risking > >>>>>>>>>>>>>>>>>>>>>>>>>>>> data loss without any operator-visible > >> signal. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MB1/MB2/MB3: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have addressed these changes in the KIP. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> 於 > >>>> 2026年4月29日 > >>>>>>>>>>> 下午4:04 > >>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi David > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I agree with the direction of moving the > >>>> 'age' > >>>>>>>>>>>>> resolution > >>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Heartbeat API to the Metadata API to keep > >> the > >>>>>> control > >>>>>>>>>>>>> plane > >>>>>>>>>>>>>>>>>>> clean. > >>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>> main > >>>>>>>>>>>>>>>>>>>>>>>>>>>> trade-off, as we noted before, is > >> introducing > >>>>>>>>>>> inter-broker > >>>>>>>>>>>>>>>>> clock > >>>>>>>>>>>>>>>>>>>>> skew. > >>>>>>>>>>>>>>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Group Coordinator approach provided a > >> single > >>>>>> source of > >>>>>>>>>>>>> truth > >>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>> time. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> However, realistically, this time skew > >>>> should be > >>>>>>>>>>>>>> negligible. > >>>>>>>>>>>>>>>>>>>>> Given > >>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the max.age threshold will likely be > >>>> configured in > >>>>>>>>>>> minutes > >>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>> hours, a > >>>>>>>>>>>>>>>>>>>>>>>>>>>> typical NTP skew (in milliseconds) between > >>>> brokers > >>>>>>>>>> won't > >>>>>>>>>>>>>> impact > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>> fallback decision. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David Jacot via dev < > >> [email protected]> > >>>> 於 > >>>>>>>>>>>>> 2026年4月29日 > >>>>>>>>>>>>>>>>>>> 下午3:29 > >>>>>>>>>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry, I haven't really followed the > >>>> previous > >>>>>>>>>>>>>> conversation > >>>>>>>>>>>>>>>>>>> but I > >>>>>>>>>>>>>>>>>>>>>>>>>> took a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quick look at this one. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ01: I don't clearly understand the > >> flow > >>>> with > >>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>> ConsumerGroupHeartbeat > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> API after reading the KIP. There is a > >> new > >>>>>> boolean; > >>>>>>>>>> the > >>>>>>>>>>>>>> KIP > >>>>>>>>>>>>>>>>>>>>> states > >>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition ages are returned only when > >> this > >>>>>> boolean > >>>>>>>>>> is > >>>>>>>>>>>>>> set. > >>>>>>>>>>>>>>>>>>>>>>>>>> Implicitly, > >>>>>>>>>>>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> means that when the consumer receives a > >> new > >>>>>>>>>> partition, > >>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>> will > >>>>>>>>>>>>>>>>>>>>>>>>>> issue a > >>>>>>>>>>>>>>>>>>>>>>>>>>>> new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HB request with the boolean set to > >> receive > >>>> the > >>>>>> ages. > >>>>>>>>>>> Is > >>>>>>>>>>>>>> my > >>>>>>>>>>>>>>>>>>>>>>>>>>>> understanding > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct? We should perhaps clarify the > >> flow > >>>> and > >>>>>> also > >>>>>>>>>>>>>> explain > >>>>>>>>>>>>>>>>>>>>> how it > >>>>>>>>>>>>>>>>>>>>>>>>>>>> fits > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into the existing flow (e.g. list > >> offsets, > >>>> fetch > >>>>>>>>>>>>> offsets, > >>>>>>>>>>>>>>>>>>> etc.). > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DJ02: It my understanding is correct, I > >>>> wonder > >>>>>> if > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API is the > >> right > >>>>>> place > >>>>>>>>>> for > >>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>> given > >>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a new > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> round trip is done anyway. > >> Alternatively, it > >>>>>> could > >>>>>>>>>>>>> simply > >>>>>>>>>>>>>>>>>>>>> include > >>>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metadata. Generally, we should be rather > >>>>>> cautious > >>>>>>>>>>> about > >>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>>>>>>>>>>> overloading > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ConsumerGroupHeartbeat API with > >>>> unrelated > >>>>>>>>>>> concepts. > >>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>> API > >>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control plane API for assigning or > >> revoking > >>>>>>>>>>> partitions. > >>>>>>>>>>>>>> The > >>>>>>>>>>>>>>>>>>> fact > >>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> don't want to add it to the > >> corresponding > >>>>>> Streams > >>>>>>>>>> API > >>>>>>>>>>>>>> also > >>>>>>>>>>>>>>>>>>>>> suggests > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> something is not quite right. What > >> would we > >>>> do > >>>>>> if we > >>>>>>>>>>>>>> want to > >>>>>>>>>>>>>>>>>>>>>>>>>> support > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Streams in the future? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> David > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Apr 29, 2026 at 12:28 AM > >> Muralidhar > >>>>>> Basani > >>>>>>>>>>> via > >>>>>>>>>>>>>> dev > >>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jiunn, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for this great kip. Good to > >> know > >>>>>> about > >>>>>>>>>> the > >>>>>>>>>>>>>> gap. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-0 - why a new v2 version bump for > >>>>>>>>>>>>>> RequestPartitionAges > >>>>>>>>>>>>>>>>>>>>> field. > >>>>>>>>>>>>>>>>>>>>>>>>>> Can a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tagged field (for ex: on response, > >>>>>> PartitionAges on > >>>>>>>>>>>>>>>>>>>>>>>>>> TopicPartitions) > >>>>>>>>>>>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> used here and avoid version bump? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-1 - For the new config, is there a > >>>>>> recommended > >>>>>>>>>>>>> value > >>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>> ConfigDef > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> validator? Probably it should based on > >> the > >>>>>>>>>>>>>>>>>>> metadata.max.age.ms > >>>>>>>>>>>>>>>>>>>>> ? > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Sizing > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> instructions can be part of javadocs I > >>>> guess. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-2 - (minor) As there are no changes > >> to > >>>> Kafka > >>>>>>>>>>>>> Streams, > >>>>>>>>>>>>>>>>>>> would > >>>>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>>>>> better > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to add this new config > >>>>>>>>>>>>> auto.offset.reset.latest.max.age > >>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> StreamsConfig block list > >>>>>>>>>>>>>>>>>>>>>>>>>> (NON_CONFIGURABLE_CONSUMER_DEFAULT_CONFIGS) > >>>>>>>>>>>>>>>>>>>>>>>>>>>> for a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clear warning, incase users configure > >> it? > >>>> This > >>>>>> is > >>>>>>>>>> the > >>>>>>>>>>>>>> most > >>>>>>>>>>>>>>>>>>>>>>>>>> familiar > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config and users might easily > >>>>>> mistakenly > >>>>>>>>>>>>>> configure > >>>>>>>>>>>>>>>>>>>>> it. Or > >>>>>>>>>>>>>>>>>>>>>>>>>>>> may be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's not worth it to add. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mb-3 - (minor) The phrasing "the > >> consumer > >>>> falls > >>>>>>>>>> back > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>> earliest" > >>>>>>>>>>>>>>>>>>>>>>>>>>>> reads as > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> if the config were being changed > >>>> per-partition > >>>>>>>>>> which > >>>>>>>>>>>>>> isn't > >>>>>>>>>>>>>>>>>>>>>>>>>> supported. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> May > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be rephrasing to something like > >> "consumer > >>>>>> resolves > >>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> initial > >>>>>>>>>>>>>>>>>>>>>>>>>>>> position to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> start offset for that partition" as if > >>>>>> earliest was > >>>>>>>>>>>>>> applied > >>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> partition only and auto.offset.reset > >>>> config is > >>>>>>>>>>>>>> unchanged. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Murali > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Apr 28, 2026 at 2:48 PM 黃竣陽 < > >>>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi chia, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have updated the KIP to include this > >>>> change. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping Tsai <[email protected]> > >> 於 > >>>>>>>>>> 2026年4月28日 > >>>>>>>>>>>>>> 晚上8:03 > >>>>>>>>>>>>>>>>>>> 寫道： > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hi Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> chia_0: Should we expose the > >> partition > >>>>>> creation > >>>>>>>>>>> time > >>>>>>>>>>>>>> via > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>> Admin > >>>>>>>>>>>>>>>>>>>>>>>>>>>> API? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I assume it would be valuable for > >> users to > >>>>>>>>>> diagnose > >>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>>> troubleshoot > >>>>>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> behavior of > >>>> auto.offset.reset.latest.max.age > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 2026/04/28 10:47:58 黃竣陽 wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to start a discussion > >> on > >>>>>> KIP-1327 > >>>>>>>>>>>>>> Prevent > >>>>>>>>>>>>>>>>> Hot > >>>>>>>>>>>>>>>>>>>>> Data > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Loss > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Partition Expansion for Latest Policy > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>> > >>>> > >> https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/KY4mGQ__;!!Ayb5sqE7!qF4q1QzF1RRgP61D7A2xuEai1ky7fepKDKFFvpNBuePikH-ULmT87TvuuZzy5kau5E4y5zMZAmfQQiwZomM$ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This proposal aims to introduces > >>>>>>>>>>>>>>>>>>>>>>>>>> auto.offset.reset.latest.max.age, > >>>>>>>>>>>>>>>>>>>>>>>>>>>> a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer config that lets the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest reset policy distinguish > >> newly > >>>>>> expanded > >>>>>>>>>>>>> (hot) > >>>>>>>>>>>>>>>>>>>>> partitions > >>>>>>>>>>>>>>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> long-existing (cold) ones. Partitions > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> younger than the configured > >> threshold > >>>>>>>>>>> automatically > >>>>>>>>>>>>>> fall > >>>>>>>>>>>>>>>>>>>>> back > >>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> earliest, preventing silent data loss > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> during topic expansion without > >> forcing a > >>>>>> full > >>>>>>>>>>>>>> historical > >>>>>>>>>>>>>>>>>>>>>>>>>> reprocess. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jiunn-Yang > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > >
