Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp

[Andrew Schofield]

Hi Roman,
Thanks for the KIP. I think it’s an interesting idea, but I think the KIP 
document needs some
more details added before it’s ready for review. For example, here’s a KIP in 
the same
area which was delivered in an earlier version of Kafka. I think this is a good 
KIP to copy
for a suitable level of detail and description 
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs).

Hope this helps.

Thanks,
Andrew

> On 15 Oct 2023, at 21:02, Roman Schmitz  wrote:
>
> Hi all,
>
> While working with different customers I came across the case several times
> that we'd like to not only explicitly remove headers by name but by pattern
> / regexp. Here is a KIP for this feature!
>
> Please let me know if you have any comments, questions, or suggestions!
>
> https://cwiki.apache.org/confluence/x/oYtEE
>
> Thanks,
> Roman

KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp

[Roman Schmitz]

Hi all,

While working with different customers I came across the case several times
that we'd like to not only explicitly remove headers by name but by pattern
/ regexp. Here is a KIP for this feature!

Please let me know if you have any comments, questions, or suggestions!

https://cwiki.apache.org/confluence/x/oYtEE

Thanks,
Roman

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

[David Jacot]

Hi Erik,

Our current target is to have a preview in 3.7. This is subject to change
though.

Best,
David

Le dim. 15 oct. 2023 à 13:02, Erik van Oosten 
a écrit :

> Thanks Philip,
>
> That sounds pretty good. Meanwhile I'll continue to study KIP-848. It is
> a bit too much to digest in 1 go.
>
> Do you have a rough timeline for when the new consumer implementation
> can be tried out in non-production environments?
>
> Kind regards,
>  Erik.
>
>
> Op 14-10-2023 om 20:48 schreef Philip Nee:
> > Hi Erik,
> >
> > Thanks for the KIP, again.  I am also very much interested in the idea of
> > this KIP, and I want to let you know that we are rewriting the kafka
> > consumer using an event-driven approach, so I think the new impl would
> make
> > this KIP much easier to implement.  In a nutshell, the network IO will
> > become completely asynchronous to the application thread, so that the
> > blocking APIs won't stale the network send/receive.  In the new impl, the
> > main role of poll are 1. check if there are any background events such as
> > error or callback invocation, 2. notify the background that the user is
> > polling, and 3. check if there is any data to return to the user.
> > Because the background thread and application thread are inherently
> working
> > in an async fashion, it is possible to continue to process and commit
> > during the revocation period; however, we have to be very careful with
> the
> > state of partition ownership as you mentioned in the KIP.
> >
> > Please keep an eye out on the new consumer implementation, in case if you
> > are interested in digging in, it is the PrototypeKafkaConsumer module.
> It
> > is not fully functional but we are working full speed to get this to a
> good
> > state.
> >
> > Thanks - I am still reading to KIP and your previous KIP to see if I can
> > make more constructive suggestions here.
> > P
> >
> > On Fri, Oct 13, 2023 at 11:54 PM Erik van Oosten
> >  wrote:
> >
> >> Hello David,
> >>
> >> Thanks, I am happy to hear we agree on the problem. All the tiny details
> >> of an implementation are less important.
> >>
> >> I will read KIP-848 first to answer you question about its relation with
> >> KIP-983. But for sure it makes sense to complete the implementation of
> >> KIP-848 first.
> >>
> >> Kind regards,
> >>   Erik.
> >>
> >>
> >> Op 13-10-2023 om 21:00 schreef David Jacot:
> >>> Hi Erik,
> >>>
> >>> Thanks for the KIP. I haven’t fully read the KIP yet but I agree with
> the
> >>> weaknesses that you point out in it. I will continue to read it.
> >>>
> >>> For your information, we are working full speed on implementing KIP-848
> >>> while also changing the internal threading model of consumer. Those
> >> changes
> >>> are already extremely large so I would rather prefer to complete them
> >>> before adding more on top of them. Moreover, I think that this KIP
> should
> >>> build on top of KIP-848 now. Would you agree with this?
> >>>
> >>>
> >>> Best,
> >>> David
> >>>
> >>> Le ven. 13 oct. 2023 à 20:44, Erik van Oosten >> .invalid>
> >>> a écrit :
> >>>
>  Thanks Philip,
> 
>  No worries, I am not in a hurry. Knowing this is not forgotten is
> enough
>  for me. If there is anything I can do to help the process please let
> me
>  know.
> 
>  Kind regards,
> Erik.
> 
> 
>  Op 13-10-2023 om 20:29 schreef Philip Nee:
> > Hi Erik,
> >
> > Sorry for the delay, I have not finished reviewing the KIP, but I
> also
>  have
> > not forgotten about it!
> >
> > In general, KIP review process can be lengthy, so I think mailing
> list
> >> is
> > the best bet to get the committer's attention.
> >
> > P
> >
> > On Fri, Oct 13, 2023 at 10:55 AM Erik van Oosten
> >   wrote:
> >
> >> Hi client developers,
> >>
> >> The text is updated so that it is more clear that you can only use
> >> auto-commit when doing synchronous processing (approach 1). I am
> >> assuming that auto-commit commits whatever was consumed in the
> >> previous
> >> poll.
> >>
> >> I am wondering why this KIP doesn't get more attention. Is async
> >> processing not something that the kafka client wants to support?
> >>
> >> Kind regards,
> >> Erik.
> >>
> >>
> >> Op 25-09-2023 om 18:17 schreef Erik van Oosten:
> >>> Hi Viktor,
> >>>
> >>> Good questions!
> >>>
> >>> 1. Auto-commits would only work with approach 1 in the KIP. Any
> async
> >>> solution is incompatible with auto-commits. Do you think the text
> >> will
> >>> improve when this is mentioned?
> >>>
> >>> 2. That is entirely correct. If you use async commits you can await
> >>> completion by doing a single sync commit with an empty offsets Map
> >>> (this will work as of Kafka 3.6.0).
> >>>
> >>> Is there anything I can do to make the text clearer?
> >>>
> >>> Kind regards,
> >>>

Re: [DISCUSS] KIP-983: Full speed async processing during rebalance

[Erik van Oosten]

Thanks Philip,

That sounds pretty good. Meanwhile I'll continue to study KIP-848. It is 
a bit too much to digest in 1 go.


Do you have a rough timeline for when the new consumer implementation 
can be tried out in non-production environments?


Kind regards,
    Erik.


Op 14-10-2023 om 20:48 schreef Philip Nee:

Hi Erik,

Thanks for the KIP, again.  I am also very much interested in the idea of
this KIP, and I want to let you know that we are rewriting the kafka
consumer using an event-driven approach, so I think the new impl would make
this KIP much easier to implement.  In a nutshell, the network IO will
become completely asynchronous to the application thread, so that the
blocking APIs won't stale the network send/receive.  In the new impl, the
main role of poll are 1. check if there are any background events such as
error or callback invocation, 2. notify the background that the user is
polling, and 3. check if there is any data to return to the user.
Because the background thread and application thread are inherently working
in an async fashion, it is possible to continue to process and commit
during the revocation period; however, we have to be very careful with the
state of partition ownership as you mentioned in the KIP.

Please keep an eye out on the new consumer implementation, in case if you
are interested in digging in, it is the PrototypeKafkaConsumer module.  It
is not fully functional but we are working full speed to get this to a good
state.

Thanks - I am still reading to KIP and your previous KIP to see if I can
make more constructive suggestions here.
P

On Fri, Oct 13, 2023 at 11:54 PM Erik van Oosten
 wrote:


Hello David,

Thanks, I am happy to hear we agree on the problem. All the tiny details
of an implementation are less important.

I will read KIP-848 first to answer you question about its relation with
KIP-983. But for sure it makes sense to complete the implementation of
KIP-848 first.

Kind regards,
  Erik.


Op 13-10-2023 om 21:00 schreef David Jacot:

Hi Erik,

Thanks for the KIP. I haven’t fully read the KIP yet but I agree with the
weaknesses that you point out in it. I will continue to read it.

For your information, we are working full speed on implementing KIP-848
while also changing the internal threading model of consumer. Those

changes

are already extremely large so I would rather prefer to complete them
before adding more on top of them. Moreover, I think that this KIP should
build on top of KIP-848 now. Would you agree with this?


Best,
David

Le ven. 13 oct. 2023 à 20:44, Erik van Oosten
.invalid>

a écrit :


Thanks Philip,

No worries, I am not in a hurry. Knowing this is not forgotten is enough
for me. If there is anything I can do to help the process please let me
know.

Kind regards,
   Erik.


Op 13-10-2023 om 20:29 schreef Philip Nee:

Hi Erik,

Sorry for the delay, I have not finished reviewing the KIP, but I also

have

not forgotten about it!

In general, KIP review process can be lengthy, so I think mailing list

is

the best bet to get the committer's attention.

P

On Fri, Oct 13, 2023 at 10:55 AM Erik van Oosten
  wrote:


Hi client developers,

The text is updated so that it is more clear that you can only use
auto-commit when doing synchronous processing (approach 1). I am
assuming that auto-commit commits whatever was consumed in the

previous

poll.

I am wondering why this KIP doesn't get more attention. Is async
processing not something that the kafka client wants to support?

Kind regards,
Erik.


Op 25-09-2023 om 18:17 schreef Erik van Oosten:

Hi Viktor,

Good questions!

1. Auto-commits would only work with approach 1 in the KIP. Any async
solution is incompatible with auto-commits. Do you think the text

will

improve when this is mentioned?

2. That is entirely correct. If you use async commits you can await
completion by doing a single sync commit with an empty offsets Map
(this will work as of Kafka 3.6.0).

Is there anything I can do to make the text clearer?

Kind regards,
   Erik.


Op 25-09-2023 om 17:04 schreef Viktor Somogyi-Vass:

Hi Erik,

I'm still trying to wrap my head around the KIP, however I have a

few

questions that weren't clear to me regarding offset commits:
1. Would auto-commits interfere with the behavior defined in your

KIP

or

would it work the same as manual commits?
2. As I see you don't separate offset commits by whether they're

sync

or

async. For sync commits timing isn't really a problem but how would

you

change work in case of async offset commits? There can be a few

caveats

there as you may not know whether a commit is finished or not until

your

callback is called.

Thanks,
Viktor

On Sat, Sep 23, 2023 at 4:00 PM Erik van Oosten
  wrote:


Hi all,

I would like to start the discussion on KIP-983: Full speed async
processing during rebalance [1].

The idea is that we can prevent the drop in throughput during a
cooperative rebalance.

I am curious to your ideas and 

Re: Apache Kafka 3.7.0 Release

[Stanislav Kozlovski]

Hey Chris,

Thanks for the catch! It was indeed copied and I wasn't sure what to make
of the bullet point, so I kept it. What you say makes sense - I removed it.

I also added KIP-976!

Cheers!

On Sat, Oct 14, 2023 at 9:35 PM Chris Egerton 
wrote:

> Hi Stanislav,
>
> Thanks for putting this together! I think the "Ensure that release
> candidates include artifacts for the new Connect test-plugins module"
> section (which I'm guessing was copied over from the 3.6.0 release plan?)
> can be removed; we made sure that those artifacts were present for 3.6.0,
> and I don't anticipate any changes that would make them likelier to be
> accidentally dropped in subsequent releases than any other Maven artifacts
> that we publish.
>
> Also, can we add KIP-976 (
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-976%3A+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect
> )
> to the release plan? The vote thread for it passed last week and I've
> published a complete PR (https://github.com/apache/kafka/pull/14538), so
> it
> shouldn't be too difficult to get things merged in time for 3.7.0.
>
> Cheers,
>
> Chris
>
> On Sat, Oct 14, 2023 at 3:26 PM Stanislav Kozlovski
>  wrote:
>
> > Thanks for letting me drive it, folks.
> >
> > I've created the 3.7.0 release page here:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.7.0
> > It outlines the key milestones and important dates for the release.
> >
> > In particular, since the last two releases slipped their originally
> > targeted release date by taking an average of 46 days after code freeze
> (as
> > opposed to the minimum which is 14 days), I pulled the dates forward to
> try
> > and catch up with the original release schedule.
> > You can refer to the last release during the Christmas holiday season -
> > Apache
> > Kafka 3.4
> >  -
> > to
> > see sample dates.
> >
> > The currently proposed dates are:
> >
> > *KIP Freeze - 18th November *(Saturday)
> > *This is 1 month and four days from now - rather short - but I'm afraid
> is
> > the only lever that's easy to pull forward.*
> > As usual, a KIP must be accepted by this date in order to be considered
> for
> > this release. Note, any KIP that may not be implemented in a week, or
> that
> > might destabilize the release, should be deferred.
> >
> > *Feature Freeze - 8th December* (Friday)
> > *This follows 3 weeks after the KIP Freeze, as has been the case in our
> > latest releases.*
> > By this point, we want all major features to be merged & us to be working
> > on stabilisation. Minor features should have PRs, the release branch
> should
> > be cut; anything not in this state will be automatically moved to the
> next
> > release in JIRA
> >
> > *Code Freeze - 20th December* (Wednesday)
> >
> > *Critically, this is before the holiday season and ends in the middle of
> > the week, to give contributors more time and flexibility to address any
> > last-minute without eating into the time people usually take holidays. It
> > comes 12 days after the Feature Freeze.This is two days shorter than the
> > usual code freeze window. I don't have a strong opinion and am open to
> > extend it to Friday, or trade off a day/two with the KF<->FF date range.*
> >
> > *Release -* *after January 3rd*.
> > *It comes after a minimum of two weeks of stabilization, so the earliest
> we
> > can start releasing is January 3rd. We will move as fast as we can and
> aim
> > completing it as early in January as possible.*
> >
> > As for the initially-populated KIPs in the release plan, I did the
> > following:
> >
> > I kept 4 KIPs that were mentioned in 3.6, saying they would have minor
> > parts finished in 3.7 (as the major ones went out in 3.6)
> > - KIP-405 Tiered Storage mentioned a major part went out with 3.6 and the
> > remainder will come with 3.7
> > - KIP-890 mentioned Part 1 shipped in 3.6. I am assuming the remainder
> will
> > come in 3.7, and have contacted the author to confirm.
> > - KIP-926 was partially implemented in 3.6. I am assuming the remainder
> > will come in 3.7, and have contacted the author to confirm.
> > - KIP-938 mentioned that the majority was completed and a small remainder
> > re: ForwardingManager metrics will come in 3.7. I have contacted the
> author
> > to confirm.
> >
> > I then went through the JIRA filter which looks at open issues with a Fix
> > Version of 3.7 and added KIP-770, KIP-858, and KIP-980.
> > I also found a fair amount of JIRAs that were targeting the 3.7 release
> but
> > consecutively had no activity on them for the past few releases. For most
> > of those, I pinged the author and explicitly asked if it's going to aim
> to
> > make it to 3.7. I have not included those here and will not until I hear
> > confirmation.
> >
> > Please review the plan and provide any additional information or updates
> > regarding KIPs that target this release version (3.7).
> > If you have authored any 

Re: [VOTE] KIP-953: partition method to be overloaded to accept headers as well.

[Jack Tomy]

Hey contributors,

Please vote or veto the proposal.

Please don't ghost. Humble request.

Thanks

On Thu, Sep 28, 2023 at 7:23 AM Jack Tomy  wrote:

> Bumping up.
>
> On Sun, Sep 17, 2023 at 9:34 AM Jack Tomy  wrote:
>
>> Hey Ismael, Sagar and everyone,
>>
>> Sorry I seem to have interpreted the thread wrong. Before we go ahead
>> with the DTO based approach I have few reasons not to go with it.
>> a. It is not following the pattern we are following today. But here I
>> agree that patterns are to be changed for good.
>> b. The client is not supposed to modify the record object (This is my
>> understanding, If this is not a necessary requirement, please call it
>> out.), passing the entire object lets the client do that. To avoid that,
>> there has to be a way to deep copy the record object each time, this adds
>> unnecessary requirements on the record object to support the deepcopy
>> implementation. I see a lot of complexity and coupling coming in here due
>> to this N I believe it's a strong reason not to go ahead with the DTO
>> approach.
>>
>> Please let me know what you think.
>>
>> Thanks.
>>
>>
>>
>>
>>
>> On Wed, Sep 6, 2023 at 7:06 AM Sagar  wrote:
>>
>>> Hey Jack,
>>>
>>> The way I interpreted this thread, it seems like there's more alignment
>>> on
>>> the DTO based approach. I spent some time on the suggestion that Ismael
>>> had
>>> regarding the usage of ProducerRecord. Did you get a chance to look at
>>> the
>>> reply I had posted and whether that makes sense? Also, checking out the
>>> AdminClient APIs examples provided by Ismael will give you more context.
>>> Let me know what you think.
>>>
>>> Thanks!
>>> Sagar.
>>>
>>> On Thu, Aug 31, 2023 at 12:49 PM Jack Tomy 
>>> wrote:
>>>
>>> > Hey everyone,
>>> >
>>> > As I see devs favouring the current style of implementation, and that
>>> is
>>> > inline with existing code. I would like to go ahead with the same
>>> approach
>>> > as mentioned in the KIP.
>>> > Can I get a few more votes so that I can take the KIP forward.
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> > On Sun, Aug 27, 2023 at 1:38 PM Sagar 
>>> wrote:
>>> >
>>> > > Hi Ismael,
>>> > >
>>> > > Thanks for pointing us towards the direction of a DTO based
>>> approach. The
>>> > > AdminClient examples seem very neat and extensible in that sense.
>>> > > Personally, I was trying to think only along the lines of how the
>>> current
>>> > > Partitioner interface has been designed, i.e having all requisite
>>> > > parameters as separate arguments (Topic, Key, Value etc).
>>> > >
>>> > > Regarding this question of yours:
>>> > >
>>> > > A more concrete question: did we consider having the method
>>> `partition`
>>> > > > take `ProduceRecord` as one of its parameters and `Cluster` as the
>>> > other?
>>> > >
>>> > >
>>> > > No, I don't think in the discussion thread it was brought up and as I
>>> > said
>>> > > it appears that could be due to an attempt to keep the new method's
>>> > > signature similar to the existing one within Partitioner. If I
>>> understood
>>> > > the intent of the question correctly, are you trying to hint here
>>> that
>>> > > `ProducerRecord` already contains all the arguments that the
>>> `partition`
>>> > > method accepts and also has a `headers` field within it. So, instead
>>> of
>>> > > adding another method for the `headers` field, why not create a new
>>> > method
>>> > > taking ProducerRecord directly?
>>> > >
>>> > > If my understanding is correct, then it seems like a very clean way
>>> of
>>> > > adding support for `headers`. Anyways, the partition method within
>>> > > KafkaProducer already takes a ProducerRecord as an argument so that
>>> makes
>>> > > it easier. Keeping that in mind, should this new method's (which
>>> takes a
>>> > > ProducerRecord as an input) default implementation invoke the
>>> existing
>>> > > method ? One challenge I see there is that the existing partition
>>> method
>>> > > expects serialized keys and values while ProducerRecord doesn't have
>>> > access
>>> > > to those (It directly operates on K, V).
>>> > >
>>> > > Thanks!
>>> > > Sagar.
>>> > >
>>> > >
>>> > > On Sun, Aug 27, 2023 at 8:51 AM Ismael Juma 
>>> wrote:
>>> > >
>>> > > > A more concrete question: did we consider having the method
>>> `partition`
>>> > > > take `ProduceRecord` as one of its parameters and `Cluster` as the
>>> > other?
>>> > > >
>>> > > > Ismael
>>> > > >
>>> > > > On Sat, Aug 26, 2023 at 12:50 PM Greg Harris
>>> > > >> > > > >
>>> > > > wrote:
>>> > > >
>>> > > > > Hey Ismael,
>>> > > > >
>>> > > > > > The mention of "runtime" is specific to Connect. When it comes
>>> to
>>> > > > > clients,
>>> > > > > one typically compiles and runs with the same version or runs
>>> with a
>>> > > > newer
>>> > > > > version than the one used for compilation. This is standard
>>> practice
>>> > in
>>> > > > > Java and not something specific to Kafka.
>>> > > > >
>>> > > > > When I said "older runtimes" I was being lazy, and should have
>>> said
>>> > > > 

[VOTE] KIP-988 Streams StandbyUpdateListener

[Colt McNealy]

Hello there,

I'd like to call a vote on KIP-988 (co-authored by my friend and colleague
Eduwer Camacaro). We are hoping to get it in before the 3.7.0 release.

https://cwiki.apache.org/confluence/display/KAFKA/KIP-988%3A+Streams+Standby+Task+Update+Listener

Cheers,
Colt McNealy

*Founder, LittleHorse.dev*

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

[Colt McNealy]

Thanks, Guozhang. I've updated the KIP and will start a vote.

Colt McNealy

*Founder, LittleHorse.dev*


On Sat, Oct 14, 2023 at 10:27 AM Guozhang Wang 
wrote:

> Thanks for the summary, that looks good to me.
>
> Guozhang
>
> On Fri, Oct 13, 2023 at 8:57 PM Colt McNealy  wrote:
> >
> > Hello there!
> >
> > Thanks everyone for the comments. There's a lot of back-and-forth going
> on,
> > so I'll do my best to summarize what everyone's said in TLDR format:
> >
> > 1. Rename `onStandbyUpdateStart()` -> `onUpdateStart()`,  and do
> similarly
> > for the other methods.
> > 2. Keep `SuspendReason.PROMOTED` and `SuspendReason.MIGRATED`.
> > 3. Remove the `earliestOffset` parameter for performance reasons.
> >
> > If that's all fine with everyone, I'll update the KIP and we—well, mostly
> > Edu (:  —will open a PR.
> >
> > Cheers,
> > Colt McNealy
> >
> > *Founder, LittleHorse.dev*
> >
> >
> > On Fri, Oct 13, 2023 at 7:58 PM Eduwer Camacaro 
> > wrote:
> >
> > > Hello everyone,
> > >
> > > Thanks for all your feedback for this KIP!
> > >
> > > I think that the key to choosing proper names for this API is
> understanding
> > > the terms used inside the StoreChangelogReader. Currently, this class
> has
> > > two possible states: ACTIVE_RESTORING and STANDBY_UPDATING. In my
> opinion,
> > > using StandbyUpdateListener for the interface fits better on these
> terms.
> > > Same applies for onUpdateStart/Suspended.
> > >
> > > StoreChangelogReader uses "the same mechanism" for active task
> restoration
> > > and standby task updates, but this is an implementation detail. Under
> > > normal circumstances (no rebalances or task migrations), the changelog
> > > reader will be in STANDBY_UPDATING, which means it will be updating
> standby
> > > tasks as long as there are new records in the changelog topic. That's
> why I
> > > prefer onStandbyUpdated instead of onBatchUpdated, even if it doesn't
> 100%
> > > align with StateRestoreListener, but either one is fine.
> > >
> > > Edu
> > >
> > > On Fri, Oct 13, 2023 at 8:53 PM Guozhang Wang <
> guozhang.wang...@gmail.com>
> > > wrote:
> > >
> > > > Hello Colt,
> > > >
> > > > Thanks for writing the KIP! I have read through the updated KIP and
> > > > overall it looks great. I only have minor naming comments (well,
> > > > aren't naming the least boring stuff to discuss and that takes the
> > > > most of the time for KIPs :P):
> > > >
> > > > 1. I tend to agree with Sophie regarding whether or not to include
> > > > "Standby" in the functions of "onStandbyUpdateStart/Suspended", since
> > > > it is also more consistent with the functions of
> > > > "StateRestoreListener" where we do not name it as
> > > > "onStateRestoreState" etc.
> > > >
> > > > 2. I know in community discussions we sometimes say "a standby is
> > > > promoted to active", but in the official code / java docs we did not
> > > > have a term of "promotion", since what the code does is really
> recycle
> > > > the task (while keeping its state stores open), and create a new
> > > > active task that takes in the recycled state stores and just changing
> > > > the other fields like task type etc. After thinking about this for a
> > > > bit, I tend to feel that "promoted" is indeed a better name for user
> > > > facing purposes while "recycle" is more of a technical detail inside
> > > > the code and could be abstracted away from users. So I feel keeping
> > > > the name "PROMOTED" is fine.
> > > >
> > > > 3. Regarding "earliestOffset", it does feel like we cannot always
> > > > avoid another call to the Kafka API. And on the other hand, I also
> > > > tend to think that such bookkeeping may be better done at the app
> > > > level than from the Streams' public API level. I.e. the app could
> keep
> > > > a "first ever starting offset" per "topic-partition-store" key, and a
> > > > when we have rolling restart and hence some standby task keeps
> > > > "jumping" from one client to another via task assignment, the app
> > > > would update this value just one when it finds the
> > > > ""topic-partition-store" was never triggered before. What do you
> > > > think?
> > > >
> > > > 4. I do not have a strong opinion either, but what about
> > > "onBatchUpdated" ?
> > > >
> > > >
> > > > Guozhang
> > > >
> > > > On Wed, Oct 11, 2023 at 9:31 PM Colt McNealy 
> > > wrote:
> > > > >
> > > > > Sohpie—
> > > > >
> > > > > Thank you very much for such a detailed review of the KIP. It might
> > > > > actually be longer than the original KIP in the first place!
> > > > >
> > > > > 1. Ack'ed and fixed.
> > > > >
> > > > > 2. Correct, this is a confusing passage and requires context:
> > > > >
> > > > > One thing on our list of TODO's regarding reliability is to
> determine
> > > how
> > > > > to configure `session.timeout.ms`. In our Kubernetes Environment,
> an
> > > > > instance of our Streams App can be terminated, restarted, and get
> back
> > > > into
> > > > > the "RUNNING" Streams state in about 20 seconds. We have two
>