Re: KIP-560 Discuss

[Boyang Chen]

No worry! Could you address Matthias' comments in this mailing thread?
Seems we still have some gaps.

On Mon, Jul 6, 2020 at 7:14 AM Sang wn Lee  wrote:

> I'm sorry.
> I just modified the KIP!
>
> On 2020/03/07 20:09:47, "Matthias J. Sax"  wrote:
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA512
> >
> > Thanks for the KIP Sang!
> >
> > I have a couple of more comments about the wiki page:
> >
> > (1) The "Public Interface" section should only list the new stuff. This
> > KIP does not change anything with regard to the existing options
> > `--input-topic` or `--intermediate-topic` and thus it's just "noise" to
> > have them in this section. Only list the new option
> > `allInputTopicsOption`.
> >
> > (2) Don't post code, ie, the implementation of private methods. KIPs
> > should only describe public interface changes.
> >
> > (3) The KIP should describe that we intend to use
> > `describeConsumerGroups` calls to discover the topic names -- atm, it's
> > unclear from the KIP how the new feature actually works.
> >
> > (4) If the new flag is used, we will discover input and intermediate
> > topics. Hence, the name is miss leading. We could call it
> > `--all-user-topics` and explain in the description that "user topics"
> > are input and intermediate topics for this case (in general, also output
> > topics are "user topics" but there is nothing to be done for output
> > topics). Thoughts?
> >
> >
> > - -Matthias
> >
> >
> > On 1/27/20 6:35 AM, Sang wn Lee wrote:
> > > thank you John Roesle
> > >
> > > It is a good idea "—all-input-topics"
> > >
> > > I agree with you
> > >
> > > I'll update right away
> > >
> > >
> > > On 2020/01/24 14:14:17, "John Roesler" 
> > > wrote:
> > >
> > >> Hi all, thanks for the explanation. I was also not sure how the
> > >> kip would be possible to implement.
> > >>
> > >> No that it does seem plausible, my only feedback is that the
> > >> command line option could align better with the existing one.
> > >> That is, the existing option is called “—input-topics”, so it
> > >> seems like the new one should be called “—all-input-topics”.
> > >>
> > >> Thanks, John
> > >>
> > >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> > >>> Thanks Sophie for the explanation! I read Sang's PR and
> > >>> basically he did exactly what you proposed (check it here
> > >>>  in case I'm
> > >>> wrong).
> > >>>
> > >>> I think Sophie's response answers Gwen's question already,
> > >>> while in the meantime for a KIP itself we are not required to
> > >>> mention all the internal details about how to make the changes
> > >>> happen (like how to actually get the external topics),
> > >>> considering the change scope is pretty small as well. But
> > >>> again, it would do no harm if we mention it inside Proposed
> > >>> Change session specifically so that people won't get confused
> > >>> about how.
> > >>>
> > >>>
> > >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman
> > >>>  wrote:
> > >>>
> >  Hi all,
> > 
> >  I think what Gwen is trying to ask (correct me if I'm wrong)
> >  is how we can infer which topics are associated with Streams
> >  from the admin client's topic list. I agree that this
> >  doesn't seem possible, since as she pointed out the topics
> >  list (or even description) lacks the specific information we
> >  need.
> > 
> >  What we could do instead is use the admin client's
> >  `describeConsumerGroups` API to get the information on the
> >  Streams app's consumer group specifically -- note that the
> >  Streams application.id config is also used as the consumer
> >  group id, so each app forms a group to read from the input
> >  topics. We could compile a list of these topics just by
> >  looking at each member's assignment (and even check for a
> >  StreamsPartitionAssignor to verify that this is indeed a
> >  Streams app group, if we're being paranoid).
> > 
> >  The reset tool actually already gets the consumer group
> >  description, in order to validate there are no active
> >  consumers in the group. We may as well grab the list of
> >  topics from it while it's there. Or did you have something
> >  else in mind?
> > 
> >  On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee
> >   wrote:
> > 
> > > Thank you
> > >
> > > I understand you
> > >
> > > 1. admin client has topic list 2. applicationId can only
> > > have one stream, so It won't be a problem! 3. For example,
> > > --input-topic [reg] Allowing reg solves some inconvenience
> > >
> > >
> > > On 2020/01/18 18:15:23, Gwen Shapira 
> > > wrote:
> > >> I am not sure I follow. Afaik:
> > >>
> > >> 1. Topics don't include client ID information 2. Even if
> > >> you did, the same ID could be used for topics that are
> > >> not
> > > Kafka
> > >> Streams input
> > >>
> > >> The r

Re: KIP-560 Discuss

[Sang wn Lee]

I'm sorry.
I just modified the KIP!

On 2020/03/07 20:09:47, "Matthias J. Sax"  wrote: 
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> Thanks for the KIP Sang!
> 
> I have a couple of more comments about the wiki page:
> 
> (1) The "Public Interface" section should only list the new stuff. This
> KIP does not change anything with regard to the existing options
> `--input-topic` or `--intermediate-topic` and thus it's just "noise" to
> have them in this section. Only list the new option
> `allInputTopicsOption`.
> 
> (2) Don't post code, ie, the implementation of private methods. KIPs
> should only describe public interface changes.
> 
> (3) The KIP should describe that we intend to use
> `describeConsumerGroups` calls to discover the topic names -- atm, it's
> unclear from the KIP how the new feature actually works.
> 
> (4) If the new flag is used, we will discover input and intermediate
> topics. Hence, the name is miss leading. We could call it
> `--all-user-topics` and explain in the description that "user topics"
> are input and intermediate topics for this case (in general, also output
> topics are "user topics" but there is nothing to be done for output
> topics). Thoughts?
> 
> 
> - -Matthias
> 
> 
> On 1/27/20 6:35 AM, Sang wn Lee wrote:
> > thank you John Roesle
> >
> > It is a good idea "—all-input-topics"
> >
> > I agree with you
> >
> > I'll update right away
> >
> >
> > On 2020/01/24 14:14:17, "John Roesler" 
> > wrote:
> >
> >> Hi all, thanks for the explanation. I was also not sure how the
> >> kip would be possible to implement.
> >>
> >> No that it does seem plausible, my only feedback is that the
> >> command line option could align better with the existing one.
> >> That is, the existing option is called “—input-topics”, so it
> >> seems like the new one should be called “—all-input-topics”.
> >>
> >> Thanks, John
> >>
> >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> >>> Thanks Sophie for the explanation! I read Sang's PR and
> >>> basically he did exactly what you proposed (check it here
> >>>  in case I'm
> >>> wrong).
> >>>
> >>> I think Sophie's response answers Gwen's question already,
> >>> while in the meantime for a KIP itself we are not required to
> >>> mention all the internal details about how to make the changes
> >>> happen (like how to actually get the external topics),
> >>> considering the change scope is pretty small as well. But
> >>> again, it would do no harm if we mention it inside Proposed
> >>> Change session specifically so that people won't get confused
> >>> about how.
> >>>
> >>>
> >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman
> >>>  wrote:
> >>>
>  Hi all,
> 
>  I think what Gwen is trying to ask (correct me if I'm wrong)
>  is how we can infer which topics are associated with Streams
>  from the admin client's topic list. I agree that this
>  doesn't seem possible, since as she pointed out the topics
>  list (or even description) lacks the specific information we
>  need.
> 
>  What we could do instead is use the admin client's
>  `describeConsumerGroups` API to get the information on the
>  Streams app's consumer group specifically -- note that the
>  Streams application.id config is also used as the consumer
>  group id, so each app forms a group to read from the input
>  topics. We could compile a list of these topics just by
>  looking at each member's assignment (and even check for a
>  StreamsPartitionAssignor to verify that this is indeed a
>  Streams app group, if we're being paranoid).
> 
>  The reset tool actually already gets the consumer group
>  description, in order to validate there are no active
>  consumers in the group. We may as well grab the list of
>  topics from it while it's there. Or did you have something
>  else in mind?
> 
>  On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee
>   wrote:
> 
> > Thank you
> >
> > I understand you
> >
> > 1. admin client has topic list 2. applicationId can only
> > have one stream, so It won't be a problem! 3. For example,
> > --input-topic [reg] Allowing reg solves some inconvenience
> >
> >
> > On 2020/01/18 18:15:23, Gwen Shapira 
> > wrote:
> >> I am not sure I follow. Afaik:
> >>
> >> 1. Topics don't include client ID information 2. Even if
> >> you did, the same ID could be used for topics that are
> >> not
> > Kafka
> >> Streams input
> >>
> >> The regex idea sounds doable, but I'm not sure it solves
> >> much?
> >>
> >>
> >> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee
> >> 
>  wrote:
> >>
> >>> Thank you Gwen Shapira! We'll add a flag to clear all
> >>> topics by clientId It is ‘reset-all-external-topics’
> >>>
> >>> I also want to use regex on the input topic flag to
> >>> clear all
> >>>

Re: KIP-560 Discuss

[Sang wn Lee]

I'm sorry.
I just modified the KIP.

On 2020/05/26 20:11:46, Boyang Chen  wrote: 
> Hey Sang, unfortunately we couldn't make it in 2.6. Do you still plan to
> work on this KIP?
> 
> On Thu, May 14, 2020 at 6:49 PM Boyang Chen 
> wrote:
> 
> > Hey Sang, seems this thread has been quiet, are you still working on this
> > KIP?
> >
> > On Sat, Mar 7, 2020 at 3:54 PM Matthias J. Sax 
> > wrote:
> >
> >> Thanks for the KIP Sang!
> >>
> >> I have a couple of more comments about the wiki page:
> >>
> >> (1) The "Public Interface" section should only list the new stuff. This
> >> KIP does not change anything with regard to the existing options
> >> `--input-topic` or `--intermediate-topic` and thus it's just "noise" to
> >> have them in this section. Only list the new option
> >> `allInputTopicsOption`.
> >>
> >> (2) Don't post code, ie, the implementation of private methods. KIPs
> >> should only describe public interface changes.
> >>
> >> (3) The KIP should describe that we intend to use
> >> `describeConsumerGroups` calls to discover the topic names -- atm, it's
> >> unclear from the KIP how the new feature actually works.
> >>
> >> (4) If the new flag is used, we will discover input and intermediate
> >> topics. Hence, the name is miss leading. We could call it
> >> `--all-user-topics` and explain in the description that "user topics"
> >> are input and intermediate topics for this case (in general, also output
> >> topics are "user topics" but there is nothing to be done for output
> >> topics). Thoughts?
> >>
> >>
> >> -Matthias
> >>
> >> On 1/27/20 6:35 AM, Sang wn Lee wrote:
> >> > thank you John Roesle
> >> >
> >> > It is a good idea
> >> > "—all-input-topics"
> >> >
> >> > I agree with you
> >> >
> >> > I'll update right away
> >> >
> >> >
> >> > On 2020/01/24 14:14:17, "John Roesler"  wrote:
> >> >> Hi all, thanks for the explanation. I was also not sure how the kip
> >> would be possible to implement.
> >> >>
> >> >> No that it does seem plausible, my only feedback is that the command
> >> line option could align better with the existing one. That is, the existing
> >> option is called “—input-topics”, so it seems like the new one should be
> >> called “—all-input-topics”.
> >> >>
> >> >> Thanks,
> >> >> John
> >> >>
> >> >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> >> >>> Thanks Sophie for the explanation! I read Sang's PR and basically he
> >> did
> >> >>> exactly what you proposed (check it here
> >> >>>  in case I'm wrong).
> >> >>>
> >> >>> I think Sophie's response answers Gwen's question already, while in
> >> the
> >> >>> meantime for a KIP itself we are not required to mention all the
> >> internal
> >> >>> details about how to make the changes happen (like how to actually
> >> get the
> >> >>> external topics), considering the change scope is pretty small as
> >> well. But
> >> >>> again, it would do no harm if we mention it inside Proposed Change
> >> session
> >> >>> specifically so that people won't get confused about how.
> >> >>>
> >> >>>
> >> >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman <
> >> sop...@confluent.io>
> >> >>> wrote:
> >> >>>
> >>  Hi all,
> >> 
> >>  I think what Gwen is trying to ask (correct me if I'm wrong) is how
> >> we can
> >>  infer which topics are associated with
> >>  Streams from the admin client's topic list. I agree that this
> >> doesn't seem
> >>  possible, since as she pointed out the
> >>  topics list (or even description) lacks the specific information we
> >> need.
> >> 
> >>  What we could do instead is use the admin client's
> >>  `describeConsumerGroups` API to get the information
> >>  on the Streams app's consumer group specifically -- note that the
> >> Streams
> >>  application.id config is also used
> >>  as the consumer group id, so each app forms a group to read from the
> >> input
> >>  topics. We could compile a list
> >>  of these topics just by looking at each member's assignment (and
> >> even check
> >>  for a StreamsPartitionAssignor
> >>  to verify that this is indeed a Streams app group, if we're being
> >>  paranoid).
> >> 
> >>  The reset tool actually already gets the consumer group description,
> >> in
> >>  order to validate there are no active
> >>  consumers in the group. We may as well grab the list of topics from
> >> it
> >>  while it's there. Or did you have something
> >>  else in mind?
> >> 
> >>  On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee 
> >> wrote:
> >> 
> >> > Thank you
> >> >
> >> > I understand you
> >> >
> >> > 1. admin client has topic list
> >> > 2. applicationId can only have one stream, so It won't be a problem!
> >> > 3. For example, --input-topic [reg]
> >> > Allowing reg solves some inconvenience
> >> >
> >> >
> >> > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> >> >> I am not sure 

Re: KIP-560 Discuss

[Boyang Chen]

Hey Sang, unfortunately we couldn't make it in 2.6. Do you still plan to
work on this KIP?

On Thu, May 14, 2020 at 6:49 PM Boyang Chen 
wrote:

> Hey Sang, seems this thread has been quiet, are you still working on this
> KIP?
>
> On Sat, Mar 7, 2020 at 3:54 PM Matthias J. Sax 
> wrote:
>
>> Thanks for the KIP Sang!
>>
>> I have a couple of more comments about the wiki page:
>>
>> (1) The "Public Interface" section should only list the new stuff. This
>> KIP does not change anything with regard to the existing options
>> `--input-topic` or `--intermediate-topic` and thus it's just "noise" to
>> have them in this section. Only list the new option
>> `allInputTopicsOption`.
>>
>> (2) Don't post code, ie, the implementation of private methods. KIPs
>> should only describe public interface changes.
>>
>> (3) The KIP should describe that we intend to use
>> `describeConsumerGroups` calls to discover the topic names -- atm, it's
>> unclear from the KIP how the new feature actually works.
>>
>> (4) If the new flag is used, we will discover input and intermediate
>> topics. Hence, the name is miss leading. We could call it
>> `--all-user-topics` and explain in the description that "user topics"
>> are input and intermediate topics for this case (in general, also output
>> topics are "user topics" but there is nothing to be done for output
>> topics). Thoughts?
>>
>>
>> -Matthias
>>
>> On 1/27/20 6:35 AM, Sang wn Lee wrote:
>> > thank you John Roesle
>> >
>> > It is a good idea
>> > "—all-input-topics"
>> >
>> > I agree with you
>> >
>> > I'll update right away
>> >
>> >
>> > On 2020/01/24 14:14:17, "John Roesler"  wrote:
>> >> Hi all, thanks for the explanation. I was also not sure how the kip
>> would be possible to implement.
>> >>
>> >> No that it does seem plausible, my only feedback is that the command
>> line option could align better with the existing one. That is, the existing
>> option is called “—input-topics”, so it seems like the new one should be
>> called “—all-input-topics”.
>> >>
>> >> Thanks,
>> >> John
>> >>
>> >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
>> >>> Thanks Sophie for the explanation! I read Sang's PR and basically he
>> did
>> >>> exactly what you proposed (check it here
>> >>>  in case I'm wrong).
>> >>>
>> >>> I think Sophie's response answers Gwen's question already, while in
>> the
>> >>> meantime for a KIP itself we are not required to mention all the
>> internal
>> >>> details about how to make the changes happen (like how to actually
>> get the
>> >>> external topics), considering the change scope is pretty small as
>> well. But
>> >>> again, it would do no harm if we mention it inside Proposed Change
>> session
>> >>> specifically so that people won't get confused about how.
>> >>>
>> >>>
>> >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman <
>> sop...@confluent.io>
>> >>> wrote:
>> >>>
>>  Hi all,
>> 
>>  I think what Gwen is trying to ask (correct me if I'm wrong) is how
>> we can
>>  infer which topics are associated with
>>  Streams from the admin client's topic list. I agree that this
>> doesn't seem
>>  possible, since as she pointed out the
>>  topics list (or even description) lacks the specific information we
>> need.
>> 
>>  What we could do instead is use the admin client's
>>  `describeConsumerGroups` API to get the information
>>  on the Streams app's consumer group specifically -- note that the
>> Streams
>>  application.id config is also used
>>  as the consumer group id, so each app forms a group to read from the
>> input
>>  topics. We could compile a list
>>  of these topics just by looking at each member's assignment (and
>> even check
>>  for a StreamsPartitionAssignor
>>  to verify that this is indeed a Streams app group, if we're being
>>  paranoid).
>> 
>>  The reset tool actually already gets the consumer group description,
>> in
>>  order to validate there are no active
>>  consumers in the group. We may as well grab the list of topics from
>> it
>>  while it's there. Or did you have something
>>  else in mind?
>> 
>>  On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee 
>> wrote:
>> 
>> > Thank you
>> >
>> > I understand you
>> >
>> > 1. admin client has topic list
>> > 2. applicationId can only have one stream, so It won't be a problem!
>> > 3. For example, --input-topic [reg]
>> > Allowing reg solves some inconvenience
>> >
>> >
>> > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
>> >> I am not sure I follow. Afaik:
>> >>
>> >> 1. Topics don't include client ID information
>> >> 2. Even if you did, the same ID could be used for topics that are
>> not
>> > Kafka
>> >> Streams input
>> >>
>> >> The regex idea sounds doable, but I'm not sure it solves much?
>> >>
>> >>
>> >> On Sat, Jan 18, 2020, 7:12 AM 

Re: KIP-560 Discuss

[Boyang Chen]

Hey Sang, seems this thread has been quiet, are you still working on this
KIP?

On Sat, Mar 7, 2020 at 3:54 PM Matthias J. Sax 
wrote:

> Thanks for the KIP Sang!
>
> I have a couple of more comments about the wiki page:
>
> (1) The "Public Interface" section should only list the new stuff. This
> KIP does not change anything with regard to the existing options
> `--input-topic` or `--intermediate-topic` and thus it's just "noise" to
> have them in this section. Only list the new option `allInputTopicsOption`.
>
> (2) Don't post code, ie, the implementation of private methods. KIPs
> should only describe public interface changes.
>
> (3) The KIP should describe that we intend to use
> `describeConsumerGroups` calls to discover the topic names -- atm, it's
> unclear from the KIP how the new feature actually works.
>
> (4) If the new flag is used, we will discover input and intermediate
> topics. Hence, the name is miss leading. We could call it
> `--all-user-topics` and explain in the description that "user topics"
> are input and intermediate topics for this case (in general, also output
> topics are "user topics" but there is nothing to be done for output
> topics). Thoughts?
>
>
> -Matthias
>
> On 1/27/20 6:35 AM, Sang wn Lee wrote:
> > thank you John Roesle
> >
> > It is a good idea
> > "—all-input-topics"
> >
> > I agree with you
> >
> > I'll update right away
> >
> >
> > On 2020/01/24 14:14:17, "John Roesler"  wrote:
> >> Hi all, thanks for the explanation. I was also not sure how the kip
> would be possible to implement.
> >>
> >> No that it does seem plausible, my only feedback is that the command
> line option could align better with the existing one. That is, the existing
> option is called “—input-topics”, so it seems like the new one should be
> called “—all-input-topics”.
> >>
> >> Thanks,
> >> John
> >>
> >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> >>> Thanks Sophie for the explanation! I read Sang's PR and basically he
> did
> >>> exactly what you proposed (check it here
> >>>  in case I'm wrong).
> >>>
> >>> I think Sophie's response answers Gwen's question already, while in the
> >>> meantime for a KIP itself we are not required to mention all the
> internal
> >>> details about how to make the changes happen (like how to actually get
> the
> >>> external topics), considering the change scope is pretty small as
> well. But
> >>> again, it would do no harm if we mention it inside Proposed Change
> session
> >>> specifically so that people won't get confused about how.
> >>>
> >>>
> >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman <
> sop...@confluent.io>
> >>> wrote:
> >>>
>  Hi all,
> 
>  I think what Gwen is trying to ask (correct me if I'm wrong) is how
> we can
>  infer which topics are associated with
>  Streams from the admin client's topic list. I agree that this doesn't
> seem
>  possible, since as she pointed out the
>  topics list (or even description) lacks the specific information we
> need.
> 
>  What we could do instead is use the admin client's
>  `describeConsumerGroups` API to get the information
>  on the Streams app's consumer group specifically -- note that the
> Streams
>  application.id config is also used
>  as the consumer group id, so each app forms a group to read from the
> input
>  topics. We could compile a list
>  of these topics just by looking at each member's assignment (and even
> check
>  for a StreamsPartitionAssignor
>  to verify that this is indeed a Streams app group, if we're being
>  paranoid).
> 
>  The reset tool actually already gets the consumer group description,
> in
>  order to validate there are no active
>  consumers in the group. We may as well grab the list of topics from it
>  while it's there. Or did you have something
>  else in mind?
> 
>  On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee 
> wrote:
> 
> > Thank you
> >
> > I understand you
> >
> > 1. admin client has topic list
> > 2. applicationId can only have one stream, so It won't be a problem!
> > 3. For example, --input-topic [reg]
> > Allowing reg solves some inconvenience
> >
> >
> > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> >> I am not sure I follow. Afaik:
> >>
> >> 1. Topics don't include client ID information
> >> 2. Even if you did, the same ID could be used for topics that are
> not
> > Kafka
> >> Streams input
> >>
> >> The regex idea sounds doable, but I'm not sure it solves much?
> >>
> >>
> >> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee 
>  wrote:
> >>
> >>> Thank you
> >>> Gwen Shapira!
> >>> We'll add a flag to clear all topics by clientId
> >>> It is ‘reset-all-external-topics’
> >>>
> >>> I also want to use regex on the input topic flag to clear all
>  matching
> >>> to

Re: KIP-560 Discuss

[Matthias J. Sax]

Thanks for the KIP Sang!

I have a couple of more comments about the wiki page:

(1) The "Public Interface" section should only list the new stuff. This
KIP does not change anything with regard to the existing options
`--input-topic` or `--intermediate-topic` and thus it's just "noise" to
have them in this section. Only list the new option `allInputTopicsOption`.

(2) Don't post code, ie, the implementation of private methods. KIPs
should only describe public interface changes.

(3) The KIP should describe that we intend to use
`describeConsumerGroups` calls to discover the topic names -- atm, it's
unclear from the KIP how the new feature actually works.

(4) If the new flag is used, we will discover input and intermediate
topics. Hence, the name is miss leading. We could call it
`--all-user-topics` and explain in the description that "user topics"
are input and intermediate topics for this case (in general, also output
topics are "user topics" but there is nothing to be done for output
topics). Thoughts?


-Matthias

On 1/27/20 6:35 AM, Sang wn Lee wrote:
> thank you John Roesle
> 
> It is a good idea
> "—all-input-topics"
> 
> I agree with you
> 
> I'll update right away
> 
> 
> On 2020/01/24 14:14:17, "John Roesler"  wrote: 
>> Hi all, thanks for the explanation. I was also not sure how the kip would be 
>> possible to implement. 
>>
>> No that it does seem plausible, my only feedback is that the command line 
>> option could align better with the existing one. That is, the existing 
>> option is called “—input-topics”, so it seems like the new one should be 
>> called “—all-input-topics”. 
>>
>> Thanks,
>> John
>>
>> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
>>> Thanks Sophie for the explanation! I read Sang's PR and basically he did
>>> exactly what you proposed (check it here
>>>  in case I'm wrong).
>>>
>>> I think Sophie's response answers Gwen's question already, while in the
>>> meantime for a KIP itself we are not required to mention all the internal
>>> details about how to make the changes happen (like how to actually get the
>>> external topics), considering the change scope is pretty small as well. But
>>> again, it would do no harm if we mention it inside Proposed Change session
>>> specifically so that people won't get confused about how.
>>>
>>>
>>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman 
>>> wrote:
>>>
 Hi all,

 I think what Gwen is trying to ask (correct me if I'm wrong) is how we can
 infer which topics are associated with
 Streams from the admin client's topic list. I agree that this doesn't seem
 possible, since as she pointed out the
 topics list (or even description) lacks the specific information we need.

 What we could do instead is use the admin client's
 `describeConsumerGroups` API to get the information
 on the Streams app's consumer group specifically -- note that the Streams
 application.id config is also used
 as the consumer group id, so each app forms a group to read from the input
 topics. We could compile a list
 of these topics just by looking at each member's assignment (and even check
 for a StreamsPartitionAssignor
 to verify that this is indeed a Streams app group, if we're being
 paranoid).

 The reset tool actually already gets the consumer group description, in
 order to validate there are no active
 consumers in the group. We may as well grab the list of topics from it
 while it's there. Or did you have something
 else in mind?

 On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee  wrote:

> Thank you
>
> I understand you
>
> 1. admin client has topic list
> 2. applicationId can only have one stream, so It won't be a problem!
> 3. For example, --input-topic [reg]
> Allowing reg solves some inconvenience
>
>
> On 2020/01/18 18:15:23, Gwen Shapira  wrote:
>> I am not sure I follow. Afaik:
>>
>> 1. Topics don't include client ID information
>> 2. Even if you did, the same ID could be used for topics that are not
> Kafka
>> Streams input
>>
>> The regex idea sounds doable, but I'm not sure it solves much?
>>
>>
>> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee 
 wrote:
>>
>>> Thank you
>>> Gwen Shapira!
>>> We'll add a flag to clear all topics by clientId
>>> It is ‘reset-all-external-topics’
>>>
>>> I also want to use regex on the input topic flag to clear all
 matching
>>> topics.
>>>
>>> On 2020/01/17 19:29:09, Gwen Shapira  wrote:
 Seem like a very nice improvement to me. But I have to admit that I
 don't understand how this will how - how could you infer the input
 topics?

 On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee 
>>> wrote:
>
> Hello,
>
> Starting this thread to discuss KIP-5

Re: KIP-560 Discuss

[Matthias J. Sax]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Thanks for the KIP Sang!

I have a couple of more comments about the wiki page:

(1) The "Public Interface" section should only list the new stuff. This
KIP does not change anything with regard to the existing options
`--input-topic` or `--intermediate-topic` and thus it's just "noise" to
have them in this section. Only list the new option
`allInputTopicsOption`.

(2) Don't post code, ie, the implementation of private methods. KIPs
should only describe public interface changes.

(3) The KIP should describe that we intend to use
`describeConsumerGroups` calls to discover the topic names -- atm, it's
unclear from the KIP how the new feature actually works.

(4) If the new flag is used, we will discover input and intermediate
topics. Hence, the name is miss leading. We could call it
`--all-user-topics` and explain in the description that "user topics"
are input and intermediate topics for this case (in general, also output
topics are "user topics" but there is nothing to be done for output
topics). Thoughts?


- -Matthias


On 1/27/20 6:35 AM, Sang wn Lee wrote:
> thank you John Roesle
>
> It is a good idea "—all-input-topics"
>
> I agree with you
>
> I'll update right away
>
>
> On 2020/01/24 14:14:17, "John Roesler" 
> wrote:
>
>> Hi all, thanks for the explanation. I was also not sure how the
>> kip would be possible to implement.
>>
>> No that it does seem plausible, my only feedback is that the
>> command line option could align better with the existing one.
>> That is, the existing option is called “—input-topics”, so it
>> seems like the new one should be called “—all-input-topics”.
>>
>> Thanks, John
>>
>> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
>>> Thanks Sophie for the explanation! I read Sang's PR and
>>> basically he did exactly what you proposed (check it here
>>>  in case I'm
>>> wrong).
>>>
>>> I think Sophie's response answers Gwen's question already,
>>> while in the meantime for a KIP itself we are not required to
>>> mention all the internal details about how to make the changes
>>> happen (like how to actually get the external topics),
>>> considering the change scope is pretty small as well. But
>>> again, it would do no harm if we mention it inside Proposed
>>> Change session specifically so that people won't get confused
>>> about how.
>>>
>>>
>>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman
>>>  wrote:
>>>
 Hi all,

 I think what Gwen is trying to ask (correct me if I'm wrong)
 is how we can infer which topics are associated with Streams
 from the admin client's topic list. I agree that this
 doesn't seem possible, since as she pointed out the topics
 list (or even description) lacks the specific information we
 need.

 What we could do instead is use the admin client's
 `describeConsumerGroups` API to get the information on the
 Streams app's consumer group specifically -- note that the
 Streams application.id config is also used as the consumer
 group id, so each app forms a group to read from the input
 topics. We could compile a list of these topics just by
 looking at each member's assignment (and even check for a
 StreamsPartitionAssignor to verify that this is indeed a
 Streams app group, if we're being paranoid).

 The reset tool actually already gets the consumer group
 description, in order to validate there are no active
 consumers in the group. We may as well grab the list of
 topics from it while it's there. Or did you have something
 else in mind?

 On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee
  wrote:

> Thank you
>
> I understand you
>
> 1. admin client has topic list 2. applicationId can only
> have one stream, so It won't be a problem! 3. For example,
> --input-topic [reg] Allowing reg solves some inconvenience
>
>
> On 2020/01/18 18:15:23, Gwen Shapira 
> wrote:
>> I am not sure I follow. Afaik:
>>
>> 1. Topics don't include client ID information 2. Even if
>> you did, the same ID could be used for topics that are
>> not
> Kafka
>> Streams input
>>
>> The regex idea sounds doable, but I'm not sure it solves
>> much?
>>
>>
>> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee
>> 
 wrote:
>>
>>> Thank you Gwen Shapira! We'll add a flag to clear all
>>> topics by clientId It is ‘reset-all-external-topics’
>>>
>>> I also want to use regex on the input topic flag to
>>> clear all
 matching
>>> topics.
>>>
>>> On 2020/01/17 19:29:09, Gwen Shapira
>>>  wrote:
 Seem like a very nice improvement to me. But I have
 to admit that I don't understand how this will how -
 how could you infer the input topics?

 On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee
 
>>> wrote

Re: KIP-560 Discuss

[Sang wn Lee]

thank you John Roesle

It is a good idea
"—all-input-topics"

I agree with you

I'll update right away


On 2020/01/24 14:14:17, "John Roesler"  wrote: 
> Hi all, thanks for the explanation. I was also not sure how the kip would be 
> possible to implement. 
> 
> No that it does seem plausible, my only feedback is that the command line 
> option could align better with the existing one. That is, the existing option 
> is called “—input-topics”, so it seems like the new one should be called 
> “—all-input-topics”. 
> 
> Thanks,
> John
> 
> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> > Thanks Sophie for the explanation! I read Sang's PR and basically he did
> > exactly what you proposed (check it here
> >  in case I'm wrong).
> > 
> > I think Sophie's response answers Gwen's question already, while in the
> > meantime for a KIP itself we are not required to mention all the internal
> > details about how to make the changes happen (like how to actually get the
> > external topics), considering the change scope is pretty small as well. But
> > again, it would do no harm if we mention it inside Proposed Change session
> > specifically so that people won't get confused about how.
> > 
> > 
> > On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman 
> > wrote:
> > 
> > > Hi all,
> > >
> > > I think what Gwen is trying to ask (correct me if I'm wrong) is how we can
> > > infer which topics are associated with
> > > Streams from the admin client's topic list. I agree that this doesn't seem
> > > possible, since as she pointed out the
> > > topics list (or even description) lacks the specific information we need.
> > >
> > > What we could do instead is use the admin client's
> > > `describeConsumerGroups` API to get the information
> > > on the Streams app's consumer group specifically -- note that the Streams
> > > application.id config is also used
> > > as the consumer group id, so each app forms a group to read from the input
> > > topics. We could compile a list
> > > of these topics just by looking at each member's assignment (and even 
> > > check
> > > for a StreamsPartitionAssignor
> > > to verify that this is indeed a Streams app group, if we're being
> > > paranoid).
> > >
> > > The reset tool actually already gets the consumer group description, in
> > > order to validate there are no active
> > > consumers in the group. We may as well grab the list of topics from it
> > > while it's there. Or did you have something
> > > else in mind?
> > >
> > > On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee  wrote:
> > >
> > > > Thank you
> > > >
> > > > I understand you
> > > >
> > > > 1. admin client has topic list
> > > > 2. applicationId can only have one stream, so It won't be a problem!
> > > > 3. For example, --input-topic [reg]
> > > > Allowing reg solves some inconvenience
> > > >
> > > >
> > > > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> > > > > I am not sure I follow. Afaik:
> > > > >
> > > > > 1. Topics don't include client ID information
> > > > > 2. Even if you did, the same ID could be used for topics that are not
> > > > Kafka
> > > > > Streams input
> > > > >
> > > > > The regex idea sounds doable, but I'm not sure it solves much?
> > > > >
> > > > >
> > > > > On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee 
> > > wrote:
> > > > >
> > > > > > Thank you
> > > > > > Gwen Shapira!
> > > > > > We'll add a flag to clear all topics by clientId
> > > > > > It is ‘reset-all-external-topics’
> > > > > >
> > > > > > I also want to use regex on the input topic flag to clear all
> > > matching
> > > > > > topics.
> > > > > >
> > > > > > On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > > > > > > Seem like a very nice improvement to me. But I have to admit that 
> > > > > > > I
> > > > > > > don't understand how this will how - how could you infer the input
> > > > > > > topics?
> > > > > > >
> > > > > > > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee  > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Starting this thread to discuss KIP-560:
> > > > > > > > wiki link :
> > > > > > > >
> > > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > > > > > > >
> > > > > > > > I'm newbie
> > > > > > > > I would like to receive feedback on the following features!
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 

Re: KIP-560 Discuss

[John Roesler]

Hi all, thanks for the explanation. I was also not sure how the kip would be 
possible to implement. 

No that it does seem plausible, my only feedback is that the command line 
option could align better with the existing one. That is, the existing option 
is called “—input-topics”, so it seems like the new one should be called 
“—all-input-topics”. 

Thanks,
John

On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote:
> Thanks Sophie for the explanation! I read Sang's PR and basically he did
> exactly what you proposed (check it here
>  in case I'm wrong).
> 
> I think Sophie's response answers Gwen's question already, while in the
> meantime for a KIP itself we are not required to mention all the internal
> details about how to make the changes happen (like how to actually get the
> external topics), considering the change scope is pretty small as well. But
> again, it would do no harm if we mention it inside Proposed Change session
> specifically so that people won't get confused about how.
> 
> 
> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman 
> wrote:
> 
> > Hi all,
> >
> > I think what Gwen is trying to ask (correct me if I'm wrong) is how we can
> > infer which topics are associated with
> > Streams from the admin client's topic list. I agree that this doesn't seem
> > possible, since as she pointed out the
> > topics list (or even description) lacks the specific information we need.
> >
> > What we could do instead is use the admin client's
> > `describeConsumerGroups` API to get the information
> > on the Streams app's consumer group specifically -- note that the Streams
> > application.id config is also used
> > as the consumer group id, so each app forms a group to read from the input
> > topics. We could compile a list
> > of these topics just by looking at each member's assignment (and even check
> > for a StreamsPartitionAssignor
> > to verify that this is indeed a Streams app group, if we're being
> > paranoid).
> >
> > The reset tool actually already gets the consumer group description, in
> > order to validate there are no active
> > consumers in the group. We may as well grab the list of topics from it
> > while it's there. Or did you have something
> > else in mind?
> >
> > On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee  wrote:
> >
> > > Thank you
> > >
> > > I understand you
> > >
> > > 1. admin client has topic list
> > > 2. applicationId can only have one stream, so It won't be a problem!
> > > 3. For example, --input-topic [reg]
> > > Allowing reg solves some inconvenience
> > >
> > >
> > > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> > > > I am not sure I follow. Afaik:
> > > >
> > > > 1. Topics don't include client ID information
> > > > 2. Even if you did, the same ID could be used for topics that are not
> > > Kafka
> > > > Streams input
> > > >
> > > > The regex idea sounds doable, but I'm not sure it solves much?
> > > >
> > > >
> > > > On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee 
> > wrote:
> > > >
> > > > > Thank you
> > > > > Gwen Shapira!
> > > > > We'll add a flag to clear all topics by clientId
> > > > > It is ‘reset-all-external-topics’
> > > > >
> > > > > I also want to use regex on the input topic flag to clear all
> > matching
> > > > > topics.
> > > > >
> > > > > On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > > > > > Seem like a very nice improvement to me. But I have to admit that I
> > > > > > don't understand how this will how - how could you infer the input
> > > > > > topics?
> > > > > >
> > > > > > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee  > >
> > > > > wrote:
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > Starting this thread to discuss KIP-560:
> > > > > > > wiki link :
> > > > > > >
> > > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > > > > > >
> > > > > > > I'm newbie
> > > > > > > I would like to receive feedback on the following features!
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: KIP-560 Discuss

[Boyang Chen]

Thanks Sophie for the explanation! I read Sang's PR and basically he did
exactly what you proposed (check it here
 in case I'm wrong).

I think Sophie's response answers Gwen's question already, while in the
meantime for a KIP itself we are not required to mention all the internal
details about how to make the changes happen (like how to actually get the
external topics), considering the change scope is pretty small as well. But
again, it would do no harm if we mention it inside Proposed Change session
specifically so that people won't get confused about how.


On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman 
wrote:

> Hi all,
>
> I think what Gwen is trying to ask (correct me if I'm wrong) is how we can
> infer which topics are associated with
> Streams from the admin client's topic list. I agree that this doesn't seem
> possible, since as she pointed out the
> topics list (or even description) lacks the specific information we need.
>
> What we could do instead is use the admin client's
> `describeConsumerGroups` API to get the information
> on the Streams app's consumer group specifically -- note that the Streams
> application.id config is also used
> as the consumer group id, so each app forms a group to read from the input
> topics. We could compile a list
> of these topics just by looking at each member's assignment (and even check
> for a StreamsPartitionAssignor
> to verify that this is indeed a Streams app group, if we're being
> paranoid).
>
> The reset tool actually already gets the consumer group description, in
> order to validate there are no active
> consumers in the group. We may as well grab the list of topics from it
> while it's there. Or did you have something
> else in mind?
>
> On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee  wrote:
>
> > Thank you
> >
> > I understand you
> >
> > 1. admin client has topic list
> > 2. applicationId can only have one stream, so It won't be a problem!
> > 3. For example, --input-topic [reg]
> > Allowing reg solves some inconvenience
> >
> >
> > On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> > > I am not sure I follow. Afaik:
> > >
> > > 1. Topics don't include client ID information
> > > 2. Even if you did, the same ID could be used for topics that are not
> > Kafka
> > > Streams input
> > >
> > > The regex idea sounds doable, but I'm not sure it solves much?
> > >
> > >
> > > On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee 
> wrote:
> > >
> > > > Thank you
> > > > Gwen Shapira!
> > > > We'll add a flag to clear all topics by clientId
> > > > It is ‘reset-all-external-topics’
> > > >
> > > > I also want to use regex on the input topic flag to clear all
> matching
> > > > topics.
> > > >
> > > > On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > > > > Seem like a very nice improvement to me. But I have to admit that I
> > > > > don't understand how this will how - how could you infer the input
> > > > > topics?
> > > > >
> > > > > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee  >
> > > > wrote:
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > Starting this thread to discuss KIP-560:
> > > > > > wiki link :
> > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > > > > >
> > > > > > I'm newbie
> > > > > > I would like to receive feedback on the following features!
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: KIP-560 Discuss

[Sophie Blee-Goldman]

Hi all,

I think what Gwen is trying to ask (correct me if I'm wrong) is how we can
infer which topics are associated with
Streams from the admin client's topic list. I agree that this doesn't seem
possible, since as she pointed out the
topics list (or even description) lacks the specific information we need.

What we could do instead is use the admin client's
`describeConsumerGroups` API to get the information
on the Streams app's consumer group specifically -- note that the Streams
application.id config is also used
as the consumer group id, so each app forms a group to read from the input
topics. We could compile a list
of these topics just by looking at each member's assignment (and even check
for a StreamsPartitionAssignor
to verify that this is indeed a Streams app group, if we're being paranoid).

The reset tool actually already gets the consumer group description, in
order to validate there are no active
consumers in the group. We may as well grab the list of topics from it
while it's there. Or did you have something
else in mind?

On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee  wrote:

> Thank you
>
> I understand you
>
> 1. admin client has topic list
> 2. applicationId can only have one stream, so It won't be a problem!
> 3. For example, --input-topic [reg]
> Allowing reg solves some inconvenience
>
>
> On 2020/01/18 18:15:23, Gwen Shapira  wrote:
> > I am not sure I follow. Afaik:
> >
> > 1. Topics don't include client ID information
> > 2. Even if you did, the same ID could be used for topics that are not
> Kafka
> > Streams input
> >
> > The regex idea sounds doable, but I'm not sure it solves much?
> >
> >
> > On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee  wrote:
> >
> > > Thank you
> > > Gwen Shapira!
> > > We'll add a flag to clear all topics by clientId
> > > It is ‘reset-all-external-topics’
> > >
> > > I also want to use regex on the input topic flag to clear all matching
> > > topics.
> > >
> > > On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > > > Seem like a very nice improvement to me. But I have to admit that I
> > > > don't understand how this will how - how could you infer the input
> > > > topics?
> > > >
> > > > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee 
> > > wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > Starting this thread to discuss KIP-560:
> > > > > wiki link :
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > > > >
> > > > > I'm newbie
> > > > > I would like to receive feedback on the following features!
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: KIP-560 Discuss

[Sang wn Lee]

Thank you

I understand you

1. admin client has topic list
2. applicationId can only have one stream, so It won't be a problem!
3. For example, --input-topic [reg] 
Allowing reg solves some inconvenience


On 2020/01/18 18:15:23, Gwen Shapira  wrote: 
> I am not sure I follow. Afaik:
> 
> 1. Topics don't include client ID information
> 2. Even if you did, the same ID could be used for topics that are not Kafka
> Streams input
> 
> The regex idea sounds doable, but I'm not sure it solves much?
> 
> 
> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee  wrote:
> 
> > Thank you
> > Gwen Shapira!
> > We'll add a flag to clear all topics by clientId
> > It is ‘reset-all-external-topics’
> >
> > I also want to use regex on the input topic flag to clear all matching
> > topics.
> >
> > On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > > Seem like a very nice improvement to me. But I have to admit that I
> > > don't understand how this will how - how could you infer the input
> > > topics?
> > >
> > > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee 
> > wrote:
> > > >
> > > > Hello,
> > > >
> > > > Starting this thread to discuss KIP-560:
> > > > wiki link :
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > > >
> > > > I'm newbie
> > > > I would like to receive feedback on the following features!
> > > >
> > > >
> > >
> >
> 

Re: KIP-560 Discuss

[Gwen Shapira]

I am not sure I follow. Afaik:

1. Topics don't include client ID information
2. Even if you did, the same ID could be used for topics that are not Kafka
Streams input

The regex idea sounds doable, but I'm not sure it solves much?


On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee  wrote:

> Thank you
> Gwen Shapira!
> We'll add a flag to clear all topics by clientId
> It is ‘reset-all-external-topics’
>
> I also want to use regex on the input topic flag to clear all matching
> topics.
>
> On 2020/01/17 19:29:09, Gwen Shapira  wrote:
> > Seem like a very nice improvement to me. But I have to admit that I
> > don't understand how this will how - how could you infer the input
> > topics?
> >
> > On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee 
> wrote:
> > >
> > > Hello,
> > >
> > > Starting this thread to discuss KIP-560:
> > > wiki link :
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> > >
> > > I'm newbie
> > > I would like to receive feedback on the following features!
> > >
> > >
> >
>

Re: KIP-560 Discuss

[Sang wn Lee]

Thank you
Gwen Shapira!
We'll add a flag to clear all topics by clientId
It is ‘reset-all-external-topics’

I also want to use regex on the input topic flag to clear all matching topics.

On 2020/01/17 19:29:09, Gwen Shapira  wrote: 
> Seem like a very nice improvement to me. But I have to admit that I
> don't understand how this will how - how could you infer the input
> topics?
> 
> On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee  wrote:
> >
> > Hello,
> >
> > Starting this thread to discuss KIP-560:
> > wiki link :
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
> >
> > I'm newbie
> > I would like to receive feedback on the following features!
> >
> >
> 

Re: KIP-560 Discuss

[Gwen Shapira]

Seem like a very nice improvement to me. But I have to admit that I
don't understand how this will how - how could you infer the input
topics?

On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee  wrote:
>
> Hello,
>
> Starting this thread to discuss KIP-560:
> wiki link :
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool
>
> I'm newbie
> I would like to receive feedback on the following features!
>
>

KIP-560 Discuss

[Sang wn Lee]

Hello,

Starting this thread to discuss KIP-560:
wiki link :
https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+infer+external+topic+partitions+in+stream+reset+tool

I'm newbie
I would like to receive feedback on the following features!