I'm sorry. I just modified the KIP!
On 2020/03/07 20:09:47, "Matthias J. Sax" <mj...@apache.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Thanks for the KIP Sang! > > I have a couple of more comments about the wiki page: > > (1) The "Public Interface" section should only list the new stuff. This > KIP does not change anything with regard to the existing options > `--input-topic` or `--intermediate-topic` and thus it's just "noise" to > have them in this section. Only list the new option > `allInputTopicsOption`. > > (2) Don't post code, ie, the implementation of private methods. KIPs > should only describe public interface changes. > > (3) The KIP should describe that we intend to use > `describeConsumerGroups` calls to discover the topic names -- atm, it's > unclear from the KIP how the new feature actually works. > > (4) If the new flag is used, we will discover input and intermediate > topics. Hence, the name is miss leading. We could call it > `--all-user-topics` and explain in the description that "user topics" > are input and intermediate topics for this case (in general, also output > topics are "user topics" but there is nothing to be done for output > topics). Thoughts? > > > - -Matthias > > > On 1/27/20 6:35 AM, Sang wn Lee wrote: > > thank you John Roesle > > > > It is a good idea "—all-input-topics" > > > > I agree with you > > > > I'll update right away > > > > > > On 2020/01/24 14:14:17, "John Roesler" <vvcep...@apache.org> > > wrote: > > > >> Hi all, thanks for the explanation. I was also not sure how the > >> kip would be possible to implement. > >> > >> No that it does seem plausible, my only feedback is that the > >> command line option could align better with the existing one. > >> That is, the existing option is called “—input-topics”, so it > >> seems like the new one should be called “—all-input-topics”. > >> > >> Thanks, John > >> > >> On Fri, Jan 24, 2020, at 01:42, Boyang Chen wrote: > >>> Thanks Sophie for the explanation! I read Sang's PR and > >>> basically he did exactly what you proposed (check it here > >>> <https://github.com/apache/kafka/pull/7948/files> in case I'm > >>> wrong). > >>> > >>> I think Sophie's response answers Gwen's question already, > >>> while in the meantime for a KIP itself we are not required to > >>> mention all the internal details about how to make the changes > >>> happen (like how to actually get the external topics), > >>> considering the change scope is pretty small as well. But > >>> again, it would do no harm if we mention it inside Proposed > >>> Change session specifically so that people won't get confused > >>> about how. > >>> > >>> > >>> On Thu, Jan 23, 2020 at 8:26 PM Sophie Blee-Goldman > >>> <sop...@confluent.io> wrote: > >>> > >>>> Hi all, > >>>> > >>>> I think what Gwen is trying to ask (correct me if I'm wrong) > >>>> is how we can infer which topics are associated with Streams > >>>> from the admin client's topic list. I agree that this > >>>> doesn't seem possible, since as she pointed out the topics > >>>> list (or even description) lacks the specific information we > >>>> need. > >>>> > >>>> What we could do instead is use the admin client's > >>>> `describeConsumerGroups` API to get the information on the > >>>> Streams app's consumer group specifically -- note that the > >>>> Streams application.id config is also used as the consumer > >>>> group id, so each app forms a group to read from the input > >>>> topics. We could compile a list of these topics just by > >>>> looking at each member's assignment (and even check for a > >>>> StreamsPartitionAssignor to verify that this is indeed a > >>>> Streams app group, if we're being paranoid). > >>>> > >>>> The reset tool actually already gets the consumer group > >>>> description, in order to validate there are no active > >>>> consumers in the group. We may as well grab the list of > >>>> topics from it while it's there. Or did you have something > >>>> else in mind? > >>>> > >>>> On Sat, Jan 18, 2020 at 6:17 PM Sang wn Lee > >>>> <ssangdd...@gmail.com> wrote: > >>>> > >>>>> Thank you > >>>>> > >>>>> I understand you > >>>>> > >>>>> 1. admin client has topic list 2. applicationId can only > >>>>> have one stream, so It won't be a problem! 3. For example, > >>>>> --input-topic [reg] Allowing reg solves some inconvenience > >>>>> > >>>>> > >>>>> On 2020/01/18 18:15:23, Gwen Shapira <g...@confluent.io> > >>>>> wrote: > >>>>>> I am not sure I follow. Afaik: > >>>>>> > >>>>>> 1. Topics don't include client ID information 2. Even if > >>>>>> you did, the same ID could be used for topics that are > >>>>>> not > >>>>> Kafka > >>>>>> Streams input > >>>>>> > >>>>>> The regex idea sounds doable, but I'm not sure it solves > >>>>>> much? > >>>>>> > >>>>>> > >>>>>> On Sat, Jan 18, 2020, 7:12 AM Sang wn Lee > >>>>>> <ssangdd...@gmail.com> > >>>> wrote: > >>>>>> > >>>>>>> Thank you Gwen Shapira! We'll add a flag to clear all > >>>>>>> topics by clientId It is ‘reset-all-external-topics’ > >>>>>>> > >>>>>>> I also want to use regex on the input topic flag to > >>>>>>> clear all > >>>> matching > >>>>>>> topics. > >>>>>>> > >>>>>>> On 2020/01/17 19:29:09, Gwen Shapira > >>>>>>> <g...@confluent.io> wrote: > >>>>>>>> Seem like a very nice improvement to me. But I have > >>>>>>>> to admit that I don't understand how this will how - > >>>>>>>> how could you infer the input topics? > >>>>>>>> > >>>>>>>> On Thu, Jan 16, 2020 at 10:03 AM Sang wn Lee > >>>>>>>> <ssangdd...@gmail.com > >>>>> > >>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hello, > >>>>>>>>> > >>>>>>>>> Starting this thread to discuss KIP-560: wiki link > >>>>>>>>> : > >>>>>>>>> > >>>>>>> > >>>>> > >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-560%3A+Auto+i > nfer+external+topic+partitions+in+stream+reset+tool > >>>>>>>>> > >>>>>>>>> > >>>> > I'm > >>>> > newbie > >>>>>>>>> I would like to receive feedback on the following > >>>>>>>>> features! > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > -----BEGIN PGP SIGNATURE----- > > iQIzBAEBCgAdFiEEI8mthP+5zxXZZdDSO4miYXKq/OgFAl5j/4sACgkQO4miYXKq > /Og4Yw/+LwLEcesv7r9JCVWhtgSe0iDrDFs3vA5p9uCzHfS/e7rm33hvGaaRSikC > A5yfayLi/88E6VzZWlY65ry8JPprCxepbalQHPUSBfcx+jwShHQOD7VwDavnAu2s > eKhmbstjtrNPtIdzdL0ine88f47ubgQpFbu5sbTFtGPGAkLuCD/QIo+tAghcVNil > ZTY/Pw602rH/CI9+jt3lenrm54Cju99bruslIhoP4eBUtiWthns2QwfGPkEOEyv6 > Oa0fm8CXJEPtvp6idCMJ9in7GlZvNsSmbHTGKWJtoTx89fdnZNjaZuFdtRK+sNE6 > w+fYtQnY7VrLAkQ/zcLgFQE2dqNSR0ZAiC2VjmMoS9Qg5UKJOpmtI0CvIAJ2PVVY > o0OnLo41wcG5W7SsumOvtwP38YnflnPjnLR0cZg74R7TVT4VmaKku9unRhUftipQ > m8aL4dNfcweB7CqhXthIABCsHUPj3VgA5HP18KmZqnCN5k4JXkJv6zU8ePm3kWGx > NowG0gf8Hiy5WbAt/HC2v53xYjjlrf1uI978NWeLqJ7Oi3K4rOi6rhg2IGGl23ym > eYt2ElUu+aIU0SB54lHtjOPbXLRKLdhhx5Zrsm5xKU3kKHGTg7kNR1aLE3LeR+M7 > i/ux5jA/tZ+l89eRyOCsnEyzTVSZC9f7zjV+acQdR08gBVeXKFE= > =LCV/ > -----END PGP SIGNATURE----- >
