Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-13 Thread Arjun Satish
Paul, Looks like the last note gives the message that this change expects people to update the quota configurations. Can we make it clear that this change will not impact quota limits, and that these default client ids are not a reasonable way to configure quotas before or after this change. Than

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-06 Thread Paul Davidson
Thanks Arjun. I've updated the KIP using your suggestion - just a few slight changes. On Fri, May 3, 2019 at 4:48 PM Arjun Satish wrote: > Maybe we can say something like: > > This change can have an indirect impact on resource usage by a Connector. > For example, systems that were enforcing quo

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-03 Thread Arjun Satish
Maybe we can say something like: This change can have an indirect impact on resource usage by a Connector. For example, systems that were enforcing quotas using a "consumer-[id]" client id will now have to update their configs to enforce quota on "connector-consumer-[id]". For systems that were no

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-03 Thread Paul Davidson
Thanks Arjun. I updated the KIP to mention the impact on quotas. Please let me know if you think I need more detail. The paragraph I added was: Since the default client.id values are changing, this will also affect any > user that has quotas defined against the current defaults. The current > defa

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-02 Thread Arjun Satish
Paul, You might want to make a note on the KIP regarding the impact on quotas. Thanks, On Thu, May 2, 2019 at 9:48 AM Paul Davidson wrote: > Thanks for the votes everyone! KIP-411 is now accepted with: > > +3 binding votes (Randall, Jason, Gwen) , and > +3 non-binding votes (Ryanne, Arjun, Mag

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-02 Thread Paul Davidson
Thanks for the votes everyone! KIP-411 is now accepted with: +3 binding votes (Randall, Jason, Gwen) , and +3 non-binding votes (Ryanne, Arjun, Magesh) Regards, Paul On Wed, May 1, 2019 at 10:07 PM Arjun Satish wrote: > Good point, Gwen. We always set a non empty value for client id: > > http

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Arjun Satish
Good point, Gwen. We always set a non empty value for client id: https://github.com/apache/kafka/blob/2.2.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L668 . But more importantly, connect client ids (for consumers, for example) were already of the form "consumer-[0-

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Gwen Shapira
I'm confused. Surely the default quota applies on empty client IDs too? otherwise it will be very difficult to enforce? So setting the client name will only change something if there's already a quota for that client? On the other hand, I fully support switching to "easy-to-wildcard" template for

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Arjun Satish
I just realized that setting the client.id on the will now trigger any quota restrictions ( https://kafka.apache.org/documentation/#design_quotasconfig) on the broker. It seems like this PR will enforce quota policies that will either require admins to set limits for each task (since the chosen for

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Gwen Shapira
hell yeah! +1 On Fri, Apr 5, 2019 at 9:08 AM Paul Davidson wrote: > Hi all, > > Since we seem to have agreement in the discussion I would like to start the > vote on KIP-411. > > See: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-05-01 Thread Jason Gustafson
Looks helpful. +1 -Jason On Tue, Apr 30, 2019 at 5:54 PM Randall Hauch wrote: > Nice simple improvement. Thanks, Paul! > > +1 (binding) > > Randall > > On Mon, Apr 29, 2019 at 5:06 PM Magesh Nandakumar > wrote: > > > Looks good to me and a very useful feature. > > > > +1 ( non-binding) > > > >

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-30 Thread Randall Hauch
Nice simple improvement. Thanks, Paul! +1 (binding) Randall On Mon, Apr 29, 2019 at 5:06 PM Magesh Nandakumar wrote: > Looks good to me and a very useful feature. > > +1 ( non-binding) > > On Mon, Apr 29, 2019, 4:05 PM Arjun Satish wrote: > > > Thanks, Paul! This is very useful. > > > > +1 (n

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-29 Thread Magesh Nandakumar
Looks good to me and a very useful feature. +1 ( non-binding) On Mon, Apr 29, 2019, 4:05 PM Arjun Satish wrote: > Thanks, Paul! This is very useful. > > +1 (non-binding) > > Best, > Arjun > > On Fri, Apr 12, 2019 at 4:13 PM Ryanne Dolan > wrote: > > > +1 (non binding) > > > > Thanks > > Ryanne

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-29 Thread Arjun Satish
Thanks, Paul! This is very useful. +1 (non-binding) Best, Arjun On Fri, Apr 12, 2019 at 4:13 PM Ryanne Dolan wrote: > +1 (non binding) > > Thanks > Ryanne > > On Fri, Apr 12, 2019, 11:11 AM Paul Davidson > wrote: > > > Just a reminder that KIP-411 is open for voting. No votes received yet! >

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-12 Thread Ryanne Dolan
+1 (non binding) Thanks Ryanne On Fri, Apr 12, 2019, 11:11 AM Paul Davidson wrote: > Just a reminder that KIP-411 is open for voting. No votes received yet! > > On Fri, Apr 5, 2019 at 9:07 AM Paul Davidson > wrote: > > > Hi all, > > > > Since we seem to have agreement in the discussion I would

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-12 Thread Paul Davidson
Just a reminder that KIP-411 is open for voting. No votes received yet! On Fri, Apr 5, 2019 at 9:07 AM Paul Davidson wrote: > Hi all, > > Since we seem to have agreement in the discussion I would like to start > the vote on KIP-411. > > See: > https://cwiki.apache.org/confluence/display/KAFKA/KI

[VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

2019-04-05 Thread Paul Davidson
Hi all, Since we seem to have agreement in the discussion I would like to start the vote on KIP-411. See: https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+IDs+distinct Also see the related PR: https://github.com/apache/kafka/pull/6097 Th