Paul,
Looks like the last note gives the message that this change expects people
to update the quota configurations. Can we make it clear that this change
will not impact quota limits, and that these default client ids are not a
reasonable way to configure quotas before or after this change.
Thanks very much and apologies for the confusion!
Best,
On Mon, May 6, 2019 at 10:33 AM Paul Davidson
<pdavid...@salesforce.com.invalid> wrote:
> Thanks Arjun. I've updated the KIP using your suggestion - just a few
> slight changes.
>
> On Fri, May 3, 2019 at 4:48 PM Arjun Satish <arjun.sat...@gmail.com>
> wrote:
>
> > Maybe we can say something like:
> >
> > This change can have an indirect impact on resource usage by a Connector.
> > For example, systems that were enforcing quotas using a "consumer-[id]"
> > client id will now have to update their configs to enforce quota on
> > "connector-consumer-[id]". For systems that were not enforcing any
> > limitations or using default quotas, there should be no change expected.
> >
> > Best,
> >
> > On Fri, May 3, 2019 at 1:38 PM Paul Davidson
> > <pdavid...@salesforce.com.invalid> wrote:
> >
> > > Thanks Arjun. I updated the KIP to mention the impact on quotas. Please
> > let
> > > me know if you think I need more detail. The paragraph I added was:
> > >
> > > Since the default client.id values are changing, this will also affect
> > any
> > > > user that has quotas defined against the current defaults. The
> current
> > > > default client.id values are of the form: consumer-{count} and
> > > > producer-{count}.
> > >
> > >
> > > Thanks,
> > >
> > > Paul
> > >
> > > On Thu, May 2, 2019 at 5:36 PM Arjun Satish <arjun.sat...@gmail.com>
> > > wrote:
> > >
> > > > Paul,
> > > >
> > > > You might want to make a note on the KIP regarding the impact on
> > quotas.
> > > >
> > > > Thanks,
> > > >
> > > > On Thu, May 2, 2019 at 9:48 AM Paul Davidson
> > > > <pdavid...@salesforce.com.invalid> wrote:
> > > >
> > > > > Thanks for the votes everyone! KIP-411 is now accepted with:
> > > > >
> > > > > +3 binding votes (Randall, Jason, Gwen) , and
> > > > > +3 non-binding votes (Ryanne, Arjun, Magesh)
> > > > >
> > > > > Regards,
> > > > >
> > > > > Paul
> > > > >
> > > > > On Wed, May 1, 2019 at 10:07 PM Arjun Satish <
> arjun.sat...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Good point, Gwen. We always set a non empty value for client id:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/kafka/blob/2.2.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L668
> > > > > > .
> > > > > >
> > > > > > But more importantly, connect client ids (for consumers, for
> > example)
> > > > > were
> > > > > > already of the form "consumer-[0-9]+", and from now on they will
> be
> > > > > > "connector-consumer-[connector_name]-[0-9]+". So, at least for
> > > connect
> > > > > > consumers/producers, we would have already been hitting the
> default
> > > > quota
> > > > > > limits and nothing changes for them. You can correct me if I'm
> > > missing
> > > > > > something, but seems like this doesn't *break* backward
> > > compatibility?
> > > > > >
> > > > > > I suppose this change only gives us a better way to manage that
> > quota
> > > > > > limit.
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > On Wed, May 1, 2019 at 9:16 PM Gwen Shapira <g...@confluent.io>
> > > wrote:
> > > > > >
> > > > > > > I'm confused. Surely the default quota applies on empty client
> > IDs
> > > > too?
> > > > > > > otherwise it will be very difficult to enforce?
> > > > > > > So setting the client name will only change something if
> there's
> > > > > already
> > > > > > a
> > > > > > > quota for that client?
> > > > > > >
> > > > > > > On the other hand, I fully support switching to
> > "easy-to-wildcard"
> > > > > > template
> > > > > > > for the client id.
> > > > > > >
> > > > > > > On Wed, May 1, 2019 at 8:50 PM Arjun Satish <
> > > arjun.sat...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I just realized that setting the client.id on the will now
> > > trigger
> > > > > any
> > > > > > > > quota restrictions (
> > > > > > > > https://kafka.apache.org/documentation/#design_quotasconfig)
> > on
> > > > the
> > > > > > > > broker.
> > > > > > > > It seems like this PR will enforce quota policies that will
> > > either
> > > > > > > require
> > > > > > > > admins to set limits for each task (since the chosen format
> is
> > > > > > > > connector-*-id), or fallback to some default value.
> > > > > > > >
> > > > > > > > Maybe we should mention this in the backward compatibility
> > > section
> > > > > for
> > > > > > > the
> > > > > > > > KIP. At the same time, since there is no way atm to turn off
> > this
> > > > > > > feature,
> > > > > > > > should this feature be merged and released in the upcoming
> > v2.3?
> > > > This
> > > > > > is
> > > > > > > > something the committers can comment better.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, May 1, 2019 at 5:13 PM Gwen Shapira <
> g...@confluent.io
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > hell yeah!
> > > > > > > > > +1
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Apr 5, 2019 at 9:08 AM Paul Davidson
> > > > > > > > > <pdavid...@salesforce.com.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > > > Since we seem to have agreement in the discussion I would
> > > like
> > > > to
> > > > > > > start
> > > > > > > > > the
> > > > > > > > > > vote on KIP-411.
> > > > > > > > > >
> > > > > > > > > > See:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+IDs+distinct
> > > > > > > > > >
> > > > > > > > > > Also see the related PR:
> > > > > https://github.com/apache/kafka/pull/6097
> > > > > > > > > >
> > > > > > > > > > Thanks to everyone who contributed!
> > > > > > > > > >
> > > > > > > > > > Paul
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > *Gwen Shapira*
> > > > > > > > > Product Manager | Confluent
> > > > > > > > > 650.450.2760 | @gwenshap
> > > > > > > > > Follow us: Twitter <https://twitter.com/ConfluentInc> |
> blog
> > > > > > > > > <http://www.confluent.io/blog>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > *Gwen Shapira*
> > > > > > > Product Manager | Confluent
> > > > > > > 650.450.2760 | @gwenshap
> > > > > > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > > > > > <http://www.confluent.io/blog>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
