Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Colin McCabe]

On Fri, Dec 15, 2017, at 08:56, dan wrote:
> i didn't think it would be appropriate to return `num.partitions` as a
> Config for a Topic since that is not a valid config for that resource
> type.
> 
> yes, that is another option i have on the wiki in the Rejected
> Alternatives section (though, its really just an alternative until something 
> else
> passes voting :). we still have the issue with return type here, NewTopic has
> the info i want to return, but not necessarily the name i'd want...

Hi Dan,

Hmm.  I don't think NewTopic is really the right thing to return,
though.  NewTopic is structured so that it can be vague... you can
specify just replication factor, or a full assignment.  You can specify
some configuration keys, or no configuration keys at all.

NewTopic will probably grow more optional fields in the future.  But the
key thing is that NewTopic represents a request to create something, not
a description of something that actually exists.

> it could
> make sense for a createTopics() to return a TopicDescription and we could
> add the config/replication info to that class, but we'd then need to add
> all this info for all describeTopics() calls. not sure of the
> ramifications/overhead here.

Replication info is already part of TopicDescription, right?  There is a
list of TopicPartitionInfo there.

Config already encapsulates everything you would want to know about
topic configs, I think.

We don't really need to have a new type that encloses both descriptions
and configs.  We can just have

CreateTopicsResult {
  KafkaFuture description(String topic);
  KafkaFuture config(String topic);
  Map values();
  KafkaFuture all();
}

And then people who care about the config of the topic they just created
can call result.config("myTopicName").get(), and those who don't care
can ignore it.

This would be really nice... it lets you figure out what the partition
assignment and config actually are when you actually create a topic. 
Plus it extends the existing createTopics(validateOnly=true)
functionality to be even more useful.

best,
Colin


> 
> dan
> 
> On Wed, Dec 13, 2017 at 10:10 AM, Colin McCabe 
> wrote:
> 
> > On Wed, Dec 13, 2017, at 10:00, dan wrote:
> > > > Why not just return
> > > > org.apache.kafka.clients.admin.Config like describeConfigs does?
> > >
> > > brokers have a `num.partitions` config that does not map to a valid
> > > `Config` entry for a topic.
> >
> > Hi Dan,
> >
> > Sorry if I'm misunderstanding something, but why not map it to
> > num.partitions?
> >
> > >
> > > another added benefit to using `NewTopic` may be (future kip) having the
> > > cluster return the actual replica mappings it would create (i have no
> > > idea if this is actually possible)
> >
> > A better way of doing that would probably be extending
> > CreateTopicsRequest so that it returns partition assignment information
> > to the caller.  Then using validOnly = true to get this information.
> >
> > Actually, come to think of it, maybe we should be doing that for this
> > KIP too.  Why not have CreateTopicsRequest return the config that was
> > used, plus the partition assignment that was made?  We don't create
> > topics that often, so the extra space on the wire should not be a
> > concern.
> >
> > best,
> > Colin
> >
> > >
> > > dan
> > >
> > > On Wed, Dec 13, 2017 at 9:55 AM, Colin McCabe 
> > wrote:
> > >
> > > > On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote:
> > > > > re: API versions, I actually wasn't sure if we needed it or not. I'm
> > fine
> > > > > if people would prefer just bumping it, but I was actually curious
> > if we
> > > > > could get away without bumping it. I don't know the behavior of the
> > > > > broker code paths for this well enough to know what types of errors
> > those
> > > > > non-null assertions get converted into.
> > > >
> > > > There's no advantage to trying to keep the API version number the same,
> > > > though.  Since we have bidirectional client compatibility now, the
> > > > clients and the server will just negotiate whatever version they need.
> > > > New clients can still talk to older brokers that don't support this
> > > > feature.
> > > >
> > > > If you don't bump the API version, the best case scenario is that you
> > > > get a disconnect exception and the end-user is left confused about why.
> > > > The worse-case scenario is that you crash the broker (but probably not,
> > > > since you'd just get an NPE in serde, I think).  If you bump the
> > version
> > > > number, you can provide a proper UnsupportedVersionException when the
> > > > feature is not supported.
> > > >
> > > > > For the return type, NewTopic seems reasonable and kind of intuitive
> > --
> > > > > basically a description of the NewTopic you would get. The only
> > reason I
> > > > > would be wary of reusing it is that what we don't want people doing
> > is
> > > > > taking that and passing it directly into 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

i didn't think it would be appropriate to return `num.partitions` as a
Config for a Topic since that is not a valid config for that resource type.

yes, that is another option i have on the wiki in the Rejected Alternatives
section (though, its really just an alternative until something else passes
voting :). we still have the issue with return type here, NewTopic has the
info i want to return, but not necessarily the name i'd want... it could
make sense for a createTopics() to return a TopicDescription and we could
add the config/replication info to that class, but we'd then need to add
all this info for all describeTopics() calls. not sure of the
ramifications/overhead here.

dan

On Wed, Dec 13, 2017 at 10:10 AM, Colin McCabe  wrote:

> On Wed, Dec 13, 2017, at 10:00, dan wrote:
> > > Why not just return
> > > org.apache.kafka.clients.admin.Config like describeConfigs does?
> >
> > brokers have a `num.partitions` config that does not map to a valid
> > `Config` entry for a topic.
>
> Hi Dan,
>
> Sorry if I'm misunderstanding something, but why not map it to
> num.partitions?
>
> >
> > another added benefit to using `NewTopic` may be (future kip) having the
> > cluster return the actual replica mappings it would create (i have no
> > idea if this is actually possible)
>
> A better way of doing that would probably be extending
> CreateTopicsRequest so that it returns partition assignment information
> to the caller.  Then using validOnly = true to get this information.
>
> Actually, come to think of it, maybe we should be doing that for this
> KIP too.  Why not have CreateTopicsRequest return the config that was
> used, plus the partition assignment that was made?  We don't create
> topics that often, so the extra space on the wire should not be a
> concern.
>
> best,
> Colin
>
> >
> > dan
> >
> > On Wed, Dec 13, 2017 at 9:55 AM, Colin McCabe 
> wrote:
> >
> > > On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote:
> > > > re: API versions, I actually wasn't sure if we needed it or not. I'm
> fine
> > > > if people would prefer just bumping it, but I was actually curious
> if we
> > > > could get away without bumping it. I don't know the behavior of the
> > > > broker code paths for this well enough to know what types of errors
> those
> > > > non-null assertions get converted into.
> > >
> > > There's no advantage to trying to keep the API version number the same,
> > > though.  Since we have bidirectional client compatibility now, the
> > > clients and the server will just negotiate whatever version they need.
> > > New clients can still talk to older brokers that don't support this
> > > feature.
> > >
> > > If you don't bump the API version, the best case scenario is that you
> > > get a disconnect exception and the end-user is left confused about why.
> > > The worse-case scenario is that you crash the broker (but probably not,
> > > since you'd just get an NPE in serde, I think).  If you bump the
> version
> > > number, you can provide a proper UnsupportedVersionException when the
> > > feature is not supported.
> > >
> > > > For the return type, NewTopic seems reasonable and kind of intuitive
> --
> > > > basically a description of the NewTopic you would get. The only
> reason I
> > > > would be wary of reusing it is that what we don't want people doing
> is
> > > > taking that and passing it directly into AdminClient.createTopics
> since
> > > > we don't want them explicitly overriding all the defaults.
> > >
> > > Yeah.  Another thing is that NewTopic has a lot of stuff related to
> > > replication that doesn't seem relevant here.  For example, when
> creating
> > > NewTopic, you have the option of either setting replicationFactor, or
> > > setting up a specific replica assignment.  Why not just return
> > > org.apache.kafka.clients.admin.Config like describeConfigs does?
> > >
> > > best,
> > > Colin
> > >
> > > >
> > > > -Ewen
> > > >
> > > > On Tue, Dec 12, 2017 at 2:32 PM, dan  wrote:
> > > >
> > > > > Colin/Ewen,
> > > > >
> > > > > i will add changes to bump the API version.
> > > > >
> > > > > any preferences on the return type for the new method? tbh it seems
> > > like
> > > > > returning a NewTopic could make sense because the ConfigResource
> for a
> > > > > TOPIC type does not let me encode `numPartitions`
> > > > >
> > > > > thanks
> > > > > dan
> > > > >
> > > > > On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe 
> > > wrote:
> > > > >
> > > > > > Hi Dan,
> > > > > >
> > > > > > The KIP looks good overall.
> > > > > >
> > > > > > On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > > > > > > I think the key point is when the kafka admin and user creating
> > > topics
> > > > > > > differ. I think a more realistic example of Dan's point (2) is
> for
> > > > > > > retention. I know that realistically, admins aren't just going
> to
> > > > > > > randomly
> > > > > > > drop the broker 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Colin McCabe]

On Wed, Dec 13, 2017, at 10:00, dan wrote:
> > Why not just return
> > org.apache.kafka.clients.admin.Config like describeConfigs does?
> 
> brokers have a `num.partitions` config that does not map to a valid
> `Config` entry for a topic.

Hi Dan,

Sorry if I'm misunderstanding something, but why not map it to
num.partitions?

> 
> another added benefit to using `NewTopic` may be (future kip) having the
> cluster return the actual replica mappings it would create (i have no
> idea if this is actually possible)

A better way of doing that would probably be extending
CreateTopicsRequest so that it returns partition assignment information
to the caller.  Then using validOnly = true to get this information.

Actually, come to think of it, maybe we should be doing that for this
KIP too.  Why not have CreateTopicsRequest return the config that was
used, plus the partition assignment that was made?  We don't create
topics that often, so the extra space on the wire should not be a
concern.

best,
Colin

> 
> dan
> 
> On Wed, Dec 13, 2017 at 9:55 AM, Colin McCabe  wrote:
> 
> > On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote:
> > > re: API versions, I actually wasn't sure if we needed it or not. I'm fine
> > > if people would prefer just bumping it, but I was actually curious if we
> > > could get away without bumping it. I don't know the behavior of the
> > > broker code paths for this well enough to know what types of errors those
> > > non-null assertions get converted into.
> >
> > There's no advantage to trying to keep the API version number the same,
> > though.  Since we have bidirectional client compatibility now, the
> > clients and the server will just negotiate whatever version they need.
> > New clients can still talk to older brokers that don't support this
> > feature.
> >
> > If you don't bump the API version, the best case scenario is that you
> > get a disconnect exception and the end-user is left confused about why.
> > The worse-case scenario is that you crash the broker (but probably not,
> > since you'd just get an NPE in serde, I think).  If you bump the version
> > number, you can provide a proper UnsupportedVersionException when the
> > feature is not supported.
> >
> > > For the return type, NewTopic seems reasonable and kind of intuitive --
> > > basically a description of the NewTopic you would get. The only reason I
> > > would be wary of reusing it is that what we don't want people doing is
> > > taking that and passing it directly into AdminClient.createTopics since
> > > we don't want them explicitly overriding all the defaults.
> >
> > Yeah.  Another thing is that NewTopic has a lot of stuff related to
> > replication that doesn't seem relevant here.  For example, when creating
> > NewTopic, you have the option of either setting replicationFactor, or
> > setting up a specific replica assignment.  Why not just return
> > org.apache.kafka.clients.admin.Config like describeConfigs does?
> >
> > best,
> > Colin
> >
> > >
> > > -Ewen
> > >
> > > On Tue, Dec 12, 2017 at 2:32 PM, dan  wrote:
> > >
> > > > Colin/Ewen,
> > > >
> > > > i will add changes to bump the API version.
> > > >
> > > > any preferences on the return type for the new method? tbh it seems
> > like
> > > > returning a NewTopic could make sense because the ConfigResource for a
> > > > TOPIC type does not let me encode `numPartitions`
> > > >
> > > > thanks
> > > > dan
> > > >
> > > > On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe 
> > wrote:
> > > >
> > > > > Hi Dan,
> > > > >
> > > > > The KIP looks good overall.
> > > > >
> > > > > On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > > > > > I think the key point is when the kafka admin and user creating
> > topics
> > > > > > differ. I think a more realistic example of Dan's point (2) is for
> > > > > > retention. I know that realistically, admins aren't just going to
> > > > > > randomly
> > > > > > drop the broker defaults from 1w to 1d without warning anyone
> > (they'd
> > > > > > likely be fired...). But as a user, I may not know the broker
> > configs,
> > > > if
> > > > > > admins have overridden them, etc. I may want a *minimum* of, e.g.,
> > 2d.
> > > > > > But if the broker defaults are higher such that the admins are
> > > > confident
> > > > > the
> > > > > > cluster can handle 1w, I'd rather just fall back on the default
> > value.
> > > > >
> > > > > Right.  I think this API addresses a similar set of use-cases as
> > adding
> > > > > the "validateOnly" boolean for createTopics.  You shouldn't have to
> > > > > create a topic to know whether it was possible to create it, or what
> > the
> > > > > retention will end up being, etc. etc.
> > > > >
> > > > > > Now, there's arguably a better solution for that case -- allow
> > topic
> > > > > > configs to express a *minimum* value (or maximum depending on the
> > > > > > particular config), with the broker config taking 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

> Why not just return
> org.apache.kafka.clients.admin.Config like describeConfigs does?

brokers have a `num.partitions` config that does not map to a valid
`Config` entry for a topic.

another added benefit to using `NewTopic` may be (future kip) having the
cluster return the actual replica mappings it would create (i have no idea
if this is actually possible)

dan

On Wed, Dec 13, 2017 at 9:55 AM, Colin McCabe  wrote:

> On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote:
> > re: API versions, I actually wasn't sure if we needed it or not. I'm fine
> > if people would prefer just bumping it, but I was actually curious if we
> > could get away without bumping it. I don't know the behavior of the
> > broker code paths for this well enough to know what types of errors those
> > non-null assertions get converted into.
>
> There's no advantage to trying to keep the API version number the same,
> though.  Since we have bidirectional client compatibility now, the
> clients and the server will just negotiate whatever version they need.
> New clients can still talk to older brokers that don't support this
> feature.
>
> If you don't bump the API version, the best case scenario is that you
> get a disconnect exception and the end-user is left confused about why.
> The worse-case scenario is that you crash the broker (but probably not,
> since you'd just get an NPE in serde, I think).  If you bump the version
> number, you can provide a proper UnsupportedVersionException when the
> feature is not supported.
>
> > For the return type, NewTopic seems reasonable and kind of intuitive --
> > basically a description of the NewTopic you would get. The only reason I
> > would be wary of reusing it is that what we don't want people doing is
> > taking that and passing it directly into AdminClient.createTopics since
> > we don't want them explicitly overriding all the defaults.
>
> Yeah.  Another thing is that NewTopic has a lot of stuff related to
> replication that doesn't seem relevant here.  For example, when creating
> NewTopic, you have the option of either setting replicationFactor, or
> setting up a specific replica assignment.  Why not just return
> org.apache.kafka.clients.admin.Config like describeConfigs does?
>
> best,
> Colin
>
> >
> > -Ewen
> >
> > On Tue, Dec 12, 2017 at 2:32 PM, dan  wrote:
> >
> > > Colin/Ewen,
> > >
> > > i will add changes to bump the API version.
> > >
> > > any preferences on the return type for the new method? tbh it seems
> like
> > > returning a NewTopic could make sense because the ConfigResource for a
> > > TOPIC type does not let me encode `numPartitions`
> > >
> > > thanks
> > > dan
> > >
> > > On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe 
> wrote:
> > >
> > > > Hi Dan,
> > > >
> > > > The KIP looks good overall.
> > > >
> > > > On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > > > > I think the key point is when the kafka admin and user creating
> topics
> > > > > differ. I think a more realistic example of Dan's point (2) is for
> > > > > retention. I know that realistically, admins aren't just going to
> > > > > randomly
> > > > > drop the broker defaults from 1w to 1d without warning anyone
> (they'd
> > > > > likely be fired...). But as a user, I may not know the broker
> configs,
> > > if
> > > > > admins have overridden them, etc. I may want a *minimum* of, e.g.,
> 2d.
> > > > > But if the broker defaults are higher such that the admins are
> > > confident
> > > > the
> > > > > cluster can handle 1w, I'd rather just fall back on the default
> value.
> > > >
> > > > Right.  I think this API addresses a similar set of use-cases as
> adding
> > > > the "validateOnly" boolean for createTopics.  You shouldn't have to
> > > > create a topic to know whether it was possible to create it, or what
> the
> > > > retention will end up being, etc. etc.
> > > >
> > > > > Now, there's arguably a better solution for that case -- allow
> topic
> > > > > configs to express a *minimum* value (or maximum depending on the
> > > > > particular config), with the broker config taking precedence if it
> has
> > > a
> > > > > smaller value (or larger in the case of maximums). This lets you
> > > express
> > > > > your minimum requirements but allows the cluster to do more if
> that's
> > > the
> > > > > default. However, that would represent a much more significant and
> > > > > invasive change, and honestly I think it is more likely to confuse
> > > users.
> > > >
> > > > There always need to be topic defaults, though.  If we add a foobar
> > > > configuration for topics, existing topics will need to get
> grandfathered
> > > > in with a default foobar.  And they won't be able to set min and max
> > > > ranges, because foobars didn't exist back when the old topics were
> > > > created.
> > > >
> > > > >
> > > > > @Dan, regarding compatibility, this changes behavior without
> revving
> > > the
> > > > > 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Colin McCabe]

On Tue, Dec 12, 2017, at 19:02, Ewen Cheslack-Postava wrote:
> re: API versions, I actually wasn't sure if we needed it or not. I'm fine
> if people would prefer just bumping it, but I was actually curious if we
> could get away without bumping it. I don't know the behavior of the
> broker code paths for this well enough to know what types of errors those
> non-null assertions get converted into.

There's no advantage to trying to keep the API version number the same,
though.  Since we have bidirectional client compatibility now, the
clients and the server will just negotiate whatever version they need. 
New clients can still talk to older brokers that don't support this
feature.

If you don't bump the API version, the best case scenario is that you
get a disconnect exception and the end-user is left confused about why. 
The worse-case scenario is that you crash the broker (but probably not,
since you'd just get an NPE in serde, I think).  If you bump the version
number, you can provide a proper UnsupportedVersionException when the
feature is not supported.

> For the return type, NewTopic seems reasonable and kind of intuitive --
> basically a description of the NewTopic you would get. The only reason I
> would be wary of reusing it is that what we don't want people doing is
> taking that and passing it directly into AdminClient.createTopics since
> we don't want them explicitly overriding all the defaults.

Yeah.  Another thing is that NewTopic has a lot of stuff related to
replication that doesn't seem relevant here.  For example, when creating
NewTopic, you have the option of either setting replicationFactor, or
setting up a specific replica assignment.  Why not just return
org.apache.kafka.clients.admin.Config like describeConfigs does?

best,
Colin

> 
> -Ewen
> 
> On Tue, Dec 12, 2017 at 2:32 PM, dan  wrote:
> 
> > Colin/Ewen,
> >
> > i will add changes to bump the API version.
> >
> > any preferences on the return type for the new method? tbh it seems like
> > returning a NewTopic could make sense because the ConfigResource for a
> > TOPIC type does not let me encode `numPartitions`
> >
> > thanks
> > dan
> >
> > On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe  wrote:
> >
> > > Hi Dan,
> > >
> > > The KIP looks good overall.
> > >
> > > On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > > > I think the key point is when the kafka admin and user creating topics
> > > > differ. I think a more realistic example of Dan's point (2) is for
> > > > retention. I know that realistically, admins aren't just going to
> > > > randomly
> > > > drop the broker defaults from 1w to 1d without warning anyone (they'd
> > > > likely be fired...). But as a user, I may not know the broker configs,
> > if
> > > > admins have overridden them, etc. I may want a *minimum* of, e.g., 2d.
> > > > But if the broker defaults are higher such that the admins are
> > confident
> > > the
> > > > cluster can handle 1w, I'd rather just fall back on the default value.
> > >
> > > Right.  I think this API addresses a similar set of use-cases as adding
> > > the "validateOnly" boolean for createTopics.  You shouldn't have to
> > > create a topic to know whether it was possible to create it, or what the
> > > retention will end up being, etc. etc.
> > >
> > > > Now, there's arguably a better solution for that case -- allow topic
> > > > configs to express a *minimum* value (or maximum depending on the
> > > > particular config), with the broker config taking precedence if it has
> > a
> > > > smaller value (or larger in the case of maximums). This lets you
> > express
> > > > your minimum requirements but allows the cluster to do more if that's
> > the
> > > > default. However, that would represent a much more significant and
> > > > invasive change, and honestly I think it is more likely to confuse
> > users.
> > >
> > > There always need to be topic defaults, though.  If we add a foobar
> > > configuration for topics, existing topics will need to get grandfathered
> > > in with a default foobar.  And they won't be able to set min and max
> > > ranges, because foobars didn't exist back when the old topics were
> > > created.
> > >
> > > >
> > > > @Dan, regarding compatibility, this changes behavior without revving
> > the
> > > > request version number, which normally we only do for things that are
> > > > reasonably considered bugfixes or were it has no compatibility
> > > > implications. In this case, older brokers talking to newer AdminClients
> > > > will presumably return some error. Do we know what the non-null
> > assertion
> > > > gets converted to and if we're happy with the behavior (i.e. will
> > > > applications be able to do something reasonable, distinguish it from
> > some
> > > > completely unrelated error, etc)? Similarly, it's obviously only one
> > > > implementation using the KIP-4 APIs, but do we know what client-side
> > > > validation AdminClient is already 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Ewen Cheslack-Postava]

re: API versions, I actually wasn't sure if we needed it or not. I'm fine
if people would prefer just bumping it, but I was actually curious if we
could get away without bumping it. I don't know the behavior of the broker
code paths for this well enough to know what types of errors those non-null
assertions get converted into.

For the return type, NewTopic seems reasonable and kind of intuitive --
basically a description of the NewTopic you would get. The only reason I
would be wary of reusing it is that what we don't want people doing is
taking that and passing it directly into AdminClient.createTopics since we
don't want them explicitly overriding all the defaults.

-Ewen

On Tue, Dec 12, 2017 at 2:32 PM, dan  wrote:

> Colin/Ewen,
>
> i will add changes to bump the API version.
>
> any preferences on the return type for the new method? tbh it seems like
> returning a NewTopic could make sense because the ConfigResource for a
> TOPIC type does not let me encode `numPartitions`
>
> thanks
> dan
>
> On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe  wrote:
>
> > Hi Dan,
> >
> > The KIP looks good overall.
> >
> > On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > > I think the key point is when the kafka admin and user creating topics
> > > differ. I think a more realistic example of Dan's point (2) is for
> > > retention. I know that realistically, admins aren't just going to
> > > randomly
> > > drop the broker defaults from 1w to 1d without warning anyone (they'd
> > > likely be fired...). But as a user, I may not know the broker configs,
> if
> > > admins have overridden them, etc. I may want a *minimum* of, e.g., 2d.
> > > But if the broker defaults are higher such that the admins are
> confident
> > the
> > > cluster can handle 1w, I'd rather just fall back on the default value.
> >
> > Right.  I think this API addresses a similar set of use-cases as adding
> > the "validateOnly" boolean for createTopics.  You shouldn't have to
> > create a topic to know whether it was possible to create it, or what the
> > retention will end up being, etc. etc.
> >
> > > Now, there's arguably a better solution for that case -- allow topic
> > > configs to express a *minimum* value (or maximum depending on the
> > > particular config), with the broker config taking precedence if it has
> a
> > > smaller value (or larger in the case of maximums). This lets you
> express
> > > your minimum requirements but allows the cluster to do more if that's
> the
> > > default. However, that would represent a much more significant and
> > > invasive change, and honestly I think it is more likely to confuse
> users.
> >
> > There always need to be topic defaults, though.  If we add a foobar
> > configuration for topics, existing topics will need to get grandfathered
> > in with a default foobar.  And they won't be able to set min and max
> > ranges, because foobars didn't exist back when the old topics were
> > created.
> >
> > >
> > > @Dan, regarding compatibility, this changes behavior without revving
> the
> > > request version number, which normally we only do for things that are
> > > reasonably considered bugfixes or were it has no compatibility
> > > implications. In this case, older brokers talking to newer AdminClients
> > > will presumably return some error. Do we know what the non-null
> assertion
> > > gets converted to and if we're happy with the behavior (i.e. will
> > > applications be able to do something reasonable, distinguish it from
> some
> > > completely unrelated error, etc)? Similarly, it's obviously only one
> > > implementation using the KIP-4 APIs, but do we know what client-side
> > > validation AdminClient is already doing and whether old AdminClients
> > > talking to new brokers will see a change in behavior (or do they do
> > > client-side validation such that old clients simply wouldn't have
> access
> > > to this new functionality)?
> >
> > I think we should bump the API version for this or add a new API key.
> > Nothing good is going to happen by pretending like this is compatible
> > with existing brokers.
> >
> > Also, I think it would be better just to have a separate function in
> > AdminClient rather than overloading the behavior of NULL in
> > describeConfigs.  It's not really that much more effort to have another
> > AdminClient function, and it's a lot simpler for devs to understand than
> > magical NULL behavior in describeConfigs.
> >
> > best,
> > Colin
> >
> > >
> > > -Ewen
> > >
> > > On Mon, Dec 11, 2017 at 2:11 PM, dan  wrote:
> > >
> > > > Dong,
> > > >
> > > > I agree that it *may* be better for a user to be explicit, however
> > there
> > > > are a couple reasons they may not.
> > > > 1) a user doesn't even know what the options are. imagine writing a
> > tool
> > > > for users to create topics that steps them through things:
> > > >
> > > > $ kafka-topics.sh --create
> > > > Give your topic a name: 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

Colin/Ewen,

i will add changes to bump the API version.

any preferences on the return type for the new method? tbh it seems like
returning a NewTopic could make sense because the ConfigResource for a
TOPIC type does not let me encode `numPartitions`

thanks
dan

On Mon, Dec 11, 2017 at 7:22 PM, Colin McCabe  wrote:

> Hi Dan,
>
> The KIP looks good overall.
>
> On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> > I think the key point is when the kafka admin and user creating topics
> > differ. I think a more realistic example of Dan's point (2) is for
> > retention. I know that realistically, admins aren't just going to
> > randomly
> > drop the broker defaults from 1w to 1d without warning anyone (they'd
> > likely be fired...). But as a user, I may not know the broker configs, if
> > admins have overridden them, etc. I may want a *minimum* of, e.g., 2d.
> > But if the broker defaults are higher such that the admins are confident
> the
> > cluster can handle 1w, I'd rather just fall back on the default value.
>
> Right.  I think this API addresses a similar set of use-cases as adding
> the "validateOnly" boolean for createTopics.  You shouldn't have to
> create a topic to know whether it was possible to create it, or what the
> retention will end up being, etc. etc.
>
> > Now, there's arguably a better solution for that case -- allow topic
> > configs to express a *minimum* value (or maximum depending on the
> > particular config), with the broker config taking precedence if it has a
> > smaller value (or larger in the case of maximums). This lets you express
> > your minimum requirements but allows the cluster to do more if that's the
> > default. However, that would represent a much more significant and
> > invasive change, and honestly I think it is more likely to confuse users.
>
> There always need to be topic defaults, though.  If we add a foobar
> configuration for topics, existing topics will need to get grandfathered
> in with a default foobar.  And they won't be able to set min and max
> ranges, because foobars didn't exist back when the old topics were
> created.
>
> >
> > @Dan, regarding compatibility, this changes behavior without revving the
> > request version number, which normally we only do for things that are
> > reasonably considered bugfixes or were it has no compatibility
> > implications. In this case, older brokers talking to newer AdminClients
> > will presumably return some error. Do we know what the non-null assertion
> > gets converted to and if we're happy with the behavior (i.e. will
> > applications be able to do something reasonable, distinguish it from some
> > completely unrelated error, etc)? Similarly, it's obviously only one
> > implementation using the KIP-4 APIs, but do we know what client-side
> > validation AdminClient is already doing and whether old AdminClients
> > talking to new brokers will see a change in behavior (or do they do
> > client-side validation such that old clients simply wouldn't have access
> > to this new functionality)?
>
> I think we should bump the API version for this or add a new API key.
> Nothing good is going to happen by pretending like this is compatible
> with existing brokers.
>
> Also, I think it would be better just to have a separate function in
> AdminClient rather than overloading the behavior of NULL in
> describeConfigs.  It's not really that much more effort to have another
> AdminClient function, and it's a lot simpler for devs to understand than
> magical NULL behavior in describeConfigs.
>
> best,
> Colin
>
> >
> > -Ewen
> >
> > On Mon, Dec 11, 2017 at 2:11 PM, dan  wrote:
> >
> > > Dong,
> > >
> > > I agree that it *may* be better for a user to be explicit, however
> there
> > > are a couple reasons they may not.
> > > 1) a user doesn't even know what the options are. imagine writing a
> tool
> > > for users to create topics that steps them through things:
> > >
> > > $ kafka-topics.sh --create
> > > Give your topic a name: my-fav-topic
> > > How many partitions do you want [12]:
> > > What is the minimum in set replica size [2]:
> > > What is the maximum message size [1MB]:
> > > ...
> > >
> > > 2) a user wants to use broker defaults within reason. say they thinke
> they
> > > want min.cleanable.dirty.ratio=.5 and the default is .6. maybe thats
> fine,
> > > or even better for them. since the person maintaining the actual
> cluster
> > > has put thought in to this config. and as the maintainer keeps working
> on
> > > making the cluster run better they can change and tune things on the
> > > cluster level as needed.
> > >
> > > dan
> > >
> > >
> > > On Wed, Dec 6, 2017 at 11:51 AM, Dong Lin  wrote:
> > >
> > > > Hey Dan,
> > > >
> > > > I think you are saying that, if user can read the default config
> before
> > > > creating the topic, then this user can make better decision in what
> > > configs
> > > > need to be overwritten. The question 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Colin McCabe]

Hi Dan,

The KIP looks good overall.

On Mon, Dec 11, 2017, at 18:28, Ewen Cheslack-Postava wrote:
> I think the key point is when the kafka admin and user creating topics
> differ. I think a more realistic example of Dan's point (2) is for
> retention. I know that realistically, admins aren't just going to
> randomly
> drop the broker defaults from 1w to 1d without warning anyone (they'd
> likely be fired...). But as a user, I may not know the broker configs, if
> admins have overridden them, etc. I may want a *minimum* of, e.g., 2d.
> But if the broker defaults are higher such that the admins are confident the
> cluster can handle 1w, I'd rather just fall back on the default value.

Right.  I think this API addresses a similar set of use-cases as adding
the "validateOnly" boolean for createTopics.  You shouldn't have to
create a topic to know whether it was possible to create it, or what the
retention will end up being, etc. etc.

> Now, there's arguably a better solution for that case -- allow topic
> configs to express a *minimum* value (or maximum depending on the
> particular config), with the broker config taking precedence if it has a
> smaller value (or larger in the case of maximums). This lets you express
> your minimum requirements but allows the cluster to do more if that's the
> default. However, that would represent a much more significant and
> invasive change, and honestly I think it is more likely to confuse users.

There always need to be topic defaults, though.  If we add a foobar
configuration for topics, existing topics will need to get grandfathered
in with a default foobar.  And they won't be able to set min and max
ranges, because foobars didn't exist back when the old topics were
created.

> 
> @Dan, regarding compatibility, this changes behavior without revving the
> request version number, which normally we only do for things that are
> reasonably considered bugfixes or were it has no compatibility
> implications. In this case, older brokers talking to newer AdminClients
> will presumably return some error. Do we know what the non-null assertion
> gets converted to and if we're happy with the behavior (i.e. will
> applications be able to do something reasonable, distinguish it from some
> completely unrelated error, etc)? Similarly, it's obviously only one
> implementation using the KIP-4 APIs, but do we know what client-side
> validation AdminClient is already doing and whether old AdminClients
> talking to new brokers will see a change in behavior (or do they do
> client-side validation such that old clients simply wouldn't have access
> to this new functionality)?

I think we should bump the API version for this or add a new API key. 
Nothing good is going to happen by pretending like this is compatible
with existing brokers.

Also, I think it would be better just to have a separate function in
AdminClient rather than overloading the behavior of NULL in
describeConfigs.  It's not really that much more effort to have another
AdminClient function, and it's a lot simpler for devs to understand than
magical NULL behavior in describeConfigs.

best,
Colin

> 
> -Ewen
> 
> On Mon, Dec 11, 2017 at 2:11 PM, dan  wrote:
> 
> > Dong,
> >
> > I agree that it *may* be better for a user to be explicit, however there
> > are a couple reasons they may not.
> > 1) a user doesn't even know what the options are. imagine writing a tool
> > for users to create topics that steps them through things:
> >
> > $ kafka-topics.sh --create
> > Give your topic a name: my-fav-topic
> > How many partitions do you want [12]:
> > What is the minimum in set replica size [2]:
> > What is the maximum message size [1MB]:
> > ...
> >
> > 2) a user wants to use broker defaults within reason. say they thinke they
> > want min.cleanable.dirty.ratio=.5 and the default is .6. maybe thats fine,
> > or even better for them. since the person maintaining the actual cluster
> > has put thought in to this config. and as the maintainer keeps working on
> > making the cluster run better they can change and tune things on the
> > cluster level as needed.
> >
> > dan
> >
> >
> > On Wed, Dec 6, 2017 at 11:51 AM, Dong Lin  wrote:
> >
> > > Hey Dan,
> > >
> > > I think you are saying that, if user can read the default config before
> > > creating the topic, then this user can make better decision in what
> > configs
> > > need to be overwritten. The question here is, how user is going to use
> > this
> > > config to make the decision.
> > >
> > > In my understanding, user will compare the default value with expected
> > > value, and override the config to be expected value if they are
> > different.
> > > If this is the only way that the default value can affect user's
> > decision,
> > > then it seems OK for user to directly override the config to the expected
> > > value. I am wondering if this solution has some drawback.
> > >
> > > On the other hand, maybe there is a more advanced 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Ewen Cheslack-Postava]

I think the key point is when the kafka admin and user creating topics
differ. I think a more realistic example of Dan's point (2) is for
retention. I know that realistically, admins aren't just going to randomly
drop the broker defaults from 1w to 1d without warning anyone (they'd
likely be fired...). But as a user, I may not know the broker configs, if
admins have overridden them, etc. I may want a *minimum* of, e.g., 2d. But
if the broker defaults are higher such that the admins are confident the
cluster can handle 1w, I'd rather just fall back on the default value.

Now, there's arguably a better solution for that case -- allow topic
configs to express a *minimum* value (or maximum depending on the
particular config), with the broker config taking precedence if it has a
smaller value (or larger in the case of maximums). This lets you express
your minimum requirements but allows the cluster to do more if that's the
default. However, that would represent a much more significant and invasive
change, and honestly I think it is more likely to confuse users.

@Dan, regarding compatibility, this changes behavior without revving the
request version number, which normally we only do for things that are
reasonably considered bugfixes or were it has no compatibility
implications. In this case, older brokers talking to newer AdminClients
will presumably return some error. Do we know what the non-null assertion
gets converted to and if we're happy with the behavior (i.e. will
applications be able to do something reasonable, distinguish it from some
completely unrelated error, etc)? Similarly, it's obviously only one
implementation using the KIP-4 APIs, but do we know what client-side
validation AdminClient is already doing and whether old AdminClients
talking to new brokers will see a change in behavior (or do they do
client-side validation such that old clients simply wouldn't have access to
this new functionality)?

-Ewen

On Mon, Dec 11, 2017 at 2:11 PM, dan  wrote:

> Dong,
>
> I agree that it *may* be better for a user to be explicit, however there
> are a couple reasons they may not.
> 1) a user doesn't even know what the options are. imagine writing a tool
> for users to create topics that steps them through things:
>
> $ kafka-topics.sh --create
> Give your topic a name: my-fav-topic
> How many partitions do you want [12]:
> What is the minimum in set replica size [2]:
> What is the maximum message size [1MB]:
> ...
>
> 2) a user wants to use broker defaults within reason. say they thinke they
> want min.cleanable.dirty.ratio=.5 and the default is .6. maybe thats fine,
> or even better for them. since the person maintaining the actual cluster
> has put thought in to this config. and as the maintainer keeps working on
> making the cluster run better they can change and tune things on the
> cluster level as needed.
>
> dan
>
>
> On Wed, Dec 6, 2017 at 11:51 AM, Dong Lin  wrote:
>
> > Hey Dan,
> >
> > I think you are saying that, if user can read the default config before
> > creating the topic, then this user can make better decision in what
> configs
> > need to be overwritten. The question here is, how user is going to use
> this
> > config to make the decision.
> >
> > In my understanding, user will compare the default value with expected
> > value, and override the config to be expected value if they are
> different.
> > If this is the only way that the default value can affect user's
> decision,
> > then it seems OK for user to directly override the config to the expected
> > value. I am wondering if this solution has some drawback.
> >
> > On the other hand, maybe there is a more advanced way that the default
> > value can affect the user's decision. It may be useful to understand this
> > use-case more specifically. Could you help provide a specific example
> here?
> >
> > Thanks,
> > Dong
> >
> >
> > On Wed, Dec 6, 2017 at 11:12 AM, dan  wrote:
> >
> > > Rajini,
> > >
> > > that was not my intent, the intent was to give a user of this api an
> > > insight in to what their topic will look like once created. as things
> > stand
> > > now a user is unable to (easily) have any knowledge of what their topic
> > > configs will be before doing a `CREATE_TOPICS`. as i mentioned in the
> > KIP,
> > > another option would be to have the `CreateTopicsOptions.
> > > validateOnly=true`
> > > version return data, but seems more invasive/questionable.
> > >
> > > dan
> > >
> > > On Wed, Dec 6, 2017 at 5:10 AM, Rajini Sivaram <
> rajinisiva...@gmail.com>
> > > wrote:
> > >
> > > > Hi Dan,
> > > >
> > > > Thank you for the KIP. KIP-226 (https://cwiki.apache.org/
> > > > confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration)
> > > proposes
> > > > to add an option to include all synonyms of a config option when
> > > describing
> > > > configs. This includes any defaults. For example (using Dong's
> > example),
> > > if
> > > > you have 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

Dong,

I agree that it *may* be better for a user to be explicit, however there
are a couple reasons they may not.
1) a user doesn't even know what the options are. imagine writing a tool
for users to create topics that steps them through things:

$ kafka-topics.sh --create
Give your topic a name: my-fav-topic
How many partitions do you want [12]:
What is the minimum in set replica size [2]:
What is the maximum message size [1MB]:
...

2) a user wants to use broker defaults within reason. say they thinke they
want min.cleanable.dirty.ratio=.5 and the default is .6. maybe thats fine,
or even better for them. since the person maintaining the actual cluster
has put thought in to this config. and as the maintainer keeps working on
making the cluster run better they can change and tune things on the
cluster level as needed.

dan


On Wed, Dec 6, 2017 at 11:51 AM, Dong Lin  wrote:

> Hey Dan,
>
> I think you are saying that, if user can read the default config before
> creating the topic, then this user can make better decision in what configs
> need to be overwritten. The question here is, how user is going to use this
> config to make the decision.
>
> In my understanding, user will compare the default value with expected
> value, and override the config to be expected value if they are different.
> If this is the only way that the default value can affect user's decision,
> then it seems OK for user to directly override the config to the expected
> value. I am wondering if this solution has some drawback.
>
> On the other hand, maybe there is a more advanced way that the default
> value can affect the user's decision. It may be useful to understand this
> use-case more specifically. Could you help provide a specific example here?
>
> Thanks,
> Dong
>
>
> On Wed, Dec 6, 2017 at 11:12 AM, dan  wrote:
>
> > Rajini,
> >
> > that was not my intent, the intent was to give a user of this api an
> > insight in to what their topic will look like once created. as things
> stand
> > now a user is unable to (easily) have any knowledge of what their topic
> > configs will be before doing a `CREATE_TOPICS`. as i mentioned in the
> KIP,
> > another option would be to have the `CreateTopicsOptions.
> > validateOnly=true`
> > version return data, but seems more invasive/questionable.
> >
> > dan
> >
> > On Wed, Dec 6, 2017 at 5:10 AM, Rajini Sivaram 
> > wrote:
> >
> > > Hi Dan,
> > >
> > > Thank you for the KIP. KIP-226 (https://cwiki.apache.org/
> > > confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration)
> > proposes
> > > to add an option to include all synonyms of a config option when
> > describing
> > > configs. This includes any defaults. For example (using Dong's
> example),
> > if
> > > you have topicA with min.cleanable.dirty.ratio=0.6 as an override and
> the
> > > broker default is 0.5, AdminClient#describeConfigs with synonyms would
> > > return the actual value in use as the config value (I,e.
> > > min.cleanable.dirty.ratio=0.6). And the synonyms list would contain
> (in
> > > the
> > > order of precedence in which these configs are applied):
> > >
> > >1. min.cleanable.dirty.ratio=0.6, config source=TOPIC_CONFIG
> > >2. log.min.cleanable.dirty.ratio=0.5, config
> > > source=STATIC_BROKER_CONFIG
> > >
> > >
> > > KIP-226 doesn't give you exactly what you are proposing in this KIP,
> but
> > it
> > > gives the mapping of configs. My concern with this KIP is that it
> assumes
> > > that if the broker config is static, i.e. if the current value of
> > > log.min.cleanable.dirty.ratio=0.6, you can safely create a topic with
> > > default min.cleanable.dirty.ratio relying on that the value to be
> applied
> > > all the time. This will not work with dynamic broker configs if the
> > broker
> > > defaults can be updated at any time.
> > >
> > >
> > > On Mon, Dec 4, 2017 at 11:22 PM, dan  wrote:
> > >
> > > > for point 1 i agree, its not too strong. only addition i could come
> up
> > > with
> > > > is that it allows any utility to have better forwards compatability.
> a
> > > cli
> > > > written that can inspect how a topic will be created would be able to
> > > give
> > > > insight/expectations about configs that didn't exist at compilation
> > time.
> > > >
> > > > for point 2, i am thinking about a situation where the user creating
> > > topics
> > > > and the user managing brokers are different people with different
> > > > skills/abilities.
> > > >
> > > > so in the given example lets assume a user creating the topic does
> not
> > > > *really* understand what that config means, so they are likely to
> just
> > > > leave it as default. and are happy for their admin to change these on
> > the
> > > > broker as needed.
> > > >
> > > > but say we have another user who is creating a topic who has much
> more
> > > > experience and has done testing, they will be able to determine what
> > the
> > > > value is 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Dong Lin]

Hey Dan,

I think you are saying that, if user can read the default config before
creating the topic, then this user can make better decision in what configs
need to be overwritten. The question here is, how user is going to use this
config to make the decision.

In my understanding, user will compare the default value with expected
value, and override the config to be expected value if they are different.
If this is the only way that the default value can affect user's decision,
then it seems OK for user to directly override the config to the expected
value. I am wondering if this solution has some drawback.

On the other hand, maybe there is a more advanced way that the default
value can affect the user's decision. It may be useful to understand this
use-case more specifically. Could you help provide a specific example here?

Thanks,
Dong


On Wed, Dec 6, 2017 at 11:12 AM, dan  wrote:

> Rajini,
>
> that was not my intent, the intent was to give a user of this api an
> insight in to what their topic will look like once created. as things stand
> now a user is unable to (easily) have any knowledge of what their topic
> configs will be before doing a `CREATE_TOPICS`. as i mentioned in the KIP,
> another option would be to have the `CreateTopicsOptions.
> validateOnly=true`
> version return data, but seems more invasive/questionable.
>
> dan
>
> On Wed, Dec 6, 2017 at 5:10 AM, Rajini Sivaram 
> wrote:
>
> > Hi Dan,
> >
> > Thank you for the KIP. KIP-226 (https://cwiki.apache.org/
> > confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration)
> proposes
> > to add an option to include all synonyms of a config option when
> describing
> > configs. This includes any defaults. For example (using Dong's example),
> if
> > you have topicA with min.cleanable.dirty.ratio=0.6 as an override and the
> > broker default is 0.5, AdminClient#describeConfigs with synonyms would
> > return the actual value in use as the config value (I,e.
> > min.cleanable.dirty.ratio=0.6). And the synonyms list would contain (in
> > the
> > order of precedence in which these configs are applied):
> >
> >1. min.cleanable.dirty.ratio=0.6, config source=TOPIC_CONFIG
> >2. log.min.cleanable.dirty.ratio=0.5, config
> > source=STATIC_BROKER_CONFIG
> >
> >
> > KIP-226 doesn't give you exactly what you are proposing in this KIP, but
> it
> > gives the mapping of configs. My concern with this KIP is that it assumes
> > that if the broker config is static, i.e. if the current value of
> > log.min.cleanable.dirty.ratio=0.6, you can safely create a topic with
> > default min.cleanable.dirty.ratio relying on that the value to be applied
> > all the time. This will not work with dynamic broker configs if the
> broker
> > defaults can be updated at any time.
> >
> >
> > On Mon, Dec 4, 2017 at 11:22 PM, dan  wrote:
> >
> > > for point 1 i agree, its not too strong. only addition i could come up
> > with
> > > is that it allows any utility to have better forwards compatability. a
> > cli
> > > written that can inspect how a topic will be created would be able to
> > give
> > > insight/expectations about configs that didn't exist at compilation
> time.
> > >
> > > for point 2, i am thinking about a situation where the user creating
> > topics
> > > and the user managing brokers are different people with different
> > > skills/abilities.
> > >
> > > so in the given example lets assume a user creating the topic does not
> > > *really* understand what that config means, so they are likely to just
> > > leave it as default. and are happy for their admin to change these on
> the
> > > broker as needed.
> > >
> > > but say we have another user who is creating a topic who has much more
> > > experience and has done testing, they will be able to determine what
> the
> > > value is on the cluster and check to see if it matches expectations or
> > > needs to be set. possibly if this is set to something incorrect for
> their
> > > usecase they will have a reason to verify and speak with the admin
> about
> > > their usecase.
> > >
> > >
> > > moreover, i think without this added capability it makes it
> > > difficult/impossible to accurately use broker defaults for topics.
> right
> > > now a user is left to either decide configs a priori and lose this
> > > functionality, or guess/check what they need to set and end in a
> possibly
> > > bad situation until they can get their *live* topic configured.
> > >
> > > dan
> > >
> > > On Mon, Dec 4, 2017 at 2:50 PM, Dong Lin  wrote:
> > >
> > > > Hey Dan,
> > > >
> > > > Thanks again for the update:)
> > > >
> > > > I am not sure I fully understand the points (1) and (2) in the
> "Always
> > > > Configure ALL Configs For a Topic". In my previous question, I don't
> > mean
> > > > that users should specify full list of topics configs. I mean that
> user
> > > can
> > > > specify the full list of topic configs he/she 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

Rajini,

that was not my intent, the intent was to give a user of this api an
insight in to what their topic will look like once created. as things stand
now a user is unable to (easily) have any knowledge of what their topic
configs will be before doing a `CREATE_TOPICS`. as i mentioned in the KIP,
another option would be to have the `CreateTopicsOptions.validateOnly=true`
version return data, but seems more invasive/questionable.

dan

On Wed, Dec 6, 2017 at 5:10 AM, Rajini Sivaram 
wrote:

> Hi Dan,
>
> Thank you for the KIP. KIP-226 (https://cwiki.apache.org/
> confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration) proposes
> to add an option to include all synonyms of a config option when describing
> configs. This includes any defaults. For example (using Dong's example), if
> you have topicA with min.cleanable.dirty.ratio=0.6 as an override and the
> broker default is 0.5, AdminClient#describeConfigs with synonyms would
> return the actual value in use as the config value (I,e.
> min.cleanable.dirty.ratio=0.6). And the synonyms list would contain (in
> the
> order of precedence in which these configs are applied):
>
>1. min.cleanable.dirty.ratio=0.6, config source=TOPIC_CONFIG
>2. log.min.cleanable.dirty.ratio=0.5, config
> source=STATIC_BROKER_CONFIG
>
>
> KIP-226 doesn't give you exactly what you are proposing in this KIP, but it
> gives the mapping of configs. My concern with this KIP is that it assumes
> that if the broker config is static, i.e. if the current value of
> log.min.cleanable.dirty.ratio=0.6, you can safely create a topic with
> default min.cleanable.dirty.ratio relying on that the value to be applied
> all the time. This will not work with dynamic broker configs if the broker
> defaults can be updated at any time.
>
>
> On Mon, Dec 4, 2017 at 11:22 PM, dan  wrote:
>
> > for point 1 i agree, its not too strong. only addition i could come up
> with
> > is that it allows any utility to have better forwards compatability. a
> cli
> > written that can inspect how a topic will be created would be able to
> give
> > insight/expectations about configs that didn't exist at compilation time.
> >
> > for point 2, i am thinking about a situation where the user creating
> topics
> > and the user managing brokers are different people with different
> > skills/abilities.
> >
> > so in the given example lets assume a user creating the topic does not
> > *really* understand what that config means, so they are likely to just
> > leave it as default. and are happy for their admin to change these on the
> > broker as needed.
> >
> > but say we have another user who is creating a topic who has much more
> > experience and has done testing, they will be able to determine what the
> > value is on the cluster and check to see if it matches expectations or
> > needs to be set. possibly if this is set to something incorrect for their
> > usecase they will have a reason to verify and speak with the admin about
> > their usecase.
> >
> >
> > moreover, i think without this added capability it makes it
> > difficult/impossible to accurately use broker defaults for topics. right
> > now a user is left to either decide configs a priori and lose this
> > functionality, or guess/check what they need to set and end in a possibly
> > bad situation until they can get their *live* topic configured.
> >
> > dan
> >
> > On Mon, Dec 4, 2017 at 2:50 PM, Dong Lin  wrote:
> >
> > > Hey Dan,
> > >
> > > Thanks again for the update:)
> > >
> > > I am not sure I fully understand the points (1) and (2) in the "Always
> > > Configure ALL Configs For a Topic". In my previous question, I don't
> mean
> > > that users should specify full list of topics configs. I mean that user
> > can
> > > specify the full list of topic configs he/she wants to override.
> > >
> > > Your KIP proposes to allow user to query the default topic configs. In
> my
> > > understanding, users want to know the default configs only if he/she
> > > already has a specific list of (config, value) pairs for which he/she
> > wants
> > > to override. The workflow will be that user queries the default
> configs,
> > > compares the default value with that specific list, and selectively
> > > override the configs whose value is different from the default value.
> > > Therefore, the alternative solution does not suffer the problem (1)
> > because
> > > user needs to know that specific list of (config, value) pair anyway.
> > Does
> > > this make sense?
> > >
> > > Regarding problem (2), I think you are saying that if the user wants to
> > set
> > > the min.cleanable.dirty.ratio to be 0.5 and the default value is
> > currently
> > > set to 0.5, then user would not want to explicitly override the config
> > > during topic creation. The purpose is for the min.cleanable.dirty.ratio
> > of
> > > this topic to be changed to 0.6 if admin change the default
> > > min.cleanable.dirty.ratio 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Rajini Sivaram]

Hi Dan,

Thank you for the KIP. KIP-226 (https://cwiki.apache.org/
confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration) proposes
to add an option to include all synonyms of a config option when describing
configs. This includes any defaults. For example (using Dong's example), if
you have topicA with min.cleanable.dirty.ratio=0.6 as an override and the
broker default is 0.5, AdminClient#describeConfigs with synonyms would
return the actual value in use as the config value (I,e.
min.cleanable.dirty.ratio=0.6). And the synonyms list would contain (in the
order of precedence in which these configs are applied):

   1. min.cleanable.dirty.ratio=0.6, config source=TOPIC_CONFIG
   2. log.min.cleanable.dirty.ratio=0.5, config source=STATIC_BROKER_CONFIG


KIP-226 doesn't give you exactly what you are proposing in this KIP, but it
gives the mapping of configs. My concern with this KIP is that it assumes
that if the broker config is static, i.e. if the current value of
log.min.cleanable.dirty.ratio=0.6, you can safely create a topic with
default min.cleanable.dirty.ratio relying on that the value to be applied
all the time. This will not work with dynamic broker configs if the broker
defaults can be updated at any time.


On Mon, Dec 4, 2017 at 11:22 PM, dan  wrote:

> for point 1 i agree, its not too strong. only addition i could come up with
> is that it allows any utility to have better forwards compatability. a cli
> written that can inspect how a topic will be created would be able to give
> insight/expectations about configs that didn't exist at compilation time.
>
> for point 2, i am thinking about a situation where the user creating topics
> and the user managing brokers are different people with different
> skills/abilities.
>
> so in the given example lets assume a user creating the topic does not
> *really* understand what that config means, so they are likely to just
> leave it as default. and are happy for their admin to change these on the
> broker as needed.
>
> but say we have another user who is creating a topic who has much more
> experience and has done testing, they will be able to determine what the
> value is on the cluster and check to see if it matches expectations or
> needs to be set. possibly if this is set to something incorrect for their
> usecase they will have a reason to verify and speak with the admin about
> their usecase.
>
>
> moreover, i think without this added capability it makes it
> difficult/impossible to accurately use broker defaults for topics. right
> now a user is left to either decide configs a priori and lose this
> functionality, or guess/check what they need to set and end in a possibly
> bad situation until they can get their *live* topic configured.
>
> dan
>
> On Mon, Dec 4, 2017 at 2:50 PM, Dong Lin  wrote:
>
> > Hey Dan,
> >
> > Thanks again for the update:)
> >
> > I am not sure I fully understand the points (1) and (2) in the "Always
> > Configure ALL Configs For a Topic". In my previous question, I don't mean
> > that users should specify full list of topics configs. I mean that user
> can
> > specify the full list of topic configs he/she wants to override.
> >
> > Your KIP proposes to allow user to query the default topic configs. In my
> > understanding, users want to know the default configs only if he/she
> > already has a specific list of (config, value) pairs for which he/she
> wants
> > to override. The workflow will be that user queries the default configs,
> > compares the default value with that specific list, and selectively
> > override the configs whose value is different from the default value.
> > Therefore, the alternative solution does not suffer the problem (1)
> because
> > user needs to know that specific list of (config, value) pair anyway.
> Does
> > this make sense?
> >
> > Regarding problem (2), I think you are saying that if the user wants to
> set
> > the min.cleanable.dirty.ratio to be 0.5 and the default value is
> currently
> > set to 0.5, then user would not want to explicitly override the config
> > during topic creation. The purpose is for the min.cleanable.dirty.ratio
> of
> > this topic to be changed to 0.6 if admin change the default
> > min.cleanable.dirty.ratio to 0.6 in the future.
> >
> > But I am not sure this is a reasonable example. If user wants to
> > compare default value of min.cleanable.dirty.ratio with its expected
> value
> > 0.5, then it seems reasonable for user to override it to be 0.5 during
> > topic creation if the default value is currently 0.6. The question is,
> why
> > would user not want to override the default value to be 0.5 if the
> default
> > value is changed from 0.5 to 0.6 later?
> >
> > Thanks,
> > Dong
> >
> >
> > On Mon, Dec 4, 2017 at 2:36 PM, dan  wrote:
> >
> > > updated again :)
> > >
> > > by having users always set all configs you lose the ability to use the
> > > broker defaults as intended, since topic 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

for point 1 i agree, its not too strong. only addition i could come up with
is that it allows any utility to have better forwards compatability. a cli
written that can inspect how a topic will be created would be able to give
insight/expectations about configs that didn't exist at compilation time.

for point 2, i am thinking about a situation where the user creating topics
and the user managing brokers are different people with different
skills/abilities.

so in the given example lets assume a user creating the topic does not
*really* understand what that config means, so they are likely to just
leave it as default. and are happy for their admin to change these on the
broker as needed.

but say we have another user who is creating a topic who has much more
experience and has done testing, they will be able to determine what the
value is on the cluster and check to see if it matches expectations or
needs to be set. possibly if this is set to something incorrect for their
usecase they will have a reason to verify and speak with the admin about
their usecase.


moreover, i think without this added capability it makes it
difficult/impossible to accurately use broker defaults for topics. right
now a user is left to either decide configs a priori and lose this
functionality, or guess/check what they need to set and end in a possibly
bad situation until they can get their *live* topic configured.

dan

On Mon, Dec 4, 2017 at 2:50 PM, Dong Lin  wrote:

> Hey Dan,
>
> Thanks again for the update:)
>
> I am not sure I fully understand the points (1) and (2) in the "Always
> Configure ALL Configs For a Topic". In my previous question, I don't mean
> that users should specify full list of topics configs. I mean that user can
> specify the full list of topic configs he/she wants to override.
>
> Your KIP proposes to allow user to query the default topic configs. In my
> understanding, users want to know the default configs only if he/she
> already has a specific list of (config, value) pairs for which he/she wants
> to override. The workflow will be that user queries the default configs,
> compares the default value with that specific list, and selectively
> override the configs whose value is different from the default value.
> Therefore, the alternative solution does not suffer the problem (1) because
> user needs to know that specific list of (config, value) pair anyway. Does
> this make sense?
>
> Regarding problem (2), I think you are saying that if the user wants to set
> the min.cleanable.dirty.ratio to be 0.5 and the default value is currently
> set to 0.5, then user would not want to explicitly override the config
> during topic creation. The purpose is for the min.cleanable.dirty.ratio of
> this topic to be changed to 0.6 if admin change the default
> min.cleanable.dirty.ratio to 0.6 in the future.
>
> But I am not sure this is a reasonable example. If user wants to
> compare default value of min.cleanable.dirty.ratio with its expected value
> 0.5, then it seems reasonable for user to override it to be 0.5 during
> topic creation if the default value is currently 0.6. The question is, why
> would user not want to override the default value to be 0.5 if the default
> value is changed from 0.5 to 0.6 later?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 2:36 PM, dan  wrote:
>
> > updated again :)
> >
> > by having users always set all configs you lose the ability to use the
> > broker defaults as intended, since topic configs are overlaid. example in
> > the kip doc.
> >
> > dan
> >
> > On Mon, Dec 4, 2017 at 11:47 AM, Dong Lin  wrote:
> >
> > > Hey Dan,
> > >
> > > Thanks for the update. I just want to push the discussion a bit
> further.
> > > Another alternative, which currently is not described in the KIP, is
> for
> > > user to always create the topic with the full list of configs it may
> want
> > > to override. Can you help me understand what is the drawback of this
> > > approach?
> > >
> > > Thanks,
> > > Dong
> > >
> > > On Mon, Dec 4, 2017 at 11:30 AM, dan  wrote:
> > >
> > > > Dong,
> > > >
> > > > i added a section on current state and workarounds along with my
> > > arguments
> > > > for why they are less than optimal to the wiki. but the jist of it is
> > you
> > > > can end up with messages in your topic in an incorrect/invalid state
> if
> > > you
> > > > do this.
> > > >
> > > > thanks,
> > > > dan
> > > >
> > > > On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin 
> wrote:
> > > >
> > > > > Hey Dan,
> > > > >
> > > > > Thanks for the KIP. Can you help me understand the motivation by
> > > > providing
> > > > > a use-case that can not be easily completed without this KIP?
> > > > >
> > > > > It seems that most users will simply create the topic without
> > worrying
> > > > > about the default configs. If a user has specific requirement for
> the
> > > > > default configs, he/she can use 

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Dong Lin]

Hey Dan,

Thanks again for the update:)

I am not sure I fully understand the points (1) and (2) in the "Always
Configure ALL Configs For a Topic". In my previous question, I don't mean
that users should specify full list of topics configs. I mean that user can
specify the full list of topic configs he/she wants to override.

Your KIP proposes to allow user to query the default topic configs. In my
understanding, users want to know the default configs only if he/she
already has a specific list of (config, value) pairs for which he/she wants
to override. The workflow will be that user queries the default configs,
compares the default value with that specific list, and selectively
override the configs whose value is different from the default value.
Therefore, the alternative solution does not suffer the problem (1) because
user needs to know that specific list of (config, value) pair anyway. Does
this make sense?

Regarding problem (2), I think you are saying that if the user wants to set
the min.cleanable.dirty.ratio to be 0.5 and the default value is currently
set to 0.5, then user would not want to explicitly override the config
during topic creation. The purpose is for the min.cleanable.dirty.ratio of
this topic to be changed to 0.6 if admin change the default
min.cleanable.dirty.ratio to 0.6 in the future.

But I am not sure this is a reasonable example. If user wants to
compare default value of min.cleanable.dirty.ratio with its expected value
0.5, then it seems reasonable for user to override it to be 0.5 during
topic creation if the default value is currently 0.6. The question is, why
would user not want to override the default value to be 0.5 if the default
value is changed from 0.5 to 0.6 later?

Thanks,
Dong


On Mon, Dec 4, 2017 at 2:36 PM, dan  wrote:

> updated again :)
>
> by having users always set all configs you lose the ability to use the
> broker defaults as intended, since topic configs are overlaid. example in
> the kip doc.
>
> dan
>
> On Mon, Dec 4, 2017 at 11:47 AM, Dong Lin  wrote:
>
> > Hey Dan,
> >
> > Thanks for the update. I just want to push the discussion a bit further.
> > Another alternative, which currently is not described in the KIP, is for
> > user to always create the topic with the full list of configs it may want
> > to override. Can you help me understand what is the drawback of this
> > approach?
> >
> > Thanks,
> > Dong
> >
> > On Mon, Dec 4, 2017 at 11:30 AM, dan  wrote:
> >
> > > Dong,
> > >
> > > i added a section on current state and workarounds along with my
> > arguments
> > > for why they are less than optimal to the wiki. but the jist of it is
> you
> > > can end up with messages in your topic in an incorrect/invalid state if
> > you
> > > do this.
> > >
> > > thanks,
> > > dan
> > >
> > > On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin  wrote:
> > >
> > > > Hey Dan,
> > > >
> > > > Thanks for the KIP. Can you help me understand the motivation by
> > > providing
> > > > a use-case that can not be easily completed without this KIP?
> > > >
> > > > It seems that most users will simply create the topic without
> worrying
> > > > about the default configs. If a user has specific requirement for the
> > > > default configs, he/she can use AdminClient.describeConfigs() and
> > > > AdminClient.alterConfigs() after the topic has been created. This
> seems
> > > to
> > > > work well. Did I miss something?
> > > >
> > > > Thanks,
> > > > Dong
> > > >
> > > >
> > > > On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
> > > >
> > > > > I would like to start a discussion about KIP-234
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> > > > >
> > > > > thanks
> > > > > dan
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

updated again :)

by having users always set all configs you lose the ability to use the
broker defaults as intended, since topic configs are overlaid. example in
the kip doc.

dan

On Mon, Dec 4, 2017 at 11:47 AM, Dong Lin  wrote:

> Hey Dan,
>
> Thanks for the update. I just want to push the discussion a bit further.
> Another alternative, which currently is not described in the KIP, is for
> user to always create the topic with the full list of configs it may want
> to override. Can you help me understand what is the drawback of this
> approach?
>
> Thanks,
> Dong
>
> On Mon, Dec 4, 2017 at 11:30 AM, dan  wrote:
>
> > Dong,
> >
> > i added a section on current state and workarounds along with my
> arguments
> > for why they are less than optimal to the wiki. but the jist of it is you
> > can end up with messages in your topic in an incorrect/invalid state if
> you
> > do this.
> >
> > thanks,
> > dan
> >
> > On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin  wrote:
> >
> > > Hey Dan,
> > >
> > > Thanks for the KIP. Can you help me understand the motivation by
> > providing
> > > a use-case that can not be easily completed without this KIP?
> > >
> > > It seems that most users will simply create the topic without worrying
> > > about the default configs. If a user has specific requirement for the
> > > default configs, he/she can use AdminClient.describeConfigs() and
> > > AdminClient.alterConfigs() after the topic has been created. This seems
> > to
> > > work well. Did I miss something?
> > >
> > > Thanks,
> > > Dong
> > >
> > >
> > > On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
> > >
> > > > I would like to start a discussion about KIP-234
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> > > >
> > > > thanks
> > > > dan
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Dong Lin]

Hey Dan,

Thanks for the update. I just want to push the discussion a bit further.
Another alternative, which currently is not described in the KIP, is for
user to always create the topic with the full list of configs it may want
to override. Can you help me understand what is the drawback of this
approach?

Thanks,
Dong

On Mon, Dec 4, 2017 at 11:30 AM, dan  wrote:

> Dong,
>
> i added a section on current state and workarounds along with my arguments
> for why they are less than optimal to the wiki. but the jist of it is you
> can end up with messages in your topic in an incorrect/invalid state if you
> do this.
>
> thanks,
> dan
>
> On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin  wrote:
>
> > Hey Dan,
> >
> > Thanks for the KIP. Can you help me understand the motivation by
> providing
> > a use-case that can not be easily completed without this KIP?
> >
> > It seems that most users will simply create the topic without worrying
> > about the default configs. If a user has specific requirement for the
> > default configs, he/she can use AdminClient.describeConfigs() and
> > AdminClient.alterConfigs() after the topic has been created. This seems
> to
> > work well. Did I miss something?
> >
> > Thanks,
> > Dong
> >
> >
> > On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
> >
> > > I would like to start a discussion about KIP-234
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> > >
> > > thanks
> > > dan
> > >
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

Dong,

i added a section on current state and workarounds along with my arguments
for why they are less than optimal to the wiki. but the jist of it is you
can end up with messages in your topic in an incorrect/invalid state if you
do this.

thanks,
dan

On Mon, Dec 4, 2017 at 10:53 AM, Dong Lin  wrote:

> Hey Dan,
>
> Thanks for the KIP. Can you help me understand the motivation by providing
> a use-case that can not be easily completed without this KIP?
>
> It seems that most users will simply create the topic without worrying
> about the default configs. If a user has specific requirement for the
> default configs, he/she can use AdminClient.describeConfigs() and
> AdminClient.alterConfigs() after the topic has been created. This seems to
> work well. Did I miss something?
>
> Thanks,
> Dong
>
>
> On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
>
> > I would like to start a discussion about KIP-234
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> >
> > thanks
> > dan
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Dong Lin]

Hey Dan,

Thanks for the KIP. Can you help me understand the motivation by providing
a use-case that can not be easily completed without this KIP?

It seems that most users will simply create the topic without worrying
about the default configs. If a user has specific requirement for the
default configs, he/she can use AdminClient.describeConfigs() and
AdminClient.alterConfigs() after the topic has been created. This seems to
work well. Did I miss something?

Thanks,
Dong


On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:

> I would like to start a discussion about KIP-234
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
>
> thanks
> dan
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

updated to be more explicit

On Mon, Dec 4, 2017 at 9:55 AM, Ted Yu  wrote:

> In the example, resources is derived from ConfigResource.
> It would be clearer if you show the derivation.
>
> Cheers
>
> On Mon, Dec 4, 2017 at 9:51 AM, dan  wrote:
>
> > I added sample usage as well as a WIP commit to the KIP.
> >
> > dan
> >
> > On Mon, Dec 4, 2017 at 9:36 AM, Ted Yu  wrote:
> >
> > > Can you add some code snippet showing how ConfigResource delivers topic
> > > defaults with the proposed change ?
> > >
> > >
> > > Cheers
> > >
> > > On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
> > >
> > > > I would like to start a discussion about KIP-234
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> > > >
> > > > thanks
> > > > dan
> > > >
> > >
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Ted Yu]

In the example, resources is derived from ConfigResource.
It would be clearer if you show the derivation.

Cheers

On Mon, Dec 4, 2017 at 9:51 AM, dan  wrote:

> I added sample usage as well as a WIP commit to the KIP.
>
> dan
>
> On Mon, Dec 4, 2017 at 9:36 AM, Ted Yu  wrote:
>
> > Can you add some code snippet showing how ConfigResource delivers topic
> > defaults with the proposed change ?
> >
> >
> > Cheers
> >
> > On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
> >
> > > I would like to start a discussion about KIP-234
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> > >
> > > thanks
> > > dan
> > >
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[dan]

I added sample usage as well as a WIP commit to the KIP.

dan

On Mon, Dec 4, 2017 at 9:36 AM, Ted Yu  wrote:

> Can you add some code snippet showing how ConfigResource delivers topic
> defaults with the proposed change ?
>
>
> Cheers
>
> On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:
>
> > I would like to start a discussion about KIP-234
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
> >
> > thanks
> > dan
> >
>

Re: [DISCUSS] KIP-234: add support for getting topic defaults from AdminClient

[Ted Yu]

Can you add some code snippet showing how ConfigResource delivers topic
defaults with the proposed change ?


Cheers

On Mon, Dec 4, 2017 at 9:25 AM, dan  wrote:

> I would like to start a discussion about KIP-234
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 234%3A+add+support+for+getting+topic+defaults+from+AdminClient
>
> thanks
> dan
>