, June 11, 2015 1:28 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
Discussion aside, was there any significant material change besides
the additions below? If so, then we can avoid the overhead of another
vote unless
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
Discussion aside, was there any significant material change besides
the additions below? If so, then we can avoid the overhead of another
vote unless someone wants to down-vote these changes.
Joel
From: Ashish Singh [asi...@cloudera.com]
Sent: Friday, May 29, 2015 8:36 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
+1 on discussing this on next KIP hangout. I will update KIP-24
: Friday, May 29, 2015 8:36 AM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
+1 on discussing this on next KIP hangout. I will update KIP-24 before that.
On Fri, May 29, 2015 at 3:40 AM, Andrii Biletskyi
andrii.bilets
: RE: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
I've made two changes to the document:
- Removed the TMR evolution piece since we agreed to retain this.
- Added two new API's to the admin client spec. (Alter and Describe config).
Please review.
Aditya
[gharatmayures...@gmail.com]
Sent: Thursday, May 21, 2015 8:29 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
administrative
operations (Thread 2)
Hi Jun,
Thanks a lot. I get it now.
Point 4) will actually enable clients to who don't want to create
From: Jun Rao [j...@confluent.io]
Sent: Thursday, May 28, 2015 5:32 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
administrative
operations (Thread 2)
There is a reasonable use case of ISR in KAFKA-2225
: Thursday, May 21, 2015 8:29 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
Hi Jun,
Thanks a lot. I get it now.
Point 4) will actually enable clients to who don't want to create a topic
with default partitions
:29 PM
To: dev@kafka.apache.org
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized administrative
operations (Thread 2)
Hi Jun,
Thanks a lot. I get it now.
Point 4) will actually enable clients to who don't want to create a topic
with default partitions, if it does not exist and then can
Hi Jun,
Thanks a lot. I get it now.
Point 4) will actually enable clients to who don't want to create a topic
with default partitions, if it does not exist and then can manually create
the topic with their own configs(#partitions).
Thanks,
Mayuresh
On Thu, May 21, 2015 at 6:16 PM, Jun Rao
For ListTopics, we decided not to add a ListTopics request for now and just
rely on passing in an empty list to TMR. We can revisit this in the future
if it becomes an issue.
Thanks,
Jun
On Wed, May 13, 2015 at 3:31 PM, Joel Koshy jjkosh...@gmail.com wrote:
Just had a few minor questions
Joel,
- DecreasePartitionNotAllowed. Yeah, that's kind of subcase of
InvalidPartitions...
But since client can always request topic metadata and check what exactly is
was wrong with Partitions argument, I think, yes, we can remove
DecreasePartitionNotAllowed
and use InvalidPartitions instead.
Just had a few minor questions before I join the vote thread.
Apologies if these have been discussed:
- Do we need DecreasePartitionsNotAllowed? i.e., can we just return
InvalidPartitions instead?
- AdminClient.listTopics: should we allow listing all partitions? Or
do you intend for the
Guys,
I've updated the wiki to reflect all previously discussed items
(regarding the schema only - this is included to phase 1).
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations
I think we can have the final discussion today (for
The following is a description of some of my concerns on allowing the same
topic multiple times in AlterTopicRequest.
ATP has an array of entries, each corresponding to a topic. We allow
multiple changes to a topic in a single entry. Those changes may fail to
apply independently (e.g., the config
Guys,
A quick summary of our today's meeting.
There were no additional issues/questions. The only item about which
we are not 100% sure is multiple instructions for one topic in one
request case.
It was proposed by Jun to explain reasons behind not allowing users doing
that again
here in mailing
Guys,
It seems that there are no open questions left so prior to our weekly call
let me summarize what I'm going to implement as part of phase one for KIP-4.
1. Add 3 new Wire Protocol requests - Create-, Alter- and DeleteTopicRequest
2. Topic requests are batch requests, errors are returned
Okay, I had some doubts in terms of reassign-partitions case. I was
not sure whether we need ISR to check post condition of partition
reassignment. But I think we can rely on assigned replicas - the workflow
in reassignPartitions is the following:
1. Update AR in ZK with OAR + RAR.
...
10. Update
Yes, to verify if a partition reassignment completes or not, we just need
to make sure AR == RAR. So, we don't need ISR for this. It's probably still
useful to know ISR for monitoring in general though.
Thanks,
Jun
On Mon, Apr 27, 2015 at 4:15 AM, Andrii Biletskyi
andrii.bilets...@stealth.ly
Andrii,
Another thing. We decided not to add the lag info in TMR. To be consistent,
we probably also want to remove ISR from TMR since only the leader knows
it. We can punt on adding any new request from getting ISR. ISR is mostly
useful for monitoring. Currently, one can determine if a replica
Andrii,
Thanks for the update.
For your second point, I agree that if a single AlterTopicRequest can make
multiple changes, there is no need to support the same topic included more
than once in the request.
Now about the semantics in your first question. I was thinking that we can
do the
Jun,
I like your approach to AlterTopicReques semantics! Sounds like
we linearize all request fields to ReplicaAssignment - I will definitely
try this out to ensure there are no other pitfalls.
With regards to multiple instructions in one batch per topic. For me
this sounds reasonable too. We
Guys,
Can we come to some agreement in terms of the second item from
the email above? This blocks me from updating and uploading the
patch. Also the new schedule for the weekly calls doesn't work very
well for me - it's 1 am in my timezone :) - so I'd rather we confirm
everything that is possible
As said above, I spent some time thinking about AlterTopicRequest
semantics and batching.
Firstly, about AlterTopicRequest. Our goal here is to see whether we
can suggest some simple semantics and at the same time let users
change different things in one instruction (hereinafter instruction - is
1. Yes, this will be much easier. Okay, let's add it.
2, Okay. This will differ a little bit from the way currently
kafka-topics.sh handles alter-topic command, but I think it's
a reasonable restriction.
I'll update KIP acordingly to our weekly call.
Thanks,
Andrii Biletskyi
On Mon, Apr 20,
Hi all,
I've updated KIP-4 page to include all previously discussed items such as:
new error codes, merged alter-topic and reassign-partitions requests, added
TMR_V1.
It'd be great if we concentrate on the Errors+Wire Protocol schema and
discuss
any remaining issues today, since first patch will
Hey Andrii, thanks for all the hard work on this, it has come a long way.
A couple questions and comments on this.
For the errors, can we do the following:
1. Remove IllegalArgument from the name, we haven't used that convention
for other errors.
2. Normalize this list with the existing errors.
Guys,
Thank you for your time. A short summary of our discussion.
Answering previous items:
1. 2. I will double check existing error codes to align the list of
errors that needs to be added.
3. We agreed to think again about the batch requests semantics.
The main concern is that users would
1. Yes, lag is probably only going to be useful for the admin client.
However, so is isr. It seems to me that we should get lag and isr from the
same request. I was thinking that we can just extend TMR by changing
replicas from an array of int to an array of (int, lag) pairs. Is that too
1. agreed
2. agree new error
3. having discrete operations for tasks makes sense, combining them is
confusing for users I think. + 1 for let user change only one thing at a
time
4. lets be consistent both to the new code and existing code. lets not
confuse the user but give them the right error
1. For the lags, we can add a new field lags per partition. It will
return for each replica that's not in isr, the replica id and the lag in
messages. Also, if TMR is sent to a non-leader, the response can just
include an empty array for isr and lags.
2. So, we will just return a topic level
Guys,
Thanks for the discussion!
Summary:
1. Q: How KAFKA-1367 (isr is inconsistent in brokers' metadata cache) can
affect implementation?
A: We can fix this issue for the leading broker - ReplicaManager (or
Partition)
component should have accurate isr list, then with
Jun,
404. Great, thanks!
500. If I understand correctly KAFKA-1367 says ISR part of TMR may
be inconsistent. If so, then I believe all admin commands but describeTopic
are not affected. Let me emphasize that it's about AdminClient operations,
not about Wire Protocol requests. What I mean:
To
Andrii,
404. Jay and I chatted a bit. We agreed to leave createTopicRequest as
async for now.
There is another thing.
500. Currently, we have this issue (KAFKA-1367) that the ISR in the
metadata cache can be out of sync. The reason is that ISR is really
maintained at the leader. We can
Hi all,
I wasn't able to send email to our thread (it says we exceeded message size
limit :)).
So I'm starting the new one.
Jun,
Thanks again for the review. Answering your comments:
201. I'm not sure I understand how can we evolve Cluster in backward
compatible way. In my understanding topic
Hi all,
A summary of our discussion:
201. Q: Cluster updates in backward compatible way.
A: Add topicConfigs map property and change constructor, this
shouldn't break Consumer/Producer since TMR is used in NetworkClient,
not directly by Consumer/Producer.
300. Q: Can we merge AlterTopic
Hm, actually the ticket you linked, Guozhang, brings as back
to the problem what should be considered a post-condition for
each of the admin commands.
In my understanding:
1) CreateTopic - broker created /brokers/topics/topic
(Not the controller picked up changes from zk and broadcasted
Andrii,
I think you are right. It seems that only ReassignPartitions needs a
separate verification request.
Thanks,
Jun
On Thu, Mar 19, 2015 at 9:22 AM, Andrii Biletskyi
andrii.bilets...@stealth.ly wrote:
Guys,
I like this idea too. Let's stick with that. I'll update KIP accordingly.
I
Great.
I want to elaborate this a bit more, to see we are on the same page
concerning the client code.
So with all topic commands being async a client (AdminClient in our
case or any other other client people would like to implement) to support
a blocking operation (which seems to be a natural
I think while loop is fine for supporting blocking, just that we need to
add back off to avoid bombarding brokers with DescribeTopic requests.
Also I have linked KAFKA-1125
https://issues.apache.org/jira/browse/KAFKA-1125 to your proposal, and
when KAFKA-1694 is done this ticket can also be
For 1), 2) and 3), blocking would probably mean that the new metadata is
propagated to every broker. To achieve that, the client can keep issuing
the describe topic request to every broker until it sees the new metadata
in the response.
Thanks,
Jun
On Fri, Mar 20, 2015 at 12:16 PM, Andrii
Jun,
I see your point. But wouldn't that lead to a fat client implementations?
Suppose someone would like to implement client for Admin Wire protocol.
Not only people will have to code quite complicated logic like send
describe
request to each broker (again state machin?) but it will also mean
Andrii,
A few points.
1. Create/Alter can typically complete quickly. So, it's possible to make
the request block until it's completed. However, currently, doing this at
the broker is a bit involved. To make Create block, we will need to add
some callbacks in KafkaController. This is possible.
+1 on broker writing to ZK for async handling. I was thinking that in the
end state the admin requests would be eventually sent to controller either
through re-routing or clients discovering them, instead of letting
controller listen on ZK admin path. But thinking about it a second time, I
think
Andri,
Thanks for the summary.
1. I just realized that in order to start working on KAFKA-1927, we will
need to merge the changes to OffsetCommitRequest (from 0.8.2) to trunk.
This is planned to be done as part of KAFKA-1634. So, we will need Guozhang
and Joel's help to wrap this up.
2.
On Wed, Mar 18, 2015 at 9:34 AM, Jun Rao j...@confluent.io wrote:
Andri,
Thanks for the summary.
1. I just realized that in order to start working on KAFKA-1927, we will
need to merge the changes to OffsetCommitRequest (from 0.8.2) to trunk.
This is planned to be done as part of KAFKA-1634.
Joel,
I'm totally behind your arguments concerning adding irrelevant stuff to
TopicMetadataRequest. And also about having a bloated request.
Personally I'd go with a separate ClusterMetadataRequest (CMR), actually
this was our initial proposal. But since the second part of the request -
brokers
I'm +1 on Jun's suggestion as long as it can work for all the requests.
On Wed, Mar 18, 2015 at 3:35 PM, Jun Rao j...@confluent.io wrote:
Andrii,
I think we agreed on the following.
(a) Admin requests can be sent to and handled by any broker.
(b) Admin requests are processed
Andrii,
I think we agreed on the following.
(a) Admin requests can be sent to and handled by any broker.
(b) Admin requests are processed asynchronously, at least for now. That is,
when the client gets a response, it just means that the request is
initiated, but not necessarily completed. Then,
+1 as well. I think it helps to keep the rerouting approach orthogonal
to this KIP.
On Wed, Mar 18, 2015 at 03:40:48PM -0700, Jay Kreps wrote:
I'm +1 on Jun's suggestion as long as it can work for all the requests.
On Wed, Mar 18, 2015 at 3:35 PM, Jun Rao j...@confluent.io wrote:
Andrii,
Yes that is what I was alluding to when I said that if we finally do
request rerouting in Kafka then the field would add little to no
value. I wasn't sure if we agreed that we _will_ do rerouting or
whether we agreed to evaluate it (KAFKA-1912). Andrii can you update
the KIP with this?
Thanks,
Gwen,
Yes, looks like KAFKA-1927 will leave TopicMetadataRequest/Response.
But I believe, KIP is still tightly related with KAFKA-1927 since we are
not only
going to update TopicMetadataRequest there but we will introduce a bunch
of new requests too. And it probably makes sense to do those
(Thanks Andrii for the summary)
For (1) yes we will circle back on that shortly after syncing up in
person. I think it is close to getting committed although development
for KAFKA-1927 can probably begin without it.
There is one more item we covered at the hangout. i.e., whether we
want to add
Got it. Thanks for clarifying!
On Wed, Mar 18, 2015 at 11:54 AM, Andrii Biletskyi
andrii.bilets...@stealth.ly wrote:
Gwen,
Yes, looks like KAFKA-1927 will leave TopicMetadataRequest/Response.
But I believe, KIP is still tightly related with KAFKA-1927 since we are
not only
going to update
Joel, Andril,
I think we agreed that those admin requests can be issued to any broker.
Because of that, there doesn't seem to be a strong need to know the
controller. So, perhaps we can proceed by not making any change to the
format of TMR right now. When we start using create topic request in
Jun,
I might be wrong but didn't we agree we will let any broker from the
cluster handle *long-running* admin requests (at this time preferredReplica
and
reassignPartitions), via zk admin path. Thus CreateTopics etc should be
sent
only to the controller.
Thanks,
Andrii Biletskyi
On Wed, Mar 18,
I may be missing some context but hopefully this will also be covered
today: I thought the earlier proposal where there was an explicit
ClusterMetadata request was clearer and explicit. During the course of
this thread I think the conclusion was that the main need was for
controller information
Guys,
Thanks for a great discussion!
Here are the actions points:
1. Q: Get rid of all scala requests objects, use java protocol definitions.
A: Gwen kindly took that (KAFKA-1927). It's important to speed up
review procedure
there since this ticket blocks other important changes.
Jun,
101. Okay, if you say that such use case is important. I also think
using clientId for these purposes is fine - if we already have this field
as part of all Wire protocol messages, why not use that.
I will update KIP-4 page if nobody has other ideas (which may come up
during the call today).
Joel,
You are right, I removed ClusterMetadata because we have partially
what we need in TopicMetadata. Also, as Jay pointed out earlier, we
would like to have orthogonal API, but at the same time we need
to be backward compatible.
But I like your idea and even have some other arguments for this
101. There may be a use case where you only want the topics to be created
manually by admins. Currently, you can do that by disabling auto topic
creation and issue topic creation from the TopicCommand. If we disable auto
topic creation completely on the broker and don't have a way to distinguish
Jun,
Answering your questions:
101. If I understand you correctly, you are saying future producer versions
(which
will be ported to TMR_V1) won't be able to automatically create topic (if
we
unconditionally remove topic creation from there). But we need to this
preserve logic.
Ok, about your
Andrii,
101. That's what I was thinking too, but it may not be that simple. In
TopicMetadataRequest_V1,
we can let it not trigger auto topic creation. Then, in the producer side,
if it gets an UnknownTopicException, it can explicitly issue a
createTopicRequest for auto topic creation. On the
Jun,
Thanks for you comments. Answers inline:
100. There are a few fields such as ReplicaAssignment,
ReassignPartitionRequest,
and PartitionsSerialized that are represented as a string, but contain
composite structures in json. Could we flatten them out directly in the
protocol definition as
Folks,
Just want to elaborate a bit more on the create-topic metadata and batching
describe-topic based on config / metadata in my previous email as we work
on KAFKA-1694. The main motivation is to have some sort of topic management
mechanisms, which I think is quite important in a multi-tenant /
---03/12/2015 09:39:50 AM---Folks, Just want to elaborate a bit more
on the create-topic metadata and batching
From: Guozhang Wang wangg...@gmail.com
To: dev@kafka.apache.org dev@kafka.apache.org
Date: 03/12/2015 09:39 AM
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
wangg...@gmail.com
To: dev@kafka.apache.org dev@kafka.apache.org
Date: 03/12/2015 09:39 AM
Subject:Re: [DISCUSS] KIP-4 - Command line and centralized
administrative operations
Folks,
Just want to elaborate a bit more on the create-topic metadata and batching
describe
Andrii,
A few more comments.
100. There are a few fields such as ReplicaAssignment,
ReassignPartitionRequest,
and PartitionsSerialized that are represented as a string, but contain
composite structures in json. Could we flatten them out directly in the
protocol definition as arrays/records?
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
administrative operations
--
Folks,
Just want to elaborate a bit more on the create-topic metadata and
batching
describe-topic based on config / metadata in my previous email as we
work
...@gmail.com
To: dev@kafka.apache.org dev@kafka.apache.org
Date: 03/12/2015 09:39 AM
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
administrative operations
--
Folks,
Just want to elaborate a bit more on the create
@kafka.apache.org
Date: 03/12/2015 09:39 AM
Subject: Re: [DISCUSS] KIP-4 - Command line and centralized
administrative operations
--
Folks,
Just want to elaborate a bit more on the create-topic metadata and
batching
describe-topic based on config / metadata in my
Found KIP-11
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface)
It actually specifies changes to the Metadata protocol, so making sure
both KIPs are consistent in this regard will be good.
On Thu, Mar 12, 2015 at 12:21 PM, Gwen Shapira gshap...@cloudera.com
Specifically for ownership, I think the plan is to add ACL (it sounds
like you are describing ACL) via an external system (Argus, Sentry).
I remember KIP-11 described this, but I can't find the KIP any longer.
Regardless, I think KIP-4 focuses on getting information that already
exists from Kafka
Hi,
As said above - I list again all comments from this thread so we
can see what's left and finalize all pending issues.
Comments from Jay:
1. This is much needed functionality, but there are a lot of the so let's
really think these protocols through. We really want to end up with a set
of well
Gwen,
My main motivation is not authenticate via ownership, but rather query
topics via ownership, and more generally query topics via patterns,
where a pattern could be a config value, metadata k-v pair, etc. Does it
make sense?
Guozhang
On Thu, Mar 12, 2015 at 12:26 PM, Gwen Shapira
Hi all,
Today I uploaded the patch that covers some of the discussed and agreed
items:
- removed MaybeOf optional type
- switched to java protocol definitions
- simplified messages (normalized configs, removed topic marked for
deletion)
I also updated the KIP-4 with respective changes and wrote
Thanks for the updated wiki. A few comments below:
1. Error description in response: I think if some errorCode could indicate
several different error cases then we should really change it to multiple
codes. In general the errorCode itself would be precise and sufficient for
describing the server
It would also be good to think through how we can use those new admin
requests in the producer/consumer client as well. Currently, both the
producer and the consumer use TopicMetadataRequest to obtain the metadata,
which will trigger a topic creation if auto topic creation is enabled. This
is a
Thanks for sending that out Joe - I don't think I will be able to make
it today, so if notes can be sent out afterward that would be great.
On Mon, Mar 02, 2015 at 09:16:13AM -0800, Gwen Shapira wrote:
Thanks for sending this out Joe. Looking forward to chatting with everyone :)
On Mon, Mar
Hey, I just sent out a google hangout invite to all pmc, committers and
everyone I found working on a KIP. If I missed anyone in the invite please
let me know and can update it, np.
We should do this every Tuesday @ 2pm Eastern Time. Maybe we can get INFRA
help to make a google account so we can
Thanks for sending this out Joe. Looking forward to chatting with everyone :)
On Mon, Mar 2, 2015 at 6:46 AM, Joe Stein joe.st...@stealth.ly wrote:
Hey, I just sent out a google hangout invite to all pmc, committers and
everyone I found working on a KIP. If I missed anyone in the invite please
Jay / Joe
We're happy to send out a Webex for this purpose. We could record the
sessions if there is interest and publish them out.
Thanks
Jeff
On Tue, Feb 24, 2015 at 11:28 AM, Jay Kreps jay.kr...@gmail.com wrote:
Let's try to get the technical hang-ups sorted out, though. I really think
Let's stay on Google hangouts that will also record and make the sessions
available on youtube.
-Jay
On Tue, Feb 24, 2015 at 11:49 AM, Jeff Holoman jholo...@cloudera.com
wrote:
Jay / Joe
We're happy to send out a Webex for this purpose. We could record the
sessions if there is interest and
Hi all,
I've updated KIP page, fixed / aligned document structure. Also I added some
very initial proposal for AdminClient so we have something to start from
while
discussing the KIP.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations
Weekly would be great maybe like every Tuesday ~ 1pm ET / 10am PT
I don't mind google hangout but there is always some issue or whatever so
we know the apache irc channel works. We can start there and see how it
goes? We can pull transcripts too and associate to tickets if need be makes
it
Let's try to get the technical hang-ups sorted out, though. I really think
there is some benefit to live discussion vs writing. I am hopeful that if
we post instructions and give ourselves a few attempts we can get it
working.
Tuesday at that time would work for me...any objections?
-Jay
On
We'd talked about doing a Google Hangout to chat about this. What about
generalizing that a little further...I actually think it would be good for
everyone spending a reasonable chunk of their week on Kafka stuff to maybe
sync up once a week. I think we could use time to talk through design
stuff,
Hey Andrii,
Generally we can do good error handling without needing custom server-side
messages. I.e. generally the client has the context to know that if it got
an error that the topic doesn't exist to say Topic X doesn't exist rather
than error code 14 (or whatever). Maybe there are specific
Jay,
Re error messages: you are right, in most cases client will have enough
context to show descriptive error message. My concern is that we will have
to
add lots of new error codes for each possible error. Of course, we could
reuse
some of existing like UknownTopicOrPartitionCode, but we will
Hi all,
I'm trying to address some of the issues which were mentioned earlier about
Admin RQ/RP format. One of those was about batching operations. What if we
follow TopicCommand approach and let people specify topic-name by regexp -
would that cover most of the use cases?
Secondly, is what
@Guozhang:
Thanks for your comments, I've answered some of those. The main thing is
having merged request for create-alter-delete-describe - I have some
concerns about this approach.
@*Jay*:
I see that introduced ClusterMetadaRequest is also one of the concerns. We
can solve it if we implement
For the Sample usage section, please consider
https://github.com/airbnb/kafkat. We find that tool to be very easy to
use, and extremely useful for our administration tasks.
Chi
On Mon, Feb 9, 2015 at 9:03 AM, Guozhang Wang wangg...@gmail.com wrote:
I feel the benefits of lowering the
Hey Andrii,
To answer your earlier question we just really can't be adding any more
scala protocol objects. These things are super hard to maintain because
they hand code the byte parsing and don't have good versioning support.
Since we are already planning on converting we definitely don't want
Hey Jay,
I would like to continue this discussion as it seem there is no progress
here.
First of all, could you please explain what did you mean in 2? How exactly
are we going to migrate to the new java protocol definitions. And why it's
a blocker for centralized CLI?
I agree with you, this
Jay,
Thanks for answering. You understood correctly, most of my comments were
related to your point 1) - about well thought-out apis. Also, yes, as I
understood we would like to introduce a single unified CLI tool with
centralized server-side request handling for lots of existing ones (incl.
Yeah I totally agree that we don't want to just have one do admin stuff
command that has the union of all parameters.
What I am saying is that command line tools are one client of the
administrative apis, but these will be used in a number of scenarios so
they should make logical sense even in
I think for the topics commands we can actually merge
create/alter/delete/describe as one request type since their formats are
very much similar, and keep list-topics and others like
partition-reassignment / preferred-leader-election as separate request
types, I also left some other comments on
I feel the benefits of lowering the development bar for new clients does
not worth the complexity we need to introduce in the server side, as today
the clients just need one more request type (metadata request) to send the
produce / fetch to the right brokers, whereas re-routing mechanism will
I¹m a little bit concerned about the request routers among brokers.
Typically we have a dominant percentage of produce and fetch
request/response. Routing them from one broker to another seems not wanted.
Also I think we generally have two types of requests/responses: data
related and admin
Hey Joe,
I think this is proposing several things:
1. A new command line utility. This isn't really fully specified here.
There is sample usage but I actually don't really understand what all the
commands will be. Also, presumably this will replace the existing shell
scripts, right? We obviously
1 - 100 of 105 matches
Mail list logo
