Thanks Joe. I will read the wiki page. On Tue, Nov 11, 2014 at 11:47 PM, Joe Stein <joe.st...@stealth.ly> wrote:
> I started writing this up on the wiki > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements > > Instead of starting a new thread I figure just continue this one I started. > I also added another (important) component for centralized management of > configuration as "global level" much like we have topic level. These > global configuration would be overridden (perhaps not all) from the > server.properties on start (so like in case one broker needs a different > port, sure). > > <<One concern I have is that using RQ/RP wire protocol to the > controller instead of the current way (via ZK admin path) may expose > concurrency on the admin requests, which may not be supported yet. > > Guozhang, take a look at the diagram how I am thinking of this it would be > a new handle request that will execute the tools pretty much how they are > today. My thinking is maybe to-do one at a time (so TopicCommand first I > think) and have what the TopicCommand is doing happen on server and send > the RQ/RP to the client but execute on the server. If there is something > not supported we will of course have to deal with that and implement it for > sure. Once we get one working end to end I think adding the rest will be > (more or less) concise iterations to get it done. I added your concern to > the wiki under the "gotchas" section. > > /******************************************* > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********************************************/ > > On Mon, Oct 20, 2014 at 2:15 AM, Guozhang Wang <wangg...@gmail.com> wrote: > > > One concern I have is that using RQ/RP wire protocol to the controller > > instead of the current way (via ZK admin path) may expose concurrency on > > the admin requests, which may not be supported yet. > > > > Some initial discussion about this is on KAFKA-1305. > > > > Guozhang > > > > On Sun, Oct 19, 2014 at 1:55 PM, Joe Stein <joe.st...@stealth.ly> wrote: > > > > > Maybe we should add some AdminMessage RQ/RP wire protocol structure(s) > > and > > > let the controller handle it? We could then build the CLI and Shell in > > the > > > project both as useful tools and samples for others. > > > > > > Making a http interface should be simple after KAFKA-1494 is done which > > all > > > client libraries could offer. > > > > > > I will update the design tonight/tomorrow and should be able to have > > > someone starting to work on it this week. > > > > > > /******************************************* > > > Joe Stein > > > Founder, Principal Consultant > > > Big Data Open Source Security LLC > > > http://www.stealth.ly > > > Twitter: @allthingshadoop > > > ********************************************/ > > > On Oct 19, 2014 1:21 PM, "Harsha" <ka...@harsha.io> wrote: > > > > > > > +1 for Web Api > > > > > > > > On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: > > > > > Apache Karaf has been doing this for quite a few years, albeit in > > Java > > > > > not Scala. Still, their coding approach to creating a CLI probably > > > > > captures many lessons learned over that time. > > > > > > > > > > Glen > > > > > > > > > > On 10/17/2014 08:03 PM, Joe Stein wrote: > > > > > > Hi, I have been thinking about the ease of use for operations > with > > > > Kafka. > > > > > > We have lots of tools doing a lot of different things and they > are > > > all > > > > kind > > > > > > of in different places. > > > > > > > > > > > > So, what I was thinking is to have a single interface for our > > tooling > > > > > > https://issues.apache.org/jira/browse/KAFKA-1694 > > > > > > > > > > > > This would manifest itself in two ways 1) a command line > interface > > 2) > > > > a repl > > > > > > > > > > > > We would have one entry point centrally for all Kafka commands. > > > > > > kafka <CMD> <ARGS> > > > > > > kafka createTopic --brokerList etc, > > > > > > kafka reassignPartition --brokerList etc, > > > > > > > > > > > > or execute and run the shell > > > > > > > > > > > > kafka --brokerList localhost > > > > > > kafka>use topicName; > > > > > > kafka>set acl='label'; > > > > > > > > > > > > I was thinking that all calls would be initialized through > > > > --brokerList and > > > > > > the broker can tell the KafkaCommandTool what server to connect > to > > > for > > > > > > MetaData. > > > > > > > > > > > > Thoughts? Tomatoes? > > > > > > > > > > > > /******************************************* > > > > > > Joe Stein > > > > > > Founder, Principal Consultant > > > > > > Big Data Open Source Security LLC > > > > > > http://www.stealth.ly > > > > > > Twitter: @allthingshadoop < > > http://www.twitter.com/allthingshadoop> > > > > > > ********************************************/ > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
