On Mon, May 6, 2019, at 10:21, Sönke Liebau wrote:
> Hi Colin,
> 
> it was my intention to keep the structure of the commands mostly intact
> while doing the refactoring - if that is possible, have not really checked
> yet to be honest.
> 
> But what I wanted to try and do is recreate the current parsing with
> argparse as much as possible. And in the process simply adding synonyms,
> for example make the kafka-console-producer understand a
> bootstrap-parameter in addition to broker-list.
> There is a bit of custom logic about which parameters go together etc. in
> the current classes, so output may look different here and there, but in
> principle I do believe that it should be possible to recreate the current
> structure.

Sounds like a good idea.  Thanks for the clarification.

> 
> If there is an appetite for a new, hadoop-like entrypoint anyway, then all
> of this might be "wasted" effort, or rather effort better spent though, you
> are right.

I don't think anyone is working on a new entry point right now -- or if they 
are, they haven't said anything yet :)

I just wanted to mention it as a possible approach in case you wanted to do a 
bigger project.

best,
Colin

> 
> Best regards,
> Sönke
> 
> 
> 
> On Mon, May 6, 2019 at 7:13 PM Colin McCabe <cmcc...@apache.org> wrote:
> 
> > Hi Sönke,
> >
> > #2 is a bit tough because people have come to rely on the way the commands
> > are structured right now.
> >
> > If we want to make big changes, it might be easier just to create a
> > separate tool and deprecate the old one(s).  One thing we've talked about
> > doing in the past is creating a single entry point for all the tool
> > functionality, kind of like hadoop did with the "hadoop" command  Or git
> > with the "git" command, etc.  Then we could deprecate the standalone
> > commands and remove them after enough time had passed-- kind of like the
> > old consumer.
> >
> > On the other hand, a more incremental change would be standardizing flags
> > a bit.  So for example, at least setting it up so that there is a standard
> > way of supplying bootstrap brokers, etc.  We could keep the old flags
> > around for a while as variants to ease the transition.
> >
> > best,
> > Colin
> >
> >
> > On Sun, May 5, 2019, at 00:54, Sönke Liebau wrote:
> > > Hi Colin,
> > >
> > > I totally agree! Especially the differently named bootstrap server
> > options
> > > have been annoying me a long time.
> > >
> > > I'd propose a two-step approach:
> > > 1. Add new default options objects similar to CommandLineUtils and
> > > CommandDefaultOptions (based on argparse4j) but in the clients project,
> > as
> > > this is referenced by all command line tools as far as I can tell
> > > 2. Refactor tools one by one to use these new helper classes (and thus
> > > argparse) and add standardized synonyms for parameters as necessary
> > >
> > > I think for step 1 we can get away with no KIP, as this doesn't change
> > any
> > > public interfaces or behavior.
> > > Step 2 probably needs a KIP as we are adding new parameters? We can pick
> > up
> > > KIP-14 again for that I think. A lot of work has been done on that
> > already.
> > >
> > > Does that sound useful to everybody?
> > >
> > > Best regards,
> > > Sönke
> > >
> > >
> > > On Thu, Apr 18, 2019 at 1:44 AM Colin McCabe <cmcc...@apache.org> wrote:
> > >
> > > > If we are going to standardize on one argument parsing library, it
> > should
> > > > certainly be argparse4j, I think.
> > > >  argparse4j is simply a better argument parsing library with support
> > for
> > > > more features.  One example is mutually exclusive options.  argparse4j
> > > > supports this with MutuallyExclusiveGroup.  jopt doesn't support this,
> > so
> > > > when it is needed, we have to add extra code to manually check that
> > > > mutually exclusive options are not set.
> > > >
> > > > argparse4j also has subcommands.  If you want something like "git add"
> > > > with some set of flags, and "git remove" with another, you can do this
> > with
> > > > argparse4j, but not with jopt.  This would be very helpful for
> > clearing up
> > > > confusion in a lot of our shell scripts which have accumulated dozens
> > of
> > > > arguments, most of which are only relevant to a very specific
> > operation.
> > > > But you just can't do it with jopt.
> > > >
> > > > Just to give an example, argparse4j with subcommands would allow you to
> > > > run something like ./kafka-topics.sh list --help and get just options
> > that
> > > > were relevant for listing topics, not the full dozens of options that
> > might
> > > > relate to adding topics, removing them, etc.
> > > >
> > > > To be honest, though, what would help users the most is standardizing
> > the
> > > > option flags across tools.  We should have a standard way of specifying
> > > > bootstrap brokers, for example.  (We can continue to support the old
> > > > synonyms for a while, of course.)
> > > >
> > > > best,
> > > > Colin
> > > >
> > > >
> > > > On Wed, Apr 17, 2019, at 08:56, Guozhang Wang wrote:
> > > > > I took another look at the PR itself and I think it would be great to
> > > > have
> > > > > this cleanup too -- I cannot remember at the beginning why we
> > gradually
> > > > > moved to different mechanism (argparse4j) for different cmds, if
> > there's
> > > > no
> > > > > rationales behind it we should just make them consistent.
> > > > >
> > > > > Thanks for driving this!
> > > > >
> > > > > Guozhang
> > > > >
> > > > > On Wed, Apr 17, 2019 at 7:19 AM Ryanne Dolan <ryannedo...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Sönke, I'd find this very helpful. It's annoying to keep track of
> > which
> > > > > > commands use which form -- I always seem to guess wrong.
> > > > > >
> > > > > > Though I don't think there is any reason to deprecate existing
> > forms,
> > > > e.g.
> > > > > > consumer.config vs consumer-config. I think it's perfectly
> > reasonable
> > > > to
> > > > > > have multiple spellings of the same arguments. I don't really see a
> > > > > > downside to keeping the aliases around indefinitely.
> > > > > >
> > > > > > Ryanne
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 17, 2019, 7:07 AM Sönke Liebau
> > > > > > <soenke.lie...@opencore.com.invalid> wrote:
> > > > > >
> > > > > > > Hi everybody,
> > > > > > >
> > > > > > > Jason and I were recently discussing command line argument
> > parsing on
> > > > > > > KAFKA-8131 (or rather the related pull request) [1].
> > > > > > >
> > > > > > > Command line tools and their arguments are somewhat diverse at
> > the
> > > > > > moment.
> > > > > > > Most of the tools use joptsimple for argument parsing, some newer
> > > > java
> > > > > > > tools use argparse4j instead and some tools use nothing at all.
> > > > > > > I've looked for a reason as to why there are two libraries being
> > > > used,
> > > > > > but
> > > > > > > couldn't really find anything. Paolo brought up the same
> > question on
> > > > the
> > > > > > > mailing list a while back [7], but got no response either.
> > > > > > > Does anybody know why this is the case?
> > > > > > >
> > > > > > > This results in no central place to add universal parameters like
> > > > help
> > > > > > and
> > > > > > > version, as well as the help output looking different between
> > some
> > > > of the
> > > > > > > tools.
> > > > > > > Also, there are a number of parameters that should be renamed to
> > > > adhere
> > > > > > to
> > > > > > > defaults.
> > > > > > >
> > > > > > > There have been a few discussions and initiatives around this in
> > the
> > > > > > past.
> > > > > > > Just of the top of my head (and a 5 minute jira search) there
> > are:
> > > > > > > - KIP-14 [2]
> > > > > > > - KAFKA-2111 [3]
> > > > > > > - KIP-316 [4]
> > > > > > > - KAFKA-1292 [5]
> > > > > > > - KAFKA-3530 [6]
> > > > > > > - and probably many more
> > > > > > >
> > > > > > > Would people generally be in favor of revisiting this topic?
> > > > > > >
> > > > > > > What I'd propose to do is:
> > > > > > > - comb through jira and KIPs, clean up old stuff and creae a new
> > > > umbrella
> > > > > > > issue to track this  (maybe reuse KIP-4 as well)
> > > > > > > - agree on one library for parsing command line arguments (don't
> > care
> > > > > > which
> > > > > > > one, but two is one too many I think)
> > > > > > > - refactor tools to use one library and default way of argument
> > > > parsing
> > > > > > > with central help and version parameter
> > > > > > > - add aliases for options that should be renamed according to
> > KIP-4
> > > > (and
> > > > > > > maybe others) so that both new and old work for a while,
> > deprecate
> > > > old
> > > > > > > parameters for a cycle or two and then remove them
> > > > > > >
> > > > > > > I'll shut up now and see if people would consider this useful or
> > > > have any
> > > > > > > other input :)
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Sönke
> > > > > > >
> > > > > > > [1]
> > https://github.com/apache/kafka/pull/6481#discussion_r273773003
> > > > > > > <https://issues.apache.org/jira/browse/KAFKA-8131>
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-14+-+Tools+Standardization
> > > > > > > [3] https://issues.apache.org/jira/browse/KAFKA-2111
> > > > > > > [4]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-316%3A+Command-line+overrides+for+ConnectDistributed+worker+properties
> > > > > > > [5] https://issues.apache.org/jira/browse/KAFKA-1292
> > > > > > > [6] https://issues.apache.org/jira/browse/KAFKA-3530
> > > > > > > [7]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > https://sematext.com/opensee/m/Kafka/uyzND10ObP01p77VS?subj=From+Scala+to+Java+based+tools+joptsimple+vs+argparse4j
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sönke Liebau
> > > Partner
> > > Tel. +49 179 7940878
> > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
> > >
> >
> 
> 
> -- 
> Sönke Liebau
> Partner
> Tel. +49 179 7940878
> OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
>

Reply via email to