Hi Sönke, #2 is a bit tough because people have come to rely on the way the commands are structured right now.
If we want to make big changes, it might be easier just to create a separate tool and deprecate the old one(s). One thing we've talked about doing in the past is creating a single entry point for all the tool functionality, kind of like hadoop did with the "hadoop" command Or git with the "git" command, etc. Then we could deprecate the standalone commands and remove them after enough time had passed-- kind of like the old consumer. On the other hand, a more incremental change would be standardizing flags a bit. So for example, at least setting it up so that there is a standard way of supplying bootstrap brokers, etc. We could keep the old flags around for a while as variants to ease the transition. best, Colin On Sun, May 5, 2019, at 00:54, Sönke Liebau wrote: > Hi Colin, > > I totally agree! Especially the differently named bootstrap server options > have been annoying me a long time. > > I'd propose a two-step approach: > 1. Add new default options objects similar to CommandLineUtils and > CommandDefaultOptions (based on argparse4j) but in the clients project, as > this is referenced by all command line tools as far as I can tell > 2. Refactor tools one by one to use these new helper classes (and thus > argparse) and add standardized synonyms for parameters as necessary > > I think for step 1 we can get away with no KIP, as this doesn't change any > public interfaces or behavior. > Step 2 probably needs a KIP as we are adding new parameters? We can pick up > KIP-14 again for that I think. A lot of work has been done on that already. > > Does that sound useful to everybody? > > Best regards, > Sönke > > > On Thu, Apr 18, 2019 at 1:44 AM Colin McCabe <cmcc...@apache.org> wrote: > > > If we are going to standardize on one argument parsing library, it should > > certainly be argparse4j, I think. > > argparse4j is simply a better argument parsing library with support for > > more features. One example is mutually exclusive options. argparse4j > > supports this with MutuallyExclusiveGroup. jopt doesn't support this, so > > when it is needed, we have to add extra code to manually check that > > mutually exclusive options are not set. > > > > argparse4j also has subcommands. If you want something like "git add" > > with some set of flags, and "git remove" with another, you can do this with > > argparse4j, but not with jopt. This would be very helpful for clearing up > > confusion in a lot of our shell scripts which have accumulated dozens of > > arguments, most of which are only relevant to a very specific operation. > > But you just can't do it with jopt. > > > > Just to give an example, argparse4j with subcommands would allow you to > > run something like ./kafka-topics.sh list --help and get just options that > > were relevant for listing topics, not the full dozens of options that might > > relate to adding topics, removing them, etc. > > > > To be honest, though, what would help users the most is standardizing the > > option flags across tools. We should have a standard way of specifying > > bootstrap brokers, for example. (We can continue to support the old > > synonyms for a while, of course.) > > > > best, > > Colin > > > > > > On Wed, Apr 17, 2019, at 08:56, Guozhang Wang wrote: > > > I took another look at the PR itself and I think it would be great to > > have > > > this cleanup too -- I cannot remember at the beginning why we gradually > > > moved to different mechanism (argparse4j) for different cmds, if there's > > no > > > rationales behind it we should just make them consistent. > > > > > > Thanks for driving this! > > > > > > Guozhang > > > > > > On Wed, Apr 17, 2019 at 7:19 AM Ryanne Dolan <ryannedo...@gmail.com> > > wrote: > > > > > > > Sönke, I'd find this very helpful. It's annoying to keep track of which > > > > commands use which form -- I always seem to guess wrong. > > > > > > > > Though I don't think there is any reason to deprecate existing forms, > > e.g. > > > > consumer.config vs consumer-config. I think it's perfectly reasonable > > to > > > > have multiple spellings of the same arguments. I don't really see a > > > > downside to keeping the aliases around indefinitely. > > > > > > > > Ryanne > > > > > > > > > > > > > > > > > > > > On Wed, Apr 17, 2019, 7:07 AM Sönke Liebau > > > > <soenke.lie...@opencore.com.invalid> wrote: > > > > > > > > > Hi everybody, > > > > > > > > > > Jason and I were recently discussing command line argument parsing on > > > > > KAFKA-8131 (or rather the related pull request) [1]. > > > > > > > > > > Command line tools and their arguments are somewhat diverse at the > > > > moment. > > > > > Most of the tools use joptsimple for argument parsing, some newer > > java > > > > > tools use argparse4j instead and some tools use nothing at all. > > > > > I've looked for a reason as to why there are two libraries being > > used, > > > > but > > > > > couldn't really find anything. Paolo brought up the same question on > > the > > > > > mailing list a while back [7], but got no response either. > > > > > Does anybody know why this is the case? > > > > > > > > > > This results in no central place to add universal parameters like > > help > > > > and > > > > > version, as well as the help output looking different between some > > of the > > > > > tools. > > > > > Also, there are a number of parameters that should be renamed to > > adhere > > > > to > > > > > defaults. > > > > > > > > > > There have been a few discussions and initiatives around this in the > > > > past. > > > > > Just of the top of my head (and a 5 minute jira search) there are: > > > > > - KIP-14 [2] > > > > > - KAFKA-2111 [3] > > > > > - KIP-316 [4] > > > > > - KAFKA-1292 [5] > > > > > - KAFKA-3530 [6] > > > > > - and probably many more > > > > > > > > > > Would people generally be in favor of revisiting this topic? > > > > > > > > > > What I'd propose to do is: > > > > > - comb through jira and KIPs, clean up old stuff and creae a new > > umbrella > > > > > issue to track this (maybe reuse KIP-4 as well) > > > > > - agree on one library for parsing command line arguments (don't care > > > > which > > > > > one, but two is one too many I think) > > > > > - refactor tools to use one library and default way of argument > > parsing > > > > > with central help and version parameter > > > > > - add aliases for options that should be renamed according to KIP-4 > > (and > > > > > maybe others) so that both new and old work for a while, deprecate > > old > > > > > parameters for a cycle or two and then remove them > > > > > > > > > > I'll shut up now and see if people would consider this useful or > > have any > > > > > other input :) > > > > > > > > > > Best regards, > > > > > Sönke > > > > > > > > > > [1] https://github.com/apache/kafka/pull/6481#discussion_r273773003 > > > > > <https://issues.apache.org/jira/browse/KAFKA-8131> > > > > > [2] > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-14+-+Tools+Standardization > > > > > [3] https://issues.apache.org/jira/browse/KAFKA-2111 > > > > > [4] > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-316%3A+Command-line+overrides+for+ConnectDistributed+worker+properties > > > > > [5] https://issues.apache.org/jira/browse/KAFKA-1292 > > > > > [6] https://issues.apache.org/jira/browse/KAFKA-3530 > > > > > [7] > > > > > > > > > > > > > > > > https://sematext.com/opensee/m/Kafka/uyzND10ObP01p77VS?subj=From+Scala+to+Java+based+tools+joptsimple+vs+argparse4j > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > -- > Sönke Liebau > Partner > Tel. +49 179 7940878 > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany >
