Re: [DISCUSS] KIP-209 Connection String Support

2017-11-27 Thread Clebert Suconic
Ping...??? :) anyone interested on this feature? anything I can do before sending a vote? Notice.. if this is inside the constructor or a at the factor.. that's an easy change that shouldn't stop any further progress here. On Wed, Nov 15, 2017 at 10:07 PM, Clebert Suconic wrote: > So, just ask

Re: [DISCUSS] KIP-209 Connection String Support

2017-11-15 Thread Clebert Suconic
So, just asking for guidance here... should I go for a vote.. or... if you guys think this should be retired.. that's fine.. I am using my spare time on this.. and I'm about to fill it up that slot soon :) I'm trying to figure out if there is interest on this. On Mon, Nov 13, 2017 at 9:51 AM, Cl

Re: [DISCUSS] KIP-209 Connection String Support

2017-11-13 Thread Clebert Suconic
It would work out. it's just a matter to find what's easier for users... a builder would work equally. I listed the rejected alternative in case the constructor wasn't accepted. I don't think it adds complexity. it's the same quite frankly. refactoring a method out to a builder of factory is a si

Re: [DISCUSS] KIP-209 Connection String Support

2017-11-13 Thread charly molter
Hi Clebert, You mention in rejected alternatives "builders" but you don't give a justification. Could you explain why a utility method or class doesn't work out? There's already 5 constructors in KafkaConsumer and I'm afraid this change would just make the API more complex. Thanks, Charly On M

Re: [DISCUSS] KIP-209 Connection String Support

2017-11-13 Thread Clebert Suconic
I was waiting for more input after my last update.. what should be done now? On Sat, Nov 4, 2017 at 2:19 PM, Clebert Suconic wrote: > I have updated KIP-209. > > > Determined the \", \\ and \; meanings > > Also introduced the possibility of using $ to translate as system properties. > > I"m not

Re: [DISCUSS] KIP-209 Connection String Support

2017-11-04 Thread Clebert Suconic
I have updated KIP-209. Determined the \", \\ and \; meanings Also introduced the possibility of using $ to translate as system properties. I"m not using an BNF formal language here, as I don't think it's needed.. it seems pretty obvious what it would be accomplished here. On Fri, Oct 27, 2017

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-27 Thread Colin McCabe
On Tue, Oct 24, 2017, at 22:51, Michael André Pearce wrote: > Fair enough on URL encoding but as mentioned it is important to be able > to escape, I agree with backslash option. > > I would still like some form of prefix to the string to denote it is for > kafka. I don't think a prefix is necessa

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-25 Thread Clebert Suconic
> > This will cause a lot of compatibility problems, right? If I switch > back and forth between two Kafka versions, they will support slightly > different sets of configuration parameters. It seems saner to simply > ignore configuration parameters that we don't understand, like we do > now. I

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-25 Thread Clebert Suconic
+1... I will update the KIP by the weekend... (I am taking this on my spare time.. although I can rush it if anyone needs it sooner). On Tue, Oct 24, 2017 at 12:27 PM, Colin McCabe wrote: > Hi Clebert, > > As some other people mentioned, a comma is probably not a great choice > for the entry sep

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-24 Thread Michael André Pearce
Fair enough on URL encoding but as mentioned it is important to be able to escape, I agree with backslash option. I would still like some form of prefix to the string to denote it is for kafka. E.g. kafka:: (if semicolon separators) Sent from my iPad > On 24 Oct 2017, at 17:27, Colin McCabe

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-24 Thread Colin McCabe
Hi Clebert, As some other people mentioned, a comma is probably not a great choice for the entry separator. We have a lot of configuration values that already include commas. How about using a semicolon instead? You also need an escaping system in case someone needs a semicolon (or whatever) th

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-19 Thread Michael André Pearce
Just another point to why I’d propose the below change to the string format I propose , is an ability to encode the strings easily. We should note that it’s quite typical for serializers to user a schematic registry where one of their properties they will need to set would be in some form like:

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-19 Thread Michael André Pearce
Hi Clebert Great kip! Instead of ‘;’ to separate the host sections with the params section could it be a ‘?’ And like wise ‘,’ param separator could this be ‘&’ (keep the ‘,’ for host separator just makes easier to distinguish) Also this was it makes it easier to encode params etc as can just

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-19 Thread Clebert Suconic
Do I have to do anything here? I wonder how long I need to wait before proposing the vote. On Tue, Oct 17, 2017 at 1:17 PM, Clebert Suconic wrote: > I had these updates in already... you just changed the names at the > string.. but it was pretty much the same thing I think... I had taken > you s

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-17 Thread Clebert Suconic
I had these updates in already... you just changed the names at the string.. but it was pretty much the same thing I think... I had taken you suggestions though. The Exceptions.. these would be implementation details... all I wanted to make sure is that users would get the name of the invalid par

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-17 Thread Clebert Suconic
I had tweaked that section a bit.. although I though it was clear what the benefit. Since it was a bit obvious I thought describing the feature and the API simplification would been enough.. I am hoping it would be clearer now. On Tue, Oct 17, 2017 at 4:37 AM, Tom Bentley wrote: > Hi Clebert, >

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-17 Thread Tom Bentley
Hi Clebert, The motivation section is written as more of a summary and doesn't really give any motivation for this change. Can you explain why it would be beneficial for Kafka to have this change? For example, if you have use cases where the current way of instantiating a producer, consumer or adm

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-17 Thread Satish Duggana
You may need to update KIP with the details discussed in this thread in proposed changes section. >>My proposed format for the connection string would be: >>IP1:host1,IP2:host2,...IPN:hostn;parameterName=value1;parameterName2=value2;... parameterNameN=valueN Format should be host1:port1,host2:por

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Clebert Suconic
That works. On Mon, Oct 16, 2017 at 6:59 PM Ted Yu wrote: > Can't you use IllegalArgumentException ? > > Some example in current code base: > > clients/src/main/java/org/apache/kafka/clients/Metadata.java: > throw new IllegalArgumentException("Max time to wait for metadata updates > should not

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Ted Yu
Can't you use IllegalArgumentException ? Some example in current code base: clients/src/main/java/org/apache/kafka/clients/Metadata.java: throw new IllegalArgumentException("Max time to wait for metadata updates should not be < 0 milliseconds"); On Mon, Oct 16, 2017 at 3:06 PM, Clebert Suconic

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Clebert Suconic
I updated the wiki with the list on the proposed arguments. I also changed it to include a new Exception class that would be named InvalidParameterException (since I couldn't find an existing Exception class that I could reuse into this). (I could review the name or the exception of course.. just

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Jakub Scholz
Hi Clebert, I think it would be good if this could cover not only KafkaConsumer and KafkaProducer but also the AdminClient. So that all three can be configured the same way. The bootstrap servers are a list - you can provide multiple bootstrap servers. Maybe you add an example of how that will be

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Ted Yu
bq. I was waiting my email to go through the servers http://search-hadoop.com/ indexes mailing lists actively. The delay is very short. FYI On Mon, Oct 16, 2017 at 2:44 PM, Clebert Suconic wrote: > On Mon, Oct 16, 2017 at 5:41 PM, Ted Yu wrote: > > Please update link for Discussion thread and

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Clebert Suconic
On Mon, Oct 16, 2017 at 5:41 PM, Ted Yu wrote: > Please update link for Discussion thread and JIRA sure thing... I was waiting my email to go through the servers... so I could get the link. > > There're two TBD's for Invalid conversion and parameters. Can you fill them > out ? will do.. I was

Re: [DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Ted Yu
Please update link for Discussion thread and JIRA There're two TBD's for Invalid conversion and parameters. Can you fill them out ? Thanks On Mon, Oct 16, 2017 at 2:30 PM, Clebert Suconic wrote: > I would like to start a discussion about KIP-209 > (https://cwiki.apache.org/confluence/display/K

[DISCUSS] KIP-209 Connection String Support

2017-10-16 Thread Clebert Suconic
I would like to start a discussion about KIP-209 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-209+-+Connection+String+Support) This is an extension of my previous thread: http://mail-archives.apache.org/mod_mbox/kafka-dev/201710.mbox/%3cCAKF+bsoFbN13D-u20tUsP6G+aHX4BUNk=s8m4kyjxat_oyv...