On Mon, Dec 11, 2017 at 4:01 PM, Gwen Shapira <g...@confluent.io> wrote:
> Thanks, Ewen :) > > One thing that wasn't clear to me from the wiki: Will standalone connect > also have a Kafka cluster ID? While it is true that only tasks have > producers and consumers, I think we assumed that all tasks on one > stand-alone will use one Kafka cluster? > Yeah, maybe not clear enough in the KIP, but this is what I was getting at -- while I think it's possible to use different clusters for worker, producer, and consumer, I don't think this is really expected or a use case worth bending backwards to support perfectly. In standalone mode, technically a value is not required because a default is included and we only utilize the value currently for the producers/consumers in tasks. But I don't think it is unreasonable to require a valid setting at the worker level, even if you override the bootstrap.servers for producer and consumer. > > Another suggestion is not to block the REST API on the connection, but > rather not return the cluster ID until we know it (return null instead). So > clients will need to poll rather than block. Not sure this is better, but > you didn't really discuss this, so wanted to raise the option. > It's mentioned briefly in https://cwiki.apache.org/confluence/display/KAFKA/KIP-238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API#KIP-238:ExposeKafkaclusterIDinConnectRESTAPI-ProposedChanges I think the tradeoff of blocking the server from being "started" until we can at least make one request to the cluster isn't unreasonable since if you can't do that, you're not going to be able to do any useful work anyway. Anyone who might otherwise be using this endpoint to monitor health (which it is useful for since it doesn't require any other external services to be running just to give a response) can just interpret connection refused or timeouts as an unhealthy state, as they should anyway. -Ewen > > Gwen > > > On Mon, Dec 11, 2017 at 3:42 PM Ewen Cheslack-Postava <e...@confluent.io> > wrote: > > > I'd like to start discussion on a simple KIP to expose Kafka cluster ID > > info in the Connect REST API: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 238%3A+Expose+Kafka+cluster+ID+in+Connect+REST+API > > > > Hopefully straightforward, though there are some details on how this > > affects startup behavior that might warrant discussion. > > > > -Ewen > > >
