Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

Very late to the game here, but a few thoughts: 1. Regarding whether KIP is necessary, I don't mind doing it for documentation sake, but I would classify any mishandling of connector names here as a bug. Which doesn't require a KIP to fix. 2. For support of characters, Kafka has some history of

Re: [DISCUSS] KIP-174 - Deprecate and remove internal converter configs in WorkerConfig

Sorry I lost track of this thread. If things are in good shape we should probably vote on this and get the deprecation commit through. It seems like a good idea as this has been confusing to users from day one. -Ewen On Wed, Aug 9, 2017 at 5:18 AM, UMESH CHAUDHARY wrote: >

Re: [VOTE] KIP-208: Add SSL support to Kafka Connect REST interface

Jakub, I left a few comments in the discuss thread, but I'll also reply here just to bump the VOTE thread's visibility. I would like to resolve the few comments I left, but I am effectively +1 on this, the comments I left were mainly details. Committers that could help with the necessary votes

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Jason, Thanks much for all the feedback. I have updated the KIP based on the latest discussion. Can you help check whether it looks good? Thanks, Dong On Thu, Jan 4, 2018 at 5:36 PM, Dong Lin wrote: > Hey Jun, > > Hmm... thinking about this more, I am not sure

[jira] [Created] (KAFKA-6424) QueryableStateIntegrationTest#queryOnRebalance should be accept raw text

Ted Yu created KAFKA-6424: - Summary: QueryableStateIntegrationTest#queryOnRebalance should be accept raw text Key: KAFKA-6424 URL: https://issues.apache.org/jira/browse/KAFKA-6424 Project: Kafka

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Hmm... thinking about this more, I am not sure that the proposed API is sufficient. For users that store offset externally, we probably need extra API to return the leader_epoch and partition_epoch for all partitions that consumers are consuming. I suppose these users currently use

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi, Dong, Yes, that's what I am thinking. OffsetEpoch will be composed of (partition_epoch, leader_epoch). Thanks, Jun On Thu, Jan 4, 2018 at 4:22 PM, Dong Lin wrote: > Hey Jun, > > Thanks much. I like the the new API that you proposed. I am not sure what > you exactly

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Thanks much. I like the the new API that you proposed. I am not sure what you exactly mean by offset_epoch. I suppose that we can use the pair of (partition_epoch, leader_epoch) as the offset_epoch, right? Thanks, Dong On Thu, Jan 4, 2018 at 4:02 PM, Jun Rao wrote:

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

Hi, Rajin, Does providing the old-secret help? My understanding is that the encoded passwd is the result of a 1-way hash with the secret. So, one can't decode the passwd with old-secret. If that's the case, one still needs to provide the unencrypted paaswd to re-encode with the new secret?

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi, Dong, Got it. The api that you proposed works. The question is whether that's the api that we want to have in the long term. My concern is that while the api change is simple, the new api seems harder to explain and use. For example, a consumer storing offsets externally now needs to call

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hey Jakub, Sorry for not getting to this sooner. Overall the proposal looks good to me, I just had a couple of questions. 1. For the configs/overrides, does this happen on a per-setting basis or if one override is included do we not use any of the original settings? I suspect that if you need to

[jira] [Created] (KAFKA-6423) Slow shutdown with many open files

Ivan Babrou created KAFKA-6423: -- Summary: Slow shutdown with many open files Key: KAFKA-6423 URL: https://issues.apache.org/jira/browse/KAFKA-6423 Project: Kafka Issue Type: Bug Affects

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Yeah I agree that ideally we don't want an ever growing global metadata version. I just think it may be more desirable to keep the consumer API simple. In my current proposal, metadata version returned in the fetch response will be stored with the offset together. More specifically, the

Jenkins build is back to normal : kafka-trunk-jdk8 #2310

See

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi, Dong, Not sure that I fully understand your latest suggestion. Returning an ever growing global metadata version itself is no ideal, but is fine. My question is whether the metadata version returned in the fetch response needs to be stored with the offset together if offsets are stored

Jenkins build is back to normal : kafka-trunk-jdk7 #3076

See

Build failed in Jenkins: kafka-trunk-jdk9 #290

See Changes: [jason] MINOR: Fix concurrency bug in MetadataCache and Metadata request when -- [...truncated 1.44 MB...] kafka.javaapi.message.ByteBufferMessageSetTest >

Re: [DISCUSS] KIP-237: More Controller Health Metrics

Thanks for the explanation Ismael. Yeah I think it makes sense that we shouldn't have to rely on class names for metrics. It seems that the current metric framework in Kafka will use the the class name in the metric name -- this logic is hard-coded in KafkaMetricsGroup class. This is probably the

Re: [DISCUSS] KIP-227: Introduce Incremental FetchRequests to Increase Partition Scalability

Hi, Collin, A few more comments on the KIP. 1. About fetch sequence number. My understanding is that the sequence number is really used to provide better fencing for requests coming from different TCP sessions for a given client. Suppose a consumer issues a fetch request and doesn't get a

Re: [DISCUSS] KIP-237: More Controller Health Metrics

Sorry, I thought I had replied to this. Relying on class names for metrics is an anti-pattern in my opinion. It couples metrics too tightly with the implementation and makes it harder to refactor. That was the reason for the suggestion. Ismael On Wed, Dec 13, 2017 at 1:51 AM, Dong Lin

Re: [VOTE] KIP-237: More Controller Health Metrics

Thanks to everyone who voted or provided feedback! This KIP is now adopted with 3 binding +1s (Ewen, Jun, Becket) and 2 non-binding +1s (Manikumar, Mickael) On Thu, Jan 4, 2018 at 6:05 AM, Mickael Maison wrote: > +1 non binding, thanks ! > > On Thu, Jan 4, 2018 at

[jira] [Created] (KAFKA-6422) When enable trace level log in mirror maker, it will throw null pointer exception and the mirror maker will shutdown

Xin Li created KAFKA-6422: - Summary: When enable trace level log in mirror maker, it will throw null pointer exception and the mirror maker will shutdown Key: KAFKA-6422 URL:

Re: [VOTE] KIP-237: More Controller Health Metrics

+1 non binding, thanks ! On Thu, Jan 4, 2018 at 4:38 AM, Becket Qin wrote: > +1. Thanks for the KIP. > > On Wed, Jan 3, 2018 at 9:28 AM, Manikumar wrote: > >> +1 (non-binding) >> >> Thanks >> >> On Wed, Jan 3, 2018 at 10:48 PM, Jun Rao

Re: [VOTE] KIP-226 - Dynamic Broker Configuration

Hi Jun/Jason, I was wondering whether it is worth adding a new property (static config in server.properties) to pass in the previous encoder password as well when changing encoder password. So you would set: - password.encoder.secret=new-password - password.encoder.old.secret=old-password