Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

Thanks, Magesh. All, I've made a few very minor changes to some JavaDocs and the signatures of the name-value pair methods in TopicSettings interface. I also described as a fifth rejected alternative why this KIP does not modify any topic settings for existing topics. All of these are pretty

Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics

Randall, I originally thought that this proposal was a config only topic settings and hence made the comment about configs being pass through. I just realized that the connectors can also override and provide the TopicSettings. With that in mind, I think the proposal looks great. Looking forward

[jira] [Created] (KAFKA-7357) Reassigned partitions but latest data is still going to old broker

Venkatesh created KAFKA-7357: Summary: Reassigned partitions but latest data is still going to old broker Key: KAFKA-7357 URL: https://issues.apache.org/jira/browse/KAFKA-7357 Project: Kafka

[jira] [Created] (KAFKA-7356) Add gradle task for dependency listing

Andy LoPresto created KAFKA-7356: Summary: Add gradle task for dependency listing Key: KAFKA-7356 URL: https://issues.apache.org/jira/browse/KAFKA-7356 Project: Kafka Issue Type: New Feature

Re: [VOTE] KIP-363: Make FunctionConversions private

Hey Joan, It looks like you've updated the KIP to "Accepted", but I only count one binding vote (Guozhang). Ted, Attila, Bill, and myself are all non-binding votes. For reference, these are all folks who hold binding votes: https://kafka.apache.org/committers . Obviously, they don't all take

Re: [VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Hey Nikolay, sorry for the silence. I'm taking another look at the KIP before voting... 1. I think the Window constructor should actually be protected. I don't know if we need a constructor that takes Instant, but if we do add one, it should definitely be protected. 2. `long

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

Hi, because I have made some significant changes on this design, so I want to reopen the discussion on this KIP: https://cwiki.apache.org/confluence/x/CgZPBQ Thanks, Yishun On Thu, Aug 16, 2018 at 5:06 PM Yishun Guan wrote: > > I see! Thanks! > > On Thu, Aug 16, 2018, 4:35 PM Guozhang Wang

Re: Kafka Producer Partition Key Selection

Hello Senthil, In our case we use NULL as message Key to achieve even distribution in producer. With that we were able to achieve very even distribution with that. Our Kafka client version is 0.10.1.0 and Kafka broker version is 1.1 Thanks, Gaurav On Wed, Aug 29, 2018 at 9:15 AM, SenthilKumar

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hi Guozhang By workflow I mean just the overall process of how the KIP is implemented. Any ideas on the ways to reduce the topic count, materializations, if there is a better way to resolve out-of-order than a highwater mark table, if the design philosophy of “keep everything encapsulated

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hi Adam, What do you mean by "additional comments on the workflow.", do you mean to let other review your PR https://github.com/apache/kafka/pull/5527 ? Is is ready for reviews? Guozhang On Tue, Aug 28, 2018 at 5:00 AM, Adam Bellemare wrote: > Okay, I will implement John's suggestion of

[jira] [Reopened] (KAFKA-7295) Fix RequestHandlerAvgIdlePercent metric calculation

[ https://issues.apache.org/jira/browse/KAFKA-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin reopened KAFKA-7295: - > Fix RequestHandlerAvgIdlePercent metric calculation > ---

[jira] [Resolved] (KAFKA-7269) KStream.merge is not documented

[ https://issues.apache.org/jira/browse/KAFKA-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7269. -- Resolution: Fixed Fix Version/s: 2.1.0 > KStream.merge is not documented >

Re: [DISCUSS] KIP-280: Enhanced log compaction

Hi all, Since there has been a rejuvenated interest in this KIP, it felt better to downgrade it back down to [DISCUSSION], as we aren't really voting on it anymore. I'll try to address the currently pending questions on the following points, so please bear with me while we go through them all:

[VOTE] KIP-358: Migrate Streams API to Duration instead of long ms times

Hello, All. Calling a vote on KIP-358 [1] [1] https://cwiki.apache.org/confluence/display/KAFKA/KIP-358%3A+Migrate+Streams+API+to+Duration+instead+of+long+ms+times signature.asc Description: This is a digitally signed message part

[jira] [Resolved] (KAFKA-7295) Fix RequestHandlerAvgIdlePercent metric calculation

[ https://issues.apache.org/jira/browse/KAFKA-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-7295. - Resolution: Fixed The issue is mitigated in https://issues.apache.org/jira/browse/KAFKA-7354 > Fix

[jira] [Resolved] (KAFKA-7354) Fix IdlePercent and NetworkProcessorAvgIdlePercent metric calculation

[ https://issues.apache.org/jira/browse/KAFKA-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Lin resolved KAFKA-7354. - Resolution: Fixed > Fix IdlePercent and NetworkProcessorAvgIdlePercent metric calculation >

Re: Kafka Producer Partition Key Selection

Why can't we override the DefaultPartitioner, and simply override paritition() method, such that it will redistribute to all partitions in round robin fashion. Round-Robin partitioner and StickyAssignor (consumer) should work nicely for any publish subscribe system. On Wed, 29 Aug 2018 at

Re: [VOTE] KIP-275 - Indicate "isClosing" in the SinkTaskContext

Hi Matt I posted in the PR about some of the commenting changes. Overall I think the whole thing looks good, so I would give it a: +1 non-binding Adam On Tue, Aug 28, 2018 at 5:13 PM, Matt Farmer wrote: > Given that voting and discussion have stalled out it seems like this is a > thing that

[jira] [Resolved] (KAFKA-7324) NPE due to lack of SASLExtensions in SASL/OAUTHBEARER

[ https://issues.apache.org/jira/browse/KAFKA-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7324. --- Resolution: Fixed > NPE due to lack of SASLExtensions in SASL/OAUTHBEARER >

[jira] [Created] (KAFKA-7355) Topic Configuration Changes are not applied until reboot

Stephane Maarek created KAFKA-7355: -- Summary: Topic Configuration Changes are not applied until reboot Key: KAFKA-7355 URL: https://issues.apache.org/jira/browse/KAFKA-7355 Project: Kafka

Re: [VOTE] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

+1 (non-binding) Thanks for the KIP. On Wed, Aug 29, 2018 at 1:41 AM Jason Gustafson wrote: > +1 Thanks for the updates. > > On Tue, Aug 28, 2018 at 1:15 AM, Viktor Somogyi-Vass < > viktorsomo...@gmail.com> wrote: > > > Sure, I've added it. I'll also do the testing today. > > > > On Mon, Aug

Re: Kafka Producer Partition Key Selection

Thanks Gaurav. Did you notice side effect mentioned in this page : https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyisdatanotevenlydistributedamongpartitionswhenapartitioningkeyisnotspecified ? --Senthil On Wed, Aug 29, 2018 at 2:02 PM Gaurav Bajaj wrote: > Hello Senthil, > > In

Re: [DISCUSS] KIP-367 Introduce close(Duration) to Producer instead of close(long, TimeUnit)

> Thanks for the KIP. Makes sense to me. Should we make a similar change to > AdminClient? Makes sense to me. Will revise KIP-367 asap -- chia-ping On 2018/08/28 20:13:19, Jason Gustafson wrote: > Thanks for the KIP. Makes sense to me. Should we make a similar change to > AdminClient? > >

Kafka Producer Partition Key Selection

Hello Experts, We want to distribute data across partitions in Kafka Cluster. Option 1 : Use Null Partition Key which can distribute data across paritions. Option 2 : Choose Key ( Random UUID ? ) which can help to distribute data 70-80%. I have seen below side effect on Confluence Page about