[jira] [Resolved] (KAFKA-5991) Change Consumer per partition lag metrics to put topic-partition-id in tags instead of metric name

[ https://issues.apache.org/jira/browse/KAFKA-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Lu resolved KAFKA-5991. - Resolution: Fixed > Change Consumer per partition lag metrics to put topic-partition-id in tags >

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

That is a good point.. I cannot think of a better option than documentation and warning, and also given that we'd probably better not reusing the function name `until` for close time. Guozhang On Tue, Jul 10, 2018 at 3:31 PM, John Roesler wrote: > I had some opportunity to reflect on the

Re: [VOTE] 2.0.0 RC2

+1 (non binding) rolling upgrade of tiny shared staging multitenacy (200+ consumer groups) cluster from 1.1 to 2.0.0-rc1 to 2.0.0-rc2. cluster looks healthy after upgrade. Lack of burrow lag suggests consumers are still happy, and incoming messages remains the same. Will monitor. On Wed, Jul 11,

[jira] [Created] (KAFKA-7149) Reduce assignment data size to improve kafka streams scalability

Ashish Surana created KAFKA-7149: Summary: Reduce assignment data size to improve kafka streams scalability Key: KAFKA-7149 URL: https://issues.apache.org/jira/browse/KAFKA-7149 Project: Kafka

[jira] [Created] (KAFKA-7148) Kafka load log very slow after goes down with outOfMemoryError

wang created KAFKA-7148: --- Summary: Kafka load log very slow after goes down with outOfMemoryError Key: KAFKA-7148 URL: https://issues.apache.org/jira/browse/KAFKA-7148 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hey Anna, Thanks for the comment. To answer your question, it seems that we can cover all case in this KIP. As stated in "Consumer Handling" section, KIP-101 based approach will be used to derive the truncation offset from the 2-tuple (offset, leaderEpoch). This approach is best effort and it is

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

I had some opportunity to reflect on the default for close time today... Note that the current "close time" is equal to the retention time, and therefore "close" today shares the default retention of 24h. It would definitely break any application that today specifies a retention time to set

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Sorry, I hit "send" before finishing. Continuing... 2) Hiding most of the consumer handling log truncation logic with minimal exposure in KafkaConsumer API. I was proposing this path. Before answering your specific questions… I want to answer to your comment “In general, maybe we should

[jira] [Resolved] (KAFKA-7141) kafka-consumer-group doesn't describe existing group

[ https://issues.apache.org/jira/browse/KAFKA-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian resolved KAFKA-7141. Resolution: Not A Problem > kafka-consumer-group doesn't describe existing group >

Re: [VOTE] 2.0.0 RC2

+1 Ran thru test suite. Checked signatures. On Tue, Jul 10, 2018 at 10:17 AM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers, > > > This is the third candidate for release of Apache Kafka 2.0.0. > > > This is a major version release of Apache Kafka. It includes 40

Re: Stream config in StreamPartitionAssignor

Sounds good. I will take a look at the doc and get back to you! From: Guozhang Wang Sent: Wednesday, July 11, 2018 2:07 AM To: dev Subject: Re: Stream config in StreamPartitionAssignor Boyang, I see. That makes sense. We should at least update the docs to

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hi Dong, Thanks for the follow up! I finally have much more clear understanding of where you are coming from. You are right. The success of findOffsets()/finding a point of non-divergence depends on whether we have enough entries in the consumer's leader epoch cache. However, I think this is a

[jira] [Created] (KAFKA-7147) ReassignPartitionsCommand should be able to connect to broker over SSL

Dong Lin created KAFKA-7147: --- Summary: ReassignPartitionsCommand should be able to connect to broker over SSL Key: KAFKA-7147 URL: https://issues.apache.org/jira/browse/KAFKA-7147 Project: Kafka

Stream processing meetup at LinkedIn (Sunnyvale) on Thursday, July 19 at 6pm

Hi everyone, We would like to invite you to a Stream Processing Meetup at LinkedIn’s Sunnyvale campus, LinkedIn Building F (LSNF) 605 West Maude Avenuee, on Thursday, July 19 at 6pm. Please RSVP here (if you intend to attend in person):

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hey Anna, Thanks much for your detailed explanation and example! It does help me understand the difference between our understanding. So it seems that the solution based on findOffsets() currently focuses mainly on the scenario that consumer has cached leaderEpoch -> offset mapping whereas I was

Re: [VOTE] KIP-264: Add a consumer metric to record raw fetch size

Bump! From: "Vahid S Hashemian" To: dev Date: 05/25/2018 10:51 AM Subject:[VOTE] KIP-264: Add a consumer metric to record raw fetch size In the absence of additional feedback on this KIP I'd like to start a vote. To summarize, the KIP simply proposes to add a consumer

[jira] [Created] (KAFKA-7146) Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups

Yishun Guan created KAFKA-7146: -- Summary: Grouping consumer requests per consumer coordinator in admin client in describeConsumerGroups Key: KAFKA-7146 URL: https://issues.apache.org/jira/browse/KAFKA-7146

Re: Kafka system tests contribution

Hi Andriy, Try looking at the logs to see why the test failed. best, Colin On Wed, May 16, 2018, at 07:08, Andriy Sorokhtey wrote: > Hi, > > Did anyone had a chance to take a look at this issue? > > 2018-05-08 15:01 GMT+03:00 Andriy Sorokhtey : > > > Hello Kafka team > > > > I’d like to

Re: Stream config in StreamPartitionAssignor

Boyang, I see. That makes sense. We should at least update the docs to reduce confusions. Guozhang On Mon, Jul 9, 2018 at 10:22 PM, Boyang Chen wrote: > Hey Guozhang, > > > thanks for the reply. Actually I'm not trying to pass config into > StreamPartitionAssignor. My debugging was about

[VOTE] 2.0.0 RC2

Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 2.0.0. This is a major version release of Apache Kafka. It includes 40 new KIPs and several critical bug fixes. Please see the 2.0.0 release plan for more details:

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

Waiting for one more binding vote to pass this minor KIP. Appreciate your vote. On Wed, Jul 4, 2018 at 7:03 PM Eno Thereska wrote: > +1 (non binding) > > On Wed, Jul 4, 2018 at 1:19 PM, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > +1 (non-binding) > > > > On Wed, Jul

[jira] [Resolved] (KAFKA-7080) WindowStoreBuilder incorrectly initializes CachingWindowStore

[ https://issues.apache.org/jira/browse/KAFKA-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-7080. - Resolution: Fixed Fixed in https://github.com/apache/kafka/pull/5257 > WindowStoreBuilder

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

Hi Guozhang, That sounds good to me. I'll include that in the KIP. Thanks, -John On Mon, Jul 9, 2018 at 6:33 PM Guozhang Wang wrote: > Let me clarify a bit on what I meant about moving `retentionPeriod` to > WindowStoreBuilder: > > In another discussion we had around KIP-319 / 330, that the

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression (Updated)

I there any update on this. The performance improvements are quite impressive and I really would like to stop forking kafka just to get this in. Thanks, Bobby On Wed, Jun 13, 2018 at 8:56 PM Dongjin Lee wrote: > Ismael, > > Oh, I forgot all of you are on working frenzy for 2.0! No problem,

Re: [DISCUSSION] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

Hi Ismael, Well, yes. If we care about headers only then you'd need to add a dummy implementation for the 2 parameter method as well. Although it is not ideal, we're using the Serializer interface everywhere and convert it to extended with ensureExtended(serializer) and delegate to the 2

[jira] [Created] (KAFKA-7145) Consumer thread getting stuck in hasNext() method

Lovenish goyal created KAFKA-7145: - Summary: Consumer thread getting stuck in hasNext() method Key: KAFKA-7145 URL: https://issues.apache.org/jira/browse/KAFKA-7145 Project: Kafka Issue

Kafka SASL handshake takes too long

Hello everyone, I hope that someone could help me with this issue. I have already posted this on: - StackOverflow: https://stackoverflow.com/questions/51249835/kafka-sasl-handshake-takes-too-long - Confluent Kafka .Net @ Github: https://github.com/confluentinc/confluent-kafka-dotnet/issues/564

Build failed in Jenkins: kafka-trunk-jdk8 #2796

See Changes: [wangguoz] MINOR: Add Scalafmt to Streams Scala API (#4965) -- [...truncated 559.63 KB...] kafka.zk.KafkaZkClientTest > testBrokerSequenceIdMethods PASSED