Re: [DISCUSS] : KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance

Hi all, I am fine with StoreQueryParameters, but then we should also change the DSL grammar accordingly. Since the PR was merged, I suppose everybody agrees on the new name and I changed the DSL grammar. Best, Bruno On Thu, Feb 6, 2020 at 1:07 PM Navinder Brar wrote: > > Hi, > > While implement

Build failed in Jenkins: kafka-2.5-jdk8 #13

See Changes: [wangguoz] KAFKA-9505: Only loop over topics-to-validate in retries (#8039) [wangguoz] KAFKA-9480: Fix bug that prevented to measure task-level process-rate [wangguoz] KAFKA-9523: Migrate BranchedMult

KAFKA-9308: Request for Review of documentation

Hi everybody, I've reworked the SSL part of the documentation a little in order to fix (among other things) KAFKA-9308[1] and would love some feedback if someone can spare a few minutes. Pull request: https://github.com/apache/kafka/pull/8009 [1] https://issues.apache.org/jira/browse/KAFKA-9308

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Sounds perfect. Thanks! -John On Mon, Feb 10, 2020, at 19:18, Sophie Blee-Goldman wrote: > Thanks John. I took out the KafkaConsumer method and moved the javadocs > to the Consumer#enforceRebalance in the KIP -- hope you're happy :P > > Also, I wanted to point out one minor change to the current

Re: [VOTE] KIP-568: Explicit rebalance triggering on the Consumer

Thanks Sophie, I'm +1 (binding) -John On Mon, Feb 10, 2020, at 20:54, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to start the voting on KIP-568. It proposes the new > Consumer#enforceRebalance API to facilitate triggering efficient rebalances. > > For reference, here is the KIP link ag

Re: [DISCUSS] : KIP-562: Allow fetching a key from a single partition rather than iterating over all the stores on an instance

Thanks, Bruno! I meant to do it, but got side-tracked. -John On Tue, Feb 11, 2020, at 03:55, Bruno Cadonna wrote: > Hi all, > > I am fine with StoreQueryParameters, but then we should also change > the DSL grammar accordingly. Since the PR was merged, I suppose > everybody agrees on the new nam

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Hi Sophie. Thanks for the KIP. I liked how focused the proposal is. Also, its motivation is clear after carefully reading the KIP and its references. Yet, I think it'd be a good idea to call out explicitly on the Rejected Alternatives section that an automatic and periodic triggering of rebalance

Re: [DISCUSS] KIP-552: Add interface to handle unused config

Ah... I've just looked at some integration tests in Streams, and see the same thing. I need to apologize to everyone in the thread for my lack of understanding, and to thank Gwen for her skepticism. Looking back at the KIP itself, I see that Artur specifically listed log messages caused by Stre

Re: [VOTE] KIP-568: Explicit rebalance triggering on the Consumer

Thanks Sophie, much required. +1 non-binding. Sent from Yahoo Mail for iPhone On Tuesday, February 11, 2020, 10:33 PM, John Roesler wrote: Thanks Sophie, I'm +1 (binding) -John On Mon, Feb 10, 2020, at 20:54, Sophie Blee-Goldman wrote: > Hey all, > > I'd like to start the voting on KIP-5

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Hi Sophie, Thanks for the KIP, makes sense to me. One quick question, I'm not sure if it's relevant or not. If a user provides a `ConsumerRebalanceListener` and a rebalance is triggered from an `enforceRebalance` call, it seems possible the listener won't get called since partition assignments

[jira] [Created] (KAFKA-9537) Abstract transformations in configurations cause unfriendly error message.

Jeremy Custenborder created KAFKA-9537: -- Summary: Abstract transformations in configurations cause unfriendly error message. Key: KAFKA-9537 URL: https://issues.apache.org/jira/browse/KAFKA-9537

Re: [DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

Hi Lucas, Yes the case you mentioned is true. I do understand KIP-501 might not fully solve this particular use case where there might blocked fetch requests. But the issue we noticed multiple times and continue to notice is 1. Fetch request comes from Follower 2. Le

[jira] [Resolved] (KAFKA-8616) Replace ApiVersionsRequest request/response with automated protocol

[ https://issues.apache.org/jira/browse/KAFKA-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-8616. Resolution: Duplicate > Replace ApiVersionsRequest request/response with automated protocol > -

[jira] [Resolved] (KAFKA-8211) Flaky Test: ResetConsumerGroupOffsetTest.testResetOffsetsExportImportPlan

[ https://issues.apache.org/jira/browse/KAFKA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8211. Resolution: Fixed Closing this one since new failures do not match the stacktrace. I've ope

[jira] [Created] (KAFKA-9538) Flaky Test `testResetOffsetsExportImportPlan`

Jason Gustafson created KAFKA-9538: -- Summary: Flaky Test `testResetOffsetsExportImportPlan` Key: KAFKA-9538 URL: https://issues.apache.org/jira/browse/KAFKA-9538 Project: Kafka Issue Type: B

Re: [VOTE] KIP-447: Producer scalability for exactly once semantics

Hey there, we are adding a small change to the KIP-447 public API. The default value of `transaction.abort.timed.out.transaction.cleanup.interval.ms` shall be changed from 1 minute to 10 seconds. The goal here is to trigger the expired transaction more frequently in order to reduce the consumer pe

[jira] [Resolved] (KAFKA-7052) ExtractField SMT throws NPE - needs clearer error message

[ https://issues.apache.org/jira/browse/KAFKA-7052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-7052. -- Fix Version/s: 2.4.1 2.3.2 2.5.0 2.2.3

[jira] [Created] (KAFKA-9539) Add leader epoch in StopReplicaRequest

David Jacot created KAFKA-9539: -- Summary: Add leader epoch in StopReplicaRequest Key: KAFKA-9539 URL: https://issues.apache.org/jira/browse/KAFKA-9539 Project: Kafka Issue Type: Improvement

[DISCUSS] KIP-570: Add leader epoch in StopReplicaRequest

Hi all, I've put together a very small KIP which proposes to add the leader epoch in the StopReplicaRequest in order to make it robust to reordering: https://cwiki.apache.org/confluence/display/KAFKA/KIP-570%3A+Add+leader+epoch+in+StopReplicaRequest Please take a look at the KIP and let me know w

Build failed in Jenkins: kafka-2.2-jdk8-old #206

See Changes: [rhauch] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node enviro

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Konstantine, Thanks for the feedback! I've updated the sections with your suggestions. I agree in particular that it's really important to make sure users don't call this unnecessarily, or for the wrong reasons: to that end I also extended the javadocs to specify that this API is for when changes

[DISCUSS] KIP-569: DescribeConfigsResponse - Update the schema to include datatype of the field

Hi all, We would like to extend the DescribeConfigsResponse schema to include the data-type of the fields. The KIP can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-569%3A+DescribeConfigsResponse+-+Update+the+schema+to+include+datatype+of+the+field Thanks Shailesh

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Thanks for the quick turnaround Sophie. My points have been addressed. I think the intended use is quite clear now. Best, Konstantine On Tue, Feb 11, 2020 at 12:57 PM Sophie Blee-Goldman wrote: > Konstantine, > Thanks for the feedback! I've updated the sections with your suggestions. I > agree

Re: [VOTE] KIP-568: Explicit rebalance triggering on the Consumer

The KIP reads quite well for me now and I think this feature will enable even more efficient load balancing for specific use cases. I'm also +1 (non-binding) - Konstantine On Tue, Feb 11, 2020 at 9:35 AM Navinder Brar wrote: > Thanks Sophie, much required. > +1 non-binding. > > > Sent from Yah

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Hey Sophie, is the `enforceRebalance` a blocking call? Could we add a code sample to the KIP on how this API should be used? Returning a future instead of a boolean might be easier as we are allowing consumer to make progress during rebalance after 429 IMHO. Boyang On Tue, Feb 11, 2020 at 1:17

Build failed in Jenkins: kafka-trunk-jdk8 #4222

See Changes: [github] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 2.84 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

[jira] [Created] (KAFKA-9540) Application getting "Could not find the standby task 0_4 while closing it" error

Badai Aqrandista created KAFKA-9540: --- Summary: Application getting "Could not find the standby task 0_4 while closing it" error Key: KAFKA-9540 URL: https://issues.apache.org/jira/browse/KAFKA-9540

Build failed in Jenkins: kafka-trunk-jdk11 #1147

See Changes: [github] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 2.85 MB...] org.apache.kafka.streams.test.ConsumerRecordFactor

[jira] [Resolved] (KAFKA-9483) Add Scala KStream#toTable to the Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-9483. Fix Version/s: 2.5.0 Resolution: Fixed > Add Scala KStream#toTable to the Streams DS

Build failed in Jenkins: kafka-2.5-jdk8 #14

See Changes: [rhauch] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 2.86 MB...] org.apache.kafka.streams.MockProcessorContextTest > sh

Re: [VOTE] KIP-568: Explicit rebalance triggering on the Consumer

Thanks for the KIP Sophie. It's a +1 (binding) for me. -Bill On Tue, Feb 11, 2020 at 4:21 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > The KIP reads quite well for me now and I think this feature will enable > even more efficient load balancing for specific use cases. > > I'm

Build failed in Jenkins: kafka-2.3-jdk8 #172

See Changes: [github] KAFKA-9181; Maintain clean separation between local and group -- [...truncated 2.94 MB...] kafka.log.LogCleanerTest > testPartialSegmentClean STARTED

Request for adding wiki edit permission for creating KIP

Hi guys, I am Xue, just joined this mailing list. Being new to Kafka developer community, I am very glad to join and any help/guide is greatly appreciated! I am working on this JIRA: https://issues.apache.org/jira/browse/KAFKA-9440 I would need to create a new KIP for this, can the administrators

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Hey Boyang, Originally I had it as a nonblocking call, but decided to change it to blocking with a timeout parameter. I'm not sure a future makes sense to return here, because the rebalance either does or does not complete within the timeout: if it does not, you will have to call poll again to com

Re: [KAFKA-557] Add emit on change support for Kafka Streams

Hi all, Bumping this. If you feel that this KIP is not too urgent. Then let me know. :) Cheers, Richard On Thu, Feb 6, 2020 at 4:55 PM Richard Yu wrote: > Hi all, > > I've had just a few thoughts regarding the forwarding of change>. As Matthias already mentioned, there are two > separate prio

[jira] [Resolved] (KAFKA-9390) Non-key joining of KTable not compatible with confluent avro serdes

[ https://issues.apache.org/jira/browse/KAFKA-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-9390. - Resolution: Fixed > Non-key joining of KTable not compatible with confluent avro serdes > ---

Re: [VOTE] KIP-447: Producer scalability for exactly once semantics

Boyang, Thanks for the update. This change makes sense to me. Guozhang On Tue, Feb 11, 2020 at 11:37 AM Boyang Chen wrote: > Hey there, > > we are adding a small change to the KIP-447 public API. The default value > of `transaction.abort.timed.out.transaction.cleanup.interval.ms` shall be > ch

Jenkins build is back to normal : kafka-trunk-jdk8 #4223

See

Re: [DISCUSS] KIP-568: Explicit rebalance triggering on the Consumer

Hello Sophie, thanks for brining up this KIP, and the great write-up summarizing the motivations of the proposal. Here are some comments: Minor: 1. If we want to make it a blocking call (I have some thoughts about this below :), to be consistent we need to consider having two overloaded function,

Re: [VOTE] KIP-568: Explicit rebalance triggering on the Consumer

Hi Sophie, Thanks for the KIP, I left some comments on the DISCUSS thread. Guozhang On Tue, Feb 11, 2020 at 3:25 PM Bill Bejeck wrote: > Thanks for the KIP Sophie. > > It's a +1 (binding) for me. > > -Bill > > On Tue, Feb 11, 2020 at 4:21 PM Konstantine Karantasis < > konstant...@confluent.i

Build failed in Jenkins: kafka-2.2-jdk8-old #207

See Changes: [rhauch] MINOR: Start using Response and replace IOException in -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variabl

Build failed in Jenkins: kafka-2.5-jdk8 #15

See Changes: [github] KAFKA-6607: Commit correct offsets for transactional input data (#8040) [matthias] MINOR: fix and improve StreamsConfig JavaDocs (#8086) [matthias] KAFKA-9483: Add Scala KStream#toTable to the

Jenkins build is back to normal : kafka-trunk-jdk11 #1148

See

Build failed in Jenkins: kafka-trunk-jdk8 #4224

See Changes: [github] MINOR: fix and improve StreamsConfig JavaDocs (#8086) [github] KAFKA-9483: Add Scala KStream#toTable to the Streams DSL (#8024) [github] MINOR: improve error reporting in DescribeConsumerG

Build failed in Jenkins: kafka-2.2-jdk8 #25

See Changes: [rhauch] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 1.82 MB...] org.apache.kafka.streams.scala.kstream.KStreamTest > pe

Build failed in Jenkins: kafka-2.4-jdk8 #139

See Changes: [rhauch] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 5.50 MB...] org.apache.kafka.streams.test.OutputVerifierTest > sh

[jira] [Resolved] (KAFKA-9487) Followup : KAFKA-9445(Allow fetching a key from a single partition); addressing code review comments

[ https://issues.apache.org/jira/browse/KAFKA-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navinder Brar resolved KAFKA-9487. -- Resolution: Fixed > Followup : KAFKA-9445(Allow fetching a key from a single partition); > add

[jira] [Created] (KAFKA-9541) Flaky Test DescribeConsumerGroupTest#testDescribeGroupWithShortInitializationTimeout

huxihx created KAFKA-9541: - Summary: Flaky Test DescribeConsumerGroupTest#testDescribeGroupWithShortInitializationTimeout Key: KAFKA-9541 URL: https://issues.apache.org/jira/browse/KAFKA-9541 Project: Kafka

[jira] [Resolved] (KAFKA-9503) TopologyTestDriver processes intermediate results in the wrong order

[ https://issues.apache.org/jira/browse/KAFKA-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-9503. - Resolution: Fixed > TopologyTestDriver processes intermediate results in the wrong order > --

Build failed in Jenkins: kafka-2.3-jdk8 #173

See Changes: [rhauch] KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent -- [...truncated 2.95 MB...] kafka.log.LogCleanerTest > testCommitMarkerRemoval STA

Build failed in Jenkins: kafka-trunk-jdk11 #1149

See Changes: [github] KAFKA-9390: Make serde pseudo-topics unique (#8054) [github] KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS -- [...truncate

Build failed in Jenkins: kafka-trunk-jdk8 #4225

See Changes: [github] KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS -- [...truncated 2.84 MB...] org.apache.kafka.streams.TopologyTestDriverTest

Build failed in Jenkins: kafka-2.5-jdk8 #16

See Changes: [rhauch] MINOR: Start using Response and replace IOException in [vvcephei] KAFKA-9390: Make serde pseudo-topics unique (#8054) [wangguoz] KAFKA-9355: Fix bug that removed RocksDB metrics after failure

[jira] [Resolved] (KAFKA-9500) Foreign-Key Join creates an invalid topology

[ https://issues.apache.org/jira/browse/KAFKA-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-9500. - Resolution: Fixed > Foreign-Key Join creates an invalid topology > --

Build failed in Jenkins: kafka-2.2-jdk8 #26

See Changes: [rhauch] MINOR: Start using Response and replace IOException in -- [...truncated 2.55 MB...] kafka.utils.CommandLineUtilsTest > testMaybeMergeOptionsDefaultOver

Build failed in Jenkins: kafka-trunk-jdk8 #4226

See Changes: [github] KAKFA-9503: Fix TopologyTestDriver output order (#8065) -- [...truncated 2.84 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldRespec

Build failed in Jenkins: kafka-trunk-jdk11 #1150

See Changes: [github] KAKFA-9503: Fix TopologyTestDriver output order (#8065) -- [...truncated 2.86 MB...] org.apache.kafka.streams.TopologyTestDriverTest > shouldPunct