Build failed in Jenkins: kafka-trunk-jdk8 #3997

See Changes: [matthias] MINOR: Add documentation for foreign-key joins (KIP-213) (#7535) [cmccabe] MINOR: Re-implement NewPartitionReassignment#of() (#7592) --

Re: [DISCUSS] KIP-514: Add a bounded flush() API to Kafka Producer

vote thread created. On Mon, Oct 21, 2019 at 9:58 AM Colin McCabe wrote: > > Hi Radai, > > It seems reasonable to me. > > best, > Colin > > > On Mon, Oct 21, 2019, at 09:52, radai wrote: > > yet another bump. > > > > can we please have a vote if there are no objections ? > > > > On Wed, Sep 25,

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

I think I agree with John's recent reasoning as well: instead of letting the storeMetadataAPI to return the staleness information, letting the client to query either active or standby and letting standby query response to include both the values + timestamp (or lag, as in diffs of timestamps)

Re: [ANNOUNCE] Apache Kafka 2.3.1

Thanks for driving the release David! Guozhang On Thu, Oct 24, 2019 at 8:27 PM David Arthur wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.3.1 > > This is a bugfix release for Kafka 2.3.0. All of the changes in this > release can be found in the

[VOTE] KIP-514 Add a bounded flush() API to Kafka Producer

Hello, I'd like to initiate a vote on KIP-514. links: the kip - https://cwiki.apache.org/confluence/display/KAFKA/KIP-514%3A+Add+a+bounded+flush%28%29+API+to+Kafka+Producer the PR - https://github.com/apache/kafka/pull/7569 Thank you

[ANNOUNCE] Apache Kafka 2.3.1

The Apache Kafka community is pleased to announce the release for Apache Kafka 2.3.1 This is a bugfix release for Kafka 2.3.0. All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kafka/2.3.1/RELEASE_NOTES.html You can download the source and binary

[jira] [Created] (KAFKA-9103) KIP-441: Add TaskLags to SubscriptionInfo

John Roesler created KAFKA-9103: --- Summary: KIP-441: Add TaskLags to SubscriptionInfo Key: KAFKA-9103 URL: https://issues.apache.org/jira/browse/KAFKA-9103 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-9089) Reassignment should be resilient to unexpected errors

[ https://issues.apache.org/jira/browse/KAFKA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9089. Resolution: Fixed > Reassignment should be resilient to unexpected errors >

Jenkins build is back to normal : kafka-trunk-jdk8 #3996

See

Build failed in Jenkins: kafka-2.4-jdk8 #37

See Changes: [matthias] MINOR: Add documentation for foreign-key joins (KIP-213) (#7535) -- [...truncated 5.13 MB...] org.apache.kafka.connect.storage.StringConverterTest >

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

+1 As someone implementing a query routing layer, there is already a need to have mechanisms in place to do healthchecks/failure detection to detect failures for queries, while Streams rebalancing eventually kicks in the background. So, pushing this complexity to the IQ client app keeps Streams

[jira] [Created] (KAFKA-9102) Increase default zk session timeout and max lag

Jason Gustafson created KAFKA-9102: -- Summary: Increase default zk session timeout and max lag Key: KAFKA-9102 URL: https://issues.apache.org/jira/browse/KAFKA-9102 Project: Kafka Issue

[jira] [Created] (KAFKA-9101) Create a fetch.max.bytes configuration for the broker

Colin McCabe created KAFKA-9101: --- Summary: Create a fetch.max.bytes configuration for the broker Key: KAFKA-9101 URL: https://issues.apache.org/jira/browse/KAFKA-9101 Project: Kafka Issue

[jira] [Created] (KAFKA-9100) The controller code allows for replicas to get added to the assignment when the replicas goes online

Jose Armando Garcia Sancio created KAFKA-9100: - Summary: The controller code allows for replicas to get added to the assignment when the replicas goes online Key: KAFKA-9100 URL:

[VOTE] KIP-541: Create a fetch.max.bytes configuration for the broker

Hi all, I'd like to start the vote on KIP-541: Create a fetch.max.bytes configuration for the broker. KIP: https://cwiki.apache.org/confluence/x/4g73Bw Discussion thread: https://lists.apache.org/thread.html/9d9dde93a07e1f1fc8d9f182f94f4bda9d016c5e9f3c8541cdc6f53b@%3Cdev.kafka.apache.org%3E

Re: [VOTE] KIP-537: Increase default zookeeper session timeout

Closing this vote. The total is +9 with +4 binding. Thanks everyone! -Jason On Wed, Oct 23, 2019 at 2:31 AM Mickael Maison wrote: > +1 (non binding) > Thanks for the KIP > > On Wed, Oct 23, 2019 at 12:02 AM Harsha Chintalapani > wrote: > > > > +1 (binding). > > Thanks, > > Harsha > > > > On

Re: [DISCUSS] KIP-541: Create a fetch.max.bytes configuration for the broker

Hi Ismael, 55 MB seems reasonable. I'll change it to that. best, Colin On Tue, Oct 22, 2019, at 07:22, Ismael Juma wrote: > Hi Colin, > > Thanks for the KIP. This is definitely needed. Have we considered having a > lower max limit by default? 75 MB is pretty high. Did we consider something >

Build failed in Jenkins: kafka-trunk-jdk11 #910

See Changes: [matthias] MINOR: Add documentation for foreign-key joins (KIP-213) (#7535) -- [...truncated 2.72 MB...] org.apache.kafka.streams.MockProcessorContextTest

[jira] [Created] (KAFKA-9099) Reassignments should be retried after unexpected errors

Jason Gustafson created KAFKA-9099: -- Summary: Reassignments should be retried after unexpected errors Key: KAFKA-9099 URL: https://issues.apache.org/jira/browse/KAFKA-9099 Project: Kafka

Re: [DISCUSS] KIP-541: Create a fetch.max.bytes configuration for the broker

On Tue, Oct 22, 2019, at 01:10, Stanislav Kozlovski wrote: > Thanks for the KIP, Colin! This is a good idea, I think it makes total > sense. > > We also have the max-size tunables on the producer-side (max.request.size). > Do we have a broker equivalent denoting the maximum size of a produce >

Re: [VOTE] KIP-538: Add a metric tracking the number of open connections with a given SSL cipher type

Hi all, With 5 binding +1 votes from Gwen Shapira, Jason Gustafson, Colin McCabe, Ismael Juma, Rajini Sivaram, and 2 non-binding +1 votes from Tom Bentley and M. Manna, the vote passes. Thanks, all. best, Colin On Thu, Oct 24, 2019, at 08:32, Rajini Sivaram wrote: > +1 (binding) > > Thanks

Re: [DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

Hi all, I've been mulling about this KIP, and I think I was on the wrong track earlier with regard to task lags. Tl;dr: I don't think we should add lags at all to the metadata API (and also not to the AssignmentInfo protocol message). Like I mentioned early on, reporting lag via

[jira] [Created] (KAFKA-9098) Name Repartition Filter, Source, and Sink Processors

Bill Bejeck created KAFKA-9098: -- Summary: Name Repartition Filter, Source, and Sink Processors Key: KAFKA-9098 URL: https://issues.apache.org/jira/browse/KAFKA-9098 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-9097) URL needs update - use https://kafka.apache.org/quickstart

Bernard A. Badger created KAFKA-9097: Summary: URL needs update - use https://kafka.apache.org/quickstart Key: KAFKA-9097 URL: https://issues.apache.org/jira/browse/KAFKA-9097 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk11 #909

See Changes: [github] MINOR: Streams upgrade system test cleanup (#7571) -- [...truncated 2.73 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Thanks for the KIP, this is something that will be appreciated by the community. +1(binding) -Bill On Thu, Oct 24, 2019 at 12:54 PM Matthias J. Sax wrote: > Thanks for the KIP! > > +1 (binding) > > > -Matthias > > On 10/24/19 6:19 AM, aishwarya kumar wrote: > > Hello All, > > > > After

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Ryanne, I think I can answer the first question: > - How is this better than using a cached remote file system, This KIP brings in another storage tier (for a total of 3: memory --- not strictly a tier, more of a cache, local storage, remote storage) and manages the movement of the data between

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

While I like the idea Sophie I don't think that it is necessary. I also think that in this case it would be better to separate the 2 option out into separate overloads. We could take a named parameter from upstream or add an extra naming option however I don't really see the advantage that would

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Thanks for the KIP! +1 (binding) -Matthias On 10/24/19 6:19 AM, aishwarya kumar wrote: > Hello All, > > After concluding discussions for this KIP, I would like to go forward with > the voting process. > > Jira Ticket: https://issues.apache.org/jira/browse/KAFKA-7658 > KIP : >

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

A few more questions: - How is this better than using a cached remote file system, e.g. mounting HDFS or S3 (yes, it's possible) and letting the OS and drivers handle everything? Maybe it's better, but the KIP doesn't address how or why, and I'd think this would be a trivial benchmark. If, for

[jira] [Created] (KAFKA-9096) Validate config inter-dependencies for KafkaConfig only after dynamic configs are loaded

Rajini Sivaram created KAFKA-9096: - Summary: Validate config inter-dependencies for KafkaConfig only after dynamic configs are loaded Key: KAFKA-9096 URL: https://issues.apache.org/jira/browse/KAFKA-9096

Re: [VOTE] KIP-538: Add a metric tracking the number of open connections with a given SSL cipher type

+1 (binding) Thanks for the KIP, Colin! Regards, Rajini On Thu, Oct 24, 2019 at 4:07 PM Ismael Juma wrote: > Awesome, +1. > > Ismael > > On Wed, Oct 23, 2019 at 3:37 PM Colin McCabe wrote: > > > Hi Ismael, > > > > That's a good idea! I added it to the KIP. > > > > best, > > Colin > > > > >

Re: [VOTE] KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'

+1 (binding). Guozhang On Thu, Oct 24, 2019 at 8:22 AM Bill Bejeck wrote: > Hi Rabi, > > Thanks for the KIP. > Just to be clear, we're going to deprecate > `UsePreviousTimeOnInvalidTimeStamp` > and add `UsePartitionOnInvalidTimeStamp` class correct? > > Otherwise, it's a +1 (binding) from me.

Re: [VOTE] KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'

Hi Rabi, Thanks for the KIP. Just to be clear, we're going to deprecate `UsePreviousTimeOnInvalidTimeStamp` and add `UsePartitionOnInvalidTimeStamp` class correct? Otherwise, it's a +1 (binding) from me. -Bill On Thu, Oct 24, 2019 at 8:09 AM Bruno Cadonna wrote: > Hi Rabi, > > Thank you for

Re: [VOTE] KIP-538: Add a metric tracking the number of open connections with a given SSL cipher type

Awesome, +1. Ismael On Wed, Oct 23, 2019 at 3:37 PM Colin McCabe wrote: > Hi Ismael, > > That's a good idea! I added it to the KIP. > > best, > Colin > > > On Wed, Oct 23, 2019, at 12:59, Ismael Juma wrote: > > Hi Colin, > > > > Thanks for the KIP. Would it make sense to have a tag with the

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Going back to initial thread with general questions on KIP. I think aspects of the user experience still need clarification: - if a user has a mix of compacted and non-compacted topics it will be hard to reason about storage needs overall. Could you give a reason why compacted topics are not

[VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Hello All, After concluding discussions for this KIP, I would like to go forward with the voting process. Jira Ticket: https://issues.apache.org/jira/browse/KAFKA-7658 KIP : https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL Thank you,

Re: [VOTE] 2.3.1 RC2

Thanks to everyone who voted! The vote for RC2 of the 2.3.1 release passes with the 6 +1s and no +0 or -1. +1 votes PMC Members: * Jason Gustafson * Guozhang Wang * Matthias Sax * Rajini Sivaram Committers: * Colin McCabe Community: * Jonathan Santilli 0 votes * No votes -1 votes * No votes

[DISCUSS] KIP-542: Partition Reassignment Throttling

Hi People, I've created a KIP to improve replication quotas by handling reassignment related throttling as a separate case with its own configurable limits and change the kafka-reassign-partitions tool to use these new configs going forward. Please have a look, I'd be happy to receive any

Re: [VOTE] KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'

Hi Rabi, Thank you for the KIP! +1 (non-binding) Best, Bruno On Thu, Oct 24, 2019 at 8:42 AM Matthias J. Sax wrote: > > +1 (binding) > > Thanks for the KIP! > > -Matthias > > On 10/23/19 9:37 AM, RABI K.C. wrote: > > Hello All, > > > > As per the discussion I want to call for a vote to go

[jira] [Created] (KAFKA-9095) Kafka Cluster Issue when API Version =1

Mantesh Janagond created KAFKA-9095: --- Summary: Kafka Cluster Issue when API Version =1 Key: KAFKA-9095 URL: https://issues.apache.org/jira/browse/KAFKA-9095 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-506: Allow setting SCRAM password via Admin interface

Hey Tom, "The intent behind sasl.scram.password.change.enabled was to give the admins the choice about whether they're willing for passwords to be sent to the broker in the clear (after all they know their deployment and network best)." I can see one advantage which is testing during development

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Eno and Viktor, IMHO, it is helpful to have HDFS(os S3) implementation to be part of this PR until it is ready to be merged. We will remove this module before merging the PR. As suggested by Viktor, we can also have a simple RI of `RemoteStorageManager` for users to understand the APIs better.

[jira] [Created] (KAFKA-9094) Validate the replicas for partition reassignments triggered through the /admin/reassign_partitions zNode

Stanislav Kozlovski created KAFKA-9094: -- Summary: Validate the replicas for partition reassignments triggered through the /admin/reassign_partitions zNode Key: KAFKA-9094 URL:

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Jun, Thanks for the clarifications. >Regarding feature branch, if the goal is faster collaboration, it seems that doing the development on your own fork is better since non-committers can push changes there. Sure, we will have the branch in our repo and collaborate with others in that branch

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Harsha, A couple more questions: 1. Can we allow RLM Followers to serve read requests? After all segments on the cold storage are closed ones, no modification is allowed. Besides KIP-392 (

[jira] [Created] (KAFKA-9093) NullPointerException in KafkaConsumer with group.instance.id

Rolef Heinrich created KAFKA-9093: - Summary: NullPointerException in KafkaConsumer with group.instance.id Key: KAFKA-9093 URL: https://issues.apache.org/jira/browse/KAFKA-9093 Project: Kafka

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Jun & Harsha, I think it would be beneficial to at least provide one simple reference implementation (file system based?) as we do with connect too. That would as a simple example and would help plugin developers to better understand the concept and the interfaces. Best, Viktor On Wed, Oct

Jenkins build is back to normal : kafka-trunk-jdk11 #908

See

Re: [VOTE] KIP-530: Consider renaming 'UsePreviousTimeOnInvalidTimeStamp' class to 'UsePartitionTimeOnInvalidTimeStamp'

+1 (binding) Thanks for the KIP! -Matthias On 10/23/19 9:37 AM, RABI K.C. wrote: > Hello All, > > As per the discussion I want to call for a vote to go forward with the > changes mentioned in KIP-530. > > Links: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028807 >