Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi Jun and David, Regarding token bucket vs, Rate behavior. We recently observed a couple of cases where a bursty workload behavior would result in long-ish pauses in between, resulting in lower overall bandwidth than the quota. I will need to debug this a bit more to be 100% sure, but it does

[jira] [Created] (KAFKA-10099) Kerberos authentication sets java authrizedId to authenticationId not autherizationId

Francois Fernando created KAFKA-10099: - Summary: Kerberos authentication sets java authrizedId to authenticationId not autherizationId Key: KAFKA-10099 URL: https://issues.apache.org/jira/browse/KAFKA-10099

[jira] [Resolved] (KAFKA-9788) Sensor name collision for group and transaction coordinator load metrics

[ https://issues.apache.org/jira/browse/KAFKA-9788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9788. Fix Version/s: 2.6.0 Resolution: Fixed > Sensor name collision for group and

Re: [DISCUSS] KIP-614: Add Prefix Scan support for State Stores

Hi Sagar, Thanks for the question, and sorry for muddying the water. I meant the Bytes/byte[] thing as advice for how you all can solve your problem in the mean time, while we work through this KIP. I don’t think it’s relevant for the KIP itself. I think the big issue here is what the type of

[jira] [Created] (KAFKA-10098) Remove unnecessary escaping in regular expression in SaslAuthenticatorTest.java

Can Cecen created KAFKA-10098: - Summary: Remove unnecessary escaping in regular expression in SaslAuthenticatorTest.java Key: KAFKA-10098 URL: https://issues.apache.org/jira/browse/KAFKA-10098 Project:

[jira] [Created] (KAFKA-10097) Avoid getting null map for task checkpoint

Boyang Chen created KAFKA-10097: --- Summary: Avoid getting null map for task checkpoint Key: KAFKA-10097 URL: https://issues.apache.org/jira/browse/KAFKA-10097 Project: Kafka Issue Type:

[jira] [Resolved] (KAFKA-9161) Close gaps in Streams configs documentation

[ https://issues.apache.org/jira/browse/KAFKA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan resolved KAFKA-9161. Resolution: Fixed > Close gaps in Streams configs documentation >

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi, David, Thanks for the reply. 11. To match the behavior in the Token bucket approach, I was thinking that requests that don't fit in the previous time windows will be accumulated in the current time window. So, the 60 extra requests will be accumulated in the latest window. Then, the client

[jira] [Resolved] (KAFKA-10096) Remove unnecessary String.format call in VersionConditionalTest.java

[ https://issues.apache.org/jira/browse/KAFKA-10096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Can Cecen resolved KAFKA-10096. --- Resolution: Won't Fix Realized that String.format is used to create line separators. > Remove

[jira] [Created] (KAFKA-10096) Remove unnecessary String.format call in VersionConditionalTest.java

Can Cecen created KAFKA-10096: - Summary: Remove unnecessary String.format call in VersionConditionalTest.java Key: KAFKA-10096 URL: https://issues.apache.org/jira/browse/KAFKA-10096 Project: Kafka

[jira] [Created] (KAFKA-10094) In MirrorSourceConnector replace two-step assignment with single call

Jakob Homan created KAFKA-10094: --- Summary: In MirrorSourceConnector replace two-step assignment with single call Key: KAFKA-10094 URL: https://issues.apache.org/jira/browse/KAFKA-10094 Project: Kafka

[jira] [Created] (KAFKA-10093) Replace iteration with call to addAll in Utils

Jakob Homan created KAFKA-10093: --- Summary: Replace iteration with call to addAll in Utils Key: KAFKA-10093 URL: https://issues.apache.org/jira/browse/KAFKA-10093 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-10092) Remove unnecessary enum modifier in NioEchoServer

Jakob Homan created KAFKA-10092: --- Summary: Remove unnecessary enum modifier in NioEchoServer Key: KAFKA-10092 URL: https://issues.apache.org/jira/browse/KAFKA-10092 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-10095) In LogCleanerManagerTest replace get().nonEmpty call with contains

Jakob Homan created KAFKA-10095: --- Summary: In LogCleanerManagerTest replace get().nonEmpty call with contains Key: KAFKA-10095 URL: https://issues.apache.org/jira/browse/KAFKA-10095 Project: Kafka

Build failed in Jenkins: kafka-2.6-jdk8 #15

See Changes: [jason] KAFKA-10080; Fix race condition on txn completion which can cause [rajinisivaram] KAFKA-10089 The stale ssl engine factory is not closed after reconfigure

[jira] [Created] (KAFKA-10091) Improve task idling

John Roesler created KAFKA-10091: Summary: Improve task idling Key: KAFKA-10091 URL: https://issues.apache.org/jira/browse/KAFKA-10091 Project: Kafka Issue Type: Task Components:

Please I Want to Subscribe

Please I Want to Subscribe

Re: [VOTE] KIP-545 support automated consumer offset sync across clusters in MM 2.0

Looks like we've got the required votes! Congrats Ning! Please note this KIP was proposed and written by Ning Zhang, not myself, but we've collaborated on the PR, testing, and community outreach. Now lets work to get the PR merged! Ryanne On Mon, May 25, 2020 at 1:47 PM Bill Bejeck wrote: >

Re: [VOTE] KIP-601: Configurable socket connection timeout in NetworkClient

Hi Cheng, Thanks for working on this. Looks good. How about "socket.connection.setup.timeout.ms" and "socket.connection.setup.timeout.max.ms" (not connections with an S)? +1 (binding) best, Colin On Wed, Jun 3, 2020, at 06:24, Rajini Sivaram wrote: > Hi Cheng, > > Thanks for the updates,

Build failed in Jenkins: kafka-2.6-jdk8 #14

See Changes: [vvcephei] KAFKA-10084: Fix EosTestDriver end offset (#8785) -- [...truncated 3.13 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTest >

[jira] [Resolved] (KAFKA-10089) The stale ssl engine factory is not closed after reconfigure

[ https://issues.apache.org/jira/browse/KAFKA-10089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-10089. Fix Version/s: 2.6.0 Reviewer: Rajini Sivaram Resolution: Fixed > The

[jira] [Resolved] (KAFKA-9777) Purgatory locking bug can lead to hanging transaction

[ https://issues.apache.org/jira/browse/KAFKA-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9777. Fix Version/s: 2.6.0 Resolution: Fixed > Purgatory locking bug can lead to hanging

[jira] [Resolved] (KAFKA-10080) IllegalStateException after duplicate CompleteCommit append to transaction log

[ https://issues.apache.org/jira/browse/KAFKA-10080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10080. - Fix Version/s: 2.6.0 Resolution: Fixed > IllegalStateException after duplicate

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi David, One quick question about the implementation (I don't think it's spelled out in the KIP): Presumably if you make, for example, a create topics request with validate only it will check for quota violation, but not count towards quota violation, right? Many thanks, Tom On Wed, Jun 3,

[jira] [Resolved] (KAFKA-8338) Improve consumer offset expiration logic to take subscription into account

[ https://issues.apache.org/jira/browse/KAFKA-8338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-8338. Resolution: Duplicate This has been done as part of KAFKA-8730 (KIP-496). > Improve consumer

Build failed in Jenkins: kafka-trunk-jdk11 #1530

See Changes: [github] KAFKA-10084: Fix EosTestDriver end offset (#8785) -- Started by an SCM change Running as SYSTEM [EnvInject] - Loading node environment variables.

Re: [VOTE] KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

+1 (binding) Thanks for the KIP, David! Regards, Rajini On Sun, May 31, 2020 at 3:29 AM Gwen Shapira wrote: > +1 (binding) > > Looks great. Thank you for the in-depth design and discussion. > > On Fri, May 29, 2020 at 7:58 AM David Jacot wrote: > > > Hi folks, > > > > I'd like to start the

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi David, 2) sorry, that was my mistake. Regards, Rajini On Wed, Jun 3, 2020 at 3:08 PM David Jacot wrote: > Hi Rajini, > > Thanks for your prompt response. > 1) Good catch, fixed. > 2) The retry mechanism will be in the client so a new field is not > required in the requests. > > Regards,

Re: [VOTE] KIP-589: Add API to update Replica state in Controller

The vote for this KIP passes with the following results: * Three binding +1 votes from Colin, Guozhang, and Jason * Two non-binding +1 votes from Jose and Boyang * No +0 or -1 votes Thanks, everyone! -David On Tue, Jun 2, 2020 at 8:56 PM Jason Gustafson wrote: > +1 I agree with Guozhang that

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi Rajini, Thanks for your prompt response. 1) Good catch, fixed. 2) The retry mechanism will be in the client so a new field is not required in the requests. Regards, David On Wed, Jun 3, 2020 at 2:43 PM Rajini Sivaram wrote: > Hi David, > > Thanks for the updates, looks good. Just a couple

Re: [DISCUSS] KIP-619 Deprecate ConsumerConfig#addDeserializerToConfig(Properties, Deserializer, Deserializer) and ProducerConfig#addSerializerToConfig(Properties, Serializer, Serializer)

When I created the KIP, the next number was 619 and not sure why the number is out of sync. At any rate, I will update the KIP number :_ On 2020/06/03 05:06:39, Cheng Tan wrote: > Hi Chia, > > Hope you are doing well. I already took KIP-619 as my KIP identification > number. Could you

Re: [VOTE] KIP-601: Configurable socket connection timeout in NetworkClient

Hi Cheng, Thanks for the updates, looks good. +1 (binding) Regards, Rajini On Wed, Jun 3, 2020 at 8:53 AM Cheng Tan wrote: > Dear Rajini, > > Thanks for the feedback. > > 1) > Because "request.timeout.ms" only affects in-flight requests, after the > API NetworkClient.ready() is invoked, the

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi David, Thanks for the updates, looks good. Just a couple of minor comments: 1) There is a typo in "*The channel will be mutated as well when `throttle_time_ms > 0`." * Should be *muted*? 2) Since the three requests will need a new field for ` *retryQuotaViolatedException*`, we should perhaps

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi all, I have updated the KIP based on our recent discussions. I have mainly changed the following points: * I have renamed the quota as suggested by Jun. * I have changed the metrics to be "token bucket" agnostic. The idea is to report the burst and the rate per principal/clientid. * I have

Build failed in Jenkins: kafka-trunk-jdk14 #164

See Changes: [github] KAFKA-10083: fix failed -- [...truncated 2.63 MB...] unit.kafka.cluster.AssignmentStateTest.testPartitionAssignmentStatus[7] failed, log available

[jira] [Created] (KAFKA-10090) Misleading warnings: The configuration was supplied but isn't a known config

Robert Wruck created KAFKA-10090: Summary: Misleading warnings: The configuration was supplied but isn't a known config Key: KAFKA-10090 URL: https://issues.apache.org/jira/browse/KAFKA-10090

Build failed in Jenkins: kafka-2.4-jdk8 #219

See Changes: [wangguoz] KAFKA-10083: fix failed -- [...truncated 2.76 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Jenkins build is back to normal : kafka-2.5-jdk8 #142

See

Jenkins build is back to normal : kafka-trunk-jdk11 #1529

See

[jira] [Created] (KAFKA-10089) The stale ssl engine factory is not closed after reconfigure

Chia-Ping Tsai created KAFKA-10089: -- Summary: The stale ssl engine factory is not closed after reconfigure Key: KAFKA-10089 URL: https://issues.apache.org/jira/browse/KAFKA-10089 Project: Kafka

Jenkins build is back to normal : kafka-2.6-jdk8 #13

See

Re: KIP-599: Throttle Create Topic, Create Partition and Delete Topic Operations

Hi Rajini, Thanks for your feedback. Please find my answers below: 1) Our main goal is to protect the controller from the extreme users (DDoS). We want to protect it from large requests or repetitive requests coming from a single user. That user could be used by multiple apps as you pointed out

Re: [VOTE] KIP-601: Configurable socket connection timeout in NetworkClient

Dear Rajini, Thanks for the feedback. 1) Because "request.timeout.ms" only affects in-flight requests, after the API NetworkClient.ready() is invoked, the connection won't get closed after "request.timeout.ms” hits. Before a) the SocketChannel is connected b) ssl handshake

[jira] [Resolved] (KAFKA-9320) Enable TLSv1.3 by default and disable some of the older protocols

[ https://issues.apache.org/jira/browse/KAFKA-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov resolved KAFKA-9320. Resolution: Fixed Fixed with the