Re: KAFKA-16221

Hi Sophie, Thank you very much. Will do a PR. Please allow some extra time, as I'd like to test the change first. Not going overboard. I have to test the real thing anyway. Kind regards, Matthias Kraaz > Hey Matthias -- I'm quite sure you should *not* do what we're doing in >

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Christo, We have localTimeMs, remoteTimeMs, and totalTimeMs as part of the FetchConsumer request metric. kafka.network:type=RequestMetrics,name={LocalTimeMs|RemoteTimeMs|TotalTimeMs},request={Produce|FetchConsumer|FetchFollower} RemoteTimeMs refers to the amount of time spent in the purgatory

Re: [DISCUSS] KIP-932: Queues for Kafka

Hi Jun, Thanks for the response. 147. I am trying to get a correspondence between the concepts and metrics of consumer groups and share groups. In both cases, the client doesn’t strictly know when the rebalance starts. All it knows is when it has work to do in order to perform its part of a

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

Hi, I’ve changed my mind on this one having read through the comments. I don’t think the exception handler should be able to mess with the headers to the detriment of the code that called the handler. While I like the hygiene of having an ImmutableHeaders interface, I feel we can use the

Re: [DISCUSS] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

Hi David, I think this KIP is a very good idea. It would be good to get rid of this cruft. Thanks, Andrew > On 2 May 2024, at 18:54, David Jacot wrote: > > Hi folks, > > I have put together a very small KIP to > deprecate offsets.commit.required.acks in 3.8 and remove it in 4.0. See the >

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Hi Christo, Thanks for the update. Questions: 1. For this "The possible state transition from DISABLED state is to the ENABLED." I think it only applies for KRaft mode. In ZK mode, the possible state is "DISABLING", right? 2. For this: "If the cluster is using Zookeeper as the control plane,

Re: [VOTE] KIP-924: customizable task assignment for Streams

I left one more nit on the discuss thread. But overall LGTM. +1 (binding) Thanks Rohan and Sophie for driving this KIP. -Matthias On 4/29/24 2:07 PM, Sophie Blee-Goldman wrote: +1 (binding) thanks for driving this KIP! On Tue, Apr 16, 2024 at 1:46 PM Rohan Desai wrote:

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Thanks Sophie. My bad. You are of course right about `TaskAssignment` and the StreamsPartitionAssignor's responsibitliy to map tasks of a instance to consumers. When I wrote my reply, I forgot about this detail. Seems you did not add `UNKNOWN_TASK_ID` error yet as proposed by Guozhang?

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Guozhang: 117. All three additions make sense to me. However, while thinking about how users would actually produce an assignment, I realized that it seems silly to make it their responsibility to distinguish between a stateless and stateful task when they return the assignment. The

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Matthias: Thanks for the naming suggestions for the error codes. I was definitely not happy with my original naming but couldn't think of anything better. I like your proposals though, will update the KIP names. I'll also add a "NONE" option as well -- much better than just passing in null for

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

I tend to agree that we should just return a pure Headers instead of introducing a new class/interface to protect overwriting them. I think a pretty good case has been made already so I won't add onto it, just wanted to voice my support. Is that the only remaining question on this KIP? Might be

Re: KAFKA-16221

Hey Matthias -- I'm quite sure you should *not* do what we're doing in https://github.com/apache/kafka/pull/15315. That's definitely a bad hack, and IIUC the only reason we accepted it was because the choice was between implementing a hacky temporary fix and blocking the entire release while we

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2866

See

[jira] [Created] (KAFKA-16659) KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER

Chia-Ping Tsai created KAFKA-16659: -- Summary: KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER Key: KAFKA-16659 URL: https://issues.apache.org/jira/browse/KAFKA-16659

Re: [DISCUSS] KIP-1036: Extend RecordDeserializationException exception

Sounds good -- exceptionOrigin makes sense to me. Let us know when you've updated the KIP. I'll cast a vote once these last small changes have been made On Thu, May 2, 2024 at 10:16 AM Frédérik Rouleau wrote: > Hi Sophie, > > I agree that the subclasses have limited value and I am not a fan of

Re: [DISCUSS] KIP-1036: Extend RecordDeserializationException exception

Hi Sophie, I agree that the subclasses have limited value and I am not a fan of "instance of" usage either. I do not see any problem with adding a field but I would rather name it something like exceptionOrigin. Any thoughts? About byteBuffer vs byte[], byteBuffer are more generic and with

Re: [DISCUSS] KIP-932: Queues for Kafka

Hi, Andrew, Thanks for the reply. 147. " it makes a judgement about whether an assignment received is equal to what it already is using." If a client receives an assignment different from what it has, it indicates the end of the rebalance. But how does the client know when the rebalance starts?

[PR] MINOR: Add code signing key for Igor [kafka-site]

soarez opened a new pull request, #600: URL: https://github.com/apache/kafka-site/pull/600 For the first PMC reviewing this, could you please update the KEYS file in https://dist.apache.org/repos/dist/release/kafka/KEYS and leave a comment in this PR? -- This is an automated message

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2865

See

[DISCUSS] Proposal about support for wildcard when creating new acls

Hello, I'd like to propose a suggestion to our resource patterns in Kafka ACLs. Currently, when adding new ACLs in Kafka, we have two types of resource patterns for topics: - LITERAL - PREFIXED However, when it comes to listing or removing ACLs, we have a couple more options: - MATCH

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

Hi Nick! I agree, the age variant is a bit nicer since the semantics are very clear from the name. If you'd rather go for the simple implementation, how about calling it `oldest-iterator-open-since-ms`? I believe this could be understood without docs. Either way, I think we should be able to open

Suggestion about support for wildcard when creating new acls

Hello, I'd like to propose a suggestion to our resource patterns in Kafka ACLs. Currently, when adding new ACLs in Kafka, we have two types of resource patterns for topics: - LITERAL - PREFIXED However, when it comes to listing or removing ACLs, we have a couple more options: - MATCH

[DISCUSS] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

Hi folks, I have put together a very small KIP to deprecate offsets.commit.required.acks in 3.8 and remove it in 4.0. See the motivation for the reason. KIP: https://cwiki.apache.org/confluence/x/9YobEg Please let me know what you think. Best, David

[jira] [Created] (KAFKA-16658) Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

David Jacot created KAFKA-16658: --- Summary: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8) Key: KAFKA-16658 URL: https://issues.apache.org/jira/browse/KAFKA-16658 Project: Kafka

Re: [VOTE] KIP-477: Add PATCH method for connector config in Connect REST API

Hello all, Thank you for your votes! I think we have reached a decision here to accept the KIP with the following votes: - 3x binding +1s from Chris Egerton, Yash Mayya, and Greg Harris. - 1x non-binding +1s from Knowles Atchison Jr. - No -1s. I will proceed with the implementation. Feel free

[jira] [Created] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

sanghyeok An created KAFKA-16657: Summary: KIP-848 does not work well on Zookeeper Mode Key: KAFKA-16657 URL: https://issues.apache.org/jira/browse/KAFKA-16657 Project: Kafka Issue Type: Bug

KAFKA-16221

Hi, Thanks for your work and sorry to bother you. My code gets the same IllegalStateException from KafkaProducer as Kafka Stream gets in KAFKA-16221: java.lang.IllegalStateException: Cannot attempt operation `abortTransaction` because the previous call to `commitTransaction` timed out and

[jira] [Resolved] (KAFKA-10804) Tune travis system tests to avoid timeouts

[ https://issues.apache.org/jira/browse/KAFKA-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10804. Resolution: Won't Fix > Tune travis system tests to avoid timeouts >

[jira] [Resolved] (KAFKA-10809) Make all system tests stably run on either Travis CI or local

[ https://issues.apache.org/jira/browse/KAFKA-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10809. Resolution: Fixed > Make all system tests stably run on either Travis CI or local >

[jira] [Created] (KAFKA-16656) Using a custom replication.policy.separator with DefaultReplicationPolicy

Lenin Joseph created KAFKA-16656: Summary: Using a custom replication.policy.separator with DefaultReplicationPolicy Key: KAFKA-16656 URL: https://issues.apache.org/jira/browse/KAFKA-16656 Project:

[jira] [Resolved] (KAFKA-10290) fix flaky core/compatibility_test_new_broker_test.py

[ https://issues.apache.org/jira/browse/KAFKA-10290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10290. Resolution: Won't Fix > fix flaky core/compatibility_test_new_broker_test.py >

[jira] [Resolved] (KAFKA-14874) Unable to create > 5000 topics for once when using Kraft

[ https://issues.apache.org/jira/browse/KAFKA-14874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-14874. Resolution: Won't Fix > Unable to create > 5000 topics for once when using Kraft >

[jira] [Resolved] (KAFKA-10805) More useful reporting from travis system tests

[ https://issues.apache.org/jira/browse/KAFKA-10805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10805. Resolution: Won't Fix > More useful reporting from travis system tests >

[jira] [Resolved] (KAFKA-10288) fix flaky client/client_compatibility_features_test.py

[ https://issues.apache.org/jira/browse/KAFKA-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10288. Resolution: Won't Fix > fix flaky client/client_compatibility_features_test.py >

[jira] [Resolved] (KAFKA-10291) fix flaky tools/log4j_appender_test.py

[ https://issues.apache.org/jira/browse/KAFKA-10291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10291. Resolution: Won't Fix > fix flaky tools/log4j_appender_test.py >

[jira] [Resolved] (KAFKA-10822) Force some stdout from system tests for Travis

[ https://issues.apache.org/jira/browse/KAFKA-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10822. Resolution: Won't Fix > Force some stdout from system tests for Travis >

[jira] [Resolved] (KAFKA-10696) Replace ProduceResponse.PartitionResponse by auto-generated PartitionProduceResponse

[ https://issues.apache.org/jira/browse/KAFKA-10696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10696. Resolution: Won't Fix > Replace ProduceResponse.PartitionResponse by auto-generated >

[jira] [Resolved] (KAFKA-7014) Guarantee the byte buffer returned by Serializer is reusable

[ https://issues.apache.org/jira/browse/KAFKA-7014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-7014. --- Resolution: Won't Fix > Guarantee the byte buffer returned by Serializer is reusable >

[jira] [Resolved] (KAFKA-6992) Configuring the batch.size to zero won't disable the batching entirely

[ https://issues.apache.org/jira/browse/KAFKA-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-6992. --- Resolution: Won't Fix > Configuring the batch.size to zero won't disable the batching

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

There is some question about whether or not we need the configuration options. My take on them is as follows: producer.id.quota.window.num No opinion. I don't know what this is used for, but I suspect that there is a good reason to have it. It is not used within the Bloom filter caching

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

Quick note: I renamed the example code. It is now at https://github.com/Claudenw/kafka/blob/KIP-936/storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerIDQuotaManagerCache.java On Thu, May 2, 2024 at 10:47 AM Claude Warren, Jr wrote: > Igor, thanks for taking the time to

Re: [DISCUSS] KIP-936 Throttle number of active PIDs

Igor, thanks for taking the time to look and to review the code. I regret that I have not pushed the latest code, but I will do so and will see what I can do about answering your Bloom filter related questions here. How would an operator know or decide to change the configuration > for the

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2864

See Changes: -- [...truncated 351225 lines...] [2024-05-02T07:37:14.138Z] > Task :connect:json:publishMavenJavaPublicationToMavenLocal [2024-05-02T07:37:14.138Z] > Task