Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging [2] https://issues.apache.org/jira/browse/KAFKA-15259 [3] https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf On Fri, Jul 19, 2024 at 12:39 PM Matthias J. Sax wrote: For catch

Re: [VOTE] KIP-1070: Deprecate MockProcessorContext

Matthias J. Sax wrote: I just realized, that there is more interfaces with a similar situation: - Transformer - TransformerSupplier - ValueTransformer - ValueTransfomerSupplier - ValueTransformerWithKey - ValueTransfromerWithKeySupplier Given that `KStream#transform` and `KStream#transformValues

[jira] [Created] (KAFKA-17178) Update KTable.transformValues to new Processor API

Matthias J. Sax created KAFKA-17178: --- Summary: Update KTable.transformValues to new Processor API Key: KAFKA-17178 URL: https://issues.apache.org/jira/browse/KAFKA-17178 Project: Kafka

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

s. This is the sort of estimation that I would expect application developers could implement, without knowing the exact serialization and protocol overhead. This could prevent user-originated oversize records from making it to the producer. Thanks, Greg On Mon, Jul 15, 2024 at 4:08 PM Matthias J.

[jira] [Created] (KAFKA-17164) Consider to enforce application.server : format at config level

Matthias J. Sax created KAFKA-17164: --- Summary: Consider to enforce application.server : format at config level Key: KAFKA-17164 URL: https://issues.apache.org/jira/browse/KAFKA-17164 Project: Kafka

Re: [VOTE] KIP-1059: Enable the Producer flush() method to clear the latest send() error

, then I believe this KIP deserves to pass. I'll continue to monitor the discussion thread in case there's an opportunity to change my mind. Best, Chris On Fri, Jun 28, 2024 at 6:32 PM Matthias J. Sax wrote: Thanks for the KIP Alieh! +1 (binding) -Matthias On 6/26/24 5:29 AM, Andrew

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

pe from any error that might take place with producing a record to Kafka, to only the ones that are thrown directly from Producer::send; is that the intended behavior here? And if so, do you have thoughts on how we might design a follow-up KIP that would catch all errors (i

Re: Proposal to Include Record Headers in ProducerInterceptor onAcknowledgement Method

If there are other suggestions, I am open to considering them as well. Best regards, Rich On Sat, Jul 13, 2024 at 1:50 PM Matthias J. Sax wrote: Hi Rich, thanks for reaching out. I don't see any reason to object to this proposal. The best way forward would be to write a KIP and collect feedback f

Re: Proposal to Include Record Headers in ProducerInterceptor onAcknowledgement Method

Hi Rich, thanks for reaching out. I don't see any reason to object to this proposal. The best way forward would be to write a KIP and collect feedback from there. -Matthias On 7/12/24 9:14 PM, Rich C. wrote: Dear Kafka Development Community, I propose enhancing the Kafka

Re: [DISCUSS] KIP-1065: Add "retry" return-option to ProductionExceptionHandler

it totally depends on the custom logic if the handler would return FAIL or CONTINUE, or maybe even error out... -Matthias On 7/8/24 10:22 PM, Matthias J. Sax wrote: Thanks for the feedback! To answer your questions: In general, yes, user topics should exist, however, we support "d

[jira] [Created] (KAFKA-17131) Cleanup `ProcessorContext` an related interfaces

Matthias J. Sax created KAFKA-17131: --- Summary: Cleanup `ProcessorContext` an related interfaces Key: KAFKA-17131 URL: https://issues.apache.org/jira/browse/KAFKA-17131 Project: Kafka Issue

Re: [VOTE] KIP-1070: Deprecate MockProcessorContext

) On Thu, Jul 11, 2024 at 9:24 AM Matthias J. Sax wrote: Hi, I want to propose a very small KIP. Skipping the DISCUSS step, and calling for a VOTE directly. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1070%3A+deprecate+MockProcessorContext -Matthias

[jira] [Resolved] (KAFKA-10356) Handle accidental deletion of sink-topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10356. - Resolution: Not A Problem > Handle accidental deletion of sink-topics as exceptio

[VOTE] KIP-1070: Deprecate MockProcessorContext

Hi, I want to propose a very small KIP. Skipping the DISCUSS step, and calling for a VOTE directly. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1070%3A+deprecate+MockProcessorContext -Matthias

[jira] [Resolved] (KAFKA-10814) improving ability of handling exception in kafka

[ https://issues.apache.org/jira/browse/KAFKA-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10814. - Resolution: Incomplete Unclear what the problem is. Closing. > improving abil

[jira] [Resolved] (KAFKA-8629) Kafka Streams Apps to support small native images through GraalVM

[ https://issues.apache.org/jira/browse/KAFKA-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8629. Resolution: Fixed GraalVM should work with KS now. > Kafka Streams Apps to support sm

[jira] [Resolved] (KAFKA-13326) Add multi-cluster support to Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-13326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13326. - Resolution: Duplicate > Add multi-cluster support to Kafka Stre

[jira] [Reopened] (KAFKA-13183) Dropping nul key/value records upstream to repartiton topic not tracked via metrics

[ https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13183: - > Dropping nul key/value records upstream to repartiton topic not tracked via > m

[jira] [Resolved] (KAFKA-13183) Dropping nul key/value records upstream to repartiton topic not tracked via metrics

[ https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13183. - Resolution: Fixed > Dropping nul key/value records upstream to repartiton to

Re: [DISCUSS] Apache Kafka 3.8.0 release

The KS system test did surface a bug. Bruno put a PR up for it, which is already approved. We should get it into 3.8 asap to unblock the release from our side. -Matthias On 7/8/24 8:51 AM, Josep Prat wrote: Hi Dongjin, KIP-390 is part of the 3.8 release because the JIRA associated with it:

Re: [DISCUSS] KIP-1065: Add "retry" return-option to ProductionExceptionHandler

serdes are probably (hopefully!) deterministic, but could deserialization fail due to network errors or other transient issues with schema registry? On Sun, Jun 30, 2024 at 12:35 PM Matthias J. Sax wrote: Hi, as a follow up to https://issues.apache.org/jira/browse/KAFKA-16508 which is related to KI

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

d behavior here? And if so, do you have thoughts on how we might design a follow-up KIP that would catch all errors (including ones reported asynchronously instead of synchronously)? I'd like it if we could leave the door open for that without painting ourselves into too much

Re: [VOTE] KIP-1041: Drop `offsets.commit.required.acks` config in 4.0 (deprecate in 3.8)

I was just made aware of this KIP, and I believe we cannot drop this config in 4.0, as we guarantee a one year deprecation period. Did not read the discuss thread. Is there any good reason for not following our guaranteed deprecation period? -Matthias On 5/13/24 5:21 AM, David Jacot wrote:

Re: [DISCUSS] KIP-1067: Remove ReplicaVerificationTool in 4.0 (deprecate in 3.9)

n 3.8 and then remove it in 4.0. ( https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794933) Maybe it is a good time to have a discussion about the deprecation cycle. Personally, the second rule is easy to follow, but the side effect is that the "deprecation cycle" is not fixed (fo

Re: [DISCUSS] KIP-1067: Remove ReplicaVerificationTool in 4.0 (deprecate in 3.9)

Hi, If I understand correctly, instead of maintaining the tool further (via KIP-752) we want to deprecate the tool? This is fine, but it seem the KIP deadline for 3.9 passed, so we cannot deprecate it in 3.9, but only in 4.0? And even if we make an exception, we usually have a deprecation

[DISCUSS] KIP-1065: Add "retry" return-option to ProductionExceptionHandler

Hi, as a follow up to https://issues.apache.org/jira/browse/KAFKA-16508 which is related to KIP-1038, I would like to prose adding a RETRY option to production error handler responses: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=311627309 Looking forward to your

[jira] [Created] (KAFKA-17057) Add "retry" option to ProductionExceptionHandler

Matthias J. Sax created KAFKA-17057: --- Summary: Add "retry" option to ProductionExceptionHandler Key: KAFKA-17057 URL: https://issues.apache.org/jira/browse/KAFKA-17057 Project: Kafka

[jira] [Resolved] (KAFKA-16508) Infinite loop if output topic does not exisit

[ https://issues.apache.org/jira/browse/KAFKA-16508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16508. - Fix Version/s: 3.9.0 Resolution: Fixed > Infinite loop if output topic d

Re: [VOTE] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Thanks for the KIP Alieh! +1 (binding) -Matthias On 6/26/24 5:29 AM, Andrew Schofield wrote: Hi Alieh, Thanks for the KIP. I think we’ve settled on a good solution. +1 (non-binding) Thanks, Andrew On 25 Jun 2024, at 13:17, Alieh Saeedi wrote: Hi all, I would like to open voting for

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Thanks Alieh, it seems this KIP can just pick between a couple of tradeoffs. Adding an overloaded `send()` as the KIP propose makes sense to me and seems to provides the cleanest solution compare to there options we discussed. Given the explicit name of the passed-in option that highlights

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

producer callback? On Mon, Jun 24, 2024 at 9:21 PM Matthias J. Sax wrote: My point it, that it does not seem to be safe to allow users to ignore errors with an implicit flush, and I think it's better to only allow it with (ie, after) an explicit flush(). My reasoning is, that users

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

action(new CommitTransactionOptions().transactionVerified(true)) I’d tend towards the enum here because I doubt we need as much flexibility as the admin API requires. Thanks, Andrew On 24 Jun 2024, at 18:39, Matthias J. Sax wrote: I am ok either way (ie, flush or commit), but I think we need

[jira] [Resolved] (KAFKA-16965) Add a "root cause" exception as a nested exception to TimeoutException for Producer

[ https://issues.apache.org/jira/browse/KAFKA-16965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16965. - Fix Version/s: 3.9.0 Resolution: Fixed > Add a "root cause&qu

Re: 回复: apply a jira account

发件人: Matthias J. Sax 发送时间: 2024年6月24日 05:43 收件人: dev@kafka.apache.org 主题: Re: apply a jira account Thanks for you interest to contribute to Kafka. Unfortunately, I could not find your account. Can you double check the Jira ID? -Matthias On 6/23/24 8:01 AM, 周 正轲 wrote: Hi Guys

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

(1) is NO and to question (2) is YES, do we need to change the current `flush` or `commitTnx` at all? Cheers, Alieh On Sat, Jun 22, 2024 at 3:21 AM Matthias J. Sax wrote: Hey Kirk, can you elaborate on a few points? Otherwise users would have to know to explicitly change their code

Re: apply a jira account

Thanks for you interest to contribute to Kafka. Unfortunately, I could not find your account. Can you double check the Jira ID? -Matthias On 6/23/24 8:01 AM, 周 正轲 wrote: Hi Guys, I want to contribute to Apache Kafka. Would you please give me the permission as a contributor? My JIRA ID is

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

Matthias J. Sax wrote: I believe the class name was picked on purpose, to make clear that it should not be used -- the problem is, that the class is in a public package and is by itself public (that's unfortunately require, given how Java works). Of course, it's also in the JavaDocs that the class

Re: [VOTE] KIP-1056 - Deprecate `default.` prefix for exception handler in StreamsConfig

+1 (binding) from my side. I understand the concerns raised, but would personally move forward with this KIP as-is. If we cannot get three votes, it would get naturally discarded. -Matthias On 6/19/24 11:33 AM, Muralidhar Basani wrote: Hi all, I would like to call a vote on KIP-1056 -

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

ould still be to put the producer into error state, and the user would need to call `abortTransaction()` to move forward. -Matthias On 6/21/24 5:26 PM, Kirk True wrote: Hi Matthias, On Jun 21, 2024, at 12:28 PM, Matthias J. Sax wrote: If we want to limit it to `RecordTooLargeException` thr

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

noticed by the producer before send() returns, so it should be possible to throw directly. Cheers, Chris On Fri, Jun 21, 2024, 14:25 Matthias J. Sax wrote: Not sure if we can change send and make it throw, given that send() is async? That is why users can register a `Callback` to begin

[jira] [Created] (KAFKA-17019) Producer TimeoutException should include root cause

Matthias J. Sax created KAFKA-17019: --- Summary: Producer TimeoutException should include root cause Key: KAFKA-17019 URL: https://issues.apache.org/jira/browse/KAFKA-17019 Project: Kafka

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

n fatal errors. But I guess we'll just have to clearly document it. In some way we are basically adding a flag to optionally restore the https://issues.apache.org/jira/browse/KAFKA-9279 bug, which is the motivation for all these changes, anyway :-). -Artem On Thu, Jun 20, 2024 at 2:18 PM Matt

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

spun up inside it. Cheers, Chris On Wed, Jun 19, 2024 at 10:32 PM Matthias J. Sax wrote: Thanks for the KIP Alieh. I actually like the KIP as-is, but think Arthem raises very good points... Seems we have four options on how to move forward? 1. add config to allow "silent error clea

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Thanks for the KIP Alieh. I actually like the KIP as-is, but think Arthem raises very good points... Seems we have four options on how to move forward? 1. add config to allow "silent error clearance" as the KIP proposes 2. change flush() to clear error and let it throw 3. add new

[jira] [Created] (KAFKA-16994) Flaky Test SlidingWindowedKStreamIntegrationTest.shouldRestoreAfterJoinRestart

Matthias J. Sax created KAFKA-16994: --- Summary: Flaky Test SlidingWindowedKStreamIntegrationTest.shouldRestoreAfterJoinRestart Key: KAFKA-16994 URL: https://issues.apache.org/jira/browse/KAFKA-16994

[jira] [Created] (KAFKA-16993) Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener.shouldInvokeUserDefinedGlobalStateRestoreListener()

Matthias J. Sax created KAFKA-16993: --- Summary: Flaky test RestoreIntegrationTest.shouldInvokeUserDefinedGlobalStateRestoreListener.shouldInvokeUserDefinedGlobalStateRestoreListener() Key: KAFKA-16993 URL: https

[jira] [Resolved] (KAFKA-16992) Flaky Test org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once_v2]

[ https://issues.apache.org/jira/browse/KAFKA-16992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16992. - Resolution: Duplicate > Flaky T

[jira] [Created] (KAFKA-16992) Flaky Test org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once_v2]

Matthias J. Sax created KAFKA-16992: --- Summary: Flaky Test org.apache.kafka.streams.integration.EOSUncleanShutdownIntegrationTest.shouldWorkWithUncleanShutdownWipeOutStateStore[exactly_once_v2] Key: KAFKA-16992

[jira] [Created] (KAFKA-16991) Flaky Test org.apache.kafka.streams.integration.PurgeRepartitionTopicIntegrationTest.shouldRestoreState

Matthias J. Sax created KAFKA-16991: --- Summary: Flaky Test org.apache.kafka.streams.integration.PurgeRepartitionTopicIntegrationTest.shouldRestoreState Key: KAFKA-16991 URL: https://issues.apache.org/jira/browse

Re: [VOTE] KIP-1038: Add Custom Error Handler to Producer

I just replied to the DISCUSS thread, proposing to DISCARD this KIP in favor of a different solution to the problem. -Matthias On 5/20/24 9:27 AM, Kirk True wrote: +1 (non-binding) Thanks Alieh! On May 20, 2024, at 6:00 AM, Walker Carlson wrote: Hey Alieh, Thanks for the KIP. +1

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

.apache.org/37/javadoc/org/apache/kafka/connect/mirror/DefaultReplicationPolicy.html#INTERNAL_TOPIC_SEPARATOR_ENABLED_CONFIG Cheers, Chris On Wed, May 8, 2024 at 12:37 AM Matthias J. Sax < mj...@apache.org wrote: Very interesting discussion. Seems a central point is the question ab

Re: [DISCUSS] Apache Kafka 3.9.0 release

In general I prefer to stick with the original time frame, but I believe an additional month would be good for KIP-848. Maybe we can cut down 2 weeks for the 4.1 and 4.2 releases each, to make up for the time? Not a must; just an idea. We can also just accept a one-time shift of the release

Re: [DISCUSS] Apache Kafka 3.8.0 release

are optional in some sense and can be descoped if time runs out. For example, we can drop support for JDK 8 without immediately refactoring everything that could benefit from the improvements in JDK9+. best, Colin On Fri, Jun 14, 2024, at 15:37, Matthias J. Sax wrote: That's my understanding, and I

Re: [VOTE] 3.7.1 RC1

Replied on https://issues.apache.org/jira/browse/KAFKA-16960 On 6/14/24 6:05 AM, Luke Chen wrote: Hi all, After running system tests, here are failing tests: 1. kafkatest.tests.core.upgrade_test failed at lz4 and snappy tests: KAFKA-16962 2.

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1 Btw: I just looked into Jira, and we have 90 tickets with "fixed version" set to 4.0.0. I would assume that most of them need to be updated to 3.9.0? https://issues.apache.org/jira/browse/KAFKA-16924?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%204.0.0 For anybody who has context

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

he new one is not set. If bothare set, the new one wins and we just log a WARN that the old one isignored in favor of the new one. -- It might be best to stick theestablished pattern?* Agree, we should follow the same pattern. Updated kip. Thanks, Murali On Fri, Jun 14, 2024 at 3:12 AM Matthias J.

Re: [DISCUSS] Apache Kafka 3.8.0 release

That's my understanding, and I would advocate strongly to keep the 4.0 release schedule as planed originally. The 3.9 one should really be an additional "out of schedule" release not impacting any other releases. -Matthias On 6/14/24 1:29 PM, David Jacot wrote: The plan sounds good to me.

Re: [QUESTION] What to do about conflicted KIP numbers?

I don't think that there is an official guideline. Personally, I would suggest that the corresponding KIP owners agree who is keeping the conflicting number, and how is changing it. For the ones changing the number, I would propose to restart a new DISCUSS thread using the new number to

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

nge. Similarly in "Test Plan" section Updated kip. 102: The "Deprecation" section should explain the behavior if both, old and new configs, are set. Updated kip. I think a ConfigException has to be thrown here if both configs are set. Thanks, Murali On Thu, Jun 13, 20

Re: [DISCUSS] KIP-1052: Align the naming convention for config and default variables in *Config classes

Might be good to restart a new discuss thread using the new KIP-1035 number to split both threads? I added the KIP to the table in the wiki page in the mean time. In Kafka Streams we did rename config names in the past in the exact same way as proposed: deprecate the old config, and add the

[jira] [Resolved] (KAFKA-16001) Migrate ConsumerNetworkThreadTest away from ConsumerTestBuilder

[ https://issues.apache.org/jira/browse/KAFKA-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16001. - Resolution: Fixed > Migrate ConsumerNetworkThreadTest away from ConsumerTestBuil

[jira] [Resolved] (KAFKA-13298) Improve documentation on EOS KStream requirements

[ https://issues.apache.org/jira/browse/KAFKA-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13298. - Fix Version/s: 3.1.0 Assignee: Andy Chambers Resolution: Fixed

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

, and second map to get back the initial key). Second map may imply a second repartitioning. However, if we do a per-task deduplication, the user may adapt to his specific use-case. Let me know what you think Ayoub Le mar. 11 juin 2024 à 20:21, Matthias J. Sax a écrit : Thanks Sebastien, that

Re: Subscribe for discussion on Kafka® development

This is self service. Please follow the instructions from the web page: https://kafka.apache.org/contact -Matthias On 6/11/24 9:38 AM, Punsak Incham wrote: I want to subscribe for discussion on Kafka® development. Best regards, A black and blue logo Description automatically generated

Re: [VOTE] KIP-1035: StateStore managed changelog offsets

Thanks Nick. +1 (binding) Looking forward to get this all merged! -Matthias On 6/12/24 9:36 AM, Nick Telford wrote: Hi everyone, I'd like to call a vote on KIP-1035[1]. Regards, Nick 1: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1035%3A+StateStore+managed+changelog+offsets

Re: [VOTE] KIP-1049: Add config log.summary.interval.ms to Kafka Streams

+1 (binding) On 6/11/24 1:17 PM, Sophie Blee-Goldman wrote: +1 (binding) Thanks for the KIP! On Tue, Jun 11, 2024 at 5:37 AM jiang dou wrote: HI I would like to start a vote for KIP-1049: Add config log.summary.interval.ms to Kafka Streams KIP：

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

Matthias J. Sax wrote: Shaswhat, any updates on this KIP? -- I still think that recommending to use `InternalFixedKeyRecordFactory` is not the best way to write test code. Changing `FixedKeyRecord` constructor (as I mentioned in my last email) might not be a good solution either. Maybe a cleaner

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

the experts in this case :) Thanks, Greg On Mon, Jun 10, 2024 at 3:47 PM Matthias J. Sax wrote: Thanks for the KIP Murali, Overall LGTM. A few comments. 100: Config names are part of the public interface, and the KIP should not say "none" in this section, but call out which configs are

[jira] [Resolved] (KAFKA-13561) Consider deprecating `StreamsBuilder#build(props)` function

[ https://issues.apache.org/jira/browse/KAFKA-13561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13561. - Resolution: Duplicate > Consider deprecating `StreamsBuilder#build(props)` funct

[jira] [Created] (KAFKA-16945) Cleanup StreamsBuilder and TopologyConfig

Matthias J. Sax created KAFKA-16945: --- Summary: Cleanup StreamsBuilder and TopologyConfig Key: KAFKA-16945 URL: https://issues.apache.org/jira/browse/KAFKA-16945 Project: Kafka Issue Type

[jira] [Resolved] (KAFKA-15600) KIP-990: Capability to PAUSE Tasks on DeserializationException

[ https://issues.apache.org/jira/browse/KAFKA-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15600. - Resolution: Won't Fix > KIP-990: Capability to PAUSE Tasks on DeserializationExcept

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

e detected. This is why I would have created two different methods, as is done for GroupBy: deduplicateByKey(...) deduplicate(...) If deduplicateByKey is used, the input stream does not need to be repartitioned. thanks Sébastien De : Matthias J. Sax Envoyé : mar

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

, it would become a non-user-facing / internal change to the "factory" we provide. -Matthias On 5/22/24 12:02 AM, Matthias J. Sax wrote: I was not aware of `InternalFixedKeyRecordFactory`. As the name indicates, it's considered an internal class, so not sure if we should recommend to u

Re: [DISCUSS] KIP-1049: Add config log.summary.interval.ms to Kafka Streams

Thanks for updating the KIP. Overall LGTM. However, "rejected alternatives" still says: add a new configuration log.summary.interval.ms to control the output interval of summary log This seems not to be rejected, but what the KIP actually proposes and should be removed? I think you can

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

be up to the user to choose any deduplicationInterval that may sufficiently cover all his late data. What do you think ? Thanks, Ayoub Le mar. 4 juin 2024 à 23:58, Matthias J. Sax a écrit : Ayoub, thanks for resurrecting this KIP. I think a built-in de-duplication operator will be ver

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Thanks for this KIP. Great to see it. I would assume it will make KIP-691 unnecessary? I don't think I fully understand the proposal yet. It's clear, that you propose to add new sub-classed to group existing exceptions. But it's not clear to me, which of the existing exceptions (which

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

Thanks for the KIP Murali, Overall LGTM. A few comments. 100: Config names are part of the public interface, and the KIP should not say "none" in this section, but call out which configs are deprecated and which ones are newly added. 101: Nit. In "Propose Changes" there is the template

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

ase. -- Cheers, Nick On Tue, 4 Jun 2024 at 20:34, Matthias J. Sax wrote: Nick, Thanks a lot for updating the KIP. I made a pass over it. Overall LGTM. A few nits and some more minor questions: 200: nit (Javadocs for `StateStore.managesOffsets()`): This is highly recommended, if possible, t

[jira] [Resolved] (KAFKA-14567) Kafka Streams crashes after ProducerFencedException

[ https://issues.apache.org/jira/browse/KAFKA-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14567. - Assignee: (was: Kirk True) Resolution: Fixed Not 100% sure either, but I feel

[jira] [Resolved] (KAFKA-16911) Kafka Streams topology optimization docs incomplete

[ https://issues.apache.org/jira/browse/KAFKA-16911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16911. - Fix Version/s: 3.8.0 Resolution: Fixed > Kafka Streams topology optimizat

[jira] [Created] (KAFKA-16911) Kafka Streams topology optimization docs incomplete

Matthias J. Sax created KAFKA-16911: --- Summary: Kafka Streams topology optimization docs incomplete Key: KAFKA-16911 URL: https://issues.apache.org/jira/browse/KAFKA-16911 Project: Kafka

[jira] [Resolved] (KAFKA-16903) Task should consider producer error previously occurred for different task

[ https://issues.apache.org/jira/browse/KAFKA-16903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16903. - Resolution: Fixed > Task should consider producer error previously occur

Re: [DISCUSS] KIP-1049: Add config log.summary.interval.ms to Kafka Streams

Jiang, Thanks for the KIP. I think it make sense. I agree with Sophie that the KIP writeup should be improved a little bit, with regard to public API (ie, config) which are changed. The only other idea I had to avoid this issue would be some internal change: we would introduce a new logger

Re: [DISCUSS] KIP-655: Add deduplication processor in kafka-streams

Ayoub, thanks for resurrecting this KIP. I think a built-in de-duplication operator will be very useful. Couple of questions: 100: `deduplicationKeySelector` Is this the best name? It might indicate that we select a "key" what is an overloaded term... Maybe we could use `Field` or `Id`

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

going to be working on 1035 starting today. I'll update the KIP first, and then call a vote. Regards, Nick On Wed, 29 May 2024 at 07:25, Bruno Cadonna wrote: Totally agree on moving forward and starting the VOTE! However, the KIP should be updated with the new info before starting the VOT

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

_TASK_ASSIGNED_MULTIPLE_TIMES, ACTIVE_AND_STANDBY_TASK_ASSIGNED_TO_SAME_KAFKASTREAMS, INVALID_STANDBY_TASK, UNKNOWN_PROCESS_ID, UNKNOWN_TASK_ID } Anything missing? (also updated all the code block headings, thanks for noticing that Bruno) On Fri, May 3, 2024 at 9

Re: Hello world - please subscribe me!

Chris, Subscription is self-service. Please follow the instruction from the web page: https://kafka.apache.org/contact -Matthias On 6/3/24 1:59 AM, Chris wrote:

[jira] [Created] (KAFKA-16863) Consider removing `default.` prefix for exception handler config

Matthias J. Sax created KAFKA-16863: --- Summary: Consider removing `default.` prefix for exception handler config Key: KAFKA-16863 URL: https://issues.apache.org/jira/browse/KAFKA-16863 Project

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

due to the expected higher latency since it happens actually before the Kafka Streams client joins the rebalance. Best, Bruno On 5/4/24 12:05 AM, Matthias J. Sax wrote: That's good questions... I could think of a few approaches, but I admit it might all be a little bit tricky to code up... H

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

/main/src/test/java/com/example/demo/MyFixedKeyProcessorTest.java On Mon, May 20, 2024 at 9:05 PM Matthias J. Sax wrote: Had a discussion on https://issues.apache.org/jira/browse/KAFKA-15242 and it was pointed out, that we also need to do something about `FixedKeyRecord`. It does not have

Re: Permission to contribute to Apache Kafka

You should be all set. On 5/21/24 8:30 AM, Harry Fallows wrote: Hello, I am following the[Getting Started guide for writing KIPs](https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals). Could someone give me the permissions to write a KIP please? My Wiki ID is

Re: Request for Authorization to Create KIP

You should be all set. On 5/21/24 6:58 PM, 黃竣陽 wrote: I want to create a KIP, and my wiki id : m1a2st and Jira id : m1a2st, Thanks for your help. jiang dou 於 2024年5月22日 上午9:01 寫道： You should send your jira ID and wiki ID， Please refer to this address ：

[jira] [Resolved] (KAFKA-15242) FixedKeyProcessor testing is unusable

[ https://issues.apache.org/jira/browse/KAFKA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15242. - Assignee: (was: Alexander Aghili) Resolution: Duplicate > FixedKeyProces

Re: [DISCUSS] KIP-1027 Add MockFixedKeyProcessorContext

would extend `FixedKeyRecord`? -- Or maybe an even better solution? I could not think of something else so far. Thoughts? On 5/3/24 9:46 AM, Matthias J. Sax wrote: Please also update the KIP. To get a wiki account created, please request it via a commet on this ticket: https

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

of a more obvious metric name. Nit: the KIP says it's a store-level metric, but I think it would be good to say explicitly that it's recorded with DEBUG level only? Yes, I've already updated the KIP with this information in the table. Regards, Nick On Sun, 31 Mar 2024 at 10:53, Matthias J. S

Re: [VOTE] KIP-989: RocksDB Iterator Metrics

+1 (binding) On 5/14/24 9:19 AM, Lucas Brutschy wrote: Hi Nick! Thanks for the KIP. +1 (binding) On Tue, May 14, 2024 at 5:16 PM Nick Telford wrote: Hi everyone, I'd like to call a vote on the Kafka Streams KIP-989: RocksDB Iterator Metrics:

Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing

+1 (binding) On 5/13/24 5:54 PM, Sophie Blee-Goldman wrote: Thanks for the KIP guys! +1 (binding) On Mon, May 13, 2024 at 6:02 AM Bill Bejeck wrote: Thanks for the KIP, this will be a great addition! +1(binding) -Bill On Fri, May 3, 2024 at 4:48 AM Bruno Cadonna wrote: Hi Damien,

Re: Creating kafka wiki id

Self-service to create an account is currently not working. Please reply on https://issues.apache.org/jira/browse/INFRA-25451 to request a wiki account. I'll update the wiki page for now until the issue is resolved. -Matthias On 5/7/24 8:25 AM, 黃竣陽 wrote: Hello, I want to create a KIP, but

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

ProductionExceptionHandler... Just explaining my reasoning -- in the end I leave it up to the KIP authors and anyone who will actually be using this feature in their applications :) On Tue, May 7, 2024 at 8:35 PM Matthias J. Sax wrote: @Loic, yes, what you describe is exactly what I had i

Re: [DISCUSS] KIP-1038: Add Custom Error Handler to Producer

rRecord as the input parameter of the handle() method as well. - increased the response types to 3 to have fail and two types of continue. - The default behaviour is having no custom handler, having the corresponding config parameter set to null. Therefore, the KIP provides no default

  1   2   3   4   5   6   7   8   9   10   >