[jira] [Resolved] (KAFKA-16350) StateUpdater does not init transaction after canceling task close action

[ https://issues.apache.org/jira/browse/KAFKA-16350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-16350. --- Resolution: Fixed > StateUpdater does not init transaction after canceling task cl

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

the TaskAssignmentUtils, I think that makes sense. LGTM. On Fri, May 3, 2024 at 2:26 AM Bruno Cadonna wrote: Hi Sophie, 117f: I think, removing the STATEFUL and STATELESS types is not enough to avoid the error Guozhang mentioned. The StreamsPartitionAssignor passes the information whethe

Re: [VOTE] KIP-1036: Extend RecordDeserializationException exception

Thanks! +1 (binding) Best, Bruno On 5/13/24 8:38 PM, Kirk True wrote: +1 (non-binding) Thanks Fred! On May 13, 2024, at 5:46 AM, Bill Bejeck wrote: Thanks for the KIP! +1 (binding) -Bill On Tue, May 7, 2024 at 6:16 PM Sophie Blee-Goldman wrote: +1 (binding) thanks for the KIP!

Re: [VOTE] KIP-924: customizable task assignment for Streams

Thanks for the KIP! Looking forward to a well-structured task assignor! +1 (binding) Best, Bruno On 5/3/24 2:44 AM, Matthias J. Sax wrote: I left one more nit on the discuss thread. But overall LGTM. +1 (binding) Thanks Rohan and Sophie for driving this KIP. -Matthias On 4/29/24 2:07

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

d enough to justify the introduced complexity? In the, for soft failures it's still a win. Just want to make sure we understand the limitations and make an educated decision. Or do I miss something? -Matthias On 5/3/24 10:20 AM, Bruno Cadonna wrote: Hi Matthias, 200: I like the idea in general. Howe

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

t we would get a large percentage of opened stores assigned as tasks anyway, this could work? -Matthias On 5/3/24 1:29 AM, Bruno Cadonna wrote: Hi Matthias, 101: Let's assume a RocksDB store, but I think the following might be true also for other store implementations. With this KIP, if Kaf

Re: [VOTE] KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing

Hi Damien, Sébastien, and Loïc, Thanks for the KIP! +1 (binding) Best, Bruno On 4/26/24 4:00 PM, Damien Gasparina wrote: Hi all, We would like to start a vote for KIP-1033: Add Kafka Streams exception handler for exceptions occurring during processing The KIP is available on

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

Hi, the KIP looks great! public static final String PROCESS_EXCEPTION_HANDLER_CLASS_CONFIG = "process.exception.handler". needs to be changed to public static final String PROCESSING_EXCEPTION_HANDLER_CLASS_CONFIG = "processing.exception.handler". The name of the constant has been

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

stale (ie, still contains the checkpoint when we opened the store when we got a task assigned). -Matthias On 4/30/24 2:40 AM, Bruno Cadonna wrote: Hi all, 100 I think we already have such a wrapper. It is called AbstractReadWriteDecorator. 101 Currently, the position is checkpoint

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

hat we've discussed (and will fix the remaining #finalAssignment mention as well, thanks Bruno. Glad to have such good proof readers! :P) On Tue, Apr 30, 2024 at 8:35 AM Bruno Cadonna wrote: Hi again, I forgot to ask whether you could add the agreement about handling invalid assignment to

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Hi again, I forgot to ask whether you could add the agreement about handling invalid assignment to the KIP. Best, Bruno On 4/30/24 2:00 PM, Bruno Cadonna wrote: Hi all, I think we are converging! 117 a) fail: Since it is an invalid consumer assignment b) pass: I agree that not assigning

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Hi all, I think we are converging! 117 a) fail: Since it is an invalid consumer assignment b) pass: I agree that not assigning a task might be reasonable in some situations c) fail: For the reasons Lucas pointed out. I am missing a good use case here. d) fail: It is invalid Somewhere in

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

the face the fact that KS is getting more and more adopted and as a more mature product should support this? -Matthias On 4/21/24 11:58 PM, Bruno Cadonna wrote: Hi all, How should we proceed here? 1. with the plain .checkpoint file 2. with a way to use the state store interface on

Re: [ANNOUNCE] New committer: Igor Soarez

Congrats! Best, Bruno Am 25. April 2024 05:18:19 MESZ schrieb Yash Mayya : >Congratulations Igor! > >On Wed, 24 Apr, 2024, 23:36 Colin McCabe, wrote: > >> Hi all, >> >> The PMC of Apache Kafka is pleased to announce a new Kafka committer, Igor >> Soarez. >> >> Igor has been a Kafka contributor

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

, I do think it makes sense to leverage it instead of creating a new metric. I will update the KIP to update the dropped-record-metric. 8. Regarding the DSL, I am aligned with Bruno, I think we could close the gaps in a future KIP. Cheers, Damien On Thu, 11 Apr 2024 at 11:56, Bruno Cadonna

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

should consider the interface approach afterwards, though. Best, Bruno On 4/17/24 3:15 PM, Bruno Cadonna wrote: Hi Nick and Sophie, I think the task ID is not enough to create a state store that can read the offsets of non-assigned tasks for lag computation during rebalancing. The state

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

probably why it doesn't follow the same naming pattern. Personally I'm fine either way (I do think ApplicationMetadata sounds a bit better but that's not a good enough reason :P) Thanks Bruno! On Wed, Apr 17, 2024 at 7:08 AM Bruno Cadonna wrote: Hi, sorry, I am late to the party. I have a c

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

Hi, sorry, I am late to the party. I have a couple of comments: (1) I would prefer Client* instead of Node* in the names. In Kafka Streams we do not really have the concept of node but we have the concept of client (admittedly, we sometimes also use instance). I would like to avoid

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

expensive, but the devil will be in the detail. I'll try to find some time to explore the idea to see if it's possible and report back, because we'll need to determine this before we can vote on the KIP. Regards, Nick On Wed, 10 Apr 2024 at 11:36, Bruno Cadonna wrote: Hi Nick

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

Congratulations, Greg! Best, Bruno On 4/15/24 7:33 AM, Claude Warren wrote: Congrats Greg! All the hard work paid off. On Mon, Apr 15, 2024 at 6:58 AM Ivan Yurchenko wrote: Congrats Greg! On Sun, Apr 14, 2024, at 22:51, Sophie Blee-Goldman wrote: Congrats Greg! Happy to have you On

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

100 Clermont-Ferrand ---- *De :* Bruno Cadonna *Envoyé :* mercredi 10 avril 2024 10:38 *À :* dev@kafka.apache.org *Objet :* [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing Warning External sender Do not click on a

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

if they're not open. If the StateManager (aka. ProcessorStateManager/GlobalStateManager) proves too expensive to hold open for closed stores, we could always have a "StubStateManager" in its place, that enables the querying of offsets, but nothing else? IDK, what do you think? Regards, Nick On T

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

this changes as well. -Matthias On 4/1/24 10:33 PM, Bruno Cadonna wrote: Hi Walker and Matthias, (2) That is exactly my point about having a compile time error versus a runtime error. The added flexibility as proposed by Matthias sounds good to me. Regarding the Named parameter, I was not aware

Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

ng along the same lines as Bruno. I'm wondering if we can make do with a task-level metric at the INFO level and the processor metric at DEBUG. IMHO, when it comes to tracking exceptions in processing, these two areas are where users will want to focus, higher level metrics wouldn't be as useful

[jira] [Reopened] (KAFKA-15538) Client support for java regex based subscription

[ https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reopened KAFKA-15538: --- > Client support for java regex based subscript

Re: [DISCUSS] KIP-1035: StateStore managed changelog offsets

Hi Nick, Thanks for breaking out the KIP from KIP-892! Here a couple of comments/questions: 1. In Kafka Streams, we have a design guideline which says to not use the "get"-prefix for getters on the public API. Could you please change getCommittedOffsets() to committedOffsets()? 2. It is

Re: [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

/enums you want to add? Best, Bruno On 4/9/24 10:17 AM, Bruno Cadonna wrote: Hi Loïc, Damien, and Sébastien, Thanks for the KIP! I find it really great that you contribute back to Kafka Streams concepts you developed for kstreamplify so that everybody can take advantage from your improvements

Re: [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for exceptions occuring during processing

Hi Loïc, Damien, and Sébastien, Thanks for the KIP! I find it really great that you contribute back to Kafka Streams concepts you developed for kstreamplify so that everybody can take advantage from your improvements. I have a couple of questions/comments: 1. and 2. I am wondering whether

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

ore experience would have a sudden inspiration for how to solve both issues with one API! Anyway, I'll slide back into the lurking shadows for now and let the discussion continue :) Cheers, Almog On Tue, Mar 26, 2024 at 4:22 AM Bruno Cadonna wrote: Hi Almog, Do you mean a API to configure

Re: [ANNOUNCE] New committer: Christo Lolov

Congrats, Christo! Well deserved! Best, Bruno On 3/26/24 1:04 PM, Luke Chen wrote: Hi, Everyone, The PMC of Apache Kafka is pleased to announce a new Kafka committer: Christo Lolov. Christo has been a Kafka contributor since 2021. He has made over 50 commits. He authored KIP-902, KIP-963,

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

ore option should go for maximum flexibility. Best, Walker On Fri, Mar 22, 2024 at 10:01 AM Bruno Cadonna wrote: Hi Walker, A couple of follow-up questions. 1. Why do you propose to explicitly pass a parameter "storeName" in StreamsBuilder#addGlobalStore? The StoreBuilder should alread

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

ught about it but I'm not convinced it is a necessary restriction. It might be useful for the already defined processors but then they might as well use the `globalTable` method. I think the add state store option should go for maximum flexibility. Best, Walker On Fri, Mar 22, 2024 at 10:01 AM Bru

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

Hi Walker, A couple of follow-up questions. 1. Why do you propose to explicitly pass a parameter "storeName" in StreamsBuilder#addGlobalStore? The StoreBuilder should already provide a name for the store, if I understand the code correctly. I would avoid using the same name for the source

Re: [DISCUSS] Apache Kafka 3.6.2 release

Thanks Manikumar! +1 Best, Bruno On 3/13/24 5:56 PM, Josep Prat wrote: +1 thanks for volunteering! Best --- Josep Prat Open Source Engineering Director, aivenjosep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari

[jira] [Reopened] (KAFKA-10199) Separate state restoration into separate threads

[ https://issues.apache.org/jira/browse/KAFKA-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reopened KAFKA-10199: --- > Separate state restoration into separate thre

Re: [DISCUSS] KIP-1024: Make the restore behavior of GlobalKTables with custom processors configureable

Hi Walker, Thanks for the KIP! Great that you are going to fix this long-standing issue! 1. I was wondering if we need the timestamp extractor as well as the key and value deserializer in Topology#addGlobalStore() that do not take a ProcessorSupplier? What about Consumed in

Re: Subscribe to Developer mailing list

Hi LoÏC, subscription to the mailing lists is self-service. See details under https://kafka.apache.org/contact Best, Bruno On 2/29/24 9:48 AM, Loic Greffier wrote: Hi @dev@kafka.apache.org , I am working as a Software Engineer at Michelin, and would like to

Re: Permissions to contribute to Apache Kafka project

Hi Paul, I gave you the permission in JIRA. Currently, all Apache projects have this issue with the wiki account creation. There is a ongoing INFRA ticket about it: https://issues.apache.org/jira/browse/INFRA-25451 Sorry about that! In any case, thank you for your interest in Apache Kafka!

Re: [ANNOUNCE] Apache Kafka 3.7.0

, Arpit Goyal, Artem Livshits, Ashwin Pankaj, ashwinpankaj, atu-sharm, bachmanity1, Bob Barrett, Bruno Cadonna, Calvin Liu, Cerchie, chern, Chris Egerton, Christo Lolov, Colin Patrick McCabe, Colt McNealy, Crispin Bernier, David Arthur, David Jacot, David Mao, Deqi Hu, Dimitar Dimitrov, Divij Vaidya

[jira] [Resolved] (KAFKA-9062) Handle stalled writes to RocksDB

[ https://issues.apache.org/jira/browse/KAFKA-9062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-9062. -- Resolution: Won't Fix > Handle stalled writes to Rock

Re: [DISCUSS] Apache Kafka 3.8.0 release

Thanks Josep! +1 Best, Bruno On 2/26/24 9:53 PM, Chris Egerton wrote: Thanks Josep, I'm +1 as well. On Mon, Feb 26, 2024 at 12:32 PM Justine Olshan wrote: Thanks Joesp. +1 from me. On Mon, Feb 26, 2024 at 3:37 AM Josep Prat wrote: Hi all, I'd like to volunteer as release manager for

[jira] [Resolved] (KAFKA-16194) KafkaConsumer.groupMetadata() should be correct when first records are returned

[ https://issues.apache.org/jira/browse/KAFKA-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-16194. --- Resolution: Fixed > KafkaConsumer.groupMetadata() should be correct when first reco

[jira] [Created] (KAFKA-16285) Make group metadata available when a new assignment is set in async Kafka consumer

Bruno Cadonna created KAFKA-16285: - Summary: Make group metadata available when a new assignment is set in async Kafka consumer Key: KAFKA-16285 URL: https://issues.apache.org/jira/browse/KAFKA-16285

Re: Improve flaky test reporting (KAFKA-12216)

Hi David, I guess you meant to say "This does not mean that we should NOT continue our effort to reduce the number of flaky tests." I totally agree with what you wrote. I am also +1 on considering all failures for unit tests. Best, Bruno On 2/12/24 9:11 AM, David Jacot wrote: Hi folks,

[jira] [Resolved] (KAFKA-16098) State updater may attempt to resume a task that is not assigned anymore

[ https://issues.apache.org/jira/browse/KAFKA-16098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-16098. --- Resolution: Fixed > State updater may attempt to resume a task that is not assigned anym

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

Congratulations Divij! Well deserved! Best, Bruno On 12/27/23 12:45 PM, Luke Chen wrote: Hi, Everyone, Divij has been a Kafka committer since June, 2023. He has remained very active and instructive in the community since becoming a committer. It's my pleasure to announce that Divij is now a

[jira] [Resolved] (KAFKA-15991) Flaky new consumer test testGroupIdNotNullAndValid

[ https://issues.apache.org/jira/browse/KAFKA-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-15991. --- Resolution: Fixed > Flaky new consumer test testGroupIdNotNullAndVa

[jira] [Created] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

Bruno Cadonna created KAFKA-16017: - Summary: Checkpointed offset is incorrect when task is revived and restoring Key: KAFKA-16017 URL: https://issues.apache.org/jira/browse/KAFKA-16017 Project

[jira] [Reopened] (KAFKA-9545) Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDeleted`

[ https://issues.apache.org/jira/browse/KAFKA-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reopened KAFKA-9545: -- > Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDele

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

concise. Best, Bruno On 12/12/23 9:08 AM, Bruno Cadonna wrote: Hi Alieh, I think using TimestampedRangeQuery to query the latest versions is totally fine. If it is not, users will report it and we can add it then. Best, Bruno On 12/11/23 6:22 PM, Alieh Saeedi wrote: Thank you all. I decided

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

eoffs I see. Not sure if three are other considerations that would tip the scale into either direction? -Matthias On 11/3/23 3:43 AM, Bruno Cadonna wrote: Hi Alieh, I like the examples! 1. Some terms like `asOf` in the descriptions still need to be replaced in the KIP. 2. In your last e-ma

Re: [ANNOUNCE] Apache Kafka 3.5.2

Congrats, Luke and thanks for running the release! Best, Bruno On 12/11/23 12:34 PM, Luke Chen wrote: The Apache Kafka community is pleased to announce the release for Apache Kafka 3.5.2 This is a bugfix release. It contains many bug fixes including upgrades the Snappy and Rocksdb

Re: [ANNOUNCE] Apache Kafka 3.6.1

Thanks for managing the release, Mickael! Best, Bruno On 12/8/23 12:44 PM, Luke Chen wrote: Hi Mickael, Thanks for running this release! Luke On Thu, Dec 7, 2023 at 7:13 PM Mickael Maison wrote: The Apache Kafka community is pleased to announce the release for Apache Kafka 3.6.1 This is

[jira] [Resolved] (KAFKA-14438) Throw error when consumer configured with empty/whitespace-only group.id for AsyncKafkaConsumer

[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-14438. --- Resolution: Fixed > Throw error when consumer configured with empty/whitespace-o

Re: [DISCUSS] KIP-997 Support fetch(fromKey, toKey, from, to) to WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

ult-time range would be "all / unbounded" ? 10: you mentioned that `WindowKeyQuery` functionality can be covered by `WindowRangeQuery`. I agree. For this case, it seems we want to deprecate `WindowKeyQuery` entirely? -Matthias On 11/16/23 1:19 AM, Bruno Cadonna wrote: Hi Hanyu, Tha

[jira] [Resolved] (KAFKA-15555) Ensure wakeups are handled correctly in PrototypeAsyncConsumer.poll()

[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-1. --- Resolution: Fixed > Ensure wakeups are handled correctly in PrototypeAsyncConsumer.p

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

s On 11/16/23 12:51 AM, Bruno Cadonna wrote: Hi, 80) We do not keep backwards compatibility with IQv1, right? I would even say that currently we do not need to keep backwards compatibility among IQv2 versions since we marked the API "Evolving" (do we only mean code compatibility here or

Re: [VOTE] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Thanks Alieh, I am +1 (binding). However, although we agreed on not specifying an order of the results by default, there is still the following sentence in the KIP: "The order of the returned records is by default ascending by timestamp. The method withDescendingTimestamps() can reverse

Re: [DISCUSS] KIP-997 Support fetch(fromKey, toKey, from, to) to WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

Hi Hanyu, Thanks for the KIP! 1) Could you please mark the pieces that you want to add to the API in the code listing in the KIP? You can add a comment like "// newly added" or similar. That would make reading the KIP a bit easier because one does not need to compare your code with the code

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

napshots for a consistent view here, before dropping this useful guarantee. Cheers, Lucas On Tue, Nov 14, 2023 at 2:20 PM Bruno Cadonna wrote: Hi Alieh, Regarding the semantics/guarantees of the query type: Do we need a snapshot semantic or can we specify a weaker but still useful semantic? A

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

guarantees as I described and if there is demand propose stricter guarantees in a follow-up KIP. Maybe there are also other simpler guarantees that make sense. Best, Bruno On 11/9/23 12:30 PM, Bruno Cadonna wrote: Hi, Thanks for the updates! First my take on previous comments: 50) I am in favor

Re: [VOTE] KIP-892: Transactional StateStores

Hi Nick! Thanks a lot for the KIP! Looking forward to the implementation! +1 (binding) Best, Bruno On 11/14/23 2:23 AM, Sophie Blee-Goldman wrote: +1 (binding) Thanks a lot for this KIP! On Mon, Nov 13, 2023 at 8:39 AM Lucas Brutschy wrote: Hi Nick, really happy with the final KIP.

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

arity and even review that ;-) Cheers, Alieh On Thu, Nov 2, 2023 at 7:13 PM Bruno Cadonna wrote: Hi Alieh, First of all, I like the examples. Is validTo in VersionedRecord exclusive or inclusive? In the javadocs you write: "@param validTothe latest timestamp that value is valid" I

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

I assume, results are returned by timestamp for each key. The KIP should be explicit about it. To be very explicit, should we rename the methods to specify the key bound? - withRange -> withKeyRange - withLowerBound -> withLowerKeyBound - withUpperBound -> withUpperKeyBou

Re: [VOTE] KIP-998: Give ProducerConfig(props, doLog) constructor protected access

Thanks for the KIP! +1 (binding) Best, Bruno On 11/3/23 7:55 AM, Chris Egerton wrote: +1 (binding) FWIW, I agree that this change should require a KIP. Gating upgrades of visibility from private or package-private to protected may be cumbersome, but it comes with the expectation that

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

(as private method) that can be called from `query()` method, but adding it to the interface seems to defeat the purpose of IQv2. Note, for existing IQv2 queries types that go against others stores, the public methods already existed when IQv2 was introduces, and thus the implementation of thes

Re: [VOTE] KIP-988 Streams StandbyUpdateListener

Hi Colt and Eduwer, Thanks for the KIP! +1 (binding) Best, Bruno On 10/26/23 7:17 PM, Matthias J. Sax wrote: +1 (binding) On 10/25/23 4:06 PM, Sophie Blee-Goldman wrote: Happy to see this -- that's a +1 (binding) from me On Mon, Oct 23, 2023 at 6:33 AM Bill Bejeck wrote: This is a

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

Congrats, Satish! Bruno On 10/29/23 2:42 PM, John Roesler wrote: Congratulations, Satish! -John On Sun, Oct 29, 2023, at 08:09, Randall Hauch wrote: Congratulations, Satish! On Sun, Oct 29, 2023 at 1:47 AM Tom Bentley wrote: Congratulations! On Sun, 29 Oct 2023 at 5:41 PM, Guozhang Wang

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

this option will force us to invoke StandbyUpdateLister once this interval is reached. On Mon, Oct 16, 2023 at 8:52 AM Bruno Cadonna wrote: Thanks for the KIP, Colt and Eduwer, Are you sure there is also not a significant performance impact for passing into the callback `currentEndOffset`? I am asking bec

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Fri, Oct 13, 2023 at 8:48 AM Nick Telford < nick.telf...@gmail.com> wrote: Hi Bruno, 4. I'll hold off on making that change until we have a consensus as to what configuration to use to control all of this, as it'll be affected by the decision on EOS isolation levels. 5. Done. I'v

[jira] [Created] (KAFKA-15625) Do not flush global state store at each commit

Bruno Cadonna created KAFKA-15625: - Summary: Do not flush global state store at each commit Key: KAFKA-15625 URL: https://issues.apache.org/jira/browse/KAFKA-15625 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15624) Reconsider synchronisation of methods in RocksDBStore

Bruno Cadonna created KAFKA-15624: - Summary: Reconsider synchronisation of methods in RocksDBStore Key: KAFKA-15624 URL: https://issues.apache.org/jira/browse/KAFKA-15624 Project: Kafka

Re: [DISCUSS] KIP-988 Streams Standby Task Update Listener

Thanks for the KIP, Colt and Eduwer, Are you sure there is also not a significant performance impact for passing into the callback `currentEndOffset`? I am asking because the comment here:

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

ED when processing.mode is anything other than at-least-once. Do you think this would be acceptable? In a later KIP, we can add support for query-time isolation levels and solve this particular problem there, which would relax this restriction. Regards, Nick On Tue, 19 Sept 2023 at 09:30, Bruno Cad

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

when processing.mode is anything other than at-least-once. Do you think this would be acceptable? In a later KIP, we can add support for query-time isolation levels and solve this particular problem there, which would relax this restriction. Regards, Nick On Tue, 19 Sept 2023 at 09:30, Bruno Cadonna wro

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

tthias J. Sax wrote: Bruno and I had some background conversation about the `get` prefix question including a few other committers. The official policy was never changed, and we should not add the `get`-prefix. It's a slip on our side in previous KIPs to add the `get`-prefix and we should actually

[jira] [Resolved] (KAFKA-15577) Reload4j | CVE-2022-45868

[ https://issues.apache.org/jira/browse/KAFKA-15577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-15577. --- Resolution: Not A Problem > Reload4j | CVE-2022-45

Re: [VOTE] KIP-960: Support single-key_single-timestamp interactive queries (IQv2) for versioned state stores

Thanks for the KIP, Alieh! +1 (binding) Best, Bruno On 10/10/23 1:14 AM, Matthias J. Sax wrote: One more nit: as discussed on the related KIP-698 thread, we should not use `get` as prefix for the getters. So it should be `K key()` and `Optional asOfTimestamp()`. Otherwise the KIP LGTM.

Re: KAFKA-15571 Review

Hi Levani, I think you found a bug and you are looking at the right place! I commented on the PR. Best, Bruno On 10/10/23 3:02 PM, Levani Kokhreidze wrote: Hello, We’ve been looking at a https://issues.apache.org/jira/browse/KAFKA-10575 but seems implementation has a bug and user defined

Re: Apache Kafka 3.7.0 Release

Thanks Stan! +1 Best, Bruno On 10/10/23 7:24 AM, Luke Chen wrote: Thanks Stanislav! On Tue, Oct 10, 2023 at 3:05 AM Josep Prat wrote: Thanks Stanislav! ——— Josep Prat Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer:

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

o omit `get`. -Matthias On 10/4/23 2:36 AM, Bruno Cadonna wrote: Hi, Regarding tombstones: As far as I understand, we need to add either a validTo field to VersionedRecord or we need to return tombstones, otherwise the result is not complete, because users could never know a record was

[jira] [Resolved] (KAFKA-10199) Separate state restoration into separate threads

[ https://issues.apache.org/jira/browse/KAFKA-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-10199. --- Resolution: Done > Separate state restoration into separate thre

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

some better name?) to allow specifying both bounds at once? The existing `RangeQuery` does the same for specifying the key-range, so might be good to add for time-range too? -Matthias On 9/6/23 5:01 AM, Bruno Cadonna wrote: In my last e-mail I missed to finish a sentence. "I think from a KIP

Re: [DISCUSS] KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2

Hi Hanyu, I agree with what others said about having a `withDescendingOrder()` method and about to document how the results are ordered. I would not add a reverse flag and adding a parameter to each method in RangeQuery. This makes the API less fluent and harder to maintain since the flag

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

Congrats, Justine! Well deserved! Best, Bruno On 9/25/23 5:28 AM, ziming deng wrote: Congratulations Justine! On Sep 25, 2023, at 00:01, Viktor Somogyi-Vass wrote: Congrats Justine! On Sun, Sep 24, 2023, 17:45 Kirk True wrote: Congratulations Justine! Thanks for all your great work!

[ANNOUNCE] New committer: Lucas Brutschy

Hi all, The PMC of Apache Kafka is pleased to announce a new Kafka committer Lucas Brutschy. Lucas' major contributions are around Kafka Streams. Lucas' significantly contributed to the state updater (https://issues.apache.org/jira/browse/KAFKA-10199) and he drives the implementation of

[ANNOUNCE] New committer: Yash Mayya

Hi all, The PMC of Apache Kafka is pleased to announce a new Kafka committer Yash Mayya. Yash's major contributions are around Connect. Yash authored the following KIPs: KIP-793: Allow sink connectors to be used with topic-mutating SMTs KIP-882: Kafka Connect REST API configuration

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

on the API for now, but I think it's important to consider how to implement the desired API, in case we come up with an API that cannot be implemented efficiently, or even at all! Thoughts? -- Nick On Wed, 13 Sept 2023 at 13:03, Bruno Cadonna wrote: Hi Nick, 6. Of course, you are right

[jira] [Resolved] (KAFKA-13973) block-cache-capacity metrics worth twice as much as normal

[ https://issues.apache.org/jira/browse/KAFKA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-13973. --- Resolution: Fixed > block-cache-capacity metrics worth twice as much as nor

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

ds-benchmark#results On Wed, 13 Sept 2023 at 09:41, Bruno Cadonna wrote: Hi Nick, Thanks for the updates and sorry for the delay on my side! 1. Making the default implementation for flush() a no-op sounds good to me. 2. I think what was bugging me here is that a third-party state store needs to

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

that downgrades should be uncommon, I think this is acceptable behaviour, as the end-state is consistent, even if it results in an undesirable state restore. Should I document the upgrade/downgrade behaviour explicitly in the KIP? -- Regards, Nick On Mon, 14 Aug 2023 at 22:31, Bruno Cadonna wrote: Hi Ni

Re: [DISCUSS] KIP-969: Support range interactive queries for versioned state stores

Hi Alieh, Thanks for the KIP! One high level comment/question: I assume you separated single key queries into two classes because versioned key queries return a single value and multi version key queries return iterators. Although, range queries always return iterators, it would make sense

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Hi Alieh, Thanks for the KIP! I think from a KIP 1. I propose to throw an IllegalArgumentException or an IllegalStateException for meaningless combinations. In any case, the KIP should specify what exception is thrown. 2. Why does not specifying a range return the latest version? I would

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

In my last e-mail I missed to finish a sentence. "I think from a KIP" should be "I think the KIP looks good!" On 9/6/23 1:59 PM, Bruno Cadonna wrote: Hi Alieh, Thanks for the KIP! I think from a KIP 1. I propose to throw an IllegalArgumentException or an I

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

f implementing your KIP: https://github.com/apache/kafka/blob/f23394336a7741bf4eb23fcde951af0a23a69bd0/streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredVersionedKeyValueStore.java#L142-L158 Best, Victoria On 2023/08/09 10:16:44 Bruno Cadonna wrote: Hi, I will use the in

[jira] [Resolved] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-15319. --- Resolution: Fixed > Upgrade rocksdb to fix CVE-2022-37

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

rks, and the complexity is drastically reduced, I may try bringing back Atomic Checkpoints into this KIP. Regards, Nick On Mon, 3 Jul 2023 at 15:27, Bruno Cadonna wrote: Hi Nick, Thanks for the insights! Very interesting! As far as I understand, you want to atomically update the state store from the

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

p" and "untilTimestamp". - About point 5: Do we need system tests here? I assumed just integration tests were enough. - Regarding long vs timestamp instance: I think yes, that 's why I used Long as timestamp. Bests, Alieh On Thu, Jul 27, 2023 at 2:28 PM Bruno C

[jira] [Created] (KAFKA-15297) Cache flush order might not be topological order

Bruno Cadonna created KAFKA-15297: - Summary: Cache flush order might not be topological order Key: KAFKA-15297 URL: https://issues.apache.org/jira/browse/KAFKA-15297 Project: Kafka Issue

Re: [VOTE] KIP-759: Unneeded repartition canceling

Hi, +1 (binding) Thanks for the KIP! Best, Bruno On 8/2/23 1:19 AM, Bill Bejeck wrote: I caught up on the discussion thread and the KIP LGTM. +1(binding) On Tue, Aug 1, 2023 at 3:07 PM Walker Carlson wrote: +1 (binding) On Mon, Jul 31, 2023 at 10:43 PM Matthias J. Sax wrote: +1

  1   2   3   4   5   6   7   >