[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752519#comment-17752519 ] Matthias J. Sax commented on KAFKA-12317: - Just catching on of comments... {quote}KAFKA-12845

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Priority: Critical (was: Major) > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Component/s: streams > Upgrade rocksdb to fix CVE-2022-37

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752150#comment-17752150 ] Matthias J. Sax commented on KAFKA-15297: - Seems we are on the same page :)  > Cache fl

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using exactly_once

2023-08-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752149#comment-17752149 ] Matthias J. Sax commented on KAFKA-15259: - The only thing I am wondering is, if we should try

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751836#comment-17751836 ] Matthias J. Sax commented on KAFKA-15302: - {quote}it's a combo of a cache snapshot

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751835#comment-17751835 ] Matthias J. Sax commented on KAFKA-15259: - Ah. Sorry. Meant https://issues.apache.org/jira

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:59 PM

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM: -- I

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM: -- I

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751761#comment-17751761 ] Matthias J. Sax commented on KAFKA-15302: - Thanks for providing more details; the `delete

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax commented on KAFKA-15259: - {quote}Should we create a new ticket for Kafka

[jira] [Commented] (KAFKA-14747) FK join should record discarded subscription responses

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751756#comment-17751756 ] Matthias J. Sax commented on KAFKA-14747: - We can discuss here on the ticket I guess. The class

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-07 Thread Matthias J. Sax
handling on the leader should typically assign the offset. I believe you can find that code here: https://github.com/apache/kafka/blob/b9a45546a7918799b6fb3c0fe63b56f47d8fcba9/core/src/main/scala/kafka/log/UnifiedLog.scala#L766 Justine On Tue, Aug 1, 2023 at 11:38 AM Matthias J. Sax wrote:

[jira] [Assigned] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15303: --- Assignee: Matthias J. Sax > Foreign key joins no longer triggered by eve

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer batches up multiple records into batches, and a single record

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax commented on KAFKA-15259: - I did sync up with [~cegerton] who worked on https

[jira] [Created] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15309: --- Summary: Add custom error handler to Producer Key: KAFKA-15309 URL: https://issues.apache.org/jira/browse/KAFKA-15309 Project: Kafka Issue Type: New

[jira] [Created] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15309: --- Summary: Add custom error handler to Producer Key: KAFKA-15309 URL: https://issues.apache.org/jira/browse/KAFKA-15309 Project: Kafka Issue Type: New

Re: Testing FixedKeyProcessor implementation using unit tests

2023-08-04 Thread Matthias J. Sax
-repro/blob/master/src/test/java/com/example/demo/MyFixedKeyProcessorTest.java Test is where I am having issues. Thanks, Zed -Original Message- From: Matthias J. Sax Sent: Tuesday, July 11, 2023 1:13 AM To: dev@kafka.apache.org Subject: Re: Testing FixedKeyProcessor implementation using

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751263#comment-17751263 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for digging into it – that's a good fine – I

[jira] [Updated] (KAFKA-15308) Wipe Stores upon OffsetOutOfRangeException in ALOS

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15308: Affects Version/s: (was: 3.4.0) (was: 3.5.0) > Wipe Sto

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-04 Thread Matthias J. Sax
e data source and replace it with CDC streaming data to Kafka. So in that case assembling data entities from Kafka topics instead of JDBC would be beneficial. Please let me know what you think. Regards, Igor On Tue, Jul 25, 2023 at 5:53 PM Matthias J. Sax wrote: Igor, thanks for the KIP. Int

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-04 Thread Matthias J. Sax
Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentation change, ie, point out that the passed in key could be `null` and similar? -Matthias On 8/2/23 3:20 PM, Guozhang Wang wrote: Thanks Florin for the writeup, One quick thing I'd like to bring up is

[jira] [Created] (KAFKA-15307) Kafka Streams configuration docs outdate

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15307: --- Summary: Kafka Streams configuration docs outdate Key: KAFKA-15307 URL: https://issues.apache.org/jira/browse/KAFKA-15307 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15307) Kafka Streams configuration docs outdate

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15307: --- Summary: Kafka Streams configuration docs outdate Key: KAFKA-15307 URL: https://issues.apache.org/jira/browse/KAFKA-15307 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751236#comment-17751236 ] Matthias J. Sax commented on KAFKA-15302: - Thanks for reporting this issue. – When you call

[jira] [Commented] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751212#comment-17751212 ] Matthias J. Sax commented on KAFKA-15303: - Thanks for filing this ticket. Kafka Streams as only

[jira] [Updated] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15303: Issue Type: Improvement (was: Bug) > Foreign key joins no longer triggered by eve

[jira] [Updated] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15303: Priority: Major (was: Critical) > Foreign key joins no longer triggered by eve

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750506#comment-17750506 ] Matthias J. Sax commented on KAFKA-15116: - Are you saying you are using a custom store

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750504#comment-17750504 ] Matthias J. Sax commented on KAFKA-15297: - The ticket description contains an example

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750503#comment-17750503 ] Matthias J. Sax commented on KAFKA-12829: - It's actually unclear when 4.0 will come along, so

[jira] [Updated] (KAFKA-15299) Support left stream-table join on foreign key

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15299: Labels: kip (was: ) > Support left stream-table join on foreign

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-01 Thread Matthias J. Sax
The _offset_ is the position of the record in the partition. The _sequence number_ is a unique ID that allows broker to de-duplicate messages. It requires the producer to implement the idempotency protocol (part of Kafka transactions); thus, sequence numbers are optional and as long as you

Re: [VOTE] KIP-759: Unneeded repartition canceling

2023-07-31 Thread Matthias J. Sax
+1 (binding) On 7/11/23 11:16 AM, Shay Lin wrote: Hi all, I'd like to call a vote on KIP-759: Unneeded repartition canceling The KIP has been under discussion for quite some time(two years). This is a valuable optimization for advanced users. I hope we can push this toward the finish line this

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-07-27 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748367#comment-17748367 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for getting back – I was just asking a general

Re: Consuming an entire partition with control messages

2023-07-27 Thread Matthias J. Sax
"1" in this kind of situation Best regards, Vincent On 13/06/2023 17:27, Matthias J. Sax wrote: Sounds like a bug in aiokafka library to me. If the last message in a topic partition is a tx-marker, the consumer should step over it, and report the correct position after the marker.

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-26 Thread Matthias J. Sax
efore feeling comfortable enough to vote on this would be a clear method signature (and maybe sample javadocs) in the "Public Interfaces" section. Thanks again for the KIP Shay! Hope I haven't dragged it out too much On Fri, Jul 21, 2023 at 3:19 PM Matthias J. Sax wrote: Some tho

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-07-26 Thread Matthias J. Sax
Thanks for the KIP Alieh. Glad to see that we can add IQ to the new versioned stores! Couple of questions: single-key lookup with timestamp (upper) bound Not sure if "bound" is the right term? In the end, it's a point lookup for a key plus timestamps, so it's an as-of timestamp (not a

Re: Streams: Global Store Processors

2023-07-26 Thread Matthias J. Sax
That's correct. And yes, it's a public contract and thus a KIP would be needed to change (or remove it). Deprecation implies that the API is kept for at least 3 releases (ie, one year), plus it can only be removed in a major release. For example, if we deprecated something in 2.1 release, we

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747630#comment-17747630 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for opening this ticket. Given that you

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Labels: kip (was: ) > Support interactive queries (IQv2) with versioned state st

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Description: KIP-960: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-960%3A

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Component/s: streams > Support interactive queries (IQv2) with versioned state st

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Issue Type: New Feature (was: Task) > Support interactive queries (IQv2) with versio

[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-07-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747210#comment-17747210 ] Matthias J. Sax commented on KAFKA-12317: - Thanks for picking it up – I think it would be good

[jira] [Created] (KAFKA-15251) Upgrade system test to use 3.5.1

2023-07-25 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15251: --- Summary: Upgrade system test to use 3.5.1 Key: KAFKA-15251 URL: https://issues.apache.org/jira/browse/KAFKA-15251 Project: Kafka Issue Type: Test

[jira] [Created] (KAFKA-15251) Upgrade system test to use 3.5.1

2023-07-25 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15251: --- Summary: Upgrade system test to use 3.5.1 Key: KAFKA-15251 URL: https://issues.apache.org/jira/browse/KAFKA-15251 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-07-25 Thread Matthias J. Sax
Igor, thanks for the KIP. Interesting proposal. I am wondering a little bit about the use-case and semantics, and if it's really required to add what you propose? Please correct me if I am wrong. In the end, a stream-table join is a "stream enrichment" (via a table lookup). Thus, it's

[jira] [Updated] (KAFKA-15242) FixedKeyProcessor testing is unusable

2023-07-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15242: Component/s: streams > FixedKeyProcessor testing is unusa

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-07-23 Thread Matthias J. Sax
configuration purposes. Thanks again for all the detailed thoughts Matthias! On Fri, Jul 21, 2023 at 11:50 AM Matthias J. Sax wrote: Thanks for the KIP. Overall I like the idea to close this gap. However, I am wondering if we should close others gaps first? In particular, IIRC, we have a few c

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-21 Thread Matthias J. Sax
ld be a clear method signature (and maybe sample javadocs) in the "Public Interfaces" section. Thanks again for the KIP Shay! Hope I haven't dragged it out too much On Fri, Jul 21, 2023 at 3:19 PM Matthias J. Sax wrote: Some thought about the API question. A. kstream.groupBy(...).aggregate(

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-21 Thread Matthias J. Sax
-759%3A+Unneeded+repartition+canceling . I will follow up with a VOTE separately. Best, Shay On Thu, Jun 29, 2023 at 4:52 PM Matthias J. Sax wrote: Shay, thanks for picking up this KIP. It's a pity that the discussion stalled for such a long time. As expressed previously, I am happy w

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-21 Thread Matthias J. Sax
I am not a clients (or threading) expert, but I tend to agree to Colin's concerns. In particular, it would be nice to see an example how you intent to use the API (I am not familiar with Kotlin or it's co-routins), to better understand what this changes help to solve to begin with. Opening

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-07-21 Thread Matthias J. Sax
Thanks for the KIP. Overall I like the idea to close this gap. However, I am wondering if we should close others gaps first? In particular, IIRC, we have a few cases for which we only have a RocksDB implementation for a store, and thus, adding an in-memory version for these stores first, to

[jira] [Comment Edited] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-07-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745690#comment-17745690 ] Matthias J. Sax edited comment on KAFKA-15116 at 7/21/23 6:18 PM

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-07-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745690#comment-17745690 ] Matthias J. Sax commented on KAFKA-15116: - {quote}The internal store is shared across stream

[jira] [Commented] (KAFKA-15190) Allow configuring a streams process ID

2023-07-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744726#comment-17744726 ] Matthias J. Sax commented on KAFKA-15190: - {quote}but although {{StreamsPartitionAssignor

[jira] [Comment Edited] (KAFKA-15190) Allow configuring a streams process ID

2023-07-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744288#comment-17744288 ] Matthias J. Sax edited comment on KAFKA-15190 at 7/18/23 6:22 PM: -- One

[jira] [Commented] (KAFKA-15190) Allow configuring a streams process ID

2023-07-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744288#comment-17744288 ] Matthias J. Sax commented on KAFKA-15190: - One more thing: the `process.id` is actually only

[jira] [Resolved] (KAFKA-13295) Long restoration times for new tasks can lead to transaction timeouts

2023-07-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13295. - Resolution: Fixed With the new restore-thread, this issue should be resolved implicilty

[jira] [Resolved] (KAFKA-13295) Long restoration times for new tasks can lead to transaction timeouts

2023-07-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13295. - Resolution: Fixed With the new restore-thread, this issue should be resolved implicilty

Re: [VOTE] KIP-941 Range queries to accept null lower and upper bounds

2023-07-10 Thread Matthias J. Sax
+1 (binding) On 7/10/23 12:13 PM, Bill Bejeck wrote: Hi Lucia, Thanks for the KIP! It will be a welcomed improvement. +1(binding) -Bill On Mon, Jul 10, 2023 at 2:40 PM Lucia Cerchie wrote: Hello everyone, I'd like to call a vote on KIP-941 <

Re: [DISCUSS] KIP-932: Queues for Kafka

2023-07-10 Thread Matthias J. Sax
xactly a specific record/batch is skipped. I don't think that this feature would be too important, but compared to the complexity of the KIP, 1 more state doesn't seem too complicated to me. Thanks, Daniel Matthias J. Sax ezt írta (időpont: 2023. júl. 10., H, 7:22): Daniel, sure. To allo

Re: Testing FixedKeyProcessor implementation using unit tests

2023-07-10 Thread Matthias J. Sax
Not sure right now, but could be a bug. Can you maybe share the full stack trace and the test program? -Matthias On 7/10/23 3:47 AM, EXT.Zlatibor.Veljkovic wrote: Hi, I am using kafka-streams-test-utils and have problem with testing FixedKeyProcessor [KIP-820

Re: [ANNOUNCE] New committer: Greg Harris

2023-07-10 Thread Matthias J. Sax
Congrats! On 7/10/23 8:45 AM, Chris Egerton wrote: Hi all, The PMC for Apache Kafka has invited Greg Harris to become a committer, and we are happy to announce that he has accepted! Greg has been contributing to Kafka since 2019. He has made over 50 commits mostly around Kafka Connect and

Re: [ANNOUNCE] New committer: Greg Harris

2023-07-10 Thread Matthias J. Sax
Congrats! On 7/10/23 8:45 AM, Chris Egerton wrote: Hi all, The PMC for Apache Kafka has invited Greg Harris to become a committer, and we are happy to announce that he has accepted! Greg has been contributing to Kafka since 2019. He has made over 50 commits mostly around Kafka Connect and

Re: [DISCUSS] KIP-932: Queues for Kafka

2023-07-09 Thread Matthias J. Sax
the responsibility of the broker, isn't it? My thought was that the broker could act differently when working with queues and read_committed isolation. Thanks, Daniel On Thu, Jul 6, 2023 at 7:26 PM Matthias J. Sax wrote: Thanks for the KIP. It seems we are in very early stage, and some very

Re: [DISCUSS] KIP-932: Queues for Kafka

2023-07-06 Thread Matthias J. Sax
Thanks for the KIP. It seems we are in very early stage, and some very important sections in the KIP are still marked as TODO. In particular, I am curious about the protocol changes, how the "queuing state" will be represented and made durable, and all the error edge case / fail-over /

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-06-29 Thread Matthias J. Sax
Shay, thanks for picking up this KIP. It's a pity that the discussion stalled for such a long time. As expressed previously, I am happy with the name `markAsPartitioned()` and also believe it's ok to just document the impact and leave it to the user to do the right thing. If we really get

Re: [DISCUSS] KIP-941: Range queries to accept null lower and upper bounds

2023-06-29 Thread Matthias J. Sax
Thanks for the KIP. LGTM. I believe you can start a vote. -Matthias On 6/26/23 11:25 AM, Lucia Cerchie wrote: Thanks for asking for clarification, Sophie; that gives me guidance on improving the KIP! Here's the updated version, including the JIRA link:

Re: [DISCUSS] KIP-941 Support async runtimes in consumer

2023-06-29 Thread Matthias J. Sax
Seems the KIP number is 947, not 941? Can you maybe start a new thread to avoid confusion? Thanks. On 6/28/23 1:11 AM, Erik van Oosten wrote: Hello developers of the Java based consumer, I submitted https://github.com/apache/kafka/pull/13914 to fix a long standing problem that the Kafka

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-06-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738782#comment-17738782 ] Matthias J. Sax commented on KAFKA-15116: - > Message A uses an internal store to st

Re: Kafka Streaming: RocksDbSessionBytesStoreSupplier seems lost data in Kubernetes

2023-06-29 Thread Matthias J. Sax
The class `RocksDbSessionBytesStoreSupplier` is in package `internal` and thus, you should not use it directly. Instead, you should use the public factory class `org.apache.kafka.streams.state.Stores` However, your usage seems correct in general. Not sure why you pass-in the supplier directly

[jira] [Commented] (KAFKA-13973) block-cache-capacity metrics worth twice as much as normal

2023-06-28 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738231#comment-17738231 ] Matthias J. Sax commented on KAFKA-13973: - The GitHub issue for RocksDB was "declined&qu

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-06-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736637#comment-17736637 ] Matthias J. Sax commented on KAFKA-15116: - > The complexity is that there is a requirem

[jira] [Commented] (KAFKA-15108) task.timeout.ms does not work when TimeoutException is thrown by streams producer

2023-06-20 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735476#comment-17735476 ] Matthias J. Sax commented on KAFKA-15108: - There are a few cases for which we cannot handle

Re: [ANNOUNCE] New committer: Divij Vaidya

2023-06-13 Thread Matthias J. Sax
Congrats! On 6/13/23 10:24 AM, Satish Duggana wrote: Congratulations Divij!! On Tue, 13 Jun 2023 at 22:41, Manyanda Chitimbo wrote: Congratulations Divij. On Tue 13 Jun 2023 at 17:50, Bruno Cadonna wrote: Hi all, The PMC of Apache Kafka is pleased to announce a new Kafka committer

Re: Consuming an entire partition with control messages

2023-06-13 Thread Matthias J. Sax
Sounds like a bug in aiokafka library to me. If the last message in a topic partition is a tx-marker, the consumer should step over it, and report the correct position after the marker. The official KafkaConsumer (ie, the Java one), does the exact same thing. -Matthias On 5/30/23 8:41 AM,

Re: [DISCUSS] KIP-923: Add A Grace Period to Stream Table Join

2023-05-25 Thread Matthias J. Sax
(if the grace period decreased) or there would be a pause until the new grace period reached. Increasing is a bit more problematic, especially if the table grace period and retention time stays the same. If the data is reprocessed after a change like that then there would be different results,

[jira] [Commented] (KAFKA-7497) Kafka Streams should support self-join on streams

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725357#comment-17725357 ] Matthias J. Sax commented on KAFKA-7497: Seems to be fixed. Cf https://issues.apache.org/jira

[jira] [Resolved] (KAFKA-7497) Kafka Streams should support self-join on streams

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7497. Resolution: Fixed > Kafka Streams should support self-join on stre

[jira] [Resolved] (KAFKA-7497) Kafka Streams should support self-join on streams

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-7497. Resolution: Fixed > Kafka Streams should support self-join on stre

[jira] [Resolved] (KAFKA-14173) TopologyTestDriver does not use mock wall clock time when sending test records

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14173. - Resolution: Not A Problem > TopologyTestDriver does not use mock wall clock time w

[jira] [Commented] (KAFKA-14173) TopologyTestDriver does not use mock wall clock time when sending test records

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725356#comment-17725356 ] Matthias J. Sax commented on KAFKA-14173: - Just discovering this ticket. I guess you would need

[jira] [Resolved] (KAFKA-14173) TopologyTestDriver does not use mock wall clock time when sending test records

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14173. - Resolution: Not A Problem > TopologyTestDriver does not use mock wall clock time w

[jira] [Updated] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10575: Labels: kip (was: ) > StateRestoreListener#onRestoreEnd should always be trigge

[jira] [Updated] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10575: Description: Part of KIP-869: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-869

[jira] [Resolved] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10575. - Fix Version/s: 3.5.0 Resolution: Fixed > StateRestoreListener#onRestore

[jira] [Resolved] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10575. - Fix Version/s: 3.5.0 Resolution: Fixed > StateRestoreListener#onRestore

[jira] [Assigned] (KAFKA-10575) StateRestoreListener#onRestoreEnd should always be triggered

2023-05-23 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-10575: --- Assignee: Guozhang Wang (was: highluck) > StateRestoreListener#onRestore

Re: [VOTE] 3.4.1 RC0

2023-05-22 Thread Matthias J. Sax
Thanks a lot! -Matthias On 5/21/23 7:27 PM, Luke Chen wrote: Hi Matthias, Yes, I agree we should get this hotfix into 3.4.1. I've backported into the 3.4 branch. I'll create a new RC for 3.4.1. Thanks. Luke On Mon, May 22, 2023 at 5:13 AM Matthias J. Sax wrote: Hi Luke, RC0 for 3.4.1

Re: [VOTE] 3.4.1 RC0

2023-05-22 Thread Matthias J. Sax
Thanks a lot! -Matthias On 5/21/23 7:27 PM, Luke Chen wrote: Hi Matthias, Yes, I agree we should get this hotfix into 3.4.1. I've backported into the 3.4 branch. I'll create a new RC for 3.4.1. Thanks. Luke On Mon, May 22, 2023 at 5:13 AM Matthias J. Sax wrote: Hi Luke, RC0 for 3.4.1

Re: [VOTE] 3.4.1 RC0

2023-05-21 Thread Matthias J. Sax
Hi Luke, RC0 for 3.4.1 includes a fix for https://issues.apache.org/jira/browse/KAFKA-14862. We recently discovered that tge fix itself introduces a regression. We have already a PR to fix-forward the regression: https://github.com/apache/kafka/pull/13734 I think we should get the open PR

Re: [VOTE] 3.4.1 RC0

2023-05-21 Thread Matthias J. Sax
Hi Luke, RC0 for 3.4.1 includes a fix for https://issues.apache.org/jira/browse/KAFKA-14862. We recently discovered that tge fix itself introduces a regression. We have already a PR to fix-forward the regression: https://github.com/apache/kafka/pull/13734 I think we should get the open PR

<    2   3   4   5   6   7   8   9   10   11   >