[jira] [Updated] (KAFKA-15385) Replace EasyMock with Mockito for AbstractStreamTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15385: Component/s: streams unit tests > Replace EasyMock with Mock

[jira] [Updated] (KAFKA-15382) Replace EasyMock with Mockito for KStreamTransformValuesTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15382: Component/s: streams unit tests > Replace EasyMock with Mock

[jira] [Updated] (KAFKA-15384) Replace EasyMock with Mockito for KTableTransformValuesTest

2023-09-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15384: Component/s: streams unit tests > Replace EasyMock with Mock

Re: AW: Table updates are not consistent when doing a join with a Stream

2023-09-04 Thread Matthias J. Sax
Your update to the KTable is async when you send data back to the KTable input topic. So your program is subject to race-conditions. So switching to the PAPI was the right move: it make the update to the state store sync and thus fixes the issue. -Matthias On 9/4/23 5:53 AM, Mauricio Lopez

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-09-04 Thread Matthias J. Sax
e inactive consumer group is cleared up after 7 days however not sure if that is the case with topics that were consumed previously and not consumed now. Does creation of new consumer group (setting a different application.id) on streams application an option here? On Thu, Aug 17, 2023 at 7:03 AM

[jira] [Updated] (KAFKA-15378) Rolling upgrade system tests are failing

2023-08-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15378: Component/s: system tests > Rolling upgrade system tests are fail

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
Thanks for splitting this part into a separate KIP! For `withKey()` we should be explicit that `null` is not allowed. (Looking into existing `KeyQuery` it seems the JavaDocs don't cover this either -- would you like to do a tiny cleanup PR for this, or fix on-the-side in one of your PRs?)

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-16 Thread Matthias J. Sax
queries+%28IQv2%29+for+versioned+state+stores> Cheers, Alieh On Thu, Aug 10, 2023 at 2:38 AM Matthias J. Sax wrote: Seems there was a lot of additional feedback. Looking forward to an updated version of the KIP. I also agree to make the queries more composable. I was considering to ra

Re: kafka streams consumer group reporting lag even on source topics removed from topology

2023-08-16 Thread Matthias J. Sax
Well, it's kinda expected behavior. It's a split brain problem. In the end, you use the same `application.id / group.id` and thus the committed offsets for the removed topics are still in `__consumer_offsets` topics and associated with the consumer group. If a tool inspects lags and compares

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755298#comment-17755298 ] Matthias J. Sax commented on KAFKA-15302: - I would need to think more about this. Overall

[jira] [Commented] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755184#comment-17755184 ] Matthias J. Sax commented on KAFKA-15338: - Sure. Thank a lot! > The metric group documentat

[jira] [Updated] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15346: Component/s: streams > Single-Key_single-timestamp IQs with versioned state sto

[jira] [Updated] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15347: Component/s: streams > Single-Key_multi-timestamp IQs with versioned state sto

[jira] [Updated] (KAFKA-15348) Range IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15348: Component/s: streams > Range IQs with versioned state sto

[jira] [Updated] (KAFKA-15344) Kafka Streams should include the message leader epoch when committing offsets

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15344: Component/s: streams > Kafka Streams should include the message leader epoch w

[jira] [Assigned] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15319: --- Assignee: Lucas Brutschy > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Fix Version/s: 3.6.0 > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15338: Labels: beginner newbie (was: ) > The metric group documentation for metrics ad

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Labels: need-kip (was: ) > Disable *-subscription-store-changelog topic creat

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Component/s: streams > Disable *-subscription-store-changelog topic creation for fore

[jira] [Resolved] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13197. - Fix Version/s: 3.6.0 3.5.2 Resolution: Fixed > KStr

[jira] [Resolved] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13197. - Fix Version/s: 3.6.0 3.5.2 Resolution: Fixed > KStr

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-10 Thread Matthias J. Sax
ontrol it with the flag, or maybe create some special aggregate that could be chained after (don't know how to do it yet :-)) Any thoughts? Regards, Igor On Wed, Aug 9, 2023 at 7:19 PM Matthias J. Sax wrote: Thanks for the details. And sorry for being a little bit picky. My goal is to real

Re: [VOTE] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
+1 (binding) On 8/10/23 12:31 PM, Florin Akermann wrote: https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams

[jira] [Updated] (KAFKA-13197) KStream-GlobalKTable join semantics don't match documentation

2023-08-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13197: Component/s: documentation streams > KStream-GlobalKTable join semant

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-10 Thread Matthias J. Sax
on the PR. Great. Yes, in general, I hope everybody agrees that we shouldn't add more details to this KIP On Thu, 10 Aug 2023 at 00:16, Matthias J. Sax wrote: Thanks for the KIP. left join KStream-GlobalTable: no longer drop left records with null-key and call KeyValueMapper with 'null' for left

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-08-09 Thread Matthias J. Sax
than the timestamp. I propose to rename the getters to getTimeFrom() and getTimeTo() as in WindowRangeQuery. 5. Please add the Test Plan section. Regarding long vs Instant: Did we miss to use Instant instead of long for all interfaces of the versioned state stores? Best, Bruno On 7/26

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-09 Thread Matthias J. Sax
d be a flag to stream-table foreign key join that would indicate if we want this join to aggregate children or not? What do you think? Regards, Igor On Fri, Aug 4, 2023 at 10:01 PM Matthias J. Sax wrote: Thanks a lot for providing more background. It's getting much clear to me now. Coupl

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-09 Thread Matthias J. Sax
s one good example that the latter can help you distinguish different scenarios whereas the former cannot. On Fri, Aug 4, 2023 at 6:32 PM Matthias J. Sax wrote: Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentation change, ie, point out that the passed in

[jira] [Commented] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752522#comment-17752522 ] Matthias J. Sax commented on KAFKA-14049: - I am still not sure if I understand this ticket

[jira] [Updated] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14049: Labels: (was: kip) > Relax Non Null Requirement for KStreamGlobalKTable Left J

[jira] [Updated] (KAFKA-14049) Relax Non Null Requirement for KStreamGlobalKTable Left Join

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14049: Labels: kip (was: beginner newbie) > Relax Non Null Requirement for KStreamGlobalKTa

[jira] [Updated] (KAFKA-14748) Relax non-null FK left-join requirement

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14748: Description: Kafka Streams enforces a strict non-null-key policy in the DSL across all

[jira] [Updated] (KAFKA-14748) Relax non-null FK left-join requirement

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14748: Labels: kip (was: ) > Relax non-null FK left-join requirem

[jira] [Updated] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12317: Description: Currently, for a stream-streams and stream-table/globalTable join

[jira] [Updated] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12317: Labels: kip (was: ) > Relax non-null key requirement for left/outer KStream jo

[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752519#comment-17752519 ] Matthias J. Sax commented on KAFKA-12317: - Just catching on of comments... {quote}KAFKA-12845

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Priority: Critical (was: Major) > Upgrade rocksdb to fix CVE-2022-37

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-08-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Component/s: streams > Upgrade rocksdb to fix CVE-2022-37

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752150#comment-17752150 ] Matthias J. Sax commented on KAFKA-15297: - Seems we are on the same page :)  > Cache fl

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using exactly_once

2023-08-08 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752149#comment-17752149 ] Matthias J. Sax commented on KAFKA-15259: - The only thing I am wondering is, if we should try

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751836#comment-17751836 ] Matthias J. Sax commented on KAFKA-15302: - {quote}it's a combo of a cache snapshot

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751835#comment-17751835 ] Matthias J. Sax commented on KAFKA-15259: - Ah. Sorry. Meant https://issues.apache.org/jira

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:59 PM

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM: -- I

[jira] [Comment Edited] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax edited comment on KAFKA-15259 at 8/7/23 11:58 PM: -- I

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751761#comment-17751761 ] Matthias J. Sax commented on KAFKA-15302: - Thanks for providing more details; the `delete

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751759#comment-17751759 ] Matthias J. Sax commented on KAFKA-15259: - {quote}Should we create a new ticket for Kafka

[jira] [Commented] (KAFKA-14747) FK join should record discarded subscription responses

2023-08-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751756#comment-17751756 ] Matthias J. Sax commented on KAFKA-14747: - We can discuss here on the ticket I guess. The class

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-07 Thread Matthias J. Sax
handling on the leader should typically assign the offset. I believe you can find that code here: https://github.com/apache/kafka/blob/b9a45546a7918799b6fb3c0fe63b56f47d8fcba9/core/src/main/scala/kafka/log/UnifiedLog.scala#L766 Justine On Tue, Aug 1, 2023 at 11:38 AM Matthias J. Sax wrote:

[jira] [Assigned] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15303: --- Assignee: Matthias J. Sax > Foreign key joins no longer triggered by eve

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer collects multiple records into batches, and a single record

[jira] [Updated] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15309: Description: The producer batches up multiple records into batches, and a single record

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751267#comment-17751267 ] Matthias J. Sax commented on KAFKA-15259: - I did sync up with [~cegerton] who worked on https

[jira] [Created] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15309: --- Summary: Add custom error handler to Producer Key: KAFKA-15309 URL: https://issues.apache.org/jira/browse/KAFKA-15309 Project: Kafka Issue Type: New

[jira] [Created] (KAFKA-15309) Add custom error handler to Producer

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15309: --- Summary: Add custom error handler to Producer Key: KAFKA-15309 URL: https://issues.apache.org/jira/browse/KAFKA-15309 Project: Kafka Issue Type: New

Re: Testing FixedKeyProcessor implementation using unit tests

2023-08-04 Thread Matthias J. Sax
-repro/blob/master/src/test/java/com/example/demo/MyFixedKeyProcessorTest.java Test is where I am having issues. Thanks, Zed -Original Message- From: Matthias J. Sax Sent: Tuesday, July 11, 2023 1:13 AM To: dev@kafka.apache.org Subject: Re: Testing FixedKeyProcessor implementation using

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751263#comment-17751263 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for digging into it – that's a good fine – I

[jira] [Updated] (KAFKA-15308) Wipe Stores upon OffsetOutOfRangeException in ALOS

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15308: Affects Version/s: (was: 3.4.0) (was: 3.5.0) > Wipe Sto

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-08-04 Thread Matthias J. Sax
e data source and replace it with CDC streaming data to Kafka. So in that case assembling data entities from Kafka topics instead of JDBC would be beneficial. Please let me know what you think. Regards, Igor On Tue, Jul 25, 2023 at 5:53 PM Matthias J. Sax wrote: Igor, thanks for the KIP. Int

Re: [DISCUSS] KIP-962 Relax non-null key requirement in Kafka Streams

2023-08-04 Thread Matthias J. Sax
Guozhang, thanks for pointing out ValueJoinerWithKey. In the end, it's just a documentation change, ie, point out that the passed in key could be `null` and similar? -Matthias On 8/2/23 3:20 PM, Guozhang Wang wrote: Thanks Florin for the writeup, One quick thing I'd like to bring up is

[jira] [Created] (KAFKA-15307) Kafka Streams configuration docs outdate

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15307: --- Summary: Kafka Streams configuration docs outdate Key: KAFKA-15307 URL: https://issues.apache.org/jira/browse/KAFKA-15307 Project: Kafka Issue Type

[jira] [Created] (KAFKA-15307) Kafka Streams configuration docs outdate

2023-08-04 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15307: --- Summary: Kafka Streams configuration docs outdate Key: KAFKA-15307 URL: https://issues.apache.org/jira/browse/KAFKA-15307 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751236#comment-17751236 ] Matthias J. Sax commented on KAFKA-15302: - Thanks for reporting this issue. – When you call

[jira] [Commented] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751212#comment-17751212 ] Matthias J. Sax commented on KAFKA-15303: - Thanks for filing this ticket. Kafka Streams as only

[jira] [Updated] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15303: Issue Type: Improvement (was: Bug) > Foreign key joins no longer triggered by eve

[jira] [Updated] (KAFKA-15303) Foreign key joins no longer triggered by events on the right side of the join after deployment with a new compatible Avro schema

2023-08-04 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15303: Priority: Major (was: Critical) > Foreign key joins no longer triggered by eve

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750506#comment-17750506 ] Matthias J. Sax commented on KAFKA-15116: - Are you saying you are using a custom store

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750504#comment-17750504 ] Matthias J. Sax commented on KAFKA-15297: - The ticket description contains an example

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750503#comment-17750503 ] Matthias J. Sax commented on KAFKA-12829: - It's actually unclear when 4.0 will come along, so

[jira] [Updated] (KAFKA-15299) Support left stream-table join on foreign key

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15299: Labels: kip (was: ) > Support left stream-table join on foreign

Re: [QUESTION] What is the difference between sequence and offset for a Record?

2023-08-01 Thread Matthias J. Sax
The _offset_ is the position of the record in the partition. The _sequence number_ is a unique ID that allows broker to de-duplicate messages. It requires the producer to implement the idempotency protocol (part of Kafka transactions); thus, sequence numbers are optional and as long as you

Re: [VOTE] KIP-759: Unneeded repartition canceling

2023-07-31 Thread Matthias J. Sax
+1 (binding) On 7/11/23 11:16 AM, Shay Lin wrote: Hi all, I'd like to call a vote on KIP-759: Unneeded repartition canceling The KIP has been under discussion for quite some time(two years). This is a valuable optimization for advanced users. I hope we can push this toward the finish line this

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-07-27 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748367#comment-17748367 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for getting back – I was just asking a general

Re: Consuming an entire partition with control messages

2023-07-27 Thread Matthias J. Sax
"1" in this kind of situation Best regards, Vincent On 13/06/2023 17:27, Matthias J. Sax wrote: Sounds like a bug in aiokafka library to me. If the last message in a topic partition is a tx-marker, the consumer should step over it, and report the correct position after the marker.

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-26 Thread Matthias J. Sax
efore feeling comfortable enough to vote on this would be a clear method signature (and maybe sample javadocs) in the "Public Interfaces" section. Thanks again for the KIP Shay! Hope I haven't dragged it out too much On Fri, Jul 21, 2023 at 3:19 PM Matthias J. Sax wrote: Some tho

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

2023-07-26 Thread Matthias J. Sax
Thanks for the KIP Alieh. Glad to see that we can add IQ to the new versioned stores! Couple of questions: single-key lookup with timestamp (upper) bound Not sure if "bound" is the right term? In the end, it's a point lookup for a key plus timestamps, so it's an as-of timestamp (not a

Re: Streams: Global Store Processors

2023-07-26 Thread Matthias J. Sax
That's correct. And yes, it's a public contract and thus a KIP would be needed to change (or remove it). Deprecation implies that the API is kept for at least 3 releases (ie, one year), plus it can only be removed in a major release. For example, if we deprecated something in 2.1 release, we

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747630#comment-17747630 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for opening this ticket. Given that you

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Labels: kip (was: ) > Support interactive queries (IQv2) with versioned state st

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Description: KIP-960: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-960%3A

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Component/s: streams > Support interactive queries (IQv2) with versioned state st

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-07-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15257: Issue Type: New Feature (was: Task) > Support interactive queries (IQv2) with versio

[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-07-25 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747210#comment-17747210 ] Matthias J. Sax commented on KAFKA-12317: - Thanks for picking it up – I think it would be good

[jira] [Created] (KAFKA-15251) Upgrade system test to use 3.5.1

2023-07-25 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15251: --- Summary: Upgrade system test to use 3.5.1 Key: KAFKA-15251 URL: https://issues.apache.org/jira/browse/KAFKA-15251 Project: Kafka Issue Type: Test

[jira] [Created] (KAFKA-15251) Upgrade system test to use 3.5.1

2023-07-25 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-15251: --- Summary: Upgrade system test to use 3.5.1 Key: KAFKA-15251 URL: https://issues.apache.org/jira/browse/KAFKA-15251 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

2023-07-25 Thread Matthias J. Sax
Igor, thanks for the KIP. Interesting proposal. I am wondering a little bit about the use-case and semantics, and if it's really required to add what you propose? Please correct me if I am wrong. In the end, a stream-table join is a "stream enrichment" (via a table lookup). Thus, it's

[jira] [Updated] (KAFKA-15242) FixedKeyProcessor testing is unusable

2023-07-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15242: Component/s: streams > FixedKeyProcessor testing is unusa

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-07-23 Thread Matthias J. Sax
configuration purposes. Thanks again for all the detailed thoughts Matthias! On Fri, Jul 21, 2023 at 11:50 AM Matthias J. Sax wrote: Thanks for the KIP. Overall I like the idea to close this gap. However, I am wondering if we should close others gaps first? In particular, IIRC, we have a few c

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-21 Thread Matthias J. Sax
ld be a clear method signature (and maybe sample javadocs) in the "Public Interfaces" section. Thanks again for the KIP Shay! Hope I haven't dragged it out too much On Fri, Jul 21, 2023 at 3:19 PM Matthias J. Sax wrote: Some thought about the API question. A. kstream.groupBy(...).aggregate(

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

2023-07-21 Thread Matthias J. Sax
-759%3A+Unneeded+repartition+canceling . I will follow up with a VOTE separately. Best, Shay On Thu, Jun 29, 2023 at 4:52 PM Matthias J. Sax wrote: Shay, thanks for picking up this KIP. It's a pity that the discussion stalled for such a long time. As expressed previously, I am happy w

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

2023-07-21 Thread Matthias J. Sax
I am not a clients (or threading) expert, but I tend to agree to Colin's concerns. In particular, it would be nice to see an example how you intent to use the API (I am not familiar with Kotlin or it's co-routins), to better understand what this changes help to solve to begin with. Opening

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

2023-07-21 Thread Matthias J. Sax
Thanks for the KIP. Overall I like the idea to close this gap. However, I am wondering if we should close others gaps first? In particular, IIRC, we have a few cases for which we only have a RocksDB implementation for a store, and thus, adding an in-memory version for these stores first, to

[jira] [Comment Edited] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-07-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745690#comment-17745690 ] Matthias J. Sax edited comment on KAFKA-15116 at 7/21/23 6:18 PM

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-07-21 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745690#comment-17745690 ] Matthias J. Sax commented on KAFKA-15116: - {quote}The internal store is shared across stream

[jira] [Commented] (KAFKA-15190) Allow configuring a streams process ID

2023-07-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744726#comment-17744726 ] Matthias J. Sax commented on KAFKA-15190: - {quote}but although {{StreamsPartitionAssignor

[jira] [Comment Edited] (KAFKA-15190) Allow configuring a streams process ID

2023-07-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744288#comment-17744288 ] Matthias J. Sax edited comment on KAFKA-15190 at 7/18/23 6:22 PM: -- One

<    2   3   4   5   6   7   8   9   10   11   >