[VOTE] KIP-825: introduce a new API to control when aggregated results are produced

Hi all, I'd like to start a vote on Kafka Streams KIP-825: https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced

[jira] [Resolved] (KAFKA-13672) Race condition in DynamicBrokerConfig

[ https://issues.apache.org/jira/browse/KAFKA-13672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-13672. --- Fix Version/s: 3.3.0 Resolution: Fixed > Race condition in DynamicBrokerConfig >

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Hi all, I just updated the KIP with option 1 as design and put option 2 and 3 in rejected alternatives. Since Matthias is strongly against `trigger`, I adopted the proposed `EmitStrategy` and dropped the "with" in the function name. So it's like this: stream .groupBy(..)

[jira] [Resolved] (KAFKA-13714) Flaky test IQv2StoreIntegrationTest

[ https://issues.apache.org/jira/browse/KAFKA-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13714. -- Fix Version/s: 3.2.0 Assignee: John Roesler Resolution: Fixed > Flaky test

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

Hi Artem, Thanks for the KIP and the patience during discussion! +1 binding from me. Luke On Thu, Mar 24, 2022 at 3:43 AM Ismael Juma wrote: > Thanks for the KIP and for taking the time to address all the feedback. +1 > (binding) > > Ismael > > On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits >

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Wow. Quite a thread... #namingIsHard :D I won't repeat all arguments which are all very good ones. I can just state my personal favorite option: stream .groupBy(..) .windowedBy(..) .withEmitStrategy(EmitStrategy.ON_WINDOW_CLOSE) .aggregate(..) .mapValues(..) Is seems

[jira] [Created] (KAFKA-13766) Use `max.poll.interval.ms` as the timeout during complete-rebalance phase

Guozhang Wang created KAFKA-13766: - Summary: Use `max.poll.interval.ms` as the timeout during complete-rebalance phase Key: KAFKA-13766 URL: https://issues.apache.org/jira/browse/KAFKA-13766 Project:

[jira] [Created] (KAFKA-13765) Describe-consumer admin should not return unstable membership information

Guozhang Wang created KAFKA-13765: - Summary: Describe-consumer admin should not return unstable membership information Key: KAFKA-13765 URL: https://issues.apache.org/jira/browse/KAFKA-13765 Project:

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.0 #191

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #95

See Changes: -- [...truncated 432983 lines...] [2022-03-24T00:39:39.079Z] > Task :metadata:testClasses UP-TO-DATE [2022-03-24T00:39:39.079Z] > Task

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Hao is right, I think that's the hindsight we have for `suppress` which since can be applied anywhere for a K(windowed)Table, incurs an awkward programming flexibility and I felt it's better to make its application scope more constraint. And I also agree with John that, unless any of us feel

[jira] [Resolved] (KAFKA-13759) Disable producer idempotence by default in producers instantiated by Connect

[ https://issues.apache.org/jira/browse/KAFKA-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantine Karantasis resolved KAFKA-13759. Resolution: Fixed > Disable producer idempotence by default in producers

[jira] [Created] (KAFKA-13764) Potential improvements for Connect incremental rebalancing logic

Chris Egerton created KAFKA-13764: - Summary: Potential improvements for Connect incremental rebalancing logic Key: KAFKA-13764 URL: https://issues.apache.org/jira/browse/KAFKA-13764 Project: Kafka

[jira] [Created] (KAFKA-13763) Improve unit testing coverage for IncrementalCooperativeAssignor

Chris Egerton created KAFKA-13763: - Summary: Improve unit testing coverage for IncrementalCooperativeAssignor Key: KAFKA-13763 URL: https://issues.apache.org/jira/browse/KAFKA-13763 Project: Kafka

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

For stream .groupBy(..) .windowedBy(..) .aggregate(..) .withEmitStrategy(EmitStrategy.ON_WINDOW_CLOSE) .mapValues(..) I think after `aggregate` it's already a table and then the emit strategy is too late to control how windowed stream is outputted to table. This is

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #796

See Changes: -- [...truncated 599099 lines...] [Pipeline] // timeout [Pipeline] } [Pipeline] // stage [Pipeline] } [2022-03-23T20:34:50.551Z] [2022-03-23T20:34:50.551Z]

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Hi, Thank you for your answers to my questions! I see the argument about conciseness of configuring a stream with methods instead of config objects. I just miss a bit the descriptive aspect. What about stream .groupBy(..) .windowedBy(..)

Re: [DISCUSS] KIP-651 - Support PEM format for SSL certificates and

Hi Rajini, On Mon, Mar 21, 2022 at 10:02 AM Rajini Sivaram wrote: > For the background on the current implementation: We use Java's keystore > loading for JKS/PKCS12 keystore files and these files require passwords. We > In Java 18: "Passwordless keystores (a keystore with no password

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

Thanks for the KIP and for taking the time to address all the feedback. +1 (binding) Ismael On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits wrote: > Hi all, > > I'd like to start a vote on > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner > . >

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Hi all, I can see both sides of this. On one hand, when we say "stream.groupBy().windowBy().count()", it seems like we're telling KS to take the raw stream, group it based on key, then window it based on time, and then compute an aggregation on the windows. In that model, "trigger()" would have

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Thanks Bruno! Argument for option 1 is: 1. Concise and descriptive. It avoids overloading existing functions and it's very clear what it's doing. Imagine if there's a autocomplete feature in Intellij or other IDE for our DSL in the future, it's not favorable to show 6 `windowedBy` functions. 2.

Re: [VOTE] KIP-653: Upgrade log4j to log4j2

Hi Dongjin, We really appreciate the super valuable work you've been doing here. Do we have evidence that customers don't use custom filters/layouts? Ismael On Wed, Mar 23, 2022 at 7:53 AM Dongjin Lee wrote: > Hi Mikael, Edoardo and Ismael, > > Sorry for being late. Frankly, I thought KIP-653

Re: [VOTE] KIP-653: Upgrade log4j to log4j2

Hi Mikael, Edoardo and Ismael, Sorry for being late. Frankly, I thought KIP-653 is not a breaking change since (as Edoardo stated) unless the user uses custom filters or layouts, log4j-1.2-api.jar 'bridge' jar can handle the cases. It is why the 'Compatibility, Deprecation, and Migration Plan'

Re: [VOTE] KIP-653: Upgrade log4j to log4j2

Hi Mickael, Thanks for your feedback. I agree with the importance of fixing the CVEs and also of not breaking compatibility in a critical layer. Regarding Apache Kafka 4.0, you suggested it would include: - log4j2 migration - idempotency enablement cleanups - removal of Java 8 and Scala 2.12

[jira] [Created] (KAFKA-13762) Kafka brokers are not coming up

Kamesh created KAFKA-13762: -- Summary: Kafka brokers are not coming up Key: KAFKA-13762 URL: https://issues.apache.org/jira/browse/KAFKA-13762 Project: Kafka Issue Type: Bug Reporter:

RE: [VOTE] KIP-653: Upgrade log4j to log4j2

Mickael, +1 from me - even if you didn't ask for a vote :-) Edo -- Edoardo Comar Event Streams for IBM Cloud From: Mickael Maison Sent: 23 March 2022 11:43 To: dev Subject: [EXTERNAL] Re: [VOTE] KIP-653:

Re: [VOTE] KIP-653: Upgrade log4j to log4j2

Hi Ismael, About 2) We can't keep shipping new releases with dependencies that have CVEs. This is negatively impacting the project and eroding the hard earned trust we have from our users. Kafka is known to be a robust, reliable and up to date project. With that in mind, and since clearly at

Re: [DISCUSS] KIP-825: introduce a new API to control when aggregated results are produced

Hi Hao, I agree with Guozhang: Great summary! Thank you! Regarding "aligned with other config class names", there is this DSL grammar John once specified https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+DSL+Grammar and we have already used it in the code. I found the grammar

Re: [VOTE] KIP-794: Strictly Uniform Sticky Partitioner

Hi Artem, Thanks for the KIP. This is a really nice improvement! +1 (binding) from me. David On Tue, Mar 22, 2022 at 9:35 PM Jun Rao wrote: > > Hi, Artem, > > Thanks for the KIP. +1 from me. > > Jun > > On Mon, Mar 21, 2022 at 4:52 PM Artem Livshits > wrote: > > > Hi all, > > > > I'd like to

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #795

See