Re: [VOTE] KIP-924: customizable task assignment for Streams

2024-05-07 Thread Rohan Desai
Thanks all! KIP-924 is accepted with 4 +1 (binding) votes from Sophia Blee-Goldman, Matthias Sax, Bruno Cadonna, and Lucas Brutschy On Tue, Apr 16, 2024 at 1:46 PM Rohan Desai wrote: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-924%3A+customizable+task+assignment+for+S

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

2024-04-26 Thread Rohan Desai
totally reasonable to fail the application in that case. We _could_ try to be more graceful and default to one of the existing assignors. But it's usually better to fail hard and fast when there is some illegal state detected imo. On Fri, Apr 19, 2024 at 4:18 PM Rohan Desai wrote: > Bruno, I

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

2024-04-19 Thread Rohan Desai
Bruno, I've incorporated your feedback into the KIP document. On Fri, Apr 19, 2024 at 3:55 PM Rohan Desai wrote: > Thanks for the feedback Bruno! For the most part I think it makes sense, > but leaving a couple follow-up thoughts/questions: > > re 4: I think Sophie's point

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

2024-04-19 Thread Rohan Desai
to me. Whether we do that or not, we can have the return type be a Set instead of a Map as well. re 6: Yes, it's a callback that's called with the final assignment. I like your suggested name. On Fri, Apr 5, 2024 at 12:17 PM Rohan Desai wrote: > Thanks for the feedback Sophie! > > re1: Tota

[VOTE] KIP-924: customizable task assignment for Streams

2024-04-16 Thread Rohan Desai
https://cwiki.apache.org/confluence/display/KAFKA/KIP-924%3A+customizable+task+assignment+for+Streams As this KIP has been open for a while, and gone through a couple rounds of review/revision, I'm calling a vote to get it approved.

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

2024-04-05 Thread Rohan Desai
the final assignment. I'll add that. re4: I like the new `NodeAssignment` type. I'll update the KIP with that. On Thu, Nov 9, 2023 at 11:18 PM Rohan Desai wrote: > Thanks for the feedback so far! I think pretty much all of it is > reasonable. I'll reply to it inline: > > > 1. Al

Re: [DISCUSS] KIP-924: customizable task assignment for Streams

2023-11-09 Thread Rohan Desai
information the interfaces are returning, that's ok at this level of discussion. On Tue, Nov 7, 2023 at 12:03 PM Rohan Desai wrote: > Hello All, > > I'd like to start a discussion on KIP-924 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-924%3A+customizable+task+assignment+for

[DISCUSS] KIP-924: customizable task assignment for Streams

2023-11-07 Thread Rohan Desai
Hello All, I'd like to start a discussion on KIP-924 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-924%3A+customizable+task+assignment+for+Streams) which proposes an interface to allow users to plug into the streams partition assignor. The motivation section in the KIP goes into some

Re: [VOTE] KIP-761: Add Total Blocked Time Metric to Streams

2022-03-22 Thread Rohan Desai
the total time spent blocked on metadata. I've implemented this in a patch here: https://github.com/apache/kafka/pull/11805 I'm hoping I can just update this KIP to include the new producer metric. On Tue, Aug 31, 2021 at 1:07 AM Rohan Desai wrote: > FYI I've updated the metric names in the

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2022-02-24 Thread Rohan Desai
the total time spent blocked on metadata. I've implemented this in a patch here: https://github.com/apache/kafka/pull/11805 I'm hoping I can just update this KIP to include the new producer metric. On Mon, Jul 12, 2021 at 12:00 PM Rohan Desai wrote: > > https://cwiki.apache.org/conf

[jira] [Created] (KAFKA-13692) stream thread blocked-time-ns-total metric does not include producer metadata wait time

2022-02-24 Thread Rohan Desai (Jira)
Rohan Desai created KAFKA-13692: --- Summary: stream thread blocked-time-ns-total metric does not include producer metadata wait time Key: KAFKA-13692 URL: https://issues.apache.org/jira/browse/KAFKA-13692

Re: [VOTE] KIP-761: Add Total Blocked Time Metric to Streams

2021-08-31 Thread Rohan Desai
FYI I've updated the metric names in the KIP to the form ".*-time-ns-total" and clarified that the times being measured are in nanoseconds. On Wed, Jul 21, 2021 at 5:09 PM Rohan Desai wrote: > Now that the discussion thread's been open for a few days, I'm calling for > a v

[jira] [Created] (KAFKA-13229) KIP-761: implement a total blocked time metric in Kafka Streams

2021-08-25 Thread Rohan Desai (Jira)
Rohan Desai created KAFKA-13229: --- Summary: KIP-761: implement a total blocked time metric in Kafka Streams Key: KAFKA-13229 URL: https://issues.apache.org/jira/browse/KAFKA-13229 Project: Kafka

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-22 Thread Rohan Desai
calls should be accounted for by the admin client iotime/iowaittime metrics. On Tue, Jul 20, 2021 at 11:49 PM Rohan Desai wrote: > > I remember now that we moved the round-trip PID's txn completion logic > into > init-transaction and commit/abort-transaction. So I think we'

[VOTE] KIP-761: Add Total Blocked Time Metric to Streams

2021-07-21 Thread Rohan Desai
Now that the discussion thread's been open for a few days, I'm calling for a vote on https://cwiki.apache.org/confluence/display/KAFKA/KIP-761%3A+Add+Total+Blocked+Time+Metric+to+Streams

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-21 Thread Rohan Desai
Jul 20, 2021 at 11:48 PM Rohan Desai wrote: > > > I had a question - it seems like from the descriptionsof > `txn-commit-time-total` and `offset-commit-time-total` that they measure > similar processes for ALOS and EOS, but only `txn-commit-time-total` is > included in `blocked-t

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-21 Thread Rohan Desai
, I've updated the KIP to reflect that the lower-level metrics are all client metrics that are then summed to compute the blocked time metric, which is a Streams metric. On Tue, Jul 20, 2021 at 11:58 AM Rohan Desai wrote: > > Similarly, I think "txn-commit-time-total" and >

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-20 Thread Rohan Desai
in the doc, but did not include it as a separate metric, and similarly, should we have a `txn-abort-time-total` as well? If yes, could you update the KIP page accordingly. `beginTransaction` is not blocking - I meant to remove that from that doc. I'll add something for abort. On Mon, Jul 1

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-20 Thread Rohan Desai
as a separate metric, and similarly, should we have a `txn-abort-time-total` as well? If yes, could you update the KIP page accordingly. Ack. On Mon, Jul 12, 2021 at 11:29 PM Rohan Desai wrote: > Hello All, > > I'd like to start a discussion on the KIP linked above which proposes som

Re: [DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-13 Thread Rohan Desai
to the metrics already implemented. Thanks in advance for your feedback! Best Regards, Rohan On Mon, Jul 12, 2021 at 12:00 PM Rohan Desai wrote: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-761%3A+Add+Total+Blocked+Time+Metric+to+Streams >

[DISCUSS] KIP-761: Add total blocked time metric to streams

2021-07-12 Thread Rohan Desai
https://cwiki.apache.org/confluence/display/KAFKA/KIP-761%3A+Add+Total+Blocked+Time+Metric+to+Streams

[jira] [Created] (KAFKA-12707) KafkaProducer should have a clearer error message on sasl.mechanism misconfiguration

2021-04-22 Thread Rohan Desai (Jira)
Rohan Desai created KAFKA-12707: --- Summary: KafkaProducer should have a clearer error message on sasl.mechanism misconfiguration Key: KAFKA-12707 URL: https://issues.apache.org/jira/browse/KAFKA-12707

[jira] [Created] (KAFKA-10585) Kafka Streams should clean up the state store directory from cleanup

2020-10-08 Thread Rohan Desai (Jira)
Rohan Desai created KAFKA-10585: --- Summary: Kafka Streams should clean up the state store directory from cleanup Key: KAFKA-10585 URL: https://issues.apache.org/jira/browse/KAFKA-10585 Project: Kafka

[jira] [Created] (KAFKA-9659) Kafka Streams / Consumer fails on "fatal exception: group.instance.id gets fenced"

2020-03-04 Thread Rohan Desai (Jira)
Rohan Desai created KAFKA-9659: -- Summary: Kafka Streams / Consumer fails on "fatal exception: group.instance.id gets fenced" Key: KAFKA-9659 URL: https://issues.apache.org/jira/browse/KAFKA-9659

[VOTE] KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers

2019-02-04 Thread Rohan Desai
Calling a vote for this since it's pretty trivial/non-controversial https://cwiki.apache.org/confluence/display/KAFKA/KIP-425%3A+Add+some+Log4J+Kafka+Appender+Properties+for+Producing+to+Secured+Brokers

Re: KIP-425

2019-02-02 Thread Rohan Desai
Please ignore. Starting a new one with the right subject. On Sat, Feb 2, 2019 at 4:23 PM Rohan Desai wrote: > Starting a mail thread to discuss KIP-425: Add some Log4J Kafka Appender > Properties for Producing to Secured Brokers >

KIP-425

2019-02-02 Thread Rohan Desai
Starting a mail thread to discuss KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers

[DISCUSS] KIP-425: Add some Log4J Kafka Appender Properties for Producing to Secured Brokers

2019-02-02 Thread Rohan Desai
https://cwiki.apache.org/confluence/display/KAFKA/KIP-425%3A+Add+some+Log4J+Kafka+Appender+Properties+for+Producing+to+Secured+Brokers

[jira] [Created] (KAFKA-7896) Add some Log4J Kafka Properties for Producing to Secured Brokers

2019-02-02 Thread Rohan Desai (JIRA)
Rohan Desai created KAFKA-7896: -- Summary: Add some Log4J Kafka Properties for Producing to Secured Brokers Key: KAFKA-7896 URL: https://issues.apache.org/jira/browse/KAFKA-7896 Project: Kafka

Request for Confluence access

2019-02-01 Thread Rohan Desai
Hello, I want to write a KIP and need access to do so in Confluence.

[jira] [Created] (KAFKA-7311) Sender should reset next batch expiry time between poll loops

2018-08-18 Thread Rohan Desai (JIRA)
Rohan Desai created KAFKA-7311: -- Summary: Sender should reset next batch expiry time between poll loops Key: KAFKA-7311 URL: https://issues.apache.org/jira/browse/KAFKA-7311 Project: Kafka

[jira] [Created] (KAFKA-7064) "Unexpected resource type GROUP" when describing broker configs using latest admin client

2018-06-15 Thread Rohan Desai (JIRA)
Rohan Desai created KAFKA-7064: -- Summary: "Unexpected resource type GROUP" when describing broker configs using latest admin client Key: KAFKA-7064 URL: https://issues.apache.org/jira/browse/

[jira] [Reopened] (KAFKA-6383) StreamThread.shutdown doesn't clean up completely when called before StreamThread.start

2018-01-02 Thread Rohan Desai (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohan Desai reopened KAFKA-6383: I missed a race condition in my fix. If we start and then shutdown a streams thread without

[jira] [Created] (KAFKA-6383) StreamThread.shutdown doesn't clean up completely when called beforeStreamThread.start

2017-12-18 Thread Rohan Desai (JIRA)
Rohan Desai created KAFKA-6383: -- Summary: StreamThread.shutdown doesn't clean up completely when called beforeStreamThread.start Key: KAFKA-6383 URL: https://issues.apache.org/jira/browse/KAFKA-6383