Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Many thanks for the feedback. Per Gwen's suggestion, I've updated the KIP to specify that the task count will be per-worker (no additional MBean tag, since each process is a worker) and per-connector (MBean tag). On Mon, Jun 17, 2019 at 8:24 PM Cyrus Vafadari wrote: > I meant to write: > I've

Build failed in Jenkins: kafka-trunk-jdk11 #639

See Changes: [wangguoz] MINOR: rename subscription construction function (#6954) -- [...truncated 2.87 MB...] org.apache.kafka.connect.runtime.ConnectorConfigTest >

Build failed in Jenkins: kafka-trunk-jdk8 #3730

See Changes: [wangguoz] KAFKA-7853: Refactor coordinator config (#6854) [jason] KAFKA-8539; Add group.instance.id to Subscription (#6936) [jason] HOTFIX: Fix optional import in ConsumerCoordinator (#6953)

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

I meant to write: I've also updated the KIP to clarify that every task must have exactly one non-null *status* at all times. On Mon, Jun 17, 2019 at 6:55 PM Cyrus Vafadari wrote: > Guozhang, > > Both of Kafka's implementations of "StatusBackingStore" immediately delete > the task from the

[jira] [Created] (KAFKA-8551) Comments for connectors() in Herder interface

Luying Liu created KAFKA-8551: - Summary: Comments for connectors() in Herder interface Key: KAFKA-8551 URL: https://issues.apache.org/jira/browse/KAFKA-8551 Project: Kafka Issue Type: Bug

Re: [VOTE] 2.3.0 RC2

+1 (binding) Verified release notes, javadoc, and run quick start with scala_2.12 binary. Guozhang On Mon, Jun 17, 2019 at 7:47 PM Satish Duggana wrote: > +1 (non-binding) > > - Ran testAll/releaseTarGzAll successfully with no failures. > - Ran through quickstart of core/streams on builds

Re: [VOTE] 2.3.0 RC2

+1 (non-binding) - Ran testAll/releaseTarGzAll successfully with no failures. - Ran through quickstart of core/streams on builds generated from 2.3.0-rc2 tag. - Ran few internal apps targeting to topics on 3 node cluster. Thanks, Satish. On Mon, Jun 17, 2019 at 7:25 PM David Arthur wrote: > >

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Guozhang, Both of Kafka's implementations of "StatusBackingStore" immediately delete the task from the backign store when you try to set it to DESTROYED, so we'd actually expect it to always be zero. A nonzero number of destroyed tasks would either indicate a new implementation of

Jenkins build is back to normal : kafka-trunk-jdk11 #638

See

Jenkins build is back to normal : kafka-trunk-jdk8 #3729

See

Re: [DISCUSS] KIP-478 Strongly Typed Processor API

Thanks for the feedback, Sophie! I actually felt a little uneasy when I wrote that remark, because it's not restricted at all in the API, it's just available to you if you choose to give your stores and context the same parameters. So, I think your use case is valid, and also perfectly

[jira] [Created] (KAFKA-8550) Connector validation fails with aliased converters

Chris Egerton created KAFKA-8550: Summary: Connector validation fails with aliased converters Key: KAFKA-8550 URL: https://issues.apache.org/jira/browse/KAFKA-8550 Project: Kafka Issue Type:

Re: Posted a new article about Kafka Streams

Hi all, Thanks for the heads-up! I've added the article to my reading list. And thanks for the reminder about your KIP, Paul. I just visited the discussion and voting threads. Also, another plug, you might both be interested in my freshly-proposed KIP-478

Re: [VOTE] KIP-401: TransformerSupplier/ProcessorSupplier StateStore connecting

I'm +1 (nonbinding) on the current iteration of the proposal. On Mon, May 27, 2019 at 1:58 PM Paul Whalen wrote: > > I spoke too early a month ago, but I believe the proposal is finalized now > and ready for voting. > > KIP: >

Re: [DISCUSS] KIP-401 TransformerSupplier/ProcessorSupplier enhancements

Hey, all, Sorry I'm late to the party. I meant to read into this KIP before, but didn't get around to it. I was just reminded when Paul mentioned it in a different thread. Please feel free to bump a discussion any time it stalls! I've just read through the whole discussion so far, and, to echo

Re: [DISCUSS] KIP-478 Strongly Typed Processor API

Hey John, thanks for writing this up! I like the proposal but there's one point that I think may be too restrictive: "A processor that happens to use a typed store is actually emitting the same types that it is storing." I can imagine someone could want to leverage this new type safety without

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Hello Cyrus, Thanks for the KIP. I just have one nit question about Connect destroyed tasks: is it an ever-increasing number? If yes, the corresponding metric value would be increasing indefinitely as well. Is that intentional? Otherwise, lgtm. Guozhang On Mon, Jun 17, 2019 at 1:14 PM Gwen

Re: Become a contributer

My user ID is anastasiavela. Anastasia On Thu, Jun 13, 2019 at 11:18 AM Anastasia Vela wrote: > Hi, > > I would like to contribute to Apache Kafka. Would I be able to get access > to handle JIRAs? > Github ID: anatasiavela > Github Email: anastasiave...@gmail.com > > Thanks, > Anastasia >

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

Woah, I wasn't aware of that Hamcrest test style. Awesome! Thanks for the updates. I look forward to hearing what others think. -John On Mon, Jun 17, 2019 at 4:12 AM Jukka Karvanen wrote: > > Wiki page updated: >

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

Sorry to join so late, but did we consider a single set of task-count metrics and using tags to scope each data point to a specific connector and worker (and in the future perhaps also user)? It will make analysis of the data easier - someone may want to breakdown tasks by both worker and

[DISCUSS] KIP-478 Strongly Typed Processor API

Hi all, I'd like to propose KIP-478 (https://cwiki.apache.org/confluence/x/2SkLBw). This proposal would add output type bounds to the Processor interface in Kafka Streams, which enables static checking of a number of useful properties: * A processor B that consumes the output of processor A is

Build failed in Jenkins: kafka-trunk-jdk11 #637

See Changes: [wangguoz] KAFKA-7853: Refactor coordinator config (#6854) [jason] KAFKA-8539; Add group.instance.id to Subscription (#6936) -- [...truncated 4.28 KB...] >

Build failed in Jenkins: kafka-trunk-jdk11 #636

See Changes: [jason] KAFKA-8457; Move `Log' reference from `Replica` into `Partition` (#6841) -- [...truncated 2.52 MB...] org.apache.kafka.connect.transforms.CastTest >

[jira] [Resolved] (KAFKA-8539) Add `group.instance.id` to Subscription class

[ https://issues.apache.org/jira/browse/KAFKA-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8539. Resolution: Fixed Fix Version/s: 2.4.0 > Add `group.instance.id` to Subscription

Re: Posted a new article about Kafka Streams

Hey Paul, Thank you so much for your input! :) Just updated my article about the iterator closing. Thank you! Best, Daniyar Yeralin > On Jun 16, 2019, at 4:11 PM, Paul Whalen wrote: > > I've only skimmed it so far, but great job! The community is in serious > need of more examples of the

[jira] [Resolved] (KAFKA-7853) Refactor ConsumerCoordinator/AbstractCoordinator to reduce constructor parameter list

[ https://issues.apache.org/jira/browse/KAFKA-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-7853. Resolution: Fixed > Refactor ConsumerCoordinator/AbstractCoordinator to reduce constructor >

Re: [VOTE] KIP-476: Add Java AdminClient interface

Andy, while I agree that the new interface is useful, I'm not convinced adding an interface requires deprecating AdminClient and changing so much client code. Why not just add the Admin interface, have AdminClient implement it, and have done? Ryanne On Mon, Jun 17, 2019 at 12:09 PM Andy Coates

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

bump

Re: [VOTE] KIP-476: Add Java AdminClient interface

Hi all, I think I've addressed all concerns. Let me know if I've not. Can I call another round of votes please? Thanks, Andy On Fri, 14 Jun 2019 at 04:55, Satish Duggana wrote: > Hi Andy, > Thanks for the KIP. This is a good change and it gives the user a better > handle on Admin client

Re: [VOTE] KIP-476: Add Java AdminClient interface

Hi All, I've updated the KIP to move the `create` factory method implementation into a new `AdminClients` utility class, rather than on the new `Admin` interface. Satish, As above, the KIP has been updated to only have the operations on the `Admin` api. As for the overhead of dynamic proxies...

[jira] [Resolved] (KAFKA-8548) Inconsistency in Kafka Documentation

[ https://issues.apache.org/jira/browse/KAFKA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8548. Resolution: Not A Problem Closing this. The docs are correct as pointed out by [~ckamal].

[jira] [Resolved] (KAFKA-8457) Remove Log dependency from Replica

[ https://issues.apache.org/jira/browse/KAFKA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Singh resolved KAFKA-8457. Resolution: Fixed Fixed in commit 57baa4079d9fc14103411f790b9a025c9f2146a4 > Remove Log

Re: [VOTE] 2.3.0 RC2

+1 binding Verified signatures, pulled down kafka_2.12-2.3.0 and ran producer/consumer perf test scripts. -David On Mon, Jun 17, 2019 at 1:48 AM Vahid Hashemian wrote: > +1 (non-binding) > > I also verifies signatures, build from source and tested the Quickstart > successfully on the built

[jira] [Created] (KAFKA-8549) Kafka Windows start up failed due to topic name conflict

prehistoricpenguin created KAFKA-8549: - Summary: Kafka Windows start up failed due to topic name conflict Key: KAFKA-8549 URL: https://issues.apache.org/jira/browse/KAFKA-8549 Project: Kafka

Re: [DISCUSS] KIP-470: TopologyTestDriver test input and output usability improvements

Wiki page updated: https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements ClientRecord removed and replaced with TestRecord in method calls. TestRecordFactory removed (time tracking functionality to be included to

[jira] [Created] (KAFKA-8548) Inconsistency in Kafka Documentation

Seweryn Habdank-Wojewodzki created KAFKA-8548: - Summary: Inconsistency in Kafka Documentation Key: KAFKA-8548 URL: https://issues.apache.org/jira/browse/KAFKA-8548 Project: Kafka

Re: Kafka streams rebalancing issue

> This leads to lockexception in >> consumer2 and consumer2 remains in livelock to create state directories for >> those two partitons. There is a fix for a `LockException` during rebalance in 0.11.0.1: https://issues.apache.org/jira/browse/KAFKA-5167 Maybe upgrading helps? Note, that you can