Re: Accessing Topology Builder

ble tracing? > > > > Also, as Matthias said, you would need to create a KIP to propose this > > change, but of course we can continue this preliminary discussion until > you > > feel confident to create the KIP. > > > > Off the top of my head, here are some ot

Re: Kafka Contributor access

Hello Srinivas, I've added you to the contributor list and assigned the ticket to you. Thanks for your interest contributing to Kafka!! Guozhang On Wed, Sep 19, 2018 at 6:08 AM, Srinivas Reddy wrote: > My Apache JIRA id is : mrsrinivas > > > > -- > Srinivas Reddy > > http://mrsrinivas.com/ > >

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

wbie%2B%2B%22%2C%20newbie)%20ORDER%20BY%20assignee%20DESC%2C%20priority%20DESC Guozhang On Mon, Sep 17, 2018 at 4:50 PM, Yishun Guan wrote: > @Guozhang Wang What do you think? > On Fri, Sep 14, 2018 at 2:39 PM Yishun Guan wrote: > > > > Hi All, > > > > Afte

Re: [ANNOUNCE] New committer: Colin McCabe

Congrats Colin! Guozhang On Tue, Sep 25, 2018 at 8:39 AM, Ray Chiang wrote: > Nice job Colin! > > -Ray > > > On 9/25/18 1:58 AM, Stanislav Kozlovski wrote: > >> Congrats Colin! >> >> On Tue, Sep 25, 2018 at 9:51 AM Edoardo Comar wrote: >> >> Congratulations Colin ! >>>

Re: [STREAMS] Qs on code of State Managers (AbstractStateManager, ProcessorStateManager and GlobalStateManagerImpl)

Hello Jacek, Your observation is valid, currently the code hierarchy of AbstractStateManager, ProcessorStateManager and GlobalStateManagerImpl are a bit intrusive to each other and would better be cleaned a bit. Feel free to create a JIRA with your observed placed to clean up and submit a PR. G

Re: [DISCUSS] KIP-378: Enable Dependency Injection for Kafka Streams handlers

Hello Wladimir, Thanks for proposing the KIP. I think the injection can currently be done by passing in the key/value pair directly into the properties which can then be accessed from the `ProcessorContext#appConfigs` or `#appConfigsWithPrefix`. For example, when constructing the properties you ca

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

> not retryable. > > > -Matthias > > On 6/25/18 10:15 AM, Guozhang Wang wrote: > > I'm wondering if StreamThreadNotStarted could be merged into > > StreamThreadNotRunning, because I think users' handling logic for the > third > > case would be likely

Re: [EXTERNAL] Incremental Cooperative Rebalancing

Hello Konstantine, Thanks for the great write-up! Here are a few quick comments I have about the proposals: 1. For "Kubernetes process death" and "Rolling bounce" case, there is another parallel work on KIP-345 [1] (cc'ed contributor) that is aimed to mitigate these two issues, but it is relying

Re: 1.0.0 KIPs Update

//github.com/apache/kafka/pull/3799 < > > >> https://github.com/apache/kafka/pull/3799> and > > https://issues.apache.org/ > > >> jira/browse/KAFKA-3480 <https://issues.apache.org/ > > jira/browse/KAFKA-3480> > > >> that don't require KIPs, is today t

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

gt;>>> > > > > > >> >>>> On Tue, 4 Jul 2017 at 23:30 Jeyhun Karimov < > > > je.kari...@gmail.com > > > > > > > > > > >> >> wrote: > > > > > >> >>>> > > > &g

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

gt; >>>> > > > > > >> >>>> On Tue, 4 Jul 2017 at 23:30 Jeyhun Karimov < > > > je.kari...@gmail.com > > > > > > > > > > >> >> wrote: > > > > > >> >>>> > > &g

1.0.0 Code Freeze Update

Hello Folks, This is another heads up on 1.0.0 progress post the feature freeze deadline: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913 We have ended up with 28 total KIPs in this release, with 5 other KIPs being postponed to subsequent releases; out of the 28 KIPs t

Re: [VOTE] KIP-202

If no on else has opinions or votes on this thread, Richard could you close the voting phase then? On Sat, Sep 23, 2017 at 4:11 PM, Ismael Juma wrote: > Thanks for the KIP, +1 (binding). > > On 19 Sep 2017 12:27 am, "Richard Yu" wrote: > > > Hello, I would like to start a VOTE thread on KIP-202

Re: [VOTE] KIP-202

sed with 5++. We are now closing the vote. > > On Mon, Sep 25, 2017 at 1:18 AM, Guozhang Wang wrote: > > > If no on else has opinions or votes on this thread, Richard could you > close > > the voting phase then? > > > > On Sat, Sep 23, 2017 at 4:11 PM, Ismael Ju

1.0.0 Branch Update

Hello Kafka developers, This is another update on the 1.0.0 release progress. We were planing to close the code freeze and make the first RC by today, but we are behind our schedule a bit. After making a pass over all the JIRAs to move non-blockers to the next release we still have 20 tickets ong

Re: Contributing to kafka

Hi Gilles, Thanks for your interest in contributing, has added you to the list. Cheers, Guozhang On Thu, Oct 5, 2017 at 1:09 PM, Gilles Degols wrote: > Hello, > > > > As I use Kafka during my day-job I would like to contribute to the project > in my spare time. According the kafka.apache.org w

Re: [VOTE] KIP-171 - Extend Consumer Group Reset Offset for Stream Application

Thanks for the KIP Jorge, overall it looks good. As Matthias mentioned in the DISCUSS thread, while working on KIP-198 we realize that the o.a.k.admin.AdminClient does not yet have all the functionalities that k.a.AdminClient (in Scala) have, for example, describeConsumerGroup(). Thus, could you

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Richard, Matthias: 0. Could you describe a bit what are the possible use cases of `allLatest`, `minKey` and `maxKey`? I'd prefer keeping the APIs to add at a minimum necessary amount, to avoid a swamp of new APIs that no one would really use but just complicated the internal code base. 1. One min

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

gt; > > > > > > On Thu, Sep 21, 2017 at 4:50 PM Damian Guy > > > wrote: > > > > > > > > > >> Hi Jeyhun, > > > > >> > > > > >> All KIP-182 API PRs have now been merged. So you can consider i

[VOTE] 1.0.0 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 1.0.0. It's worth noting that starting in this version we are using a different version protocol with three digits: *major.minor.bug-fix* Any and all testing is welcome, but the following

Re: [VOTE] 1.0.0 RC0

. > https://repository.apache.org/content/groups/staging/org/ > apache/kafka/kafka-clients/ > > I don't see 1.0.0 jars. > > Would the jars be populated later ? > > Thanks > > On Tue, Oct 10, 2017 at 6:34 PM, Guozhang Wang wrote: > > > Hello Kafka users, developers and c

Re: [VOTE] 1.0.0 RC0

> > -import org.apache.kafka.common.protocol.SecurityProtocol; > +import org.apache.kafka.common.security.auth.SecurityProtocol; > > I took a look at docs/upgrade.html but didn't see any mentioning. > > Should this be documented ? > > On Tue, Oct 10, 2017 at 6:34 PM, Guozhang Wang wrote: > > >

Re: [VOTE] 1.0.0 RC0

the upgrade.html. Guozhang On Thu, Oct 12, 2017 at 11:39 AM, Guozhang Wang wrote: > Thanks Ted, > > I'm looking into this for possible doc changes now. > > Guozhang > > On Wed, Oct 11, 2017 at 3:23 PM, Ted Yu wrote: > >> Looks like the following change is ne

Re: [VOTE] 1.0.0 RC0

oned explicitly. > Otherwise they need to dig into the code to find out. > > Just my two cents. > > On Thu, Oct 12, 2017 at 2:24 PM, Guozhang Wang wrote: > > > Ted, > > > > I can found that we do have a corresponding doc change for this renaming: > >

[VOTE] 1.0.0 RC1

Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 1.0.0. It's worth noting that starting in this version we are using a different version protocol with three digits: *major.minor.bug-fix* Any and all testing is welcome, but the followin

Re: [VOTE] 1.0.0 RC1

eant to read October 17th, > which is next Tuesday? > > I feel like this short a testing window for a 1.0 RC is a little low, as > 1.0 is clearly a big announcement of stability, and folk should be given > enough time to do thorough testing. > > Thanks > > Tom > > O

Re: Possible Feature: Topic Retention Policy

Hello Alexei, Thanks for bringing up this question. Just my 2 cents: 1. For request-response messaging, I think an alternative approach is to use a single topic for request queue, and use one temporary topic for response queue. I.e. everyone sends their request to a single topic, and wait for its

Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1

kafka/pull/4075) and reduces the controlled shutdown time from 6.5 mins > to 30 secs, with > 25K partitions, RF=2 and 5 brokers. > > It would be useful to include this fix in 1.0.0. > > Thanks, > > Jun > > > On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang wrote: > &

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

gt; > o.a.k.processor. > > > > 6. I'm not clear why we want to move `commit()` from ProcessorContext to > > RecordContext? > > > > - > Because it makes sense logically and to reduce code maintenance (both > interfaces have offset() timestamp() topic() part

[VOTE] 1.0.0 RC2

Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 1.0.0. The main PRs that gets merged in after RC1 are the following: https://github.com/apache/kafka/commit/dc6bfa553e73ffccd1e604963e076c78d8ddcd69 It's worth noting that starting in thi

Re: [kafka-clients] [VOTE] 1.0.0 RC2

7;s included in 1.0.0 is a logging >> change improvement in the controller, which does give significant perf >> benefit. However, the async ZK changes are in trunk and will be in 1.1.0. >> >> Jun >> >> On Tue, Oct 17, 2017 at 9:47 AM, Guozhang Wang >> wrot

Re: [VOTE] 1.0.0 RC2

ator) > > I also noticed another issue on Windows that I reported in KAFKA-6075. > That issue is not specific to the release candidate though. > > Thanks. > --Vahid > > > > From: Guozhang Wang > To: "dev@kafka.apache.org" , > "us...@kafka.apa

Re: [VOTE] 1.0.0 RC2

Sent the email too quick.. The filed blocker JIRA is this one: https://issues.apache.org/jira/browse/KAFKA-6093 Guozhang On Thu, Oct 19, 2017 at 8:58 PM, Guozhang Wang wrote: > Vahid, > > Thanks for testing the RC! Really appreciate it. > > Could you file another JIRA

Re: Apache Kafka Connectors and Apache Camel Components

Hi Chris, There has been some discussions of keeping an eco-system around Apache Kafka v.s. making every component of it as in AK in the community. In the past we have tried to follow the latter approach, e.g. keeping the contributed hadoop clients within AK: https://github.com/apache/kafka/tree/

Re: [VOTE] KIP-171 - Extend Consumer Group Reset Offset for Stream Application

Thanks Jorge for driving this KIP! +1 (binding). Guozhang On Mon, Oct 16, 2017 at 2:11 PM, Bill Bejeck wrote: > +1 > > Thanks, > Bill > > On Fri, Oct 13, 2017 at 6:36 PM, Ted Yu wrote: > > > +1 > > > > On Fri, Oct 13, 2017 at 3:32 PM, Matthias J. Sax > > wrote: > > > > > +1 > > > > > > > > >

Re: Request for access to the KIP wiki page

Hello Sönke, It's done. Cheers. Guozhang On Mon, Oct 23, 2017 at 11:08 AM, Sönke Liebau < soenke.lie...@opencore.com.invalid> wrote: > Hi all, > > could someone please grant my user (userid: soenkeliebau) permissions to > the KIP page of the wiki, I'd like to create a proposal. > > Thanks! >

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

>>> we can adopt > >>> > >>> public interface RichValueMapper { > >>> VR apply(final V value, > >>> final K key, > >>> final RecordContext recordContext, > >>> final Processo

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

k before vote. > > On Wed, Oct 18, 2017 at 8:26 PM, Richard Yu > wrote: > > > Is this KIP close to completion? Because we could start working on the > > code itself now. (Its at about this stage). > > > > On Mon, Oct 16, 2017 at 7:37 PM, Richard Yu > > wrote:

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

Thanks for the KIP Matt. Regarding the handle interface of ProductionExceptionHandlerResponse, could you write it on the wiki also, along with the actual added config names (e.g. what https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers described)

Re: Requesting Wiki Comment Permissions

Hi Jordan, I'd recommend leaving comments or asking questions about a wiki page directly on the mailing list here, as it gets much faster. Guozhang On Mon, Oct 23, 2017 at 1:31 PM, Jordan Pilat wrote: > Hello, > > Would somebody please be so kind as to grant me "comment" permissions on > the

Re: Metadata class doesn't "expose" topics with errors

Hello Paolo, The reason we filtered the errors in the topics in the generated Cluster is that Metadata and its "fetch()" returned Cluster is a common class that is used among all clients (producer, consumer, connect, streams, admin), and is treated as a high-level representation of the current sna

[VOTE] 1.0.0 RC3

Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 1.0.0. The main PRs that gets merged in after RC1 are the following: https://github.com/apache/kafka/commit/dc6bfa553e73ffccd1e604963e076c 78d8ddcd69 It's worth noting that starting in th

Re: [VOTE] KIP-205: Add all() and range() API to ReadOnlyWindowStore

+1. Thanks. On Mon, Oct 23, 2017 at 8:11 PM, Richard Yu wrote: > Hi all, > > I want to propose KIP-205 for the addition of new API. It is about adding > methods similar to those found in ReadOnlyKeyValueStore to the > ReadOnlyWindowStore class. As it appears the discussion has reached a > conclu

Re: [VOTE] 1.0.0 RC3

above: 1.0.0-rc3 tag > > FYI > > On Mon, Oct 23, 2017 at 6:00 PM, Guozhang Wang wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the third candidate for release of Apache Kafka 1.0.0. The main > PRs > > that gets merg

Re: [VOTE] 1.0.0 RC3

t; > On Wed, Oct 25, 2017 at 10:37 AM, Guozhang Wang > wrote: > > > Ted: > > > > Thanks for the reminder. Yes it is a typo. In fact this is the "forth" > > candidate of the release, not the "third" one :) > > > > > > Jaikiran:

Re: [DISCUSS] KIP-205: Add getAllKeys() API to ReadOnlyWindowStore

Hello Richard, Xavier: I think I am convinced my your arguments. How about keeping all() as is and rename "range" to "fetchAll" then? Guozhang On Thu, Oct 26, 2017 at 9:21 AM, Xavier Léauté wrote: > I don't feel this worth holding up the vote for, if no one else shares my > concerns. > On W

Re: [VOTE] KIP-205: Add all() and range() API to ReadOnlyWindowStore

BTW, Xavier has an argument to rename "range" as it does not take a key or key range at all. I'd propose we rename this function to "fetchAll". Guozhang On Wed, Oct 25, 2017 at 7:27 AM, Damian Guy wrote: > +1 > > On Tue, 24 Oct 2017 at 16:46 Guozhang Wang w

[VOTE] 1.0.0 RC4

Hello Kafka users, developers and client-developers, This is the fifth candidate for release of Apache Kafka 1.0.0. The main PRs that gets merged in after RC3 are the following: *https://github.com/apache/kafka/commit/def1a768a6301c14ad6611358716ab03de04e76b

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

23, 2017 at 9:00 PM Matt Farmer wrote: > > > Thanks for this feedback. I’m at a conference right now and am planning > on > > updating the KIP again with details from this conversation later this > week. > > > > I’ll shoot you a more detailed response then!

Re: Metadata class doesn't "expose" topics with errors

Paolo Patierno > Senior Software Engineer (IoT) @ Red Hat > Microsoft MVP on Azure & IoT > Microsoft Azure Advisor > > Twitter : @ppatierno<http://twitter.com/ppatierno> > Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience<

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

up after > > crashing we’ll eventually try to process the message we failed to produce > > again. > > > > I’m concerned that “putting in a queue for later” opens you up to putting > > messages into the destination topic in an unexpected order. However if &g

Re: [VOTE] 1.0.0 RC4

Oct 31, 2017 at 8:42 PM, Ismael Juma > > wrote: > > > > > > > > > +1 (binding) from me. Tested the quickstart with the source and > > binary > > > > > (Scala 2.12) artifacts, ran the tests on the source artifact and >

[RESULTS] [VOTE] Release Kafka version 1.0.0

The vote on RC4 passes with 8 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jason Gustafson * Ismael Juma * Guozhang Wang Community: * Manikumar Reddy * Ted Yu * Jeff Chao * Vahid Hashemian * Satish Guggana 0 votes * No votes -1 votes * No votes Vote thread: http://mail

[ANNOUNCE] Apache Kafka 1.0.0 Released

Hussonnois, Janek P, Gregor Uhlenheuer, Guozhang Wang, Gwen Shapira, Hamidreza Afzali, Hao Chen, Jiefang He, Holden Karau, Hooman Broujerdi, Hugo Louro, Ismael Juma, Jacek Laskowski, Jakub Scholz, James Cheng, James Chien, Jan Burkhardt, Jason Gustafson, Jeff Chao, Jeff Klukas, Jeff Widman, Jeremy

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

return context().recordContext().topic(); > > > >>> } > > > >>> > > > >>> @Override > > > >>> public int partition() { > > > >>> return context().recor

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Made a pass over the PR and left some comments. I'm +1 on the wiki design page as well. On Tue, Oct 31, 2017 at 7:13 AM, Bill Bejeck wrote: > +1 > > Thanks, > Bill > > On Tue, Oct 31, 2017 at 4:36 AM, Paolo Patierno > wrote: > > > Hi all, > > > > > > because I don't see any further discussion a

Re: [VOTE] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

The vote should stay open for at least 72 hours. The bylaws can be found here https://cwiki.apache.org/confluence/display/KAFKA/Bylaws On Wed, Nov 1, 2017 at 8:09 AM, Matt Farmer wrote: > Hello all, > > It seems like discussion around KIP-210 has gone to a lull. I've got some > candidate work un

Re: [VOTE] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

+1 (binding) from me. Thanks! On Wed, Nov 1, 2017 at 4:50 PM, Guozhang Wang wrote: > The vote should stay open for at least 72 hours. The bylaws can be found > here https://cwiki.apache.org/confluence/display/KAFKA/Bylaws > > On Wed, Nov 1, 2017 at 8:09 AM, Matt Farmer wrote: &g

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Thanks for the KIP writeup Jan. I made a first pass and here are some quick comments: 1. Could we use K0 / V0 and K1 / V1 etc since K0 and KO are a bit harder to differentiate when reading. 2. I think you missed the key type in the intrusive approach example code snippet regarding "KTable oneTo

Re: kafka-pr-jdk9-scala2.12 keeps failing

Noticed that as well, could we track down to which git commit / version upgrade caused the issue? Guozhang On Thu, Nov 2, 2017 at 6:25 PM, Ted Yu wrote: > Hi, > I took a look at recent runs under https://builds.apache. > org/job/kafka-pr-jdk9-scala2.12 > > All the recent runs failed with: > >

Re: [VOTE] KIP-215: Add topic regex support for Connect sinks

+1 binding On Fri, Nov 3, 2017 at 1:25 PM, Ewen Cheslack-Postava wrote: > +1 binding > > Thanks Jeff! > > On Wed, Nov 1, 2017 at 5:21 PM, Randall Hauch wrote: > > > +1 (non-binding) > > > > Thanks for pushing this through. Great work! > > > > Randall Hauch > > > > On Wed, Nov 1, 2017 at 9:40 AM

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

Thanks for writing up the KIP. Vito, Matthias: one thing that I wanted to figure out first is what categories of errors we want to notify the users, if we only wants to distinguish fatal v.s. retriable then probably we should rename the proposed StateStoreMigratedException / StateStoreClosedExcept

[DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

Hello folks, I have filed a new KIP on adding AdminClient into Streams for internal topic management. Looking for feedback on *https://cwiki.apache.org/confluence/display/KAFKA/KIP-220%3A+Add+AdminClient+into+Kafka+Streams%27+ClientSupplier

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

t;> > >> On 11/1/17 12:23 AM, Matt Farmer wrote: > >> > Thanks for the heads up. Yes, I think my changes are compatible with > >> that > >> > PR, but there will be a merge conflict that happens whenever one of > the > >> PRs > >>

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

vel instead of WARN level. Would that alternative be appealing to you? > > On Sun, Nov 5, 2017 at 12:32 PM Guozhang Wang wrote: > > > Thanks for the updates. I made a pass over the wiki again and it looks > > good. > > > > About whether record collector should

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

Thanks Ted, has updated the KIP. On Fri, Nov 3, 2017 at 8:01 PM, Ted Yu wrote: > Looks good overall. > > bq. the creation within StreamsPartitionAssignor > > Typo above: should be StreamPartitionAssignor > > On Fri, Nov 3, 2017 at 4:49 PM, Guozhang Wang wrote: > > &g

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

rote: > > Looks good overall. > > > > bq. the creation within StreamsPartitionAssignor > > > > Typo above: should be StreamPartitionAssignor > > > > On Fri, Nov 3, 2017 at 4:49 PM, Guozhang Wang > wrote: > > > >> Hello folks, > >> > >> I hav

Re: [DISCUSS] 0.11.0.2 bug fix release

Thanks Rajini, +1 on releasing a 0.11.0.2. I made a pass over the outstanding issues, and added another related issue ( https://issues.apache.org/jira/browse/KAFKA-4767) to KAFKA-5936 to the list. For the other outstanding ones that do not yet have assigned to anyone yet, we'd need to start worki

Re: [ANNOUNCE] New committer: Onur Karaman

I'd want to add that Onur has also been contributing a lot to the new consumer development in the past, making it happen in the 0.9 release. Very well deserved. Congrats, Onur! Guozhang On Mon, Nov 6, 2017 at 9:28 AM, Ted Yu wrote: > Congratulations, Onur! > > On Mon, Nov 6, 2017 at 9:24 AM,

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

A quick question: I think we do not yet have the `list consumer groups` func as in the old AdminClient. Without this `describe group` given the group id would not be very useful. Could you include this as well in your KIP? More specifically, you can look at kafka.admin.AdminClientfor more details o

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

e group.id from the user as command line > argument. Thus, I think the KIP is useful without "describe group" > command to. > > I am happy to include "describe group" command in the KIP. Just want to > point out, that there is no reason to insist on it IMHO. >

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

pdate and clarification. > > Sounds good to me :) > > > -Matthias > > > > On 11/6/17 12:16 AM, Guozhang Wang wrote: > > Thanks Matthias, > > > > 1) Updated the KIP page to include KAFKA-6126. > > 2) For passing configs, I agree, will make a pass ove

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

Regarding the API design (the proposed set of overloads v.s. one overload on #map to enrich the record), I think what we have represents a good trade-off between API succinctness and user convenience: on one hand we definitely want to keep as fewer overloaded functions as possible. But on the other

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

Matthias J. Sax wrote: > I would prefer to keep the current check. We could even improve it, and > do the check to more than one brokers (even all provided > bootstrap.servers) or some random servers after we got all meta data > about the cluster. > > > -Matthias > &g

Re: Invalid topology building: Processor has no access to StateStore

Hello Florian, In PAPI, you need to associate the state store with the processor node via the Topology#addStateStore(StoreBuilder); note that StateStore#init() is supposed to be triggered by the Streams library only but not by the users: it is called when the library is constructing the specified

Re: kafka-pr-jdk9-scala2.12 keeps failing

t; > > > > >> > > FAILURE: Build failed with an exception. > > >> > > > > >> > > * What went wrong: > > >> > > Could not determine java version from '9.0.1'. > > >> > > > > >> > &g

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

Instead of restoring the global store during registration, could we also do this after the rebalance callback as in the main loop? By doing this we can effectively swallow-and-retry-in-next-loop as we did for non-global stores. Since global stores are per task not per thread, we would not process t

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

> threads start processing. They don't go through a rebalance as it is manual > assignment of all partitions in the topic. > > On Thu, 9 Nov 2017 at 17:43 Guozhang Wang wrote: > > > Instead of restoring the global store during registration, could we also > do > > th

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

the offset causing the flush? > So one might aswell think to drop offsets from this RecordContext. > > Best Jan > > > > > > > > On 07.11.2017 03:18, Guozhang Wang wrote: > >> Regarding the API design (the proposed set of overloads v.s. one overload >> on

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

gt; > >>> transformations on a Source-backed KTable/KStream > > >>> could also return a Ktable/KStream instance of the type we return > from > > >>> the topology builder. > > >>> Only after any operation that exceeds projection or filter one would > > >&

Re: [DISCUSS] KIP-221: Repartition Topic Hints in Streams

A few clarification questions on the proposal details. 1. API: although the repartition only happens at the final stateful operations like agg / join, the repartition flag info was actually passed from an earlier operator like map / groupBy. So what should be the new API look like? For example, if

Re: [DISCUSS] Kafka 2.0.0 in June 2018

Thanks Ismael, the proposal looks good to me. A side note regarding: https://issues.apache.org/jira/browse/KAFKA-5637, could we resolve this ticket sooner than later to make clear about the code deprecation and support duration when moving from 1.0.x to 2.0.x? Guozhang On Fri, Nov 10, 2017 at

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

Sounds good to me for returning an object than a Long type. Guozhang On Fri, Nov 10, 2017 at 9:26 AM, Paolo Patierno wrote: > Ismael, > > > yes it makes sense. Taking a look to the other methods in the Admin > Client, there is no use case returning a "simple" type : most of them are > Void or

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

ng on the PR, I realized that parameter > > > "retry.backoff.ms" is already available in StreamsConfig. I updated > the > > > KIP accordingly. > > > > > > I also discovered, that we have a hard coded number of retries for > state > &g

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

oft Azure Advisor* > > Twitter : @ppatierno <http://twitter.com/ppatierno> > Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience <http://paolopatierno.wordpress.com/> > > > ------ > *From:* Paolo Patiern

Re: [VOTE] KIP-204 : adding records deletion operation to the new Admin Client API

rry Guozhang ... reviewing the emails thread I have misunderstood your > +1 which was not a vote but just about the wiki design. > > So the vote is still opened with 2 binding votes and 5 non binding votes. > -- > *From:* Guozhang Wang > *Sent:* Monday, N

Re: [DISCUSS] KIP-224: Add configuration parameters `retries` and `retry.backoff.ms` to Streams API

not retryable. > > > I updated the KIP and PR accordingly, ie, we don't need to consider > newly added "retry" parameter for state directory locking. > > Let me know what you think about this. I'll start the vote thread in > parallel. > > > -Matthias

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

nce cost by not having a transient network >client >3. Ideally, this check should be on the AdminClient itself but adding >such a check creates "scope creep" for this KIP. > > IMHO the combination of these reasons makes option 3 my preferred approach. > > Th

[VOTE] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

Hello folks, I have filed a new KIP on adding AdminClient into Streams for internal topic management. Please review and cast your vote on this thread. *https://cwiki.apache.org/confluence/display/KAFKA/KIP-220%3A+Add+AdminClient+into+Kafka+Streams%27+ClientSupplier

Re: [VOTE] KIP-224: Add configuration parameters `retries` to Streams API

+1 On Tue, Nov 14, 2017 at 3:02 AM, Damian Guy wrote: > +1 > > On Tue, 14 Nov 2017 at 02:40 Bill Bejeck wrote: > > > Thanks for the KIP, +1 > > > > -Bill > > > > On Mon, Nov 13, 2017 at 7:25 PM, Ted Yu wrote: > > > > > +1 > > > > > > On Mon, Nov 13, 2017 at 4:20 PM, Matthias J. Sax < > matth..

Re: [DISCUSS] KIP-220: Add AdminClient into Kafka Streams' ClientSupplier

t; > Thanks for looking into this into details! > > > > As mentioned, I would like to keep the check, but if it's too much > > overhead, I agree that it's not worth it. > > > > Thanks. > > > > -Matthias > > > > On 11/14/17 10:00 AM

Re: [VOTE] KIP-205: Add all() and range() API to ReadOnlyWindowStore

t; > > +1 for the KIP and +1 on renaming "range" to "fetchAll" > > > > Thanks, > > Bill > > > > On Thu, Oct 26, 2017 at 4:39 PM, Guozhang Wang > wrote: > > > > > BTW, Xavier has an argument to rename "range" as i

Re: [VOTE] 0.11.0.2 RC0

+1 - Ran through quick start, streams tutorials. - Ran testAll (scala 2.11, java 8) On Wed, Nov 15, 2017 at 12:27 AM, Satish Duggana wrote: > +1 (non-binding) > > - Ran testAll on source > - Verified signatures on binaries built with Scala-2.11 > - Ran through quickstart of core/streams > > > >

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Thanks for the explanation Jan. On top of my head I'm leaning towards the "more intrusive" approach to resolve the race condition issue we discussed above. Matthias has some arguments for this approach already, so I would not re-iterate them here. To me I find the "ValueMapper joinPrefixFaker" is a

Re: [VOTE] KIP-159: Introducing Rich functions to Streams

t; proposed KIP (-1 non binding of course) state that I am open for any > >>> questions > >>> regarding this. I will also do the usual thing and point out that the > >>> friends > >>> over at Hive got it correct aswell. > >>> One can not user thei

Re: [DISCUSS] KIP-222 - Add "describe consumer group" to KafkaAdminClient

> > Cheers, > > > > Tom > > > > On 6 November 2017 at 22:16, Jorge Esteban Quilcate Otoya < > > quilcate.jo...@gmail.com> wrote: > > > > > Thanks for the feedback! > > > > > > @Ted Yu: Links added. > > > > > > KIP updated. Chang

Re: [VOTE] KIP-222 - Add "describe consumer group" to KafkaAdminClient

Hello Jorge, I left some comments on the discuss thread. The wiki page itself looks good overall. Guozhang On Tue, Nov 14, 2017 at 10:02 AM, Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Added. > > El mar., 14 nov. 2017 a las 19:00, Ted Yu () > escribió: > > > Please fill i

Re: [ANNOUNCE] Apache Kafka 0.11.0.2 Released

companies worldwide, > including Capital > One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, > The New York Times, Uber, Yelp, and Zalando, among others. > > > > A big thank you for the following 20 contributors to this release! > > > Alex Good

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Hello Jan, I think I get your point about the cumbersome that CombinedKey would introduce for serialization and tooling based on serdes. What I'm still wondering is the underlying of joinPrefixFakers mapper: from your latest comment it seems this mapper will be a one-time mapper: we use this to ma

Re: SessionKeySchema#segmentsToSearch()

For `SessionKeySchema#segmentsToSearch`: for session store, multiple sessions may merge together when receiving late arrived records. When I looked at the code, it seems that we have merged the sessions during aggregations to effectively move the sessions between segments. So I'm not 100% certain w

<    1   2   3   4   5   6   7   8   9   10   >