Re: Kafka Streams : Rack Aware deployment in EC2

Thanks Matthias.. I don't think I will be able to take it up.. Will wait for it to be available in future. :) On Mon, Oct 19, 2020 at 11:56 PM Matthias J. Sax wrote: > No suggestions... Sorry. > > The task assignment algorithm cannot be customized. > > The only thing you _could_ do, is to pickup

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations, Sophie! You always provide good review comments for my PRs. Well deserved! Luke On Tue, Oct 20, 2020 at 9:23 AM Rankesh Kumar wrote: > Many congratulations, Sophie. > > Best regards, > Rankesh > > > > On 20-Oct-2020, at 12:34 AM, Gwen Shapira wrote: > > > > Congratulations, So

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Many congratulations, Sophie. Best regards, Rankesh > On 20-Oct-2020, at 12:34 AM, Gwen Shapira wrote: > > Congratulations, Sophie! > > On Mon, Oct 19, 2020 at 9:41 AM Matthias J. Sax wrote: >> >> Hi all, >> >> I am excited to announce that A. Sophie Blee-Goldman has accepted her >> invita

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Many congratulations, Chia-Ping! Best regards, Rankesh > On 20-Oct-2020, at 6:45 AM, Luke Chen wrote: > > Congratulations! Chia-Ping大大! > Well deserved! > > Luke > > On Tue, Oct 20, 2020 at 2:30 AM Mickael Maison > wrote: > >> Congrats Chia-Ping! >> >> On Mon, Oct 19, 2020 at 8:29 PM Ismae

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations! Chia-Ping大大! Well deserved! Luke On Tue, Oct 20, 2020 at 2:30 AM Mickael Maison wrote: > Congrats Chia-Ping! > > On Mon, Oct 19, 2020 at 8:29 PM Ismael Juma wrote: > > > > Congratulations Chia-Ping! > > > > Ismael > > > > On Mon, Oct 19, 2020 at 10:25 AM Guozhang Wang > wrote

kafka-streams: do not output anything while state is not stable

Hello there, Let's say I need to restart my streams app from a blank state (whether by changing app.id or using application-reset-tool). My app is designed on "at least once" paradigm, and outputs are upserts. The input topics have a few days worth of data, and the app will restart from there. If

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations, Sophie! On Mon, Oct 19, 2020 at 9:41 AM Matthias J. Sax wrote: > > Hi all, > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > invitation to become an Apache Kafka committer. > > Sophie is actively contributing to Kafka since Feb 2019 and has > accumulated

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congrats Chia-Ping! On Mon, Oct 19, 2020 at 8:29 PM Ismael Juma wrote: > > Congratulations Chia-Ping! > > Ismael > > On Mon, Oct 19, 2020 at 10:25 AM Guozhang Wang wrote: > > > Hello all, > > > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > > become an Apache Kafka

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations Sophie! Ismael On Mon, Oct 19, 2020 at 9:41 AM Matthias J. Sax wrote: > Hi all, > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > invitation to become an Apache Kafka committer. > > Sophie is actively contributing to Kafka since Feb 2019 and has > accum

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations Chia-Ping! Ismael On Mon, Oct 19, 2020 at 10:25 AM Guozhang Wang wrote: > Hello all, > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > become an Apache Kafka committer. > > Chia-Ping has been contributing to Kafka since March 2018 and has made 74 > c

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations Sophie! On Mon, Oct 19, 2020 at 8:00 PM James Cheng wrote: > > Congratulations, Sophie! > > -James > > > On Oct 19, 2020, at 9:40 AM, Matthias J. Sax wrote: > > > > Hi all, > > > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > > invitation to become an A

Re: Kafka Streams : Rack Aware deployment in EC2

No suggestions... Sorry. The task assignment algorithm cannot be customized. The only thing you _could_ do, is to pickup the ticket yourself (to get the feature maybe in 2.8 release). Not sure if you would be interested to contribute :) -Matthias On 10/19/20 11:08 AM, Tony John wrote: > Thanks

Re: Kafka Streams : Rack Aware deployment in EC2

Thanks for the quick response Matthias. We were planning to move to AWS MSK, and I think we will have to defer it until rack awareness is in place for Kafka Streams. Reason being consumers consuming from brokers in different availability zones would result in high data transfer charges. Do you have

Re: [ANNOUNCE] New committer: David Jacot

Congratulations, David! -James > On Oct 16, 2020, at 9:01 AM, Gwen Shapira wrote: > > The PMC for Apache Kafka has invited David Jacot as a committer, and > we are excited to say that he accepted! > > David Jacot has been contributing to Apache Kafka since July 2015 (!) > and has been very act

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations Chia-Ping! -James > On Oct 19, 2020, at 10:24 AM, Guozhang Wang wrote: > > Hello all, > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > become an Apache Kafka committer. > > Chia-Ping has been contributing to Kafka since March 2018 and has made 74

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations, Sophie! -James > On Oct 19, 2020, at 9:40 AM, Matthias J. Sax wrote: > > Hi all, > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > invitation to become an Apache Kafka committer. > > Sophie is actively contributing to Kafka since Feb 2019 and has > a

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congrats Sophie! Cheers, Kowshik On Mon, Oct 19, 2020 at 10:31 AM Bill Bejeck wrote: > Congratulations Sophie! > > -Bill > > On Mon, Oct 19, 2020 at 12:49 PM Leah Thomas wrote: > > > Congrats Sophie! > > > > On Mon, Oct 19, 2020 at 11:41 AM Matthias J. Sax > wrote: > > > > > Hi all, > > > >

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations On Mon, 19 Oct, 2020, 11:02 pm Bill Bejeck, wrote: > Congratulations Chia-Ping! > > -Bill > > On Mon, Oct 19, 2020 at 1:26 PM Matthias J. Sax wrote: > > > Congrats Chia-Ping! > > > > On 10/19/20 10:24 AM, Guozhang Wang wrote: > > > Hello all, > > > > > > I'm happy to announce th

Re: Client session timed out

can you try to disable automatic start and on the node just booted, start zookeeper first, check the log that it is connected and then start Kafka? I assume everything is set-up OK including in $KAFKA_HOME/config/server.properties values for broker.id, hostname, zookeeper.connect=:2181,,server2>:2

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations Sophie!! So well deserved. On Mon, Oct 19, 2020 at 6:31 PM Bill Bejeck wrote: > Congratulations Sophie! > > -Bill > > On Mon, Oct 19, 2020 at 12:49 PM Leah Thomas wrote: > > > Congrats Sophie! > > > > On Mon, Oct 19, 2020 at 11:41 AM Matthias J. Sax > wrote: > > > > > Hi all, >

Re: Kafka Streams : Rack Aware deployment in EC2

There is no such feature for Kafka Streams yet. We have one ticket for rack aware standby task placement (https://issues.apache.org/jira/browse/KAFKA-6718) but nobody is working on it. -Matthias On 10/19/20 9:20 AM, Tony John wrote: > Hi All, > > I have been trying to figure out some documenta

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations Chia-Ping! -Bill On Mon, Oct 19, 2020 at 1:26 PM Matthias J. Sax wrote: > Congrats Chia-Ping! > > On 10/19/20 10:24 AM, Guozhang Wang wrote: > > Hello all, > > > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > > become an Apache Kafka committer. > >

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations Sophie! -Bill On Mon, Oct 19, 2020 at 12:49 PM Leah Thomas wrote: > Congrats Sophie! > > On Mon, Oct 19, 2020 at 11:41 AM Matthias J. Sax wrote: > > > Hi all, > > > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > > invitation to become an Apache Kafka

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congrats Chia-Ping! On 10/19/20 10:24 AM, Guozhang Wang wrote: > Hello all, > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > become an Apache Kafka committer. > > Chia-Ping has been contributing to Kafka since March 2018 and has made 74 > commits: > > https://githu

[ANNOUNCE] New committer: Chia-Ping Tsai

Hello all, I'm happy to announce that Chia-Ping Tsai has accepted his invitation to become an Apache Kafka committer. Chia-Ping has been contributing to Kafka since March 2018 and has made 74 commits: https://github.com/apache/kafka/commits?author=chia7712 He's also authored several major impro

Re: Stream stopped running due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId

Hey Pushkar, If I remember correctly there were a number of broker-side bugs that were uncovered over the course of this ticket, any of which could result in the exception you've seen. I'm *pretty* sure that they should all be fixed in 2.5.0, but it's possible that some of the fixes only made it i

Re: Client session timed out

Yes, you have it Holen Sie sich Outlook für iOS Von: Mich Talebzadeh Gesendet: Monday, October 19, 2020 7:09:53 PM An: users@kafka.apache.org Betreff: Re: Client session timed out Ok I think it is clearer now. As I understand all your Zoo

Re: Client session timed out

Ok I think it is clearer now. As I understand all your Zookeepers and Kafkas are running. (5 nodes meaning 5 physical hosts?). Then you have to reboot one physical host. You still have the redundancy. But when the physical host comes up and your zookeeper and then Kafka come up, you have Kafka tim

Re: Client session timed out

No, sorry I‘m not so good in explaining. The scenario is: the complete Cluster is running, all zookeepers and all kafkas. And then I restart one server, the others are still running. Holen Sie sich Outlook für iOS Von: Mich Talebzadeh Gesen

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congrats Sophie! On Mon, Oct 19, 2020 at 11:41 AM Matthias J. Sax wrote: > Hi all, > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > invitation to become an Apache Kafka committer. > > Sophie is actively contributing to Kafka since Feb 2019 and has > accumulated 140 com

Re: Client session timed out

Can you pls clarify when you say you start one pair (Zookeeper and Kafka), what happens to the others. Do you keep them down? LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

[ANNOUNCE] New committer: A. Sophie Blee-Goldman

Hi all, I am excited to announce that A. Sophie Blee-Goldman has accepted her invitation to become an Apache Kafka committer. Sophie is actively contributing to Kafka since Feb 2019 and has accumulated 140 commits. She authored 4 KIPs in the lead - KIP-453: Add close() method to RocksDBConfigSe

Re: Client session timed out

Yes, it's a new setup. The kafka and zookeeper runs as services and are enabled, so they should start at system start. And kafka starts after zookeeper. And if I stop everything and start it, then it works. But then, if I restart one server with zookeeper and kafka the kafka gets the timeouts. 

Kafka Streams : Rack Aware deployment in EC2

Hi All, I have been trying to figure out some documentation on how to enable rack awareness on a Kafka Streams app. I do see the broker.rack configuration which needs to be done at the broker side and client.rack configuration for the consumers. Is there any specific configuration which is require

Re: Client session timed out

OK so the issue seems to be kafka cluster. Is this a new setup? HTH LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * *Disclaimer:* Use it at your own risk. Any an

Re: Client session timed out

Thanks for your answer, but the zookeeper ensemble is running and ports are ok. Am 19.10.20, 17:38 schrieb "Mich Talebzadeh" : Start the zookeeper ensemble first before starting Kafka cluster. They need to select a leader and ensure that they all come online OK. Check port 2181, 288

Re: Client session timed out

Start the zookeeper ensemble first before starting Kafka cluster. They need to select a leader and ensure that they all come online OK. Check port 2181, 2888, 3888 using netstat -plten|egrep '2181|2888|3888' tcp0 0 :::2181 :::* LISTEN 1005 9934134

Client session timed out

Hi, I have a 5 nodes kafka cluster with 3 zookeepers. If I restart 1 node (zookeeper and kafka) the kafka gets a Client session timed out, have not heard from server in 6007ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn) Client session timed out, have not heard from server in 6007ms for s