Re: How do I start a Flink application on my Flink+Mesos cluster?

Hi Felipe, I am glad that you were able to fix the problem yourself. > But I suppose that Mesos will allocate Slots and Task Managers dynamically. > Is that right? Yes, that is the case since Flink 1.5 [1]. > Then I put the number of "mesos.resourcemanager.tasks.cpus" to be equal or > less the

Re: error in Elasticsearch

Hi, You are not supposed to change that part of the exercise code. You have to pass the path to the input file as a program argument (e.g., --input /path/to/file). See [1] and [2] on how to configure program arguments in IntelliJ. Best, Gary [1] https://www.jetbrains.com/help/idea/run-debug-conf

Re: Checkpointing is not performing well

What is the upper limit of checkpoint size of Flink System? Regards, Ravi On Wed 11 Sep, 2019, 06:48 Vijay Bhaskar, wrote: > You crossed the upper limits of the check point system of Flink a way > high. Try to distribute events equally over time by adding some sort of > controlled back pressur

Re: Filter events based on future events

Hi Theo, I would implement this with a KeyedProcessFunction. These are the important points to consider: 1) partition the output of the Kafka source by Kafka partition (or the attribute that determines the partition). This will ensure that the data stay in order (per partition). 2) The KeyedProce

Re:

Hi, This is clearly a Scala version issue. You need to make sure that all Flink dependencies have the same version and are compiled for Scala 2.11. The "_2.11" postfix in the dependency name indicates that it is a Scala 2.11 dependency ("_2.12 indicates Scala 2.12 compatibility). Best, Fabian Am

Re: Checkpointing is not performing well

Hi, There is no upper limit for state size in Flink. There are applications with 10+ TB state. However, it is natural that checkpointing time increases with state size as more data needs to be serialized (in case of FSStateBackend) and written to stable storage. (The same is btw true for recovery

[DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

Hi all! We still maintain connectors for Kafka 0.8 and 0.9 in Flink. I would suggest to drop those with Flink 1.10 and start supporting only Kafka 0.10 onwards. Are there any concerns about this, or still a significant number of users of these versions? Best, Stephan

Re: [DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

on 2019/9/11 16:17, Stephan Ewen wrote: We still maintain connectors for Kafka 0.8 and 0.9 in Flink. I would suggest to drop those with Flink 1.10 and start supporting only Kafka 0.10 onwards. Are there any concerns about this, or still a significant number of users of these versions? Bu

Re: error in Elasticsearch

Hallo I put arguments but the same error appear .. what should i do ? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: suggestion of FLINK-10868

Hi Till, Some of our online batch tasks have strict SLA requirements, and they are not allowed to be stuck for a long time. Therefore, we take a rude way to make the job exit immediately. The way to wait for connection recovery is a better solution. Maybe we need to add a timeout to wait for JM to

Re: suggestion of FLINK-10868

Suggestion 1 makes sense. For the quick termination I think we need to think a bit more about it to find a good solution also to support strict SLA requirements. Cheers, Till On Wed, Sep 11, 2019 at 11:11 AM Anyang Hu wrote: > Hi Till, > > Some of our online batch tasks have strict SLA requirem

[ANNOUNCE] Zili Chen becomes a Flink committer

Hi everyone, I'm very happy to announce that Zili Chen (some of you might also know him as Tison Kun) accepted the offer of the Flink PMC to become a committer of the Flink project. Zili Chen has been an active community member for almost 16 months now. He helped pushing the Flip-6 effort over th

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Hi on 2019/9/11 17:22, Till Rohrmann wrote: I'm very happy to announce that Zili Chen (some of you might also know him as Tison Kun) accepted the offer of the Flink PMC to become a committer of the Flink project. Congratulations Zili Chen. regards.

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili Chen! Wesley Peng 于2019年9月11日周三 下午5:25写道： > Hi > > on 2019/9/11 17:22, Till Rohrmann wrote: > > I'm very happy to announce that Zili Chen (some of you might also know > > him as Tison Kun) accepted the offer of the Flink PMC to become a > > committer of the Flink project. >

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! > 在 2019年9月11日，下午5:26，Jeff Zhang 写道： > > Congratulations Zili Chen! > > Wesley Peng mailto:wes...@thepeng.eu>> 于2019年9月11日周三 > 下午5:25写道： > Hi > > on 2019/9/11 17:22, Till Rohrmann wrote: > > I'm very happy to announce that Zili Chen (some of you might also know > > him as T

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! Best, Terry Wang > 在 2019年9月11日，下午5:28，Dian Fu 写道： > > Congratulations! > >> 在 2019年9月11日，下午5:26，Jeff Zhang mailto:zjf...@gmail.com>> >> 写道： >> >> Congratulations Zili Chen! >> >> Wesley Peng mailto:wes...@thepeng.eu>> 于2019年9月11日周三 >> 下午5:25写道： >> Hi >> >> on 2019/9/1

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili! Thanks, Zhu Zhu Terry Wang 于2019年9月11日周三 下午5:34写道： > Congratulations! > > Best, > Terry Wang > > > > 在 2019年9月11日，下午5:28，Dian Fu 写道： > > Congratulations! > > 在 2019年9月11日，下午5:26，Jeff Zhang 写道： > > Congratulations Zili Chen! > > Wesley Peng 于2019年9月11日周三 下午5:25写道： > >> H

Re: error in Elasticsearch

Program arguments should be set to "--input /home/alaa/nycTaxiRides.gz" (without the quotes). On Wed, Sep 11, 2019 at 10:39 AM alaa wrote: > Hallo > > I put arguments but the same error appear .. what should i do ? > > > < > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/fi

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! Best, Jiayi Liao Original Message Sender: Till Rohrmann Recipient: dev; user Date: Wednesday, Sep 11, 2019 17:22 Subject: [ANNOUNCE] Zili Chen becomes a Flink committer Hi everyone, I'm very happy to announce that Zili Chen (some of you might also know him as Tison Kun)

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! --- Oytun Tez *M O T A W O R D* The World's Fastest Human Translation Platform. oy...@motaword.com — www.motaword.com On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy wrote: > Congratulations! > > > Best, > > Jiayi Liao > > Original Message > *Sender:* Till Rohrmann > *Recipient:* d

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili Chen! bupt_ljy 于2019年9月11日周三 下午6:36写道： > Congratulations! > > > Best, > > Jiayi Liao > > Original Message > *Sender:* Till Rohrmann > *Recipient:* dev; user > *Date:* Wednesday, Sep 11, 2019 17:22 > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer > > Hi everyone,

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congrats Zili! Thanks, Biao /'bɪ.aʊ/ On Wed, 11 Sep 2019 at 18:43, Oytun Tez wrote: > Congratulations! > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Human Translation Platform. > oy...@motaword.com — www.motaword.com > > > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy wrote: > >

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congrats Zili Chen :-) Cheers, Fabian Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu : > Congrats Zili! > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Wed, 11 Sep 2019 at 18:43, Oytun Tez wrote: > >> Congratulations! >> >> --- >> Oytun Tez >> >> *M O T A W O R D* >> The World's Fastest Human Tran

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili ! Best, Guowei Fabian Hueske 于2019年9月11日周三 下午7:02写道： > Congrats Zili Chen :-) > > Cheers, Fabian > > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu : > >> Congrats Zili! >> >> Thanks, >> Biao /'bɪ.aʊ/ >> >> >> >> On Wed, 11 Sep 2019 at 18:43, Oytun Tez wrote: >> >>>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! Regards, Xiaogang Guowei Ma 于2019年9月11日周三 下午7:07写道： > Congratulations Zili ! > > Best, > Guowei > > > Fabian Hueske 于2019年9月11日周三 下午7:02写道： > >> Congrats Zili Chen :-) >> >> Cheers, Fabian >> >> Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu > >: >> >>> Congrats Zili! >>

Re: error in Elasticsearch

Thank for your reply >> I run it >> and I think it run correctly now -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

RE: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations, Zili. Best, Xingcan From: SHI Xiaogang Sent: Wednesday, September 11, 2019 7:43 AM To: Guowei Ma Cc: Fabian Hueske ; Biao Liu ; Oytun Tez ; bupt_ljy ; dev ; user ; Till Rohrmann Subject: Re: [ANNOUNCE] Zili Chen becomes a Flink committer Congratulations! Rega

Job recovery from a checkpoint

Hi, We can get a job recovery from a save point nicely after a restart of our flink cluster using bin/flink run -s :savepointPath [:runArgs] The previous job states are recovered after this reload. I expect I do something similar to recover a flink from a checkpoint location after a restart of o

Re: Job recovery from a checkpoint

Hi Min First of all, Flink could resume from an externalized checkpoint with same command as restoring from savepoint. * Did you make the externalized checkpoint retained after job canceled? * Did you really pass the correct checkpoint path (including chk-xxx folder) to the command line

Kafka Schema registry

Hi. Do Flink have out of the Box Support for Kafka Schema registry for both sources and sinks? If not, does anyone knows about a implementation we can build on so we can help make it general available in a future release. Med venlig hilsen / Best regards Lasse Nedergaard

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili! Best, Jark On Wed, 11 Sep 2019 at 23:06, wrote: > Congratulations, Zili. > > > > Best, > > Xingcan > > > > *From:* SHI Xiaogang > *Sent:* Wednesday, September 11, 2019 7:43 AM > *To:* Guowei Ma > *Cc:* Fabian Hueske ; Biao Liu ; > Oytun Tez ; bupt_ljy ; dev < > d...@flin

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations! Best, Hequn On Thu, Sep 12, 2019 at 9:24 AM Jark Wu wrote: > Congratulations Zili! > > Best, > Jark > > On Wed, 11 Sep 2019 at 23:06, wrote: > > > Congratulations, Zili. > > > > > > > > Best, > > > > Xingcan > > > > > > > > *From:* SHI Xiaogang > > *Sent:* Wednesday, Septembe

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili! -- Rong On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng wrote: > Congratulations! > > Best, Hequn > > On Thu, Sep 12, 2019 at 9:24 AM Jark Wu wrote: > >> Congratulations Zili! >> >> Best, >> Jark >> >> On Wed, 11 Sep 2019 at 23:06, wrote: >> >> > Congratulations, Zili. >> >

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili！ Best, Paul Lam > 在 2019年9月12日，09:34，Rong Rong 写道： > > Congratulations Zili! > > -- > Rong > > On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng > wrote: > Congratulations! > > Best, Hequn > > On Thu, Sep 12, 2019 at 9:24 AM Jark Wu

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congrats, Zili! On Thu, Sep 12, 2019 at 9:39 AM Paul Lam wrote: > Congratulations Zili！ > > Best, > Paul Lam > > 在 2019年9月12日，09:34，Rong Rong 写道： > > Congratulations Zili! > > -- > Rong > > On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng wrote: > >> Congratulations! >> >> Best, Hequn >> >> On Thu,

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congratulations Zili！ -- From:Becket Qin Send Time:2019年9月12日(星期四) 03:43 To:Paul Lam Cc:Rong Rong ; dev ; user Subject:Re: [ANNOUNCE] Zili Chen becomes a Flink committer Congrats, Zili! On Thu, Sep 12, 2019 at 9:39 AM Paul Lam wr

Re: Checkpointing is not performing well

I meant upper limit w.r.t resources you are using. Even if you increase resources, Spiking data is always a problem which anyways you need to take care of. Best thing is to add more back pressure from source. Regards Bhaskar On Wed, Sep 11, 2019 at 1:43 PM Fabian Hueske wrote: > Hi, > > There i