Re: Question regarding state cleaning using timer

Thanks Hequn. But i want to give random TTL for each partitioned key. How can i achieve it? Regards Bhaskar On Mon, Sep 17, 2018 at 7:30 AM Hequn Cheng wrote: > Hi bhaskar, > > You need change nothing if you want to handle multi keys. Flink will do it > for you. The ValueState is a keyed state.

Re: Task managers run on separate nodes in a cluster

Thanks for the feedback Liu and Till. @Liu Yeah that would work but unfortunately we run other services on the cluster so it's not really an option. @Till Will have a look and see how much time I can dedicate to this. M On Mon, Sep 17, 2018 at 7:21 AM Till Rohrmann wrote: > Hi Martin, > > I'm no

Re: Task managers run on separate nodes in a cluster

Hi Martin, I'm not aware that the community is actively working on enabling the balanced host attribute constraint. If you wanna give it a try, then I'm happy to review your contribution. Cheers, Till On Mon, Sep 17, 2018 at 5:28 AM Renjie Liu wrote: > Hi, Martin: > I think a better solution w

Re: Task managers run on separate nodes in a cluster

Hi, Martin: I think a better solution would be to set the number of cores of each container equals to that of a physical server if this mesos cluster is dedicated to your flink cluster. On Mon, Sep 17, 2018 at 5:28 AM Martin Eden wrote: > Hi Till, > > I was able to use mesos.constraints.hard.hos

Re: Trying to figure out why a slot takes a long time to checkpoint

Hi, Julio: This happens frequently? What state backend do you use? The async checkpoint duration and sync checkpoint duration seems normal compared to others, it seems that most of the time are spent acking the checkpoint. On Sun, Sep 16, 2018 at 9:24 AM vino yang wrote: > Hi Julio, > > Yes, it

Utilising EMR's master node

Hello everyone, I'm trying to run Flink on AWS EMR following the guides from Flink doc and from AWS

Re: LocalEnvironment and Python streaming

Hi Joe, Maybe Chesnay is better suited to answer this question, Ping him for you. Thanks, vino. Joe Malt 于2018年9月15日周六 上午1:51写道： > Hi, > > Is there any way to execute a job using the LocalEnvironment when using > the Python streaming API? This would make it much easier to debug jobs. > > At th

Re: twitter source is shutting down after successfull connect

Hi Björn, >From the exception stack information, the NPE comes from the auth module of the twitter client. Did you fill in the relevant authentication information correctly? Specific instructions can refer to the official guidance document.[1] Thanks, vino. [1]: https://ci.apache.org/projects/fl

Re: Question regarding state cleaning using timer

Hi bhaskar, You need change nothing if you want to handle multi keys. Flink will do it for you. The ValueState is a keyed state. You can think of Keyed State as Operator State that has be

Re: Task managers run on separate nodes in a cluster

Hi Till, I was able to use mesos.constraints.hard.hostattribute to run all task managers on a particular host in my cluster. However, after looking a bit at the code, I'm not sure we can use mesos.constraints.hard.hostattribute for load balancing Flink task managers evenly across hosts in a Mesos

twitter source is shutting down after successfull connect

Hey, I updated to flink-connector-twitter_2.11 version 1.6.0 and now I am getting this error on startup. Used the 1.2.0 version before where everything worked. Any idea what is causing this? 17:39:07,866 INFO com.twitter.hbc.httpclient.BasicClient - New connection executed: flink-twitter-so

Fwd: why same Sliding ProcessTime TimeWindow triggered twice

-- Forwarded message - From: 远远 Date: 2018年9月16日周日 下午4:08 Subject: Re: why same Sliding ProcessTime TimeWindow triggered twice To: hi, the flink version that i test is 1.4.2, and i just run test code with local env in IDEA, and all the setting in the test code. my os is deepin(

Re: why same Sliding ProcessTime TimeWindow triggered twice

Hi, I’ve tested your code in my local environment and everything worked fine. It’s a little weird to see your output like that. I wonder if you could give more information about your environment, e.g., your flink version and execution settings. Thanks, Xingcan > On Sep 16, 2018, at 3:19 PM, 远

why same Sliding ProcessTime TimeWindow triggered twice

hi，everyone: today, i test Sliding ProcessTime TimeWindow with print some merties. i find a same sliding window be printed twice, as fllower: now ===> 2018-09-16 15:11:44 start ===> 2018-09-16 15:10:45 end ===> 2018-09-16 15:11:45 max ===> 2018-09-16 15:11:44 TimeWindow{start=1537081845000,