回复：Need help to understand memory consumption

Hi Julien, Flink would manage the default 70% fraction of free memory in TaskManager for caching data efficiently, just as you mentioned in this article "https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html;. These managed memories are persistent resident and referenced

Re: Need help to understand memory consumption

Hi Julien, AFAIK, streaming jobs put data objects on heap, so the it depends on the JVM GC to release the memory. Best, Paul Lam > 在 2018年10月12日，14:29，jpreis...@free.fr 写道： > > Hi, > > My use case is : > - I use Flink 1.4.1 in standalone cluster with 5 VM (1 VM = 1 JobManager + 1 >

Re: Window State is not being store on check-pointing

Hi Hequn, I tried with following : Configuration conf = new Configuration(); conf.setString("state.checkpoints.dir","file:///home/sohanvir/Desktop/flink/checkpoints2"); final StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment(1,conf);

Re: Window State is not being store on check-pointing

Hi sohimankotia, Have you ever enableExternalizedCheckpoints[1]? > // enable externalized checkpoints which are retained after job > cancellation > > env.getCheckpointConfig().enableExternalizedCheckpoints(ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION); Best, Hequn [1]

Re: Custom Trigger + SQL Pattern

Hi Shahar, The table function takes a single row but can output multi rows. You can split the row based on the "last" event. The code looks like: val sessionResult = > "SELECT " + > " lastUDAF(line) AS lastEvents " > "FROM MyTable " + > "GROUP BY

Re: When does Trigger.clear() get called?

Hi Andrew, You should call it manually, as the global window does not have a natural end. Best, Hequn On Wed, Oct 17, 2018 at 2:47 AM Andrew Danks wrote: > Hi Fabian & Hequn, > > Thank you for your responses. I am just responding now as I was out of > office for the last few days > > You

Re: When does Trigger.clear() get called?

Hi Fabian & Hequn, Thank you for your responses. I am just responding now as I was out of office for the last few days You mentioned that clear() is called when the time exceeds the window’s end timestamp. For my application I am using a GlobalWindow on a keyed stream -- would clear() get

Re: Custom Trigger + SQL Pattern

Im wondering how does that work, it seems that a table function still takes a single row's values as an input, am i wrong (or at least that is how the examples show)? How would the SQL look like? On Fri, Oct 12, 2018 at 9:15 PM Hequn Cheng wrote: > Hi shkob1, > > > while one is time(session

Re: Fire and Purge with Idle State

Thanks! On Fri, Oct 12, 2018 at 9:29 PM Hequn Cheng wrote: > Hi shkob1, > > Currently, the idle state retention time is only used for unbounded > operators in sql/table-api. The unbounded operators include non-window > group by, non-window join, unbounded over, etc. The retention time affects >

Window State is not being store on check-pointing

Hi, I am using following in code : 1. flink 1.4 2. running example on IDE 3. Enabled Exactly once semantics 4. Window Aggregation 5. Checkpoint is enabled at 20 Sec 6/ RocksDB as state backend Workflow : Kafka Source -> map -> keyBy -> Window(60 Sec) -> ApplyFunction -> Aggregated Record to

Report failed job submission

Hi to all, which is the correct wat to report back to the user a failure from a job submission in FLink? If everything is OK the job run API returns the job id but what if there are error in parameter validation or some other problem? Which is the right way to report back to the user the job error

Re: Flink Table API and table name

Done: https://issues.apache.org/jira/browse/FLINK-10562 On Tue, Oct 16, 2018 at 11:12 AM Timo Walther wrote: > Hi Flavio, > > yes you are right, I don't see a reason why we should not support such > table names. Feel free to open an issue for it. > > Regards, > Timo > > > Am 16.10.18 um 10:56

Re: Flink Table API and table name

Hi Flavio, yes you are right, I don't see a reason why we should not support such table names. Feel free to open an issue for it. Regards, Timo Am 16.10.18 um 10:56 schrieb miki haiat: Im not sure if it will solve this issue but can you try to register the your catalog [1]

Re: Flink Table API and table name

Im not sure if it will solve this issue but can you try to register the your catalog [1] 1. https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/table/common.html#register-an-external-catalog On Tue, Oct 16, 2018 at 11:40 AM Flavio Pompermaier wrote: > Hi to all, > in my job I'm

Flink Table API and table name

Hi to all, in my job I'm trying to read a dataset whose name/id starts with a number. It seems that when using the Table API to read that dataset, if the name starts with a number it is a problem..am I wrong? I can't find anything about table id constraints on the documentation and it seems that

Re: Can't start taskmanager in Minikube

Did you execute this command ? Note: If using MiniKube please make sure to execute minikube ssh 'sudo ip > link set docker0 promisc on' before deploying a Flink cluster. Otherwise > Flink components are not able to self reference themselves through a > Kubernetes service. On Tue, Oct 16, 2018

Can't start taskmanager in Minikube

I followed the Doc( https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/deployment/kubernetes.html#session-cluster-resource-definitions) to run flink on kubernetes, but there is an exception(java.net.UnknownHostException: flink-jobmanager: Temporary failure in name resolution). I