Re: Re:Flink batch job memory/disk leak when invoking set method on a static Configuration object.

2019-06-28 Thread Vadim Vararu
Hi, I've run it on a standalone Flink cluster. No Yarn involved. From: Haibo Sun Sent: Friday, June 28, 2019 6:13 AM To: Vadim Vararu Cc: user@flink.apache.org Subject: Re:Flink batch job memory/disk leak when invoking set method on a static Configuration object

Flink batch job memory/disk leak when invoking set method on a static Configuration object.

2019-06-27 Thread Vadim Vararu
Hi guys, I have a simple batch job with a custom output formatter that writes to a local file. public class JobHadoop { public static void main(String[] args) throws Exception { ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

Broadcast state before events stream consumption

2019-02-08 Thread Vadim Vararu
Hi all, I need to use the broadcast state mechanism (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) for the next scenario. I have a reference data stream (slow) and an events stream (fast running) and I want to do a kind of lookup in the

Flink streaming. Broadcast reference data map across nodes

2017-02-21 Thread Vadim Vararu
Hi all, I would like to do something similar to Spark's broadcast mechanism. Basically, i have a big dictionary of reference data that has to be accessible from all the nodes (in order to do some joins of log line with reference line). I did not find yet a way to do it. Any ideas?

Re: Is it OK to have very many session windows?

2017-02-20 Thread Vadim Vararu
e per pane has to be kept in memory? Regards, Timo Am 20/02/17 um 16:34 schrieb Vadim Vararu: HI guys, Is it okay to have very many (tens of thousands or hundreds of thousand) of session windows? Thanks, Vadim.

Is it OK to have very many session windows?

2017-02-20 Thread Vadim Vararu
HI guys, Is it okay to have very many (tens of thousands or hundreds of thousand) of session windows? Thanks, Vadim.