Re: SideOutput doesn't receive anything if filter is applied after the process function

2018-01-13 Thread Chen Qin
Hi Juho, I think sideoutput might deserve a seperate class which inherit form singleoutput. It might prevent lot of confusions. A more generic question is whether datastream api can be mulitple ins and mulitple outs natively. It's more like scheduling problem when you come from single process

Questions about managed operator state

2018-01-13 Thread Boris Lublinsky
Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state Refers to CheckpointedRestoring interface.

Re: Two issues when deploying Flink on DC/OS

2018-01-13 Thread Eron Wright
Hello Dongwon, Flink doesn't support a 'unique host' constraint at this time; it simply accepts adequate offers without any such consideration. Flink does support a 'host attributes' constraint to filter certain hosts, but that's not applicable here. Under the hood, Flink uses a library called

Re: Keyed State

2018-01-13 Thread Boris Lublinsky
Thanks Fabian Can you also explain a thread model? What is the paralelization between multiple keys? Is it hash based? And also are processElement 1 and 2 are executed on different threads? More specifically if processElement is an order of magnitude slower then 2, will it impact processElement

Re: Keyed State

2018-01-13 Thread Boris Lublinsky
Thanks Fabian Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ > On Jan 13, 2018, at 11:06 AM, Fabian Hueske wrote: > > Yes, that is correct. > You can treat keyed ValueState like a distributed hashmap and Flink routes > all state

Re: Keyed State

2018-01-13 Thread Fabian Hueske
Yes, that is correct. You can treat keyed ValueState like a distributed hashmap and Flink routes all state accesses to the entry for the key of the current record. 2018-01-13 17:07 GMT+01:00 Boris Lublinsky : > Can you, please confirm that my understanding is

Keyed State

2018-01-13 Thread Boris Lublinsky
Can you, please confirm that my understanding is correct? I am looking at the documentation on low level joins https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/operators/process_function.html#low-level-joins

logging question

2018-01-13 Thread jp
Hello, I am learning Flink and using the docker image along with the AMIDST library for this. Below is a sample task from AMIDST which provides INFO output up until I reach updateModel(). I pasted the short method as well and wonder what prevents the Logger from //Set-up Flink