Watermark won't advance in ProcessFunction

2019-10-27 Thread 杨力
I'm going to sort elements in a PriorityQueue and set up timers at (currentWatermark + 1), following the instructions in https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/process_function.html#timer-coalescing . However, it seems that

Cannot modify parallelism (rescale job) more than once

2019-10-27 Thread Pankaj Chand
Hello, I am trying to modify the parallelism of a streaming Flink job (wiki-edits example) multiple times on a standalone cluster (one local machine) having two TaskManagers with 3 slots each (i.e. 6 slots total). However, the "modify" command is only working once (e.g. when I change the

[ANNOUNCE] Weekly Community Update 2019/43

2019-10-27 Thread Konstantin Knauf
Dear community, happy to share this week's community update with updates on some exciting ongoing efforts like unaligned checkpoints, the contribution of a Pulsar connector and the introduction of Executors, a new initiative around queryable state, a couple of bugs and a bit more. Flink

Custom Partitioning with keyed state

2019-10-27 Thread Heidi Hazem Mohamed
Hi, What I want : I have my own partitioning technique that generates keys for DataStream tuples ,those keys range are equal to the number of nodes in the clusters like if I set the parallelism equal to 4 the generated keys will be 0,1,2 and 3 and so on and then every key should be partitioned