On 12.09.20 17:20, Alexey Trenikhun wrote:
We use union state to generate sequences, each operator generates offset0 +
number-of-tasks - task-index + task-specific-counter * number-of-tasks (e.g.
for 2 instances of operator -one instance produce even number, another odd).
Last generated sequence number is stored union list state, on restart from
where we should start to avoid collision with already generated numbers, to do
saw we calculate offset0 as max over union list state.
Is this for testing or a production use case?
With the introduction of the new source interface in FLIP-27 [1] it will
not be possible to access different state types in a sink. However,
using the enumerator and readers should work for your use case, since
the enumerator acts as a central coordinating component that would
assign ranges to the different readers. Have you looked at this new
development perchance?
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface
