Hi Averell,
If I understood right, the job graph is A (parallelism = 1) --> B
(parallelism > 1), then I think the records sending into the subtask B_i should
be the same as the order sending out from A. Therefore, could you also provide
more details on the topology ? Is there only the two operators? And could you
also provide how the message order is checked in B_i ?
Best,
Yun
------------------------------------------------------------------
From:Averell <lvhu...@gmail.com>
Send Time:2019 Oct. 31 (Thu.) 12:55
To:user <user@flink.apache.org>
Subject:Preserving (best effort) messages order between operators
Hi,
I have a source function with parallelism = 1, sending out records ordered
by event-time. These records are then re-balanced to the next operator which
has parallelism > 1. I observed that within each subtask of the 2nd
operator, the order of the messages is not maintained. Is this behaviour
expected? If it is, is there any way to avoid that? Or at least reduce that?
I have high back-pressure on that 2nd operator as the one after that is
slow. There is also high back-pressure on the 1st operator, which makes my
problem more severe (the mentioned out-of-order is high). If I could
throttle the 1st operator when back-pressure is high, then I could mitigate
the mentioned problem. But I could not find any guide on doing that.
Could you please help?
Thanks.
Averell
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
