Re: [External] Re: [External] Re:Re: Backpressure issue with Flink Sql Job

2024-07-02 Thread Ashish Khatkar via user
Hi Penny, As the join is an unwindowed join, we are performing the backfill since the data started becoming available. On Mon, Jul 1, 2024 at 3:14 PM Penny Rastogi wrote: > Hi Ashish, > > How are you performing the backfill operation? Some time window? Can you > specify details? > > I mean ,if

Re: [External] Re:Re: Backpressure issue with Flink Sql Job

2024-07-01 Thread Penny Rastogi
Hi Ashish, How are you performing the backfill operation? Some time window? Can you specify details? I mean ,if it helps, you can check out https://www.ververica.com/blog/how-to-write-fast-flink-sql . Regards On Tue, Jun 25, 2024 at 4:30 PM Ashish Khatkar via user < user@flink.apache.org>

Re: [External] Re:Re: Backpressure issue with Flink Sql Job

2024-06-25 Thread Ashish Khatkar via user
Hi Xuyang, The input records are balanced across subtasks, with debloating buffers enabled, the records this subtask receives is less as compared to other subtasks. If the differences among all subtasks are not significant, we might be > encountering an IO bottleneck. In this case, we could try

Re:Re: Backpressure issue with Flink Sql Job

2024-06-24 Thread Xuyang
Hi, Ashish. Can you confirm whether, on the subtask label page of this sink materializer node, the input records for each subtask are approximately the same? If the input records for subtask number 5 are significantly larger compared to the others, it signifies a serious data skew, and it