Hi Penny,
As the join is an unwindowed join, we are performing the backfill since the
data started becoming available.
On Mon, Jul 1, 2024 at 3:14 PM Penny Rastogi wrote:
> Hi Ashish,
>
> How are you performing the backfill operation? Some time window? Can you
> specify details?
>
> I mean ,if
Hi Ashish,
How are you performing the backfill operation? Some time window? Can you
specify details?
I mean ,if it helps, you can check out
https://www.ververica.com/blog/how-to-write-fast-flink-sql .
Regards
On Tue, Jun 25, 2024 at 4:30 PM Ashish Khatkar via user <
user@flink.apache.org>
Hi Xuyang,
The input records are balanced across subtasks, with debloating buffers
enabled, the records this subtask receives is less as compared to other
subtasks.
If the differences among all subtasks are not significant, we might be
> encountering an IO bottleneck. In this case, we could try
Hi, Ashish.
Can you confirm whether, on the subtask label page of this sink materializer
node, the input records for each subtask are approximately the same?
If the input records for subtask number 5 are significantly larger compared to
the others, it signifies a serious data skew, and it
Hi Ashish,
Can you check a few things.
1. Is your source broker count also 20 for both topics?
2. You can try increasing the state operation memory and reduce the disk
I/O.
-
- Increase the number of CU resources in a single slot.
- Set optimization parameters:
-
Hi all,
We are facing backpressure in the flink sql job from the sink and the
backpressure only comes from a single task. This causes the checkpoint to
fail despite enabling unaligned checkpoints and using debloating buffers.
We enabled flamegraph and the task spends most of the time doing
