Re: What causes a task to change parallelism?

2022-05-09 Thread Caizhi Weng
Hi! I can't see the image (if there is any) in the email. But from the description it is related to the arrow labeled GLOBAL. A global shuffle collects all records from its upstream and aggregate them in its downstream. There are several SQL patterns which lead to this type of shuffle, for exampl

What causes a task to change parallelism?

2022-05-09 Thread Jason Politis
Good evening all, We are running a job in flink SQL. We've confirmed all Kafka topics that we are sourcing from have 5 partitions. All source tasks in the larger DAG, of which we're only showing a small portion of it below, have a parallelism of 5. But for some reason, this one little guy here