Hello Xia,
Thanks for the FLIP!
Since we are introducing the mode as a configuration option,
could it make sense to have `InferMode.NONE` option also?
The `NONE` option would disable the inference.
This way we deprecate the `table.exec.hive.infer-source-parallelism`
and no additional `table.exec.hive.infer-source-parallelism.enabled`
option is required.
What do you think?
Best,
Muhammet
On 2024-04-16 07:07, Xia Sun wrote:
Hi everyone,
I would like to start a discussion on FLIP-445: Support dynamic
parallelism
inference for HiveSource[1].
FLIP-379[2] has introduced dynamic source parallelism inference for
batch
jobs, which can utilize runtime information to more accurately decide
the
source parallelism. As a follow-up task, we plan to implement the
dynamic
parallelism inference interface for HiveSource, and also switch the
default
static parallelism inference to dynamic parallelism inference.
Looking forward to your feedback and suggestions, thanks.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-445%3A+Support+dynamic+parallelism+inference+for+HiveSource
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs
Best regards,
Xia
