If you have some experience before, I'd recommend setting a good parallelism
and TM resource spec first, to give the autotuner a good starting point.
Usually, the autoscaler can tune your jobs well within a few attempts (<=3). As
for `pekko.ask.timeout`, the default value should be sufficient in most cases.
Best,
Zhanghao Chen
From: Maxim Senin via user
Sent: Thursday, April 18, 2024 5:56
To: user@flink.apache.org
Subject: Parallelism for auto-scaling, memory for auto-tuning - Flink operator
Hi.
Does it make sense to specify `parallelism` for task managers or the `job`,
and, similarly, to specify memory amount for the task managers, or it’s better
to leave it to autoscaler and autotuner to pick the best values? How many times
would the autoscaler need to restart task managers before it picks the right
values? Does `pekko.ask.timeout` need to be sufficient for task managers to get
into running state with all the restarts?
Cheers,
Maxim
COGILITY SOFTWARE CORPORATION LEGAL DISCLAIMER: The information in this email
is confidential and is intended solely for the addressee. Access to this email
by anyone else is unauthorized. If you are not the intended recipient, any
disclosure, copying, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful.
