Hi,
How can i setParallelism in a generic way that provide better performance
on any device not only mine? In this case is it better to be set to a
certain value or if i just didn't set it to any value does flink take care
of that generically and provide better execution performance?
Thanks.
Hi everyone,
I read that Flink is supposed to automatically optimize the degree of
parallelism. But I never saw any change of parallelism in the web interface
without defining dop manually (-p parameter).
Is there any of this optimization actually happening? Or how can I switch it
on?
Cheers
Hi Ahmed,
if you don't set the parallelism in your program then depending on how you
execute your program different parallelisms will be used. If you execute it
in your IDE, then the number of cores will be used as parallelism. If you
submit it to a cluster without specifying the parallelism via t
Thanks so much Till and Fabian. So if i were to set it, how can i know the
best degree to be used for my application, is it the number of cores or
what should i set it to?
Thanks
On 18 March 2016 at 13:36, Fabian Hueske wrote:
> Hi,
>
> did find the documentation for configuring the parallelism
Hi,
did find the documentation for configuring the parallelism [1]?
It explains how to set the parallelism on different levels: Cluster, Job,
Task.
Best, Fabian
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/common/index.html#parallel-execution
2016-03-18 13:34 GMT+01:00 T
matically optimize the degree of
> parallelism. But I never saw any change of parallelism in the web interface
> without defining dop manually (-p parameter).
>
> Is there any of this optimization actually happening? Or how can I switch
> it on?
>
>
> Cheers
> Malte
>
master student working on this feature, so it might be possible within the
> next 3-6 months.
>
> Regards,
> Alexander
>
> 2015-03-02 17:01 GMT+01:00 Malte Schwarzer :
>>
>> Hi everyone,
>>
>> I read that Flink is supposed to automatically optimize the degree
Just to clarify.
The pull request 410 does not optimize the degree of parallelism of
operators with respect to performance / gain.
It just sets the maximum possible parallelization for the current execution
environment.
Cheers, Fabian
2015-03-02 17:20 GMT+01:00 Max Michels :
> Hi!
>
>
