Hi,
I have recently tried to upgrade Flink from 1.2.0 to the newest version and
noticed that starting from the version 1.5 the performance is much worse
when processing fixed graphs in a standalone JVM environment (Java 8).
This affects all the use-cases when a Gelly graph (pre-built from a fixed
Hi Jakub,
I had a look at the changes of Flink 1.5 [1] and didn't find anything
obvious.
Something that might cause a different behavior is the new deployment and
process model (FLIP-6).
In Flink 1.5, there is a switch to disable it and use the previous
deployment mechanism.
You could try to disa
Thanks for your replies.
We use Flink from within a standalone Java 8 application (no Hadoop, no
clustering), so it's basically boils down to running a simple code like this:
import java.util.*;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.graph.*;
import org.ap
Hi,
In Flink 1.5 there were three big changes, that could affect performance.
1. FLIP-6 changes (As previously Yang and Fabian mentioned)
2. Credit base flow control (especially if you are using SSL)
3. Low latency network changes
I would suspect them in that order. First and second you can disa
Hi,
I can confirm that the performance drop is directly related to FLIP-6 changes.
Applying this modification to the code posted above restores the previous graph
processing speed under Flink 1.5.6:
---
org.apache.flin
Hi Jakub,
what are the cluster settings and the exact job settings you are running
your job with? I'm asking because one difference between legacy and FLIP-6
mode is that the legacy mode spreads out tasks across all available
TaskManagers whereas the FLIP-6 mode tries to bin package them on as few
Thanks for your reply, Till.
As mentioned above I execute graph processing in a straight-ahead Java
standalone environment (no cluster underneath, no specific configuration except
for parallelism), just as if you simply ran the Java class I pasted upthread
with a Flink distribution JAR (plus Ge