[
https://issues.apache.org/jira/browse/FLINK-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725870#comment-14725870
]
Stephan Ewen commented on FLINK-2603:
-------------------------------------
You stumbled upon a limitation from our use of Akka.
The source data is distributed with Akka messages, which are dropped if they
are too large. By default, that limit is 10 MB, you can increase it in the
configuration {{akka.framesize}}
(https://ci.apache.org/projects/flink/flink-docs-release-0.9/setup/config.html#distributed-coordination-via-akka)
We plan to change this such that sources and closures that are too large get
distributed via the BLOB manager.
BTW: There is an open pull request with a fix for the "pulling data back" part.
In the same way, we will address this issue.
(https://github.com/apache/flink/pull/934)
> Flink hangs before starting execution
> -------------------------------------
>
> Key: FLINK-2603
> URL: https://issues.apache.org/jira/browse/FLINK-2603
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime
> Affects Versions: master, 0.9.1
> Reporter: Greg Hogan
>
> The following simple application created from Flink's Java quickstart will
> hang before execution if the data size is to large. I have tested this on
> both 0.9.1 and master (7364ce18) and the threshold occurs with
> $ ./bin/flink run /path/to/blocks/target/blocks-1.0-SNAPSHOT.jar 19
> running in a few seconds while
> $ ./bin/flink run /path/to/blocks/target/blocks-1.0-SNAPSHOT.jar 20
> seemingly hangs forever.
> I first put this together two months ago so if it is a bug it is not a new
> bug.
> {code}
> package blocks;
> import java.util.ArrayList;
> import java.util.List;
> import org.apache.flink.api.java.ExecutionEnvironment;
> import org.apache.flink.api.java.io.DiscardingOutputFormat;
> public class Job {
> public static void main(String[] args) throws Exception {
> // set up the execution environment
> final ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
> List<Long> data = new ArrayList<>();
> int count = 1 << Integer.valueOf(args[0]);
>
> for (int i = 0 ; i < count ; i++) {
> data.add(0L);
> }
> env.fromCollection(data).output(new DiscardingOutputFormat());
> System.out.println("Ready to execute ...");
>
> // execute program
> env.execute("Flink Java API Skeleton");
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
