[
https://issues.apache.org/jira/browse/FLINK-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247022#comment-14247022
]
ASF GitHub Bot commented on FLINK-1325:
---------------------------------------
GitHub user aljoscha opened a pull request:
https://github.com/apache/incubator-flink/pull/269
[FLINK-1325] Add Java ClosureCleaner
This also adds the ExecutionConfig, where the user can enable/disable the
closure cleaner and other settings. I'm also planning to use this for the
object-reuse settings in the future.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/incubator-flink java-closure-cleaner
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/269.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #269
----
commit 685da9e47b89a56e5f900d1a7f9b5ab77bc51c67
Author: Aljoscha Krettek <[email protected]>
Date: 2014-12-15T17:37:42Z
Add ExecutionConfig
commit fe51a9c50963bb5c0a6fd0beb8db57c94685adbc
Author: Aljoscha Krettek <[email protected]>
Date: 2014-12-15T18:50:14Z
[FLINK-1325] Add Java ClosureCleaner
----
> Add a closure cleaner for Java
> ------------------------------
>
> Key: FLINK-1325
> URL: https://issues.apache.org/jira/browse/FLINK-1325
> Project: Flink
> Issue Type: Improvement
> Components: Java API
> Reporter: Stephan Ewen
> Assignee: Aljoscha Krettek
>
> The Java API could really need a simple closure cleaner.
> All functions that are implemented as anonymous subclasses hold a reference
> to the enclosing class, unless they are implemented as part of a static
> method.
> That reference (called {{this$0}}) causes serialization to fail, as it draws
> non serializable classes into the function, even in cases where the function
> makes no access to the enclosing data.
> It is possible to manually set this reference to {{null}}, using reflection,
> or using a debugger. Then the serialization succeeds.
> I suggest to add a closure cleaner that uses an ASM visitor over the
> function's code to see if there is any access to the {{this$0}} field. In
> case there is non, the field should be set to {{null}}.
> The problem can be reproduced with the simple program below:
> {code}
> public class Test {
> public void runProgram() throws Exception {
> ExecutionEnvironment env = ExecutionEnvironment
> .getExecutionEnvironment();
> env.generateSequence(1, 10)
> .map(new MapFunction<Long, Long>() {
> public Long map(Long value) {
> return value * 2;
> }
> })
> .print();
> env.execute();
> }
> public static void main(String[] args) throws Exception {
> new Test().runProgram();
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
