[
https://issues.apache.org/jira/browse/FLINK-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089784#comment-14089784
]
ASF GitHub Bot commented on FLINK-909:
--------------------------------------
GitHub user StephanEwen opened a pull request:
https://github.com/apache/incubator-flink/pull/91
[FLINK-909], [FLINK945] Remove additional superstep at the end of
iterrations
Fixes simultaneously the pitfall reported in [FLINK-909] and the memory
release bug reported in [FLINK-945].
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink iterbug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/91.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #91
----
commit 76a48df00f0fc202bdedefb33d576fd9a18a92c0
Author: Stephan Ewen <[email protected]>
Date: 2014-08-07T20:00:51Z
[FLINK-909] Remove additional empty (and non empty for iterative broadcast
variables) superstep.
[FLINK-945] Fix early memory release in iterations
----
> Pitfall due to additional superstep after the iteration has stopped
> -------------------------------------------------------------------
>
> Key: FLINK-909
> URL: https://issues.apache.org/jira/browse/FLINK-909
> Project: Flink
> Issue Type: Bug
> Reporter: GitHub Import
> Assignee: Markus Holzemer
> Labels: github-import
> Fix For: pre-apache
>
>
> Currently, after an iteration has exceeded the maximum number of iterations,
> all tasks are started again for an additional superstep during which they are
> stopped. This works if a tasks only waits for dynamic input. However, in the
> case where one has a task, e.g. a coGroup operation, which gets dynamic and
> static input the execution is not blocked. This can then lead to erroneous
> behaviour which the user is not aware of.
> I had this problem implementing ALS. Here one has a loop which gets as
> dynamic input matrix columns and as static input matrix entries. The columns
> and the entries are used to construct a matrix which represents a system of
> linear equations. If the set of columns are empty, then the matrix is
> singular and thus not solvable. During the additional superstep the task
> won't receive any columns but would still try to solve the now singular
> matrix.
> It would be good to finish the iteration without initiating this additional
> superstep.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/909
> Created by: [tillrohrmann|https://github.com/tillrohrmann]
> Labels:
> Created at: Thu Jun 05 17:50:17 CEST 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.2#6252)
