[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/378 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-11 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73908802 I added the UpdateTask message aggregation. I also had to rework the PartitionInfo creation to make it work with the concurrent task updates. This requires another

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-11 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73911265 The job that was previously failing is fixed with this change. We should merge this change ASAP, because its kinda impossible right now to seriously use flink

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-11 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73854842 There is a problem: https://travis-ci.org/apache/flink/jobs/50215407 ``` java.lang.IllegalStateException: Consumer state is FINISHED but was expected to be RUNNING.

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-11 Thread tillrohrmann
Github user tillrohrmann commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73851481 You're right. At the moment there is no aggregation of messages. I'll add it. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-10 Thread uce
Github user uce commented on a diff in the pull request: https://github.com/apache/flink/pull/378#discussion_r24415175 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java --- @@ -416,13 +426,26 @@ boolean

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-10 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73708514 Looks good to me. +1 We chatted about batching update task calls. Did you realize a problem with it or can we open an improvement issue for it? --- If your project

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-10 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/378#issuecomment-73693561 Very nice. I will have a detailed look later. @zentol Can you also test it with the Python API? I think you initially noticed the problem. --- If your project is

[GitHub] flink pull request: [FLINK-1489] Fixes blocking scheduleOrUpdateCo...

2015-02-09 Thread tillrohrmann
GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/378 [FLINK-1489] Fixes blocking scheduleOrUpdateConsumers message calls Replaces the blocking calls with futures which in case of an exception let the respective task fail. Furthermore, the