Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1360
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158334929
I think we agree that we want to set the number of task slots to the
maximum parallelism instead of the default one. I'll merge this later on.
---
If your project is set up
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158052735
Yes I think so. I guess it's due to the difference that batch programs are
executed using a `PlanExecutor` whereas streaming programs are directly
executed by the `
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158049463
Exactly. But that makes sense, right? If the users explicitly sets the
number of task slots, we shouldn't change the number of task slots
automatically.
---
If your projec
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158047994
But this is only true if the `taskManagerNumSlots` in `LocalExecutor` are
left untouched. Currently, this is the case but this is not enforced.
---
If your project
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158032582
Just checked. The batch side always uses the maximum parallelism as the
number of task slots (if they are not set explicitly). Till and me actually
thought differently. So t
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/1360#discussion_r45327211
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/LocalStreamEnvironment.java
---
@@ -91,7 +91,11 @@ public JobExe
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158024467
Alright. I will push the original pull request version again which uses the
max parallelism of all operators. Further, I will open a separate JIRA for the
batch side change.
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/1360#discussion_r45326391
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/LocalStreamEnvironment.java
---
@@ -91,7 +91,11 @@ public JobExec
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158022328
+1 for taking the max parallelism of all operators
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158020011
I would also be in favor of changing the local execution to always use the
maximum specified parallelism as the number of task slots. IMHO the current
behavior is not intuit
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158019230
Yes, in such a case it would fail. Thinking about it, you're right that it
would give a better user experience if the maximum degree of the job is taken
instead of
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-158009498
True. That's how it is handled on the batch side. Not sure about this
behavior though. If a user sets a default parallelism but uses operators with
`parallelism > defaultPar
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1360#issuecomment-157766781
I think the maximum parallelism of the job should be taken if the user has
not specified a different parallelism than the default one (`-1`). I think
that's also ho
14 matches
Mail list logo
