zhuzhurk commented on a change in pull request #7255: [FLINK-10945] Use
InputDependencyConstraint to avoid resource dead…
URL: https://github.com/apache/flink/pull/7255#discussion_r241636524
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
##########
@@ -755,23 +775,10 @@ else if (numConsumers == 0) {
// TODO The current approach may send many
update messages even though the consuming
// task has already been deployed with all
necessary information. We have to check
// whether this is a problem and fix it, if it
is.
- CompletableFuture.supplyAsync(
- () -> {
- try {
- final ExecutionGraph
executionGraph = consumerVertex.getExecutionGraph();
-
consumerVertex.scheduleForExecution(
-
executionGraph.getSlotProvider(),
-
executionGraph.isQueuedSchedulingAllowed(),
-
LocationPreferenceConstraint.ANY, // there must be at least one known location
-
Collections.emptySet());
- } catch (Throwable t) {
- consumerVertex.fail(new
IllegalStateException("Could not schedule consumer " +
- "vertex
" + consumerVertex, t));
- }
-
- return null;
- },
- executor);
+ if
(consumerVertex.checkInputDependencyConstraints()) {
Review comment:
From my understanding, the TODO comment is related to the "`consumerState ==
CREATED`" section in `scheduleOrUpdateConsumers`, which invokes
`cachePartitionInfo` first and then schedules the vertex. The
cachePartitionInfo action is needed to avoid deployment race, at the cost of
redundant partition infos to update to task, which is the concern as described
in the TODO comment.
So far the redundant partition it's not a big problem. But I think we can
optimize it later. One possible solution in my mind is to remove known
partition infos from the cache when creating InputChannelDeploymentDescriptor.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
